You don't need to add the reference. The person who originally need that reference does so. Then everyone else puts the DLLs in the GAC. Apps always look in the GAC -first-, then try to resolve the reference locally on disk... since it finds it in the GAC, it will never try to look for it elsewhere.
The original person will have to have the assembly on disk normally to -add- the reference, but thats the only time it will be required. After that, they just checkin the project files, and everyone who has the assemblies in the GAC will not have any problem.