Not exactly.

You add the class library project to the solution first (right click on Solution, Add Project, Class Library).

Then, you build whatever functionality you want into the library.

Then, you add a reference to the above project from the web site (right click on web site, Add Reference, Projects tab).