Since everyone is replying with something along the lines of deleting your cache and no one is mentioning how to ill explain for those who want to know but don't know how to. These are the steps for Windows, I am not sure how these hold up on any other OS.
Step 1: Open your steam library and find Garry's Mod Step 2: Right click Garry's Mod within the your list of games (you can also click the gear icon towards the right of the games page ) and a small menu with a few options should popup. There should be and option called Manage with a arrow pointing to the right, HOVER over the word with your mouse and another set of options should appear, CLICK on the option labeled Browse Local Files. Step 3: A file explorer window should open up containing the game files. You will need to navigate to the directory garrysmod/cache This file should contain the folders lua, workshop, DownloadCache.db there maybe some extra files but these files should already be there. Step 4: Delete everything within garrysmod/cache
When you join a server files from the server are downloaded into your cache to help speed up the loading time for then next time you join back, however if you join lots of servers over time this file gets quite large. You should delete the contents of your cache folder every once in a while.