Does the speed of the site affect the top search engines?

Yes it does. Not only does it affect your ranking, but the average user will just close the tab if it takes too long to load.