Well there are a couple things that really do need to be adjusted on the server itself for best performance. That would be the webserver and mysql configurations. If either or both of those configurations aren't done properly, you could have a very nice server that performs very poorly.
