The first thing I would do is turn off Recent Visitors. That will help reduce some of the load on the server. In addition, I believe it is recommended that you have no more than 5000 topics in a forum.
Approximately how many pageviews per month are you getting? If you don't know, then you could give the current, and we could guesstimate.
