Just took a peek at your forum, even registered so I could get a good look at the size of your forum.

It's not all that large, so even having the total # of unread topics/posts turned on should be that bad at all. I was thinking maybe the search could be doing it as sometimes it can lock while doing a large search over all posts, however on the size of your forum, that shouldn't be an issue either.

Are you logging slow queries in your mysql configuration?