You could give the moderator group no access to the moderator forum, then make a new group and give THAT access to the moderator forum and add all of your moderators to that forum...
Not sure how it'd work out, but it'd be worth a try, and you could test it by using the "become this user" feature
