When creating a new forum access to all groups should default to NOT ALLOWED except for ADMIN. Then you can grant access to the few or multiple groups.