OK, I think I figured out the issue.

If I enable a Teaser forum, it also shows all portal islands.

So, now how do I hide teaser forums (since we have levels of permissions to registered members) and if that isn't possible, how do I hide portal islands to only registered members?