If someone is assigned to a board when you edit the user, one of the options is "Remove from moderator list". This will allow you to remove a moderator. If noone is assigned to a forum, then the main admin is the moderator by default, and there is no way to remove them from this list as it's generated for display on the fly.