That tag is designed for the bot to archive your forum but not follow all the links to the threads, they do not need to archive every page. Also will stop most dumb bots from registering.
Another good thing about the tag is you will not be lagged out by the bot archiving all your pages and crawling your entire board. If you dont have a large board it is not much of a problem but my one board has 30,000 and up post's and pages.
