Originally Posted by Gizmo
Originally Posted by jgeoff
One question, though - what is to prevent Google from indexing links that are NOT on the submitted sitemap?
You could add a robots.txt rule to not spider your forums, then use a sitemap... although I'm not sure how this would work, since Id on't know if google would still crawl th edata in the sitemap...

I think that was my point back on Page 1 -- If you exclude a directory with robots.txt then it won't be crawled, and I'm pretty sure regardless of what's in your sitemap. That's just how it works (as I understand it). And that's why I keep asking. wink


GangsterBB.NET (Ver. 7.6.1.1)
PHP Version 8.1.34 / MySQL 5.7.23-23 / Apache 2.4.67
2007 Content Rulez Contest - Hon Mention
UBB.classic 6.7.2 - RIP