#87362
04/08/2004 8:05 PM
Joined: Jan 2004
Posts: 21
stranger
stranger
Joined: Jan 2004
Posts: 21
I have a few forums that I don't want indexed by search engines. I entered the following into robots.txt but the forum is still being crawled:
Disallow: /ubb/Forum9/
Disallow: /ubb/ultimatebb.php/forum/9.html
What exactly should I put after Disallow: ?
Thanks
<img src="https://www.ubbcentral.com/boards/images/graemlins/smile.gif" alt="" />
#87363
04/08/2004 8:26 PM
Joined: Jun 2006
Posts: 16,532
Likes: 150
Joined: Jun 2006
Posts: 16,532
Likes: 150
The only way to truely stop a search engine from indexing a section is to not have the function enabled...
#87364
04/08/2004 8:35 PM
Anonymous
Unregistered
Anonymous
Unregistered
if you make the forum password protected, it won't index anything but an "Enter password" page
#87365
04/08/2004 8:48 PM
Joined: Jun 2006
Posts: 16,532
Likes: 150
Joined: Jun 2006
Posts: 16,532
Likes: 150
That or install the "Post Count Restricted Forums" mod and set it to 1 lol...
#87366
04/08/2004 8:51 PM
Anonymous
Unregistered
Anonymous
Unregistered
Gizzy is of course talking about the modification at UBBDEV that HASN'T BEEN RELEASED YET!
#87367
04/09/2004 7:19 AM
Joined: Jan 2004
Posts: 21
stranger
stranger
Joined: Jan 2004
Posts: 21
OK, so robots.txt might not be the best way to go. Is it possible to add:
<meta name="robots" content="noindex,nofollow"> to individual forum headers?
#87368
04/09/2004 12:02 PM
Joined: Jun 2006
Posts: 16,532
Likes: 150
Joined: Jun 2006
Posts: 16,532
Likes: 150
That should work so long as the robot listens to meta tags <img src="https://www.ubbcentral.com/boards/images/graemlins/wink.gif" alt="" /> ... You'll find a few rogue's who don't listen to anyone; or anything for that matter.
0 members (),
166
guests, and
137
robots.
Key:
Admin ,
Global Mod ,
Mod
