Previous Thread
Next Thread
Print Thread
Hop To
#250820 11/07/2012 11:41 AM
Joined: Jun 2006
Posts: 391
addict
addict
Joined: Jun 2006
Posts: 391
Is there anyway to complete block search bots from indexing your site? I am starting a new site, but need to block them. Thanks


Greg AKA Virgil Earp at the OK Corral
Tombstone, AZ
GregK #250821 11/07/2012 12:41 PM
Joined: Jan 2004
Posts: 2,474
Likes: 3
D
Pooh-Bah
Pooh-Bah
D Offline
Joined: Jan 2004
Posts: 2,474
Likes: 3
There is smile

Create a file in your main directory (usually htdocs) called robots.txt

In it, place this text...
Quote
User-agent: *
Disallow: /

This will stop 99.9% of robots. The only ones it doesn't stop are the ones that don't follow the robots exclusion protocol.
(The naughty bots.)


Link Copied to Clipboard
ShoutChat
Comment Guidelines: Do post respectful and insightful comments. Don't flame, hate, spam.
Recent Topics
Bots
by Outdoorking - 04/13/2024 5:08 PM
Can you add html to language files?
by Baldeagle - 04/07/2024 2:41 PM
Do I need to rebuild my database?
by Baldeagle - 04/07/2024 2:58 AM
This is not a bug, but a suggestion
by Baldeagle - 04/05/2024 11:25 PM
spam issues
by ECNet - 03/19/2024 11:45 PM
Who's Online Now
0 members (), 686 guests, and 131 robots.
Key: Admin, Global Mod, Mod
Random Gallery Image
Latest Gallery Images
Los Angeles
Los Angeles
by isaac, August 6
3D Creations
3D Creations
by JAISP, December 30
Artistic structures
Artistic structures
by isaac, August 29
Stones
Stones
by isaac, August 19
Powered by UBB.threads™ PHP Forum Software 8.0.0
(Preview build 20230217)