|
Joined: Aug 2006
Posts: 1,649 Likes: 1
Pooh-Bah
|
Pooh-Bah
Joined: Aug 2006
Posts: 1,649 Likes: 1 |
I noticed that some of my "Guests" were really undefined spiders, so I added them (and some others I've heard of) to the list. Control Panel » Display Options » Who's Online: DEFAULT LIST:Alexa=ia_archiver Altavista=Scooter AllTheWeb=FAST-WebCrawler Excite=ArchitextSpider Google=Googlebot Yahoo=Yahoo! Slurp Inktomi=Slurp MSN=MSNBOT ADD:Whois.sc=SurveyBot Google-AdSense=Mediapartners-Google TAMU.edu=IRLbot InfoSeek=UltraSeek InfoSeek=InfoSeek Sidewinder Lycos=Lycos_Spider_(T-Rex) ScrubTheWeb=Scrubby Looksmart=MantraAgent SingingFish=asterias Ask.com=teoma_agent1 Of course if anyone feels ambitious, they could add many more of them
GangsterBB.NET (Ver. 7.6.1.1) PHP Version 5.6.40 / MySQL 5.7.23-23 (was 5.6.41-84.1) / Apache 2.4.54 2007 Content Rulez Contest - Hon Mention UBB.classic 6.7.2 - RIP
|
|
|
|
Joined: Jun 2006
Posts: 3,837
Carpal Tunnel
|
Carpal Tunnel
Joined: Jun 2006
Posts: 3,837 |
Thanks jgeoff
|
|
|
|
Joined: Jan 2004
Posts: 2,474 Likes: 3
Pooh-Bah
|
Pooh-Bah
Joined: Jan 2004
Posts: 2,474 Likes: 3 |
Cheers jgeoff - I did try this once before but I managed to bugger it up somehow and all the spiders turned anonymous
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
Joined: Jul 2006
Posts: 4,057 |
Cheers, just checked that link and i cant understand how to understand them, as in how to write them. Good find, and cheers for your list
BOOM !! Version v7.6.1.1 People who inspire me Isaac ME Gizmo
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
Joined: Jul 2006
Posts: 4,057 |
Ive got the following showing up as a SPIDER
80.5.160.7
after a google search it comes up with
http://en.wikipedia.org/wiki/User_talk:80.5.160.7
So i entered it like so 80.5.160.7=User talk
and the that now comes up as Anonymous
How do you sort these fellas out then ? Whats the trigger to make them spiders or Anonymous ?
BOOM !! Version v7.6.1.1 People who inspire me Isaac ME Gizmo
|
|
|
|
Joined: Jun 2006
Posts: 16,292 Likes: 116
|
Joined: Jun 2006
Posts: 16,292 Likes: 116 |
You have to enter the user agent, not the IP address (although the ip address would be cool, you can easily find netblocks that various groups use). " the list" shows "Robot identifier", thats the user agent, thats what the ubb wants. So you'd put: Service=Robot identifier
|
|
|
|
Joined: Dec 2006
Posts: 1,235
veteran
|
veteran
Joined: Dec 2006
Posts: 1,235 |
I did try this once before but I managed to bugger it up somehow and all the spiders turned anonymous Messed mine up and found that the form field doesn't like an empty line / line break after the last entry.
|
|
|
|
Joined: Jun 2006
Posts: 16,292 Likes: 116
|
Joined: Jun 2006
Posts: 16,292 Likes: 116 |
yeh, a couple files don't like extra spaces
|
|
|
|
Joined: Aug 2006
Posts: 1,649 Likes: 1
Pooh-Bah
|
Pooh-Bah
Joined: Aug 2006
Posts: 1,649 Likes: 1 |
Here's an update. This is my current complete list (of those I've noticed on my site - you may have others, so be careful):
Alexa=ia_archiver
Altavista=Scooter
AllTheWeb=FAST-WebCrawler
Excite=ArchitextSpider
Google=Googlebot
Yahoo=Yahoo! Slurp
Inktomi=Slurp
MSN=MSNBOT
Whois.sc=SurveyBot
Google-AdSense=Mediapartners-Google
TAMU.edu=IRLbot
InfoSeek=UltraSeek
InfoSeek=InfoSeek Sidewinder
Lycos=Lycos_Spider_(T-Rex)
ScrubTheWeb=Scrubby
Looksmart=MantraAgent
SingingFish=asterias
Ask.com=teoma_agent1
Voila=Voila
AskJeeves=AskJeeves
Voyager=Voyager
Voila=Voila
SnapPreviewBot=SnapPreviewBot
Snapbot=Snapbot
BoardReader=BoardReader
YodaoBot=YodaoBot
TMCrawler=TMCrawler
StackRambler(ru)=StackRambler
EntireWeb.com=Speedy Spider
Gigablast Gigabot=Gigabot
Twiceler (leech?)=Twiceler
NaverBot=NaverBot
BecomeBot=BecomeBot
MJ12bot=MJ12bot
Replace Control Panel » Display Options » Who's Online » Search Engine Agents with that list (unless yours contains some not mentioned here; if so, post them ) Hopefully these extended lists will end up in a default/updated version of a future UBB release...
GangsterBB.NET (Ver. 7.6.1.1) PHP Version 5.6.40 / MySQL 5.7.23-23 (was 5.6.41-84.1) / Apache 2.4.54 2007 Content Rulez Contest - Hon Mention UBB.classic 6.7.2 - RIP
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
Joined: Jul 2006
Posts: 4,057 |
Cheers jgeoff just wasn't long enough so Thank You jgeoff
BOOM !! Version v7.6.1.1 People who inspire me Isaac ME Gizmo
|
|
|
|
Joined: Jun 2006
Posts: 16,292 Likes: 116
|
Joined: Jun 2006
Posts: 16,292 Likes: 116 |
<snicker> I have several myself, may post later
|
|
|
|
Joined: May 2008
Posts: 753 Likes: 1
Old Hand
|
Old Hand
Joined: May 2008
Posts: 753 Likes: 1 |
is there an updated spider list?
"No matter where you go, there you are." "If you can't do something smart, Do something right" "There are three kinds of people in the world, those who can count, and those who can't"
|
|
|
|
Joined: Jun 2006
Posts: 16,292 Likes: 116
|
Joined: Jun 2006
Posts: 16,292 Likes: 116 |
Will be maintaining my list here
|
|
|
|
|