Joined: Jun 2006
Posts: 3,837
Carpal Tunnel
Carpal Tunnel
Joined: Jun 2006
Posts: 3,837
First time I have ever seen 0 spiders online on my forums. I must have upset them all LOL
Joined: Jan 2004
Posts: 2,474
Likes: 3
Pooh-Bah
Pooh-Bah
Joined: Jan 2004
Posts: 2,474
Likes: 3
That's very odd - never seen that before.
How could you account for it?
Joined: Jun 2006
Posts: 16,299
Likes: 116
Joined: Jun 2006
Posts: 16,299
Likes: 116
could be that you don't have enough user agents setup
Joined: Jun 2006
Posts: 3,837
Carpal Tunnel
Carpal Tunnel
Joined: Jun 2006
Posts: 3,837
hmmm........
my list is empty - very strange
Can someone post a nice decent list for me - thanks.
Joined: Jan 2004
Posts: 2,474
Likes: 3
Pooh-Bah
Pooh-Bah
Joined: Jan 2004
Posts: 2,474
Likes: 3
Here's one to start you off... Alexa=ia_archiver AllTheWeb=crawler@fast AllTheWeb=FAST-WebCrawler AltaVista=Mercator Altavista=Scooter Ask.com=Ask Jeeves Ask.com=teoma_agent1 AskJeeves=AskJeeves BecomeBot=BecomeBot BoardReader=BoardReader Crawler.de=Crawler EntireWeb.com=Speedy Spider Entireweb=Speedy Spider Excite=ArchitextSpider Gigablast Gigabot=Gigabot Gigabot=Gigabot Google Adsense=Mediapartners-Google Google Images=Googlebot-Image Google Mobile=Googlebot-Mobile Google=Googlebot Infoseek=InfoSeek sidewinder InfoSeek=UltraSeek Inktomi=Slurp Looksmart=MantraAgent Lycos=Lycos_Spider_(T-Rex) MJ12bot=MJ12bot MSN=MSNBOT Naver.com=Yeti/ NaverBot=NaverBot NorthernLight.com=Gulliver ScrubTheWeb=Scrubby Search Hippo=Fluffy the Spider SingingFish=asterias Snapbot=Snapbot SnapPreviewBot=SnapPreviewBot Sogou=sogou web spider StackRambler(ru)=StackRambler TAMU.edu=IRLbot Teoma=TeomaAgent TMCrawler=TMCrawler Twiceler=Twiceler-0.9 Voila=Voila Voyager=Voyager Webcrawler.com=WebCrawler Whois.sc=SurveyBot Wisenut=Zyborg Yahoo=Yahoo! Slurp YodaoBot=YodaoBot
Joined: Jun 2006
Posts: 3,837
Carpal Tunnel
Carpal Tunnel
Joined: Jun 2006
Posts: 3,837
That driv - appreciated
Joined: Jun 2006
Posts: 3,837
Carpal Tunnel
Carpal Tunnel
Joined: Jun 2006
Posts: 3,837
Alexa=ia_archiver
AllTheWeb=crawler@fast
AllTheWeb=FAST-WebCrawler
AltaVista=Mercator
Altavista=Scooter
Anzwers=AnzwersCrawl
Ask.com=Ask Jeeves
Ask.com=teoma_agent1
Ask=Teoma
Atomz=Atomz
AskJeeves=AskJeeves
BecomeBot=BecomeBot
BoardReader=BoardReader
Boitho=boitho.com
Crawler.de=Crawler
DMOZ=Robozilla
EntireWeb.com=Speedy Spider
Entireweb=Speedy Spider
Excite=ArchitextSpider
Exalead=Exabot
Factbites=Factbot
Fast=FAST
Gigablast Gigabot=Gigabot
Gigabot=Gigabot
Google Adsense=Mediapartners-Google
Google Images=Googlebot-Image
Google Mobile=Googlebot-Mobile
Google=Googlebot
Infoseek=InfoSeek sidewinder
InfoSeek=UltraSeek
Inktomi=Slurp
Looksmart=MantraAgent
LookSmart=FurlBot
Lycos=Lycos_Spider_(T-Rex)
MJ12bot=MJ12bot
MSN=MSNBOT
Microsoft Research=MSRBOT
MSN=MSNBOT
MSN Pic-Search=PSbot
Naver.com=Yeti/
NaverBot=NaverBot
NetSeer=Teemer
NorthernLight.com=Gulliver
noXtrum=noxtrumbot
Pagebull=Pagebull
ScrubTheWeb=Scrubby
Searchme=Charlotte
Seznam=SeznamBot
Search Hippo=Fluffy the Spider
SingingFish=asterias
Snapbot=Snapbot
SnapPreviewBot=SnapPreviewBot
Sogou=sogou web spider
StackRambler(ru)=StackRambler
TAMU.edu=IRLbot
Teoma=TeomaAgent
TMCrawler=TMCrawler
Twiceler=Twiceler-0.9
Voila=Voila
Voila=VoilaBot
Voyager=Voyager
Walhello=appie
Webcrawler.com=WebCrawler
Whois.sc=SurveyBot
Wisenut=Zyborg
Yahoo=Yahoo! Slurp
YodaoBot=YodaoBot
Yahoo! Blogs=Yahoo-blogs/v3.9
Yahoo! MM=Yahoo-MMcrawler
Combines the ones that I have pulled from other posts and his list
Joined: Jun 2006
Posts: 3,837
Carpal Tunnel
Carpal Tunnel
Joined: Jun 2006
Posts: 3,837
Slowly showing up now. Just wondered though if anyone knows why no locations is showing against the bots?
Joined: Jan 2004
Posts: 2,474
Likes: 3
Pooh-Bah
Pooh-Bah
Joined: Jan 2004
Posts: 2,474
Likes: 3
ips you mean? dunno - nothing daft like some 'spaces' crept into the text list is there?
Joined: Jun 2006
Posts: 3,837
Carpal Tunnel
Carpal Tunnel
Joined: Jun 2006
Posts: 3,837
No where is says... Reading a post Forum: Billericay Town Thread: LATEST PLAYER NEWS This appears for members and guests, but not spiders.
Joined: Jun 2006
Posts: 16,299
Likes: 116
Joined: Jun 2006
Posts: 16,299
Likes: 116
I'll merge this list with mine and repost later
Joined: Jun 2006
Posts: 16,299
Likes: 116
Joined: Jun 2006
Posts: 16,299
Likes: 116
Merged several lists here and cut out duplicates:
Alexa=ia_archiver
AltaVista=Mercator
Altavista=Scooter
AllTheWeb=FAST-WebCrawler
Anzwers=AnzwersCrawl
AllTheWeb=crawler@fast
Ask.com=Ask Jeeves
Ask.com=teoma_agent1
Ask=Teoma
Atomz=Atomz
AskJeeves=AskJeeves
BecomeBot=BecomeBot
BoardReader=BoardReader
Boitho=boitho.com
Crawler.de=Crawler
DMOZ=Robozilla
EntireWeb.com=Speedy Spider
Entireweb=Speedy Spider
Excite=ArchitextSpider
Exalead=Exabot
Factbites=Factbot
Fast=FAST
Gigabot=Gigabot
Google=Googlebot
Google Mobile=Googlebot-Mobile
Google Images=Googlebot-Image
Google Adsense=Mediapartners-Google
Infoseek=InfoSeek sidewinder
InfoSeek=UltraSeek
Inktomi=Slurp
Looksmart=MantraAgent
LookSmart=FurlBot
Lycos=Lycos_Spider_(T-Rex)
MJ12bot=MJ12bot
MSN=MSNBOT
Microsoft Research=MSRBOT
MSN=MSNBOT
MSN Pic-Search=PSbot
Naver.com=Yeti/
NaverBot=NaverBot
NetSeer=Teemer
NorthernLight.com=Gulliver
noXtrum=noxtrumbot
Pagebull=Pagebull
ScrubTheWeb=Scrubby
Searchme=Charlotte
Seznam=SeznamBot
Search Hippo=Fluffy the Spider
SingingFish=asterias
Snapbot=Snapbot
SnapPreviewBot=SnapPreviewBot
Sogou=sogou web spider
StackRambler(ru)=StackRambler
TAMU.edu=IRLbot
Teoma=TeomaAgent
TMCrawler=TMCrawler
Twiceler=Twiceler-0.9
Voila=Voila.fr
Voila=Voila
Voila=VoilaBot
Voyager=Voyager
Walhello=appie
Webcrawler.com=WebCrawler
Whois.sc=SurveyBot
Wisenut=Zyborg
Yahoo=Yahoo! Slurp
Yahoo=Yahoo Slurp
YodaoBot=YodaoBot
Yahoo! Blogs=Yahoo-blogs/v3.9
Yahoo! MM=Yahoo-MMcrawler
Joined: Dec 2006
Posts: 1,235
veteran
veteran
Joined: Dec 2006
Posts: 1,235
Some additional bots to squeeze in to your list:
DepSpid=DepSpid
Fast(AllTheWeb)=FAST-WebCrawler
Fast(AllTheWeb)=crawler@fast
Gigablast=GigaBot
Nutch=nutchsearch
Picsearch=psbot
Shelob=shelob
WebAlta=WebAlta
Worio=woriobot
YaCy=yacybot
Yahoo! Germany=Yahoo! DE Slurp
Yoono=yoofind
Joined: Dec 2006
Posts: 1,235
veteran
veteran
Joined: Dec 2006
Posts: 1,235
I've just put your list in and Yahoo! displays as Inktomi. Remove Yahoo=Yahoo Slurp from your list and Yahoo! will display correctly.
Joined: Jun 2006
Posts: 16,299
Likes: 116
Joined: Jun 2006
Posts: 16,299
Likes: 116
Weird that it's showing as inktomi, but I wouldn't want to remove it, because yahoo has 2 brands: Yahoo=Yahoo! Slurp Yahoo=Yahoo Slurp Wondering why the one would work and the other wouldn't... I'd figure it would have been the one with the ! that wouldn't work... I'll play with it more later today.
Joined: Jun 2006
Posts: 16,299
Likes: 116
Joined: Jun 2006
Posts: 16,299
Likes: 116
This is quite curious; all of the yahoo ones are being flagged as inktomi; and I mean all of them... When I remove it (inktomi) they just show blank lol
Joined: Jun 2006
Posts: 16,299
Likes: 116
Joined: Jun 2006
Posts: 16,299
Likes: 116
I'm pretty sure when you remove a bot from the listing in the cp that it's supposed to go back to anon right? So why would it be missing from my list, but be listed as a bot with no name lol...
Attachments
Last edited by Gizmo; 05/15/2008 9:20 AM .
Joined: Aug 2006
Posts: 1,649
Likes: 1
Pooh-Bah
Pooh-Bah
Joined: Aug 2006
Posts: 1,649
Likes: 1
Yahoo=Yahoo! Slurp works just fine for me. In fact, they're my biggest "customer" lolInktomi=Slurp is on my list, but haven't yet noticed it. The first is listed higher in my list, so maybe that's why it's showing correctly. Just put Yahoo above Inktomi (tho maybe some of the Yahoo's are actually Inktomi? If so, the logic is off a bit)
GangsterBB.NET (Ver. 7.6.1.1)
PHP Version 5.6.40 / MySQL 5.7.23-23 (was 5.6.41-84.1) / Apache 2.4.54
2007 Content Rulez Contest - Hon Mention
UBB.classic 6.7.2 - RIP
Joined: Jun 2006
Posts: 3,837
Carpal Tunnel
Carpal Tunnel
Joined: Jun 2006
Posts: 3,837
Just wondered though if anyone knows why no locations is showing against the bots?
Anyone?
Joined: Aug 2006
Posts: 1,649
Likes: 1
Pooh-Bah
Pooh-Bah
Joined: Aug 2006
Posts: 1,649
Likes: 1
No idea, sorry, it's working just fine for me...
GangsterBB.NET (Ver. 7.6.1.1)
PHP Version 5.6.40 / MySQL 5.7.23-23 (was 5.6.41-84.1) / Apache 2.4.54
2007 Content Rulez Contest - Hon Mention
UBB.classic 6.7.2 - RIP
Joined: Aug 2006
Posts: 1,649
Likes: 1
Pooh-Bah
Pooh-Bah
Joined: Aug 2006
Posts: 1,649
Likes: 1
Giz, this is a duplicate: EntireWeb.com=Speedy Spider Entireweb=Speedy Spider Also, I think some of these aren't on there (my latest additions): Amazon=AMZNKAssocBot Omgili=omgilibot Snoopy=Snoopy Kalooga=Kalooga
GangsterBB.NET (Ver. 7.6.1.1)
PHP Version 5.6.40 / MySQL 5.7.23-23 (was 5.6.41-84.1) / Apache 2.4.54
2007 Content Rulez Contest - Hon Mention
UBB.classic 6.7.2 - RIP
Joined: Jun 2006
Posts: 3,837
Carpal Tunnel
Carpal Tunnel
Joined: Jun 2006
Posts: 3,837
Updated list Yahoo=Yahoo! Slurp Yahoo! Germany=Yahoo! DE Slurp Alexa=ia_archiver AllTheWeb=crawler@fast AllTheWeb=FAST-WebCrawler AltaVista=Mercator Altavista=Scooter Amazon=AMZNKAssocBot Anzwers=AnzwersCrawl Ask.com=Ask Jeeves Ask.com=teoma_agent1 Ask=Teoma Atomz=Atomz AskJeeves=AskJeeves BecomeBot=BecomeBot BoardReader=BoardReader Boitho=boitho.com Crawler.de=Crawler DepSpid=DepSpid DMOZ=Robozilla Entireweb=Speedy Spider Excite=ArchitextSpider Exalead=Exabot Factbites=Factbot Fast=FAST Fast(AllTheWeb)=FAST-WebCrawler Fast(AllTheWeb)=crawler@fast Gigabot=Gigabot Google Adsense=Mediapartners-Google Google Images=Googlebot-Image Google Mobile=Googlebot-Mobile Google=Googlebot Google Sitemaps=Google-Sitemaps/1.0 Infoseek=InfoSeek sidewinder InfoSeek=UltraSeek Inktomi=Slurp Kalooga=Kalooga Looksmart=MantraAgent LookSmart=FurlBot Lycos=Lycos_Spider_(T-Rex) MJ12bot=MJ12bot MSN=MSNBOT Microsoft Research=MSRBOT MSN=MSNBOT MSN Pic-Search=PSbot Naver.com=Yeti/ NaverBot=NaverBot NetSeer=Teemer NorthernLight.com=Gulliver noXtrum=noxtrumbot Nutch=nutchsearch Omgili=omgilibot Pagebull=Pagebull Picsearch=psbot ScrubTheWeb=Scrubby Searchme=Charlotte Seznam=SeznamBot Search Hippo=Fluffy the Spider Shelob=shelob SingingFish=asterias Snapbot=Snapbot SnapPreviewBot=SnapPreviewBot Snoopy=Snoopy Sogou=sogou web spider StackRambler(ru)=StackRambler StumbleUpon=StumbleUpon.com TAMU.edu=IRLbot Teoma=TeomaAgent TMCrawler=TMCrawler Twiceler=Twiceler-0.9 Voila=Voila.fr Voila=Voila Voila=VoilaBot Voyager=Voyager Walhello=appie WebAlta=WebAlta Webcrawler.com=WebCrawler Whois.sc=SurveyBot Wisenut=Zyborg Worio=woriobot YaCy=yacybot YodaoBot=YodaoBot Yahoo! Blogs=Yahoo-blogs/v3.9 Yahoo! MM=Yahoo-MMcrawler Yoono=yoofind
Joined: Aug 2006
Posts: 1,649
Likes: 1
Pooh-Bah
Pooh-Bah
Joined: Aug 2006
Posts: 1,649
Likes: 1
Thanks, Ian! Also: StumbleUpon=StumbleUpon.com was on my list
GangsterBB.NET (Ver. 7.6.1.1)
PHP Version 5.6.40 / MySQL 5.7.23-23 (was 5.6.41-84.1) / Apache 2.4.54
2007 Content Rulez Contest - Hon Mention
UBB.classic 6.7.2 - RIP
Joined: Aug 2006
Posts: 1,649
Likes: 1
Pooh-Bah
Pooh-Bah
Joined: Aug 2006
Posts: 1,649
Likes: 1
Now I'm getting that blank bot name, though (which appear to actually be legit Guests -- I'm getting 0 Guests now), and Inktomi replacing Yahoo
Placing
Yahoo=Yahoo! Slurp
Yahoo! Germany=Yahoo! DE Slurp
above Inktomi corrects this.
Will have to try and figure this out later; I gotta run soon.
GangsterBB.NET (Ver. 7.6.1.1)
PHP Version 5.6.40 / MySQL 5.7.23-23 (was 5.6.41-84.1) / Apache 2.4.54
2007 Content Rulez Contest - Hon Mention
UBB.classic 6.7.2 - RIP
Joined: Jun 2006
Posts: 16,299
Likes: 116
Joined: Jun 2006
Posts: 16,299
Likes: 116
Yeh, it's definately a matching problem where inktomi is atching before yahoo is listed; so just keep the yahoo on top
...
Have a few more to merge in :sigh:
Joined: Jun 2006
Posts: 16,299
Likes: 116
Joined: Jun 2006
Posts: 16,299
Likes: 116
Latest list (that is working):
Yahoo=Yahoo! Slurp
Yahoo! Germany=Yahoo! DE Slurp
Alexa=ia_archiver
AltaVista=Mercator
Altavista=Scooter
AllTheWeb=FAST-WebCrawler
Amazon=AMZNKAssocBot
Anzwers=AnzwersCrawl
AllTheWeb=crawler@fast
Ask.com=Ask Jeeves
Ask.com=teoma_agent1
Ask=Teoma
Atomz=Atomz
AskJeeves=AskJeeves
BecomeBot=BecomeBot
BoardReader=BoardReader
Boitho=boitho.com
Crawler.de=Crawler
DepSpid=DepSpid
DMOZ=Robozilla
EntireWeb.com=Speedy Spider
Excite=ArchitextSpider
Exalead=Exabot
Factbites=Factbot
Fast=FAST
Gigabot=Gigabot
Google=Googlebot
Google Mobile=Googlebot-Mobile
Google Images=Googlebot-Image
Google Adsense=Mediapartners-Google
Google Sitemaps=Google-Sitemaps/1.0
Infoseek=InfoSeek sidewinder
InfoSeek=UltraSeek
Inktomi=Slurp
Kalooga=Kalooga
Looksmart=MantraAgent
LookSmart=FurlBot
Lycos=Lycos_Spider_(T-Rex)
MJ12bot=MJ12bot
MSN=MSNBOT
Microsoft Research=MSRBOT
MSN=MSNBOT
MSN Pic-Search=PSbot
Naver.com=Yeti/
NaverBot=NaverBot
NetSeer=Teemer
NorthernLight.com=Gulliver
noXtrum=noxtrumbot
Nutch=nutchsearch
Omgili=omgilibot
Pagebull=Pagebull
Picsearch=psbot
ScrubTheWeb=Scrubby
Searchme=Charlotte
Seznam=SeznamBot
Search Hippo=Fluffy the Spider
Shelob=shelob
SingingFish=asterias
Snapbot=Snapbot
SnapPreviewBot=SnapPreviewBot
Snoopy=Snoopy
Sogou=sogou web spider
StackRambler(ru)=StackRambler
StumbleUpon=StumbleUpon.com
TAMU.edu=IRLbot
Teoma=TeomaAgent
TMCrawler=TMCrawler
Twiceler=Twiceler-0.9
Voila=Voila.fr
Voila=Voila
Voila=VoilaBot
Voyager=Voyager
Walhello=appie
WebAlta=WebAlta
Webcrawler.com=WebCrawler
Whois.sc=SurveyBot
Wisenut=Zyborg
Worio=woriobot
YaCy=yacybot
YodaoBot=YodaoBot
Yahoo! Blogs=Yahoo-blogs/v3.9
Yahoo! MM=Yahoo-MMcrawler
Yoono=yoofind
Joined: Aug 2006
Posts: 1,649
Likes: 1
Pooh-Bah
Pooh-Bah
Joined: Aug 2006
Posts: 1,649
Likes: 1
Any idea what caused Guests to end up in the Bots section before Giz's last list? Also - I would remove the .com from StumbleUpon.com -- or at least add: StumbleUpon=StumbleUpon I'm not sure if I added the .com to the wrong side, or, whether it was another one of their bots. But without it works (because it's on my site right now lol)
GangsterBB.NET (Ver. 7.6.1.1)
PHP Version 5.6.40 / MySQL 5.7.23-23 (was 5.6.41-84.1) / Apache 2.4.54
2007 Content Rulez Contest - Hon Mention
UBB.classic 6.7.2 - RIP
Joined: Jun 2006
Posts: 16,299
Likes: 116
Joined: Jun 2006
Posts: 16,299
Likes: 116
'eh for lesser known things i throw in the .com so it's easier to figure where they're from...
Joined: Aug 2006
Posts: 1,649
Likes: 1
Pooh-Bah
Pooh-Bah
Joined: Aug 2006
Posts: 1,649
Likes: 1
then use StumbleUpon.com=StumbleUpon StumbleUpon.com may or may not be the bot's name, but I know for sure StumbleUpon is one of them...
GangsterBB.NET (Ver. 7.6.1.1)
PHP Version 5.6.40 / MySQL 5.7.23-23 (was 5.6.41-84.1) / Apache 2.4.54
2007 Content Rulez Contest - Hon Mention
UBB.classic 6.7.2 - RIP
Joined: Aug 2006
Posts: 1,649
Likes: 1
Pooh-Bah
Pooh-Bah
Joined: Aug 2006
Posts: 1,649
Likes: 1
WTH is cr4nk.ws? Doesn't appear very friendly on the surface... Just noticed this one tonight ((at least) two different IPs: 193.189.149.55 | 77.79.246.200 ) ...one Netherlands, the other Poland, but claiming to be [de]?... Suspicious...
Last edited by jgeoff; 06/03/2008 3:21 AM .
GangsterBB.NET (Ver. 7.6.1.1)
PHP Version 5.6.40 / MySQL 5.7.23-23 (was 5.6.41-84.1) / Apache 2.4.54
2007 Content Rulez Contest - Hon Mention
UBB.classic 6.7.2 - RIP
Bots
by Outdoorking - 04/13/2024 5:08 PM