UBBCentral
Posted By: Ian zero spiders online! - 05/14/2008 9:25 AM
First time I have ever seen 0 spiders online on my forums.

I must have upset them all LOL
Posted By: driv Re: zero spiders online! - 05/14/2008 10:00 AM
That's very odd - never seen that before.
How could you account for it? confused
Posted By: Gizmo Re: zero spiders online! - 05/14/2008 10:11 AM
could be that you don't have enough user agents setup wink
Posted By: Ian Re: zero spiders online! - 05/14/2008 12:50 PM
hmmm........

my list is empty - very strange frown

Can someone post a nice decent list for me - thanks.
Posted By: driv Re: zero spiders online! - 05/14/2008 1:36 PM
Here's one to start you off...

Alexa=ia_archiver
AllTheWeb=crawler@fast
AllTheWeb=FAST-WebCrawler
AltaVista=Mercator
Altavista=Scooter
Ask.com=Ask Jeeves
Ask.com=teoma_agent1
AskJeeves=AskJeeves
BecomeBot=BecomeBot
BoardReader=BoardReader
Crawler.de=Crawler
EntireWeb.com=Speedy Spider
Entireweb=Speedy Spider
Excite=ArchitextSpider
Gigablast Gigabot=Gigabot
Gigabot=Gigabot
Google Adsense=Mediapartners-Google
Google Images=Googlebot-Image
Google Mobile=Googlebot-Mobile
Google=Googlebot
Infoseek=InfoSeek sidewinder
InfoSeek=UltraSeek
Inktomi=Slurp
Looksmart=MantraAgent
Lycos=Lycos_Spider_(T-Rex)
MJ12bot=MJ12bot
MSN=MSNBOT
Naver.com=Yeti/
NaverBot=NaverBot
NorthernLight.com=Gulliver
ScrubTheWeb=Scrubby
Search Hippo=Fluffy the Spider
SingingFish=asterias
Snapbot=Snapbot
SnapPreviewBot=SnapPreviewBot
Sogou=sogou web spider
StackRambler(ru)=StackRambler
TAMU.edu=IRLbot
Teoma=TeomaAgent
TMCrawler=TMCrawler
Twiceler=Twiceler-0.9
Voila=Voila
Voyager=Voyager
Webcrawler.com=WebCrawler
Whois.sc=SurveyBot
Wisenut=Zyborg
Yahoo=Yahoo! Slurp
YodaoBot=YodaoBot
Posted By: Ian Re: zero spiders online! - 05/14/2008 1:44 PM
That driv - appreciated smile
Posted By: Ian Re: zero spiders online! - 05/14/2008 1:54 PM
Alexa=ia_archiver
AllTheWeb=crawler@fast
AllTheWeb=FAST-WebCrawler
AltaVista=Mercator
Altavista=Scooter
Anzwers=AnzwersCrawl
Ask.com=Ask Jeeves
Ask.com=teoma_agent1
Ask=Teoma
Atomz=Atomz
AskJeeves=AskJeeves
BecomeBot=BecomeBot
BoardReader=BoardReader
Boitho=boitho.com
Crawler.de=Crawler
DMOZ=Robozilla
EntireWeb.com=Speedy Spider
Entireweb=Speedy Spider
Excite=ArchitextSpider
Exalead=Exabot
Factbites=Factbot
Fast=FAST
Gigablast Gigabot=Gigabot
Gigabot=Gigabot
Google Adsense=Mediapartners-Google
Google Images=Googlebot-Image
Google Mobile=Googlebot-Mobile
Google=Googlebot
Infoseek=InfoSeek sidewinder
InfoSeek=UltraSeek
Inktomi=Slurp
Looksmart=MantraAgent
LookSmart=FurlBot
Lycos=Lycos_Spider_(T-Rex)
MJ12bot=MJ12bot
MSN=MSNBOT
Microsoft Research=MSRBOT
MSN=MSNBOT
MSN Pic-Search=PSbot
Naver.com=Yeti/
NaverBot=NaverBot
NetSeer=Teemer
NorthernLight.com=Gulliver
noXtrum=noxtrumbot
Pagebull=Pagebull
ScrubTheWeb=Scrubby
Searchme=Charlotte
Seznam=SeznamBot
Search Hippo=Fluffy the Spider
SingingFish=asterias
Snapbot=Snapbot
SnapPreviewBot=SnapPreviewBot
Sogou=sogou web spider
StackRambler(ru)=StackRambler
TAMU.edu=IRLbot
Teoma=TeomaAgent
TMCrawler=TMCrawler
Twiceler=Twiceler-0.9
Voila=Voila
Voila=VoilaBot
Voyager=Voyager
Walhello=appie
Webcrawler.com=WebCrawler
Whois.sc=SurveyBot
Wisenut=Zyborg
Yahoo=Yahoo! Slurp
YodaoBot=YodaoBot
Yahoo! Blogs=Yahoo-blogs/v3.9
Yahoo! MM=Yahoo-MMcrawler

Combines the ones that I have pulled from other posts and his list smile
Posted By: Ian Re: zero spiders online! - 05/14/2008 2:01 PM
Slowly showing up now.

Just wondered though if anyone knows why no locations is showing against the bots?
Posted By: driv Re: zero spiders online! - 05/14/2008 2:56 PM
ips you mean?
dunno - nothing daft like some 'spaces' crept into the text list is there?
Posted By: Ian Re: zero spiders online! - 05/14/2008 3:21 PM
No where is says...

Reading a post
Forum: Billericay Town
Thread: LATEST PLAYER NEWS

This appears for members and guests, but not spiders.
Posted By: Gizmo Re: zero spiders online! - 05/14/2008 11:57 PM
I'll merge this list with mine and repost later
Posted By: Gizmo Re: zero spiders online! - 05/15/2008 1:21 AM
Merged several lists here and cut out duplicates:
Code
Alexa=ia_archiver
AltaVista=Mercator
Altavista=Scooter
AllTheWeb=FAST-WebCrawler
Anzwers=AnzwersCrawl
AllTheWeb=crawler@fast
Ask.com=Ask Jeeves
Ask.com=teoma_agent1
Ask=Teoma
Atomz=Atomz
AskJeeves=AskJeeves
BecomeBot=BecomeBot
BoardReader=BoardReader
Boitho=boitho.com
Crawler.de=Crawler
DMOZ=Robozilla
EntireWeb.com=Speedy Spider
Entireweb=Speedy Spider
Excite=ArchitextSpider
Exalead=Exabot
Factbites=Factbot
Fast=FAST
Gigabot=Gigabot
Google=Googlebot
Google Mobile=Googlebot-Mobile
Google Images=Googlebot-Image
Google Adsense=Mediapartners-Google
Infoseek=InfoSeek sidewinder
InfoSeek=UltraSeek
Inktomi=Slurp
Looksmart=MantraAgent
LookSmart=FurlBot
Lycos=Lycos_Spider_(T-Rex)
MJ12bot=MJ12bot
MSN=MSNBOT
Microsoft Research=MSRBOT
MSN=MSNBOT
MSN Pic-Search=PSbot
Naver.com=Yeti/
NaverBot=NaverBot
NetSeer=Teemer
NorthernLight.com=Gulliver
noXtrum=noxtrumbot
Pagebull=Pagebull
ScrubTheWeb=Scrubby
Searchme=Charlotte
Seznam=SeznamBot
Search Hippo=Fluffy the Spider
SingingFish=asterias
Snapbot=Snapbot
SnapPreviewBot=SnapPreviewBot
Sogou=sogou web spider
StackRambler(ru)=StackRambler
TAMU.edu=IRLbot
Teoma=TeomaAgent
TMCrawler=TMCrawler
Twiceler=Twiceler-0.9
Voila=Voila.fr
Voila=Voila
Voila=VoilaBot
Voyager=Voyager
Walhello=appie
Webcrawler.com=WebCrawler
Whois.sc=SurveyBot
Wisenut=Zyborg
Yahoo=Yahoo! Slurp
Yahoo=Yahoo Slurp
YodaoBot=YodaoBot
Yahoo! Blogs=Yahoo-blogs/v3.9
Yahoo! MM=Yahoo-MMcrawler
Posted By: ScriptKeeper Re: zero spiders online! - 05/15/2008 12:16 PM
Some additional bots to squeeze in to your list:
Code
DepSpid=DepSpid
Fast(AllTheWeb)=FAST-WebCrawler
Fast(AllTheWeb)=crawler@fast
Gigablast=GigaBot
Nutch=nutchsearch
Picsearch=psbot
Shelob=shelob
WebAlta=WebAlta
Worio=woriobot
YaCy=yacybot
Yahoo! Germany=Yahoo! DE Slurp
Yoono=yoofind
Posted By: ScriptKeeper Re: zero spiders online! - 05/15/2008 12:25 PM
I've just put your list in and Yahoo! displays as Inktomi. Remove Yahoo=Yahoo Slurp from your list and Yahoo! will display correctly.
Posted By: Gizmo Re: zero spiders online! - 05/15/2008 1:00 PM
Weird that it's showing as inktomi, but I wouldn't want to remove it, because yahoo has 2 brands:
Yahoo=Yahoo! Slurp
Yahoo=Yahoo Slurp

Wondering why the one would work and the other wouldn't... I'd figure it would have been the one with the ! that wouldn't work...

I'll play with it more later today.
Posted By: Gizmo Re: zero spiders online! - 05/15/2008 1:15 PM
This is quite curious; all of the yahoo ones are being flagged as inktomi; and I mean all of them... When I remove it (inktomi) they just show blank lol
Posted By: Gizmo Re: zero spiders online! - 05/15/2008 1:18 PM
I'm pretty sure when you remove a bot from the listing in the cp that it's supposed to go back to anon right? So why would it be missing from my list, but be listed as a bot with no name lol...

Attached picture buggg.JPG
Posted By: jgeoff Re: zero spiders online! - 05/15/2008 3:15 PM

Yahoo=Yahoo! Slurp works just fine for me. In fact, they're my biggest "customer" lol

Inktomi=Slurp is on my list, but haven't yet noticed it.

The first is listed higher in my list, so maybe that's why it's showing correctly. Just put Yahoo above Inktomi (tho maybe some of the Yahoo's are actually Inktomi? If so, the logic is off a bit)
Posted By: Ian Re: zero spiders online! - 05/15/2008 3:16 PM
Originally Posted by Ian


Just wondered though if anyone knows why no locations is showing against the bots?


Anyone?
Posted By: jgeoff Re: zero spiders online! - 05/15/2008 3:25 PM

No idea, sorry, it's working just fine for me...
Posted By: jgeoff Re: zero spiders online! - 05/15/2008 3:29 PM

Giz, this is a duplicate:

EntireWeb.com=Speedy Spider
Entireweb=Speedy Spider

Also, I think some of these aren't on there (my latest additions):

Amazon=AMZNKAssocBot
Omgili=omgilibot
Snoopy=Snoopy
Kalooga=Kalooga


Posted By: Ian Re: zero spiders online! - 05/15/2008 6:40 PM
Updated list

Yahoo=Yahoo! Slurp
Yahoo! Germany=Yahoo! DE Slurp
Alexa=ia_archiver
AllTheWeb=crawler@fast
AllTheWeb=FAST-WebCrawler
AltaVista=Mercator
Altavista=Scooter
Amazon=AMZNKAssocBot
Anzwers=AnzwersCrawl
Ask.com=Ask Jeeves
Ask.com=teoma_agent1
Ask=Teoma
Atomz=Atomz
AskJeeves=AskJeeves
BecomeBot=BecomeBot
BoardReader=BoardReader
Boitho=boitho.com
Crawler.de=Crawler
DepSpid=DepSpid
DMOZ=Robozilla
Entireweb=Speedy Spider
Excite=ArchitextSpider
Exalead=Exabot
Factbites=Factbot
Fast=FAST
Fast(AllTheWeb)=FAST-WebCrawler
Fast(AllTheWeb)=crawler@fast
Gigabot=Gigabot
Google Adsense=Mediapartners-Google
Google Images=Googlebot-Image
Google Mobile=Googlebot-Mobile
Google=Googlebot
Google Sitemaps=Google-Sitemaps/1.0
Infoseek=InfoSeek sidewinder
InfoSeek=UltraSeek
Inktomi=Slurp
Kalooga=Kalooga
Looksmart=MantraAgent
LookSmart=FurlBot
Lycos=Lycos_Spider_(T-Rex)
MJ12bot=MJ12bot
MSN=MSNBOT
Microsoft Research=MSRBOT
MSN=MSNBOT
MSN Pic-Search=PSbot
Naver.com=Yeti/
NaverBot=NaverBot
NetSeer=Teemer
NorthernLight.com=Gulliver
noXtrum=noxtrumbot
Nutch=nutchsearch
Omgili=omgilibot
Pagebull=Pagebull
Picsearch=psbot
ScrubTheWeb=Scrubby
Searchme=Charlotte
Seznam=SeznamBot
Search Hippo=Fluffy the Spider
Shelob=shelob
SingingFish=asterias
Snapbot=Snapbot
SnapPreviewBot=SnapPreviewBot
Snoopy=Snoopy
Sogou=sogou web spider
StackRambler(ru)=StackRambler
StumbleUpon=StumbleUpon.com
TAMU.edu=IRLbot
Teoma=TeomaAgent
TMCrawler=TMCrawler
Twiceler=Twiceler-0.9
Voila=Voila.fr
Voila=Voila
Voila=VoilaBot
Voyager=Voyager
Walhello=appie
WebAlta=WebAlta
Webcrawler.com=WebCrawler
Whois.sc=SurveyBot
Wisenut=Zyborg
Worio=woriobot
YaCy=yacybot
YodaoBot=YodaoBot
Yahoo! Blogs=Yahoo-blogs/v3.9
Yahoo! MM=Yahoo-MMcrawler
Yoono=yoofind
Posted By: jgeoff Re: zero spiders online! - 05/15/2008 7:01 PM

Thanks, Ian!

Also:

StumbleUpon=StumbleUpon.com

was on my list
Posted By: jgeoff Re: zero spiders online! - 05/15/2008 7:06 PM

Now I'm getting that blank bot name, though (which appear to actually be legit Guests -- I'm getting 0 Guests now), and Inktomi replacing Yahoo whistle

Placing

Yahoo=Yahoo! Slurp
Yahoo! Germany=Yahoo! DE Slurp

above Inktomi corrects this.

Will have to try and figure this out later; I gotta run soon.

Posted By: Gizmo Re: zero spiders online! - 05/15/2008 11:16 PM
Yeh, it's definately a matching problem where inktomi is atching before yahoo is listed; so just keep the yahoo on top wink...

Have a few more to merge in :sigh:
Posted By: Gizmo Re: zero spiders online! - 05/15/2008 11:30 PM
Latest list (that is working):
Code
Yahoo=Yahoo! Slurp
Yahoo! Germany=Yahoo! DE Slurp
Alexa=ia_archiver
AltaVista=Mercator
Altavista=Scooter
AllTheWeb=FAST-WebCrawler
Amazon=AMZNKAssocBot
Anzwers=AnzwersCrawl
AllTheWeb=crawler@fast
Ask.com=Ask Jeeves
Ask.com=teoma_agent1
Ask=Teoma
Atomz=Atomz
AskJeeves=AskJeeves
BecomeBot=BecomeBot
BoardReader=BoardReader
Boitho=boitho.com
Crawler.de=Crawler
DepSpid=DepSpid
DMOZ=Robozilla
EntireWeb.com=Speedy Spider
Excite=ArchitextSpider
Exalead=Exabot
Factbites=Factbot
Fast=FAST
Gigabot=Gigabot
Google=Googlebot
Google Mobile=Googlebot-Mobile
Google Images=Googlebot-Image
Google Adsense=Mediapartners-Google
Google Sitemaps=Google-Sitemaps/1.0
Infoseek=InfoSeek sidewinder
InfoSeek=UltraSeek
Inktomi=Slurp
Kalooga=Kalooga
Looksmart=MantraAgent
LookSmart=FurlBot
Lycos=Lycos_Spider_(T-Rex)
MJ12bot=MJ12bot
MSN=MSNBOT
Microsoft Research=MSRBOT
MSN=MSNBOT
MSN Pic-Search=PSbot
Naver.com=Yeti/
NaverBot=NaverBot
NetSeer=Teemer
NorthernLight.com=Gulliver
noXtrum=noxtrumbot
Nutch=nutchsearch
Omgili=omgilibot
Pagebull=Pagebull
Picsearch=psbot
ScrubTheWeb=Scrubby
Searchme=Charlotte
Seznam=SeznamBot
Search Hippo=Fluffy the Spider
Shelob=shelob
SingingFish=asterias
Snapbot=Snapbot
SnapPreviewBot=SnapPreviewBot
Snoopy=Snoopy
Sogou=sogou web spider
StackRambler(ru)=StackRambler
StumbleUpon=StumbleUpon.com
TAMU.edu=IRLbot
Teoma=TeomaAgent
TMCrawler=TMCrawler
Twiceler=Twiceler-0.9
Voila=Voila.fr
Voila=Voila
Voila=VoilaBot
Voyager=Voyager
Walhello=appie
WebAlta=WebAlta
Webcrawler.com=WebCrawler
Whois.sc=SurveyBot
Wisenut=Zyborg
Worio=woriobot
YaCy=yacybot
YodaoBot=YodaoBot
Yahoo! Blogs=Yahoo-blogs/v3.9
Yahoo! MM=Yahoo-MMcrawler
Yoono=yoofind
Posted By: jgeoff Re: zero spiders online! - 05/16/2008 12:39 AM

Any idea what caused Guests to end up in the Bots section before Giz's last list?

Also - I would remove the .com from StumbleUpon.com -- or at least add:

StumbleUpon=StumbleUpon

I'm not sure if I added the .com to the wrong side, or, whether it was another one of their bots. But without it works (because it's on my site right now lol)

Posted By: Gizmo Re: zero spiders online! - 05/16/2008 12:48 AM
'eh for lesser known things i throw in the .com so it's easier to figure where they're from...
Posted By: jgeoff Re: zero spiders online! - 05/16/2008 1:06 AM

then use StumbleUpon.com=StumbleUpon

StumbleUpon.com may or may not be the bot's name, but I know for sure StumbleUpon is one of them...
Posted By: jgeoff Re: zero spiders online! - 06/03/2008 6:44 AM

WTH is cr4nk.ws? Doesn't appear very friendly on the surface... Just noticed this one tonight ((at least) two different IPs: 193.189.149.55 | 77.79.246.200 ) ...one Netherlands, the other Poland, but claiming to be [de]?... Suspicious...

© 2019 UBB.threads PHP Forum Software Community