|
Joined: Mar 2007
Posts: 307 Likes: 3
Enthusiast
|
Enthusiast
Joined: Mar 2007
Posts: 307 Likes: 3 |
Can someone please give me the list of Search Engine Agent Strings for the who's online feature?
Somehow when setting up the Forum for the first time the list was deleted.
TIA, Bill
|
|
|
|
Joined: Jun 2006
Posts: 16,365 Likes: 126
|
Joined: Jun 2006
Posts: 16,365 Likes: 126 |
Mine is: Alexa=ia_archiver
Altavista=Scooter
AllTheWeb=FAST-WebCrawler
Excite=ArchitextSpider
Gigabot=Gigabot
Google=Googlebot
Google Mobile=Googlebot-Mobile
Google Images=Googlebot-Image
Google Adsense=Mediapartners-Google
Yahoo=Yahoo! Slurp
Inktomi=Slurp
MSN=MSNBOT Our official listing is maintained at UBBDev here
|
|
|
|
Joined: Mar 2007
Posts: 307 Likes: 3
Enthusiast
|
Enthusiast
Joined: Mar 2007
Posts: 307 Likes: 3 |
Thanks Gizmo! Bill
|
|
|
|
Joined: Jun 2006
Posts: 3,839 Likes: 1
Carpal Tunnel
|
Carpal Tunnel
Joined: Jun 2006
Posts: 3,839 Likes: 1 |
this page has a fairly comprehensive listing
|
|
|
|
Joined: Jun 2006
Posts: 16,365 Likes: 126
|
Joined: Jun 2006
Posts: 16,365 Likes: 126 |
I've seen similar lists; I'm curious though how popular some of those are... I mean, I have a fairly moderate site with a lot of spider food, but I don't get too many on my boards... I really should look at the "active spiders" on my stats script
|
|
|
|
Joined: Jul 2007
Posts: 103
member
|
member
Joined: Jul 2007
Posts: 103 |
I've updated the above list with some recent crawlers that where snooping my board. So feel free to use it . Alexa=ia_archiver
Altavista=Scooter
AllTheWeb=FAST-WebCrawler
Bing=bingbot
Excite=ArchitextSpider
Gigabot=Gigabot
Google=Googlebot
Google Mobile=Googlebot-Mobile
Google Images=Googlebot-Image
Google Adsense=Mediapartners-Google
Magpie=magpie-crawler
Yahoo=Yahoo! Slurp
Yandex=YandexBot
Inktomi=Slurp
MSN=MSNBOT
|
|
|
|
Joined: Jan 2004
Posts: 2,474 Likes: 3
Pooh-Bah
|
Pooh-Bah
Joined: Jan 2004
Posts: 2,474 Likes: 3 |
Have you seen this list nims?
|
|
|
|
Joined: Jul 2007
Posts: 103
member
|
member
Joined: Jul 2007
Posts: 103 |
No I did not . This topic was what Google came up with, so thanks for pointing it out to me . I've update my board with the new list .
|
|
|
|
Joined: Apr 2004
Posts: 1,973 Likes: 154
|
Joined: Apr 2004
Posts: 1,973 Likes: 154 |
An updated list of ROBOTS/Search Agents. UBB.Threads Admin > General > Primary > Who's Online > Search Engine Agents This list: 553 total Original sourced from: http://user-agent-string.info/Source Version: uas_20130123-01I've cleaned and converted it for use with UBB.Threads 192.com=192.comAgent
50.nu=50.nu
80legs=008
80legs=80legs
abby=abby
Aboundexbot=Aboundexbot
AboutUsBot=AboutUsBot
Abrave Spider=Abrave Spider
Accelobot=Accelobot
Accelovation=heritrix
Accoona-AI-Agent=Accoona-AI-Agent
AcoonBot=Acoon
Acorn=Acorn
AddThis.com=AddThis.com
adidxbot=adidxbot
adressendeutschland.de=adressendeutschland.de
AdsBot-Google=AdsBot-Google
Ahrefs=AhrefsBot
aiHitBot=aiHitBot
akula=akula
Alexa=ia_archiver
AllTheWeb=crawler@fast
AllTheWeb=FAST-WebCrawler
Almaden=Almaden
AltaVista=Mercator
Altavista=Scooter
Amagit.COM=Amagit.COM
Amazon=AMZNKAssocBot
Amfibibot=Amfibibot
AntBot=AntBot
Anzwers=AnzwersCrawl
Apercite=Apercite
AportWorm=AportWorm
arachnode.net=arachnode.net
archive.org_bot=archive.org_bot
ASAHA Search Engine Turkey=ASAHA Search Engine Turkey
Ask.com=Ask Jeeves
Ask.com=AskJeeves
Ask.com=Teoma
Atomz=Atomz
Babaloo=BabalooSpider
BacklinkCrawler=BacklinkCrawler
Bad-Neighborhood=Bad Neighborhood
BaiDuSpider=BaiDuSpider
baypup=baypup
BDFetch=BDFetch
BecomeBot=BecomeBot
Bigsearch.ca=Bigsearch.ca
Bing=bingbot
bitlybot=bitlybot
biwec=biwec
BixoLabs=bixolabs
Blaiz-Bee=Blaiz-Bee
Blekkobot=Blekkobot
BlinkaCrawler=BlinkaCrawler
Bloggsi=Bloggsi
BlogPulse=BlogPulse
bnf.fr_bot=bnf.fr_bot
BoardReader=BoardReader
Boitho=boitho.com
bot.wsowner.com=bot.wsowner.com
botmobi=botmobi
BotOnParade=BotOnParade
bot-pge.chlooe.com=bot-pge.chlooe.com
Browsershots=Browsershots
btbot=btbot
Butterfly=Butterfly
BuzzRankingBot=BuzzRankingBot
CamontSpider=CamontSpider
CareerBot=CareerBot
Castabot=Castabot
CazoodleBot=CazoodleBot
CCBot=CCBot
ccubee=ccubee
ChangeDetection=ChangeDetection
City4you=City4you
Cityreview=Cityreview Robot
CJB.NET Proxy=CJB.NET Proxy
CligooRobot=CligooRobot
coccoc=coccoc
Combine=Combine
CompSpyBot=CompSpyBot
ConveraCrawler=ConveraCrawler
ConveraCrawler=ConveraMultiMediaCrawler
copyright sheriff=copyright sheriff
CorpusCrawler=CorpusCrawler
Covario-IDS=CovarioIDS
Covario-IDS=Covario-IDS
Crawler.de=Crawler
Crawler4j=Crawler4j
csci_b659=csci_b659
Cuil=Twiceler
CyberPatrol=CyberPatrol
DataFountains=DataFountains at Dmoz
DataparkSearch=DataparkSearch
Daumoa=Daumoa
DCPbot=DCPbot
DealGates Bot=DealGates Bot
del.icio.us Thumbnails=del.icio.us-thumbnails
DepSpid=DepSpid
discoverybot=discobot
discoverybot=discoverybot
DKIMRepBot=DKIMRepBot
DMOZ=Robozilla
DNS Mine=DNSMine
DNS-Digger-Explorer=DNS-Digger-Explorer
DomainDB=DomainDB
DontBuyLists=DBLBot
Dot TK - spider=Dot TK - spider 3.0
DotBot=DotBot
dotSemantic=dotSemantic
DripfeedBot=DripfeedBot
drupact=drupact
DuckDuckBot=DuckDuckBot
DuckDuckPreview=DuckDuckPreview
EasyBib AutoCite=EasyBib AutoCite
eCairn-Grabber=eCairn-Grabber
EdisterBot=EdisterBot
egothor=egothor
ejupiter.com=ejupiter.com
emefgebot=emefgebot
EnaBot=EnaBot
Enterprise_Search=Enterprise_Search
Entireweb=Speedy Spider
envolk=envolk
e-SocietyRobot=e-SocietyRobot
Esribot=Esribot
EuripBot=EuripBot
Eurobot=Eurobot
EventGuruBot=EventGuruBot
EvriNid=EvriNid
Exabot=Exabot
Exabot Images=Exabot-Images
Exabot Thumbnails=Exabot-Thumbnails
ExactSEEK=ExactSEEK
Excite=Architext
Ezooms=Ezooms
Facebook Dev=facebookplatform
Facebook=FacebookExternalHit
Factbites=Factbot
FairShare=FairShare
Falconsbot=Falconsbot
Fast=FAST
FAST Enterprise Crawler=FAST Enterprise Crawler
FAST MetaWeb Crawler=FAST MetaWeb Crawler
FauBot=FauBot
favorstarbot=favorstarbot
FeedCatBot=FeedCatBot
Findexa Crawler=Findexa Crawler
findlinks=findlinks
Flatland Industries Web Spider=flatlandbot
FlightDeckReportsBot=FlightDeckReportsBot
FlipboardProxy=FlipboardProxy
Flocke bot=Flocke bot
FollowSite Bot=FollowSite Bot
Fooooo_Web_Video_Crawl=Fooooo_Web_Video_Crawl
Forschungsportal=Forschungsportal
Francis=Francis
FunnelBack=FunnelBack
FyberSpider=FyberSpider
g2crawler=g2crawler
Gaisbot=Gaisbot
Gallent Search Spider=Gallent Search Spider v1.4 Robot 3
GarlikCrawler=GarlikCrawler
genieBot=genieBot
Genieo Web filter=Genieo
GeonaBot=GeonaBot
Gigablast=GigaBot
GingerCrawler=GingerCrawler
Girafabot=Girafabot
GOFORITBOT=GOFORITBOT
gonzo=gonzo
Google=Googlebot
Google Adsense=Mediapartners-Google
Google Images=Googlebot-Image
Google Mobile=Googlebot-Mobile
Google Sitemaps=Google-Sitemaps
Google Videos=Googlebot-Video
Google Web Preview=Google Web Preview
GrapeshotCrawler=grapeFX
GrapeshotCrawler=GrapeshotCrawler
GurujiBot=GurujiBot
Hailoobot=Hailoobot
HatenaScreenshot=HatenaScreenshot
HeartRails_Capture=HeartRails_Capture
HiddenMarket=HiddenMarket
Holmes=holmes
HomeTags=HomeTags
HooWWWer=HooWWWer
HostTracker.com=HostTracker.com
HuaweiSymantecSpider=HuaweiSymantecSpider
HubSpot Connect=HubSpot Connect 1.0
iaskspider=iaskspider
ICC-Crawler=ICC-Crawler
ICF_Site_Crawler=ICF_Site_Crawler
ichiro=ichiro
iCjobs=iCjobs
IlseBot=IlseBot
IlTrovatore=IlTrovatore
IlTrovatore-Setaccio=IlTrovatore-Setaccio
imbot=imbot
Influencebot=Influencebot
InfociousBot=InfociousBot b
Infohelfer=Infohelfer
Infoseek=InfoSeek sidewinder
Infoseek=Ultraseek
Inktomi=Slurp
IntegromeDB=IntegromeDB
InternetSeer=InternetSeer (Web Monitor)
Ipselonbot=Ipselonbot
IstellaBot=IstellaBot
JadynAve=JadynAve
Jambot=Jambot
JikeSpider=JikeSpider
Job Roboter Spider=Job Roboter Spider 3
JUST-CRAWLER=JUST-CRAWLER
Jyxobot=Jyxobot
Kakle Bot=kakle-spider
Kalooga=Kalooga
Karneval-Bot=Karneval-Bot
KeywenBot=KeywenBot
KeywordDensityRobot=KeywordDensityRobot
Kongulo=Kongulo
KRetrieve=KRetrieve
Krugle=Krugle
ksibot=ksibot
L.webis=L.webis
LapozzBot=LapozzBot
Larbin=Larbin
LemurWebCrawler=LabrinWebCrawler
LexxeBot=LexxeBot
libwww=libwww-perl
Lijit=Lijit
LinguaBot=LinguaBot
Linguee Bot=Linguee Bot
Link Valet Online=Link Valet Online
LinkAider=LinkAider
linkdex.com=linkdex.com
linksmanager_bot=linksmanager_bot
LinkWalker=LinkWalker
livedoor ScreenShot=livedoor ScreenShot
lmspider=lmspider
LookSmart=FurlBot
Looksmart=MantraAgent
Lycos=Lycos_Spider
Magpie=magpie-crawler
Mahiti Crawler=Mahiti Crawler
Mail.Ru=Mail.Ru
Megatext=Megatext
MeMoNewsBot=MeMoNewsBot
MetaGeneratorCrawler=MetaGeneratorCrawler
MetaJobBot=MetaJobBot
MetamojiCrawler=MetamojiCrawler
Metaspinner=Metaspinner
MetaTagRobot=MetaTagRobot
MetaURI=MetaURI
MIA Bot=MIA Bot
Microsoft Research=MSRBOT
miniRank=miniRank
MJ12bot=MJ12bot
MLBot=MLBot
MnoGoSearch=MnoGoSearch
Moatbot=Moatbot
moba-crawler=moba-crawler
MojeekBot=MojeekBot
Motoricerca-Robots.txt-Checker=Motoricerca-Robots.txt-Checker
mozDex=Mozdex
Mp3Bot=Mp3Bot
MQbot=MQbot
MSN=msnbot
MSN Media=msnbot-media
MSN News=msnbot-NewsBlogs
MSN Pic-Search=PSbot
MultiCrawler=MultiCrawler
MyFamilyBot=MyFamilyBot
Najdi.si=Najdi.si
Naver=Yeti
NaverBot=NaverBot
NaverBot=Yepi
navissobot=navissobot
NerdByNature.Bot=NerdByNature.Bot
nestReader=nestReader
NetcraftSurveyAgent=NetcraftSurveyAgent
netEstate Crawler=netEstate NE Crawler
netEstate Crawler=netEstate RSS crawler
Netintelligence LiveAssessment=Netintelligence LiveAssessment
NetResearchServer=nrsbot
Netseer=Netseer crawler
NetSeer=Teemer
NetWhatCrawler=NetWhatCrawler
NextGenSearchBot=NextGenSearchBot
nextthing.org=nextthing.org
NG-Search=NG-Search
Nigma.ru=Nigma.ru
NimbleCrawler=NimbleCrawler
nodestackbot=nodestackbot
NorthernLight.com=Gulliver
noXtrum=noxtrumbot
noyona=noyona
NPBot=NPBot
Nuhk=Nuhk
NuSearch Spider=NuSearch Spider
Nutch=heeii
Nutch=InternetArchive
Nutch=Nutch
Nutch=NutchCSV
Nutch=nutchsearch
nworm=nwormFeedFinder
Nymesis=Nymesis
oBot=oBot
Ocelli=Ocelli
Omgili=omgilibot
OmniExplorer_Bot=OmniExplorer_Bot
OnetSzukaj=OnetSzukaj
OpenAcoon=openacoon
Openbot=Openbot
OpenCalaisSemanticProxy=OpenCalaisSemanticProxy
OpenindexSpider=OpenindexDeepSpider
OpenindexSpider=OpenindexShalooowSpider
OpenindexSpider=OpenindexSpider
OpenWebSpider=OpenWebSpider
Orbiter=Orbiter
OrgbyBot=OrgbyBot
OsObot=OsObot
owsBot=owsBot
page_verifier=page_verifier
Page2RSS=Page2RSS
PageBitesHyperBot=PageBitesHyperBot
Pagebull=Pagebull
PagePeeker=PagePeeker
Panscient=panscient.com
Panscient web crawler=Panscient web crawler
PaperLiBot=PaperLiBot
ParchBot=ParchBot
Peepowbot=Peepowbot
peerindex=peerindex
Peew=Peew
percbotspider=percbotspider
pingdom.com_bot=Pingdom GIGRIB
pingdom.com_bot=pingdom.com_bot 1.4
Pixray-Seeker=Pixray-Seeker
Plukkie=Plukkie
pmoz.info ODP link checker=pmoz.info ODP link checker
Pogodak.co.yu=Pogodak.co.yu
polixea.de=polixea.de
Pompos=Pompos
posterus=posterus
PostPost=PostPost
PrivacyFinder=PrivacyFinder
ProCogBot=ProCogBot
proximic=proximic
ptd-crawler=ptd-crawler
Qirina Hurdler=Qirina Hurdler
Qseero=Qseero
Qualidator.com Bot=Qualidator.com Bot
Quantcastbot=Quantcastbot
quickobot=quickobot
R6 bot=R6_CommentReader
R6 bot=R6_FeedFetcher
RADaR-Bot=RADaR-Bot
RankurBot=RankurBot
RedBot=RedBot1.0
Robo Crawler=Robo Crawler 6.4.5
Robots_Tester=Robots_Tester
rogerbot=rogerbot
Ronzoobot=Ronzoobot
Ruky-Roboter=Ruky-Roboter
RyzeCrawler=RyzeCrawler
SAI Crawler=SAI Crawler
SanszBot=SanszBot
SBIder=SBIder
SBSearch=SBSearch
Scarlett=Scarlett
schibstedsokbot=schibstedsokbot
ScollSpider=ScollSpider
ScoutJet=ScoutJet
ScrubTheWeb=Scrubby
Search Engine World Robots.txt Validator=Search Engine World Robots.txt Validator
Search Hippo=Fluffy the Spider
Search Me=Charlotte
search.KumKie.com=search.KumKie.com
Search17Bot=Search17Bot
SearchmetricsBot=SearchmetricsBot
Semager=Semager
SemrushBot=SemrushBot
Sensis Web Crawler=Sensis Web Crawler
SEODat=SEODat
SEOENGBot=SEOENGBot
SEOkicks-Robot=SEOkicks-Robot
Setoozbot =OOZBOT
Seznam=SeznamBot
Shelob=shelob
Shim-Crawler=Shim-Crawler
ShopWiki=ShopWiki
ShowyouBot=ShowyouBot
silk=silk
Similarpages=Similar
SingingFish=asterias
Sirketce Busiverse=Busiversebot
Sirketce Busiverse=Sirketcebot
sistrix=sistrix
Sitedomain-Bot=Sitedomain-Bot
SkreemRBot=SkreemRBot
smart.apnoti.com Robot=smart.apnoti.com Robot
Snap=Snapbot
snap.com=snap.com
SnapPreviewBot=SnapPreviewBot
Snappy=Snappy
SniffRSS=SniffRSS
Snoopy=Snoopy
Sogou=Sogou develop spider
Sogou=sogou spider
Sogou=Sogou web spider
Sosospider=Sosospider
spbot=spbot
SpiderLing=SpiderLing
Spinn3r=Spinn3r
Spock Crawler=Spock Crawler
SpokeSpider=SpokeSpider
Sproose=Sproose
SrevBot=SrevBot
SSLBot=SSLBot
StackRambler=StackRambler
StatoolsBot=StatoolsBot
Steeler=Steeler
Strokebot=Strokebot
StumbleUpon=StumbleUpon.com
suggybot=suggybot
Surphace Scout=Surphace Scout
SWEBot=SWEBot
SygolBot=SygolBot
SynooBot=SynooBot
Szukacz=Szukacz
Szukankobot=Szukankobot
Tagoobot=Tagoobot
TAMU.edu=IRLbot
taptubot=taptubot
Technoratibot=Technoratibot
Teoma=TeomaAgent
TeragramCrawler=TeragramCrawler
textractor=textractor.harvester
textractor=textractor.queuekeeper
Theophrastus=Theophrastus
Thumbnail.CZ robot=Thumbnail.CZ robot
Thumbshots=Thumbshots
TinEye=TinEye
TMCrawler=TMCrawler
Topicbot=Topicbot
Toread-Crawler=Toread-Crawler
Touche=Touche
trendictionbot =trendictionbot
TurnitinBot=TurnitinBot
TutorGigBot=TutorGigBot
TwengaBot=TwengaBot
Twikle=Twikle
UASlinkChecker=UASlinkChecker
UnisterBot=UnisterBot
UnwindFetchor=UnwindFetchor
Updownerbot=Updownerbot
UptimeDog=UptimeDog
UptimeRobot=UptimeRobot
urlfan-bot=urlfan-bot
Urlfilebot (Urlbot)=Urlfilebot
Vagabondo=Vagabondo
VERASYS 2k=VERASYS 2k
Vermut=Vermut
Vespa Crawler=Vespa Crawler
VideoSurf_bot=VideoSurf_bot
virus_detector=virus_detector
Visbot=Visbot
VMBot=VMBot
void-bot=void-bot
Voila=Voila
VORTEX=VORTEX
voyager=voyager
VWBot=VWBot
Walhello=appie
WASALive-Bot=WASALive-Bot
WatchMouse=WatchMouse
WBSearchBot=WBSearchBot
Web Data Centre=WebDataCentreBot
WebAlta=WebAlta
WebarooBot=RufusBot (Rufus Web Miner)
WebarooBot=WebarooBot
WebarooBot=WebMiner
Webcrawler=WebCrawler
WebImages=WebImages
webinatorbot=webinatorbot
webmastercoffee=webmastercoffee
WebNL=WebNL
WebRankSpider=WebRankSpider
Webscope Crawler=Webscope Crawler
Web-sniffer=Web-sniffer
WebWatch=WebWatch
wectar=wectar
WeSEE:Search=WeSEE:Search
Whois=SurveyBot
Whoismindbot=Whoismindbot
WikioFeedBot=WikioFeedBot 1.0
wikiwix-bot=wikiwix-bot
Willow Internet Crawler=Willow Internet Crawler
WinkBot=WinkBot
WinWebBot=WinWebBot
WISEnut=ZyBorg
WMCAI_robot=WMCAI_robot
Woko=Woko robot
Worio=woriobot
Wotbox=Wotbox
wsAnalyzer=wsAnalyzer
wscheck.com=wscheck.com
www.fi crawler=www.fi crawler
wwwster=wwwster
XmarksFetch=XmarksFetch
XML Sitemaps Generator=XML Sitemaps Generator
Yaanb=Yaanb
YaCy=yacybot
Yahoo=Yahoo Slurp
Yahoo!=Yahoo! Slurp
Yahoo! Blogs=Yahoo-blogs
Yahoo! CHINA=Yahoo! Slurp China
Yahoo! GERMANY=Yahoo! DE Slurp
Yahoo! JAPAN=Y!J-BRI
Yahoo! JAPAN=Y!J-BRJ
Yahoo! JAPAN=Y!J-BRO
Yahoo! JAPAN=Y!J-BRW
Yahoo! JAPAN=Y!J-BSC
Yahoo! Media=Yahoo-MMcrawler
Yandex=Yandex
Yandex Antivirus=YandexAntivirus
Yandex Blogs=YandexBlogs
Yandex Catalog=YandexCatalog
Yandex Favicons=YandexFavicons
Yandex Images=YandexImages
Yandex Media=YandexMedia
Yandex Metrika=YandexMetrika
Yandex News=YandexNews
Yandex Resizer=YandexImageResizer
Yandex Video=YandexVideo
Yanga=Yanga
YioopBot=gofind
YioopBot=YioopBot
YodaoBot=YodaoBot
YodaoBot Images=YodaoBot-Image
Yoono=yoofind
Yoono Bot=Yoono Bot
YoudaoBot=YoudaoBot
YowedoBot=YowedoBot
YRSpider=YRSpider
ZACATEK_CZ=ZACATEK_CZ_BOT
Zeusbot=Zeusbot
Zookabot=Zookabot
ZoomSpider (ZSEBOT)=ZoomSpider
ZumBot=ZumBot
|
|
|
|
Joined: Dec 2003
Posts: 6,628 Likes: 85
|
Joined: Dec 2003
Posts: 6,628 Likes: 85 |
Good job. We can never keep this list updated 100% due to the constant changing robots and opinions on what is a robot or not.
Blue Man Group There is no such thing as stupid questions. Just stupid answers
|
|
|
|
Joined: Jun 2006
Posts: 998 Likes: 25
Old Hand
|
Old Hand
Joined: Jun 2006
Posts: 998 Likes: 25 |
Please enlighten me what this is for? what happen if I use it and if I don't?
Cheers
|
|
|
|
Joined: Dec 2003
Posts: 6,628 Likes: 85
|
Joined: Dec 2003
Posts: 6,628 Likes: 85 |
If you look at the who is online you will see # of registered,# of guests and # of Spiders. This defines who is a Spider and will not have them show as a guest. Since there are new Spiders almost daily it is hard to keep up with. So the new spiders not in the agent string will show as a guest.
Blue Man Group There is no such thing as stupid questions. Just stupid answers
|
|
|
|
Joined: Jun 2006
Posts: 998 Likes: 25
Old Hand
|
Old Hand
Joined: Jun 2006
Posts: 998 Likes: 25 |
OK I see, I just altered the who's online to 240 min and 120 robots shows now after a couple hours,,
|
|
|
|
Joined: Dec 2003
Posts: 6,628 Likes: 85
|
Joined: Dec 2003
Posts: 6,628 Likes: 85 |
Blue Man Group There is no such thing as stupid questions. Just stupid answers
|
|
|
|
Joined: Jun 2006
Posts: 16,365 Likes: 126
|
Joined: Jun 2006
Posts: 16,365 Likes: 126 |
I know a lot of forums that have an "Active Today" area versus a "Who's Online"
|
|
|
|
Joined: Apr 2004
Posts: 1,973 Likes: 154
|
Joined: Apr 2004
Posts: 1,973 Likes: 154 |
List of Search Engine Spiders for UBB.Threads - 2014-03-25 About:The advantage to using this list is that Search Engine Spiders get put into the correct "Search Spiders" group when viewing the forums/ubbthreads.php/online page. This translation list is not used anywhere else in UBBT except in the forums/ubbthreads.php/online page. Having a long list of Search Engine Spiders will not slow down your board -- the only exception is that depending on the server in which your site is hosted on, there may be a minimal delay in generating the "online" page. Details:Total Search Spiders: 427 Total Agent Strings: 748 Original Sourced From: http://user-agent-string.info/list-of-ua/botsSource Version: uas_20140325-01 I've cleaned and converted it for use with UBB.Threads How-To:1) COPY/PASTE this list to the "Search Engine Agents" box at: Control Panel > Display Options > Who's Online 2) Click "Update General Display Options" done. Having problems? REMOVE any blank lines from the top/bottom of your copied list. The List:192.com=192.com
200PleaseBot=200PleaseBot/1.0
360Spider=360Spider
4seohuntBot=4seohuntBot
50.nu=50.nu/0.01
008=008/0.71
008=008/0.83
008=008/0.85
^Nail=^Nail
A6-Indexer=A6-Indexer/1.0
abby=abby/1.0
Aboundexbot=Aboundexbot/0.2
Aboundexbot=Aboundexbot/0.3
AboutUsBot=AboutUsBot
AboutUsBot=AboutUsBot Johnny5/2.0
AboutUsBot=AboutUsBot/0.9
AboutUsBot=AboutUsBot/Harpy
Abrave Spider=Abrave Spider/4-1
Abrave Spider=Abrave Spider/4-2
Accelobot=Accelobot
AcoonBot=Acoon v4.1.0
AcoonBot=Acoon v4.10.1
AcoonBot=Acoon v4.10.3
AcoonBot=Acoon v4.10.4
AcoonBot=Acoon v4.10.5
AcoonBot=Acoon v4.9.5
AcoonBot=AcoonBot/4.10.5
AcoonBot=AcoonBot/4.10.6
AcoonBot=AcoonBot/4.10.7
AcoonBot=AcoonBot/4.10.8
AcoonBot=AcoonBot/4.11.0
AcoonBot=AcoonBot/4.11.1
AcoonBot=AcoonBot/4.12.1
AcoonBot=OpenAcoon v4.1.0
AcoonBot=OpenAcoon v4.10.5
AddThis.com=AddThis.com robot
ADmantX=ADmantX
adressendeutschland.de=adressendeutschland.de
AdsBot-Google=AdsBot-Google
AhrefsBot=AhrefsBot/1.0
AhrefsBot=AhrefsBot/2.0
AhrefsBot=AhrefsBot/3.0
AhrefsBot=AhrefsBot/3.1
AhrefsBot=AhrefsBot/4.0
AhrefsBot=AhrefsBot/5.0
aiHitBot=aiHitBot-BP/1.1
aiHitBot=aiHitBot-DM/2.0.2
aiHitBot=aiHitBot/1.0
aiHitBot=aiHitBot/1.0-DS
aiHitBot=aiHitBot/1.1
aiHitBot=aiHitBot/2.7
aiHitBot=aiHitBot/2.8
akula=akula/12.0rc-2
akula=akula/k311
alexa site audit=alexa site audit/1.0
Amagit.COM=Amagit.COM
amibot=amibot
AMZNKAssocBot=AMZNKAssocBot/4.0
AntBot=AntBot/1.0
Apercite=Apercite
AportWorm=AportWorm/3.2
AraBot=AraBot 1.0
arachnode.net=arachnode.net/2.5
archive.org_bot=archive.org_bot
archive.org_bot=special_archiver/3.1.1
Ask Jeeves/Teoma=Ask Jeeves/Teoma
Automattic Analytics Crawler=Automattic Analytics Crawler/0.1
BabalooSpider=BabalooSpider/1.3
backlink-check.de=backlink-check.de
BacklinkCrawler=BacklinkCrawler
BacklinkCrawler=BacklinkCrawler V
Bad-Neighborhood=Bad Neighborhood Header Detector
Bad-Neighborhood=Bad-Neighborhood Link Analyzer
Baiduspider=Baiduspider
Baiduspider JP=Baiduspider japan
Baiduspider-Image=Baiduspider-image
Baiduspider=Baiduspider/2.0
baypup=baypup/1.1
baypup=baypup/colbert
BDCbot=BDCbot/1.0
BDFetch=BDFetch
BegunAdvertising=BegunAdvertising/3.0
bingbot=bingbot SitemapProbe
bingbot=bingbot/2.0
BingPreview=BingPreview/1.0b
bitlybot=bitlybot
biwec=biwec
bixocrawler=bixo
bixocrawler=bixocrawler
bixocrawler=bixolabs/1.0
bixocrawler=ptd-crawler
bl.uk_lddc_bot=bl.uk_lddc_bot/3.1.1
Blekkobot=Blekkobot
BLEXBot=BLEXBot
BLEXBot=BLEXBot/1.0
BlinkaCrawler=BlinkaCrawler/1.0
BlogPulse=BlogPulse
BlogPulse=BlogPulseLive
bnf.fr_bot=bnf.fr_bot
bot-pge.chlooe.com=bot-pge.chlooe.com/1.0.0
bot.wsowner.com=bot.wsowner.com/1.0.0
botmobi=botmobi
BotOnParade=BotOnParade
BrainbruBot=BrainbruBot/1.0
Browsershots=Browsershots
BUbiNG=BUbiNG
Butterfly=Butterfly/1.0
Butterfly=Butterfly/1.0 a
CamontSpider=CamontSpider/1.0
CareerBot=CareerBot/1.1
Castabot=Castabot/0.1
CatchBot=CatchBot/1.0
CatchBot=CatchBot/2.0
CatchBot=CatchBot/3.0
CatchBot=CatchBot/5.0
CCBot=CCBot/1.0
CCBot=CCBot/2.0
CCResearchBot=CCResearchBot/1.0
ChangeDetection=ChangeDetection
Charlotte=Charlotte/1.1
CirrusExplorer=CirrusExplorer/1.1
cityreview=cityreview
classbot=classbot
CligooRobot=CligooRobot/2.0
CliqzBot=Cliqz Bot
Cliqzbot=Cliqzbot/0.1
CloudFlare-AlwaysOnline=CloudFlare-AlwaysOnline/1.0
CloudServerMarketSpider=CloudServerMarketSpider/1.0
CMS Crawler=CMS Crawler
coccoc=coccoc
coccoc=coccoc/1.0
CompSpyBot=CompSpyBot/1.0
copyright sheriff=copyright sheriff
CorpusCrawler=CorpusCrawler 2.0.0
CorpusCrawler=CorpusCrawler 2.0.10
CorpusCrawler=CorpusCrawler 2.0.12
CorpusCrawler=CorpusCrawler 2.0.13
CorpusCrawler=CorpusCrawler 2.0.14
CorpusCrawler=CorpusCrawler 2.0.15
CorpusCrawler=CorpusCrawler 2.0.17
CorpusCrawler=CorpusCrawler 2.0.19
CorpusCrawler=CorpusCrawler 2.0.20
CorpusCrawler=CorpusCrawler 2.0.21
CorpusCrawler=CorpusCrawler 2.0.22
CorpusCrawler=CorpusCrawler 2.0.24
CorpusCrawler=CorpusCrawler 2.0.25
CorpusCrawler=CorpusCrawler 2.0.8
CorpusCrawler=CorpusCrawler 2.0.9
Covario-IDS=Covario-IDS/1.0
Covario-IDS=CovarioIDS/1.1
crawler for netopian=crawler for netopian
Crawler4j=Crawler4j
Crowsnest=Crowsnest/0.5
Curious George=Curious George
Daumoa=Daumoa/2.0 b
Daumoa=Daumoa/3.0
DBLBot=DBLBot/1.0
DCPbot=DCPbot/1.0
DCPbot=DCPbot/1.1
DealGates Bot=DealGates Bot/1.1
discoverybot=discobot/1.0
discoverybot=discobot/1.1
discoverybot=discobot/2.0
discoverybot=discoverybot/2.0
DKIMRepBot=DKIMRepBot/1.0
Dlvr.it/1.0=Dlvr.it/1.0
DNS-Digger-Explorer=DNS-Digger-Explorer/1.0
DomainDB=DomainDB/1.1
Dot TK - spider=Dot TK - spider 3.0
DotBot=DotBot/1.1
dotSemantic=dotSemantic/1.0
DripfeedBot=DripfeedBot/2.0
drupact=drupact/0.7
DuckDuckPreview=DuckDuckPreview/1.0
EasouSpider=EasouSpider
EasyBib AutoCite=EasyBib AutoCite
eCairn-Grabber=eCairn-Grabber/1.0
EdisterBot=EdisterBot
Embedly=Embedly/0.2
emefgebot=emefgebot/beta
envolk=envolk/1.7
Esribot=Esribot/1.0
EuripBot=EuripBot/1.1
Eurobot=Eurobot/1.1
Eurobot=Eurobot/1.2
EventGuruBot=EventGuruBot/1.0
EvriNid=EvriNid/1.0.0
Exabot=Exabot-Images/3.0
Exabot=Exabot-Thumbnails
Exabot=Exabot/3.0
Exabot=Exabot/3.0/BiggerBetter
Exabot=ExaleadCloudview/5
Exabot=ExaleadCloudview/6
ExB Language Crawler=ExB Language Crawler 2.1.1
ExB Language Crawler=ExB Language Crawler 2.1.2
ExB Language Crawler=ExB Language Crawler 2.1.5
Ezooms=Ezooms/1.0
Facebook External Hit=FacebookExternalHit/1.0
Facebook External Hit=FacebookExternalHit/1.1
facebookplatform=facebookplatform/1.0
factbot=Factbot 1.09
FairShare=FairShare
Falconsbot=Falconsbot
fastbot crawler=fastbot crawler beta 2.0
fastbot crawler=fastbot.de crawler beta 2.0
FauBot=FauBot/0.1
FeedCatBot=FeedCatBot/3.0
FeedFinder/bloggz.se=FeedFinder-2.0
Feedly=Feedly/1.0
Feedly=FeedlyBot/1.0
Fetch-Guess=Fetch/2.0a
findlinks=findlinks/1.1.3-beta9
findlinks=findlinks/1.1.4-beta7
findlinks=findlinks/1.1.6-beta1
findlinks=findlinks/1.1.6-beta1 Yacy
findlinks=findlinks/1.1.6-beta2
findlinks=findlinks/1.1.6-beta3
findlinks=findlinks/1.1.6-beta4
findlinks=findlinks/1.1.6-beta5
findlinks=findlinks/1.1.6-beta6
findlinks=findlinks/2.0
findlinks=findlinks/2.0.1
findlinks=findlinks/2.0.2
findlinks=findlinks/2.0.4
findlinks=findlinks/2.0.5
findlinks=findlinks/2.0.9
findlinks=findlinks/2.1
findlinks=findlinks/2.1.3
findlinks=findlinks/2.1.5
findlinks=findlinks/2.2
findlinks=findlinks/2.5
findlinks=findlinks/2.6
firmilybot=firmilybot/0.3
Flatland Industries Web Spider=flatlandbot/baypup
FlightDeckReportsBot=FlightDeckReportsBot/2.0
FlipboardProxy=FlipboardProxy/0.0.5
Flocke bot=Flocke bot
FollowSite Bot=FollowSite Bot
Fooooo_Web_Video_Crawl=Fooooo_Web_Video_Crawl
FyberSpider=FyberSpider/1.3
Gaisbot=Gaisbot/3.0 - 06
GarlikCrawler=GarlikCrawler/1.1
GarlikCrawler=GarlikCrawler/1.1 b
GeliyooBot=GeliyooBot/1.0
GeliyooBot=GeliyooBot/1.0beta
Genieo Web filter=Genieo/1.0
Gigabot=Gigabot/1.0
Gigabot=Gigabot/3.0
GingerCrawler=GingerCrawler/1.0
Girafabot=Girafabot
gocrawl=gocrawl v0.4
GOFORITBOT=GOFORITBOT
gonzo=gonzo/1
gonzo=gonzo1
gonzo=gonzo2
Googlebot AdsBot Mobile=AdsBot-Google-Mobile
Googlebot Web Preview=Google Web Preview
Googlebot Snippet=Googlebot snippet
Googlebot-Image=Googlebot-Image/1.0
Googlebot-Mobile=Googlebot-Mobile
Googlebot-Mobile=Googlebot-Mobile/2.1
Googlebot-Video=Googlebot-Video/1.0
Googlebot=Googlebot/2.1
Googlebot AdSense=Mediapartners-Google
Grahambot=Grahambot/0.1
GrapeshotCrawler=grapeFX/0.9
GrapeshotCrawler=GrapeshotCrawler/2.0
GurujiBot=GurujiBot/1.0
Hailoobot=Hailoobot/1.2
HatenaScreenshot=HatenaScreenshot/1.0 (checker)
HeartRails_Capture=HeartRails_Capture/0.6
heritrix=heritrix/1.12.1
heritrix=heritrix/1.12.1b
heritrix=heritrix/1.14.2
heritrix=heritrix/1.14.3
heritrix=heritrix/1.14.3.r6601
heritrix=heritrix/2.0.2
heritrix=heritrix/3.0.0
heritrix=heritrix/3.1.1
Holmes=holmes/3.12.4 - morfeo
HolmesBot=HolmesBot
HomeTags=HomeTags/1.0
HostTracker.com=HostTracker.com/1.0
HostTracker=HostTracker/2.0
HuaweiSymantecSpider=HuaweiSymantecSpider/1.0
HubSpot Connect=HubSpot Connect 1.0
ia_archiver=ia_archiver
ia_archiver=ia_archiver alexa
ICC-Crawler=ICC-Crawler
ICC-Crawler=ICC-Crawler/2.0
ichiro=ichiro/3.0
ichiro=ichiro/4.0
ichiro=ichiro/5.0
iCjobs=iCjobs/3.2.3
imbot=imbot/0.1
immediatenet thumbnails=immediatenet thumbnails
Influencebot=Influencebot/0.9
Infohelfer=Infohelfer/1.0
Infohelfer=Infohelfer/1.2.0
Infohelfer=Infohelfer/1.3.0
Infohelfer=Infohelfer/1.3.3
IntegromeDB=IntegromeDB
IstellaBot=IstellaBot/1.01.18
IstellaBot=IstellaBot/1.10.2
IstellaBot=IstellaBot/1.18.81
IXEbot=IXEbot
Jabse.com Crawler=Jabse.com Crawler v.1.0
JadynAveBot=JadynAveBot
JikeSpider=JikeSpider
JikeSpider=JikeSpider b
Job Roboter Spider=Job Roboter Spider 3
JUST-CRAWLER=JUST-CRAWLER
Jyxobot=Jyxobot
Jyxobot=JyxobotRSS/0.06
Kalooga=Kalooga
Karneval-Bot=Karneval-Bot/1.06
KeywordDensityRobot=KeywordDensityRobot/0.8
KomodiaBot=KomodiaBot/1.0
Kraken=Kraken/0.1
KrOWLer=KrOWLer/0.0.1
kulturarw=kulturarw3
L.webis=L.webis/0.44
L.webis=L.webis/0.50
L.webis=L.webis/0.51
L.webis=L.webis/0.53
Leikibot=Leikibot/1.0
LemurWebCrawler=LemurWebCrawler
LexxeBot=LexxeBot/1.0
Lijit=Lijit
LinguaBot=LinguaBot/v0.001-dev
linguatools=linguatools
Linguee Bot=Linguee Bot
Link Valet Online=Link Valet Online 1.1
LinkAider=LinkAider
linkdex.com=linkdex.com/v2.0
linkdexbot=linkdexbot
linkdexbot=linkdexbot/2.0
linkdexbot=linkdexbot/2.1
LinkedInBot=LinkedInBot/1.0
LinkWalker=LinkWalker/2.0
Lipperhey Spider=Lipperhey Spider
livedoor ScreenShot=livedoor ScreenShot/0.10
LoadImpactPageAnalyzer=LoadImpactPageAnalyzer/1.3.0
LoadTimeBot=LoadTimeBot/0.7
LoadTimeBot=LoadTimeBot/0.81
LoadTimeBot=LoadTimeBot/0.9
LuminateBot=LuminateBot/1.0
magpie-crawler=magpie-crawler/1.1
Mail.Ru bot=Mail.Ru/1.0
Mail.Ru bot=Mail.RU/2.0
Mail.Ru bot=Mail.RU_Bot/2.0
Mail.Ru bot=Mail.RU_Bot/Fast/2.0
Mail.Ru bot=Mail.RU_Bot/Img/2.0
meanpathbot=meanpathbot/1.0
MeMoNewsBot=MeMoNewsBot/2.0
MetaGeneratorCrawler=MetaGeneratorCrawler/1.1
MetaGeneratorCrawler=MetaGeneratorCrawler/1.3.14
MetaGeneratorCrawler=MetaGeneratorCrawler/1.3.3
MetaHeadersBot=MetaHeadersBot
MetaJobBot=MetaJobBot
MetamojiCrawler=MetamojiCrawler/1.0
Metaspinner/0.01=Metaspinner/0.01
MetaURI API=MetaURI API/2.0
MetaURI=MetaURI
MIA Bot=MIA Bot
MiaDev=MiaDev/0.0.1
MJ12bot=MJ12bot/v1.2.1
MJ12bot=MJ12bot/v1.2.3
MJ12bot=MJ12bot/v1.2.4
MJ12bot=MJ12bot/v1.2.5
MJ12bot=MJ12bot/v1.3.0
MJ12bot=MJ12bot/v1.3.1
MJ12bot=MJ12bot/v1.3.2
MJ12bot=MJ12bot/v1.3.3
MJ12bot=MJ12bot/v1.4.0
MJ12bot=MJ12bot/v1.4.1
MJ12bot=MJ12bot/v1.4.2
MJ12bot=MJ12bot/v1.4.3
MJ12bot=MJ12bot/v1.4.4
MLBot=MLBot b
MnoGoSearch=MnoGoSearch/3.3.9
Moatbot=Moatbot/2.2
moba-crawler=moba-crawler
MojeekBot=MojeekBot/0.2
MojeekBot=MojeekBot/0.2 Relaunch
MojeekBot=MojeekBot/0.5
MojeekBot=MojeekBot/0.6
Motoricerca-Robots.txt-Checker=Motoricerca-Robots.txt-Checker/1.0
Mp3Bot=Mp3Bot/0.4
Mp3Bot=Mp3Bot/0.7
MSNBot=adidxbot/1.1
MSNBot=adidxbot/2.0
MSNBot=msnbot-media/1.1
MSNBot=msnbot-media/2.0b
MSNBot=msnbot-NewsBlogs/2.0b
MSNBot=msnbot-UDiscovery/2.0b
MSNBot=MSNBot/1.0
MSNBot=MSNBot/1.1
MSNBot=MSNBot/2.0b
MSNBot=MSNBot/2.0b + .
MSNBot=MSNBot/2.0b + ._
MSRBOT=MSRBOT
musobot=musobot/1.0
Najdi.si=Najdi.si/3.1
NalezenCzBot=NalezenCzBot/1.0
NaverBot=Yeti-Mobile/0.1
NaverBot=NaverBot/1.0
NaverBot=Yepi/1.0
NaverBot=Yeti-FeedItemCrawler/1.0
NaverBot=Yeti/1.0
NaverBot=Yeti/1.1
nekstbot=nekstbot
NerdByNature.Bot=NerdByNature.Bot
NetcraftSurveyAgent=NetcraftSurveyAgent/1.0
netEstate Crawler=netEstate NE Crawler
netEstate Crawler=netEstate RSS crawler
NetResearchServer=nrsbot/5.0
NetResearchServer=nrsbot/6.0
Netseer=Netseer crawler/2.0
NextGenSearchBot=NextGenSearchBot 1 b
Nigma.ru=Nigma.ru/3.0
NLNZ_IAHarvester2013=NLNZ_IAHarvester2013
nodestackbot=nodestackbot/0.1
Nuhk=Nuhk/2.4
Nuhk=Nuhk/2.4 b
Nutch=Nutch/1.0 at zscho.de
nworm=nwormFeedFinder
Nymesis=Nymesis/2.0
oBot=oBot
oBot=oBot - b
oBot=oBot/2.3.1
oBot=oBot/2.3.1 b
Ocelli=Ocelli/1.4
omgilibot=omgilibot/0.4
OoyyoBot=OoyyoBot
Open Web Analytics Bot=Open Web Analytics Bot 1.5.4
OpenCalaisSemanticProxy=OpenCalaisSemanticProxy
OpenindexSpider=OpenindexDeepSpider
OpenindexSpider=OpenindexShalooowSpider
OpenindexSpider=OpenindexSpider
OpenWebSpider=OpenWebSpider v0.1.2.B
OpenWebSpider=OpenWebSpider v0.1.4
OrgbyBot=OrgbyBot/1.2
OrgbyBot=OrgbyBot/1.3
OsObot=OsObot
ownCloud Server Crawler=ownCloud Server Crawler
Page2RSS=Page2RSS/0.7
page_verifier=page_verifier
PagePeeker=PagePeeker
PagePeeker=PagePeeker/2.1
Panscient web crawler=Panscient web crawler
PaperLiBot=PaperLiBot/2.1
ParchBot=ParchBot/1.0
parsijoo=parsijoo
PayPal IPN=PayPal IPN
Peeplo Screenshot Bot=Peeplo Screenshot Bot/0.20
Peepowbot=Peepowbot/1.0
peerindex=peerindex/0.1
Peew=Peew/1.0
percbotspider=percbotspider
pingdom.com_bot=pingdom.com_bot 1.4
PiplBot=PiplBot
Pixray-Seeker=Pixray-Seeker/1.1
Pixray-Seeker=Pixray-Seeker/2.0
Plukkie=Plukkie/1.1
Plukkie=Plukkie/1.2
Plukkie=Plukkie/1.3
Plukkie=Plukkie/1.4
pmoz.info ODP link checker=pmoz.info ODP link checker
Pompos=Pompos/1.3
PostPost=PostPost/1.0
ProCogBot=ProCogBot/1.0
ProCogSEOBot=ProCogSEOBot/1.0
proximic=proximic
psbot=psbot-page
psbot=psbot/0.1
Qirina Hurdler=Qirina Hurdler v. 1.05 10.11.01
Qseero=Qseero
Qseero=Qseero 1.0.0
Qualidator.com Bot=Qualidator.com Bot 1.0
Qualidator.com SiteAnalyzer 1.0=Qualidator.com SiteAnalyzer 1.0
QuerySeekerSpider=QuerySeekerSpider
quickobot=quickobot-1
R6 bot=R6_CommentReader
R6 bot=R6_FeedFetcher
RADaR-Bot=RADaR-Bot
RankurBot=RankurBot/2.1
Readability=Readability/6a54d4
Robots_Tester=Robots_Tester
Robozilla=Robozilla/1.0
rogerbot=rogerbot/1.0
Ronzoobot=Ronzoobot/1.2
Ronzoobot=Ronzoobot/1.3
Ronzoobot=Ronzoobot/1.5
Ronzoobot=Ronzoobot/1.6
RSSMicro.com RSS/Atom Feed Robot=RSSMicro.com RSS/Atom Feed Robot
Ruky-Roboter=Ruky-Roboter/1.06
RyzeCrawler=RyzeCrawler/1.1.1
SAI Crawler=SAI Crawler
SanszBot=SanszBot/1.7
SBIder=SBIder/1.0
SBSearch=SBSearch/0.9
Scarlett=Scarlett/ 1.0
Scooter=Scooter/3.3
ScoutJet=ScoutJet
ScoutJet=ScoutJet old
ScreenerBot Crawler=ScreenerBot Crawler Beta 2.0
search.KumKie.com=search.KumKie.com
Search17Bot=Search17Bot/1.1
SearchmetricsBot=SearchmetricsBot
SecurityResearchBot=SecurityResearchBot
seegnifybot=seegnifybot/1.0.0
Semager=Semager/1.4
Semager=Semager/1.4c
Semantifire=Semantifire1/0.20
SemrushBot=SemrushBot
SemrushBot=SemrushBot/0.9
SemrushBot=SemrushBot/0.91
SemrushBot=SemrushBot/0.92
SemrushBot=SemrushBot/0.95
SemrushBot=SemrushBot/0.96.2
SemrushBot=SemrushBot/0.96.3
SemrushBot=SemrushBot/0.96.4
SemrushBot=SemrushBot/0.97
SeoCheckBot=SeoCheck
SeoCheckBot=SeoCheckBot
SEODat=SEODat/0.1
SEOENGBot=SEOENGBot/1.0
SEOENGBot=SEOENGBot/1.2
SEOENGBot=SEOENGBot/1.2 new
SEOENGBot=SEOENGBot/1.2 old
SEOkicks-Robot=SEOkicks-Robot
Setoozbot=OOZBOT/0.20 c
Setoozbot=OOZBOT/0.20
Setoozbot=OOZBOT/0.20 b
Setoozbot=SETOOZBOT/0.30
Setoozbot=SETOOZBOT/0.30 a
Setoozbot=SETOOZBOT/0.30 b
Setoozbot=Setoozbot/1.0
Setoozbot=SETOOZBOT/5.0
SeznamBot=Seznam screenshot-generator 2.0
SeznamBot=Seznam screenshot-generator 2.1
SeznamBot=Seznam screenshot-generator Q3
SeznamBot=SeznamBot/2.0
SeznamBot=SeznamBot/2.0-test
SeznamBot=SeznamBot/3.0
SeznamBot=SeznamBot/3.0-alpha
SeznamBot=SeznamBot/3.0-beta
SeznamBot=SeznamBot/3.0-test
SeznamBot=SeznamBot/3.1-test
SeznamBot=SeznamBot/3.2
SeznamBot=SeznamBot/3.2-test1
Shareaholicbot=Shareaholicbot/1.0
Shelob=Shelob
ShopWiki=ShopWiki/1.0
ShowyouBot=ShowyouBot
sistrix=sistrix
SiteCondor=SiteCondor
Sitedomain-Bot=Sitedomain-Bot 1.0
smart.apnoti.com Robot=smart.apnoti.com Robot/v1.34
Snapbot=Snapbot/1.0 c
SniffRSS=SniffRSS/0.5beta
socialbm_bot=socialbm_bot
socialbm_bot=socialbm_bot/1.0
sogou spider=Sogou web spider
sogou spider=Sogou web spider/4.0
sogou spider=Sogou web spider/4.0l-2m!
sogou spider=Sogou-Test-Spider/4.0
SolomonoBot=SolomonoBot/1.04
Sosospider=Sosospider
Sosospider=Sosospider/2.0
spbot=spbot/1.0
spbot=spbot/1.1
spbot=spbot/1.2
spbot=spbot/2.0
spbot=spbot/2.0.1
spbot=spbot/2.0.2
spbot=spbot/2.0.3
spbot=spbot/2.0.4
spbot=spbot/2.1
spbot=spbot/3.0
spbot=spbot/3.1
spbot=spbot/4.0
spbot=spbot/4.0.1
spbot=spbot/4.0.2
spbot=spbot/4.0.3
spbot=spbot/4.0.4
spbot=spbot/4.0.5
spbot=spbot/4.0.6
spbot=spbot/4.0.7
spbot=spbot/4.0.8
spbot=spbot/4.0.9
spbot=spbot/4.0a
spbot=spbot/4.0b
Speedy=Speedy Spider
Speedy=Speedy Spider Beta/1.2
SpiderLing=SpiderLing
Spinn3r=Spinn3r 3.1
SSL-Crawler=SSL-Crawler
SSLBot=SSLBot/1.0
StackRambler=StackRambler/2.0
StatoolsBot=StatoolsBot
Steeler=Steeler/3.5
Strokebot=Strokebot
suggybot=suggybot/0.01a
Surphace Scout=Surphace Scout/4.0
SurveyBot=SurveyBot/2.3
SWEBot=SWEBot/1.0
SygolBot=SygolBot
Szukacz=Szukacz/1.5 b
Tagoobot=Tagoobot/3.0
taptubot=taptubot
Technoratibot=Technoratibot/7.0
Technoratibot=Technoratibot/8.0
Thumbnail.CZ robot=Thumbnail.CZ robot 1.1
ThumbShots-Bot=ThumbShots-Bot
thumbshots-de-Bot=thumbshots-de-bot
Thumbshots.ru=Thumbshots.ru
ThumbSniper=ThumbSniper
TinEye=TinEye-bot/0.02
TinEye=TinEye/1.0
TinEye=TinEye/1.1
TomTom places company search=TomTom places company search
Topicbot=Topicbot/12.0rc-2
Toread-Crawler=Toread-Crawler/1.1
Touche=Touche
trendictionbot=trendictionbot0.5.0
trendictionbot=trendictionbot0.4.2
TurnitinBot=TurnitinBot/2.1
TurnitinBot=TurnitinBot/3.0
TwengaBot=TwengaBot
TwengaBot=TwengaBot-Discover
TwengaBot=TwengaBot/1.1
Twiceler=Twiceler-0.9
Twiceler=Twiceler-0.9 b
Twikle=Twikle/1.0
UASlinkChecker=UASlinkChecker/1.0
uMBot=uMBot-FC/1.0
uMBot=uMBot-LN/1.0
UnisterBot=UnisterBot
UnwindFetchor=UnwindFetchor/1.0
Updownerbot=Updownerbot
UptimeDog=UptimeDog
UptimeRobot=UptimeRobot/1.0
URLAppendBot=URLAppendBot/1.0
urlfan-bot=urlfan-bot/1.0
Urlfilebot (Urlbot)=Urlfilebot/2.2
Vagabondo=Vagabondo/4.0
Vagabondo=Vagabondo/4.0Beta
Vedma=Vedma/0.91Beta
VideoSurf_bot=VideoSurf_bot
Visbot=Visbot/1.1
Visbot=Visbot/2.0
VoilaBot=VoilaBot BETA 1.2
voyager=voyager/1.0
voyager=voyager/2.0
WASALive-Bot=WASALive-Bot
WatchMouse=WatchMouse/18990 bc
WatchMouse=WatchMouse/18990 d2.watchmouse.com
WatchMouse=WatchMouse/18990 d3.watchmouse.com
WatchMouse=WatchMouse/18990 gab
WatchMouse=WatchMouse/18990 hk
WatchMouse=WatchMouse/18990 it
WatchMouse=WatchMouse/18990 liz
WatchMouse=WatchMouse/18990 ny
WatchMouse=WatchMouse/18990 se.watchmouse.com
WatchMouse=WatchMouse/18990 uk
WBSearchBot=WBSearchBot/1.1
Web-Monitoring=Web-Monitoring/1.0
Web-sniffer=Web-sniffer/1.0.31
WebCookies=WebCookies/1.0
WebImages=WebImages 0.3
webinatorbot=webinatorbot 1.0
webinatorbot=webinatorbot 1.1
webmastercoffee=webmastercoffee/0.7
WebNL=WebNL
WebRankSpider=WebRankSpider/1.37
WebThumbnail=WebThumbnail/2.2
WebThumbnail=WebThumbnail/3.x
WebWatch/Robot_txtChecker=WebWatch/Robot_txtChecker
WeSEE:Search=WeSEE
WeSEE:Search=WeSEE:Search
WeSEE:Search=WeSEE:Search/0.1 (Alpha)
Whoismindbot=Whoismindbot/1.0
WikioFeedBot=WikioFeedBot 1.0
wikiwix-bot=wikiwix-bot/3.0
Willow Internet Crawler=Willow Internet Crawler 2.1
WillyBot=WillyBot/1.1
WinWebBot=WinWebBot/1.0
WMCAI_robot=WMCAI_robot
Woko=Woko 3.0
Woko=Woko robot 3.0
woriobot=woriobot
Wotbox=Wotbox/2.0
Wotbox=Wotbox/2.01
wsAnalyzer=wsAnalyzer/1.0
wscheck.com=wscheck.com/1.0.0
XmarksFetch=XmarksFetch/1.0
XML Sitemaps Generator=XML Sitemaps Generator/1.0
XRL=XRL/3.00
Yaanb=Yaanb/1.5.001
Yaanb=Yaanb/1.5.001 b
yacybot=yacybot
Yahoo!=Y!J-BRI/0.0.1
Yahoo!=Y!J-BRJ/YATS
Yahoo!=Y!J-BRO/YFSJ
Yahoo!=Y!J-BRW/1.0
Yahoo!=Y!J-BSC/1.0
Yahoo!=Yahoo! Slurp
Yahoo!=Yahoo! Slurp China
Yahoo!=Yahoo! Slurp/3.0
Yahoo!=Yahoo-MMCrawler/4.0
Yahoo!=YahooCacheSystem
YandexBot=Mozilla/5.0 (compatible; YandexMedia/3.0
YandexBot=Yandex.Server/2009.5
YandexBot=Yandex.Server/2010.9
YandexBot=Yandex/1.01.001
YandexBot=YandexAntivirus/2.0
YandexBot=YandexBlogs/0.99
YandexBot=YandexBot/3.0
YandexBot=YandexBot/3.0-MirrorDetector
YandexBot=YandexCatalog/3.0
YandexBot=YandexDirect/3.0
YandexBot=YandexFavicons/1.0
YandexBot=YandexImageResizer/2.0
YandexBot=YandexImages/3.0
YandexBot=YandexMedia/3.0
YandexBot=YandexMetrika/2.0
YandexBot=YandexNews/3.0
YandexBot=YandexSomething/1.0
YandexBot=YandexVideo/3.0
YandexBot=YandexWebmaster/2.0
YandexBot=YandexZakladki/3.0
Yanga=Yanga v1.1/beta
YioopBot=gofind
YioopBot=YioopBot
YodaoBot=YodaoBot-Image/1.0
YodaoBot=YodaoBot/1.0
YoudaoBot=YoudaoBot/1.0
YowedoBot=YowedoBot/1.0
YRSpider=YRSpider
ZeerchBot=ZeerchBot ZB-1
Zookabot=Zookabot/2.0
ZookaBot=ZookaBot/2.1
Zookabot=Zookabot/2.2
Zookabot=Zookabot/2.4
Zookabot=Zookabot/2.5
ZumBot=ZumBot/1.0
|
|
|
|
Joined: Apr 2004
Posts: 1,973 Likes: 154
|
Joined: Apr 2004
Posts: 1,973 Likes: 154 |
The most current version of List of Search Engine Spiders for UBB.Threads can now be found at: https://www.ubbdev.com/spider-list/Although the source information for you to create your own updates and conversions is posted above, I plan on updating the post at that link every couple of months, making your job as an UBBT forum admin much easier.
|
|
|
|
Joined: Apr 2004
Posts: 1,973 Likes: 154
|
Joined: Apr 2004
Posts: 1,973 Likes: 154 |
The most current version of List of Search Engine Spiders for UBB.Threads can be found at: https://www.ubbdev.com/spider-list/The list has been updated 2014-10-14
|
|
|
0 members (),
1,613
guests, and
50
robots. |
Key:
Admin,
Global Mod,
Mod
|
|
|
|