Previous Thread
Next Thread
Print Thread
Hop To
#262636 04/27/2019 12:52 PM
Joined: Dec 2003
Posts: 6,222
Likes: 32
Ruben Offline OP
OP Offline
Joined: Dec 2003
Posts: 6,222
Likes: 32
I have a site that is just crawled to death by AhrefsBot.
Every day when I view WOL at least 300 minimum by the same bot.
I looked them up
They do respect robots.txt.
But it appears it is a bot for marketing research only

Does this bot do any good for me?

Seems to me it does not help for search engine indexing for visitor traffic.

Thoughts..

FWIW,
This site has the bulk of the bots as AhrefsBot..

Last edited by Ruben; 04/27/2019 1:47 PM. Reason: Added comment.

Blue Man Group
There is no such thing as stupid questions. Just stupid answers
Joined: Jun 2006
Posts: 15,979
Likes: 41
UBB.threads Developer
Offline
UBB.threads Developer
Joined: Jun 2006
Posts: 15,979
Likes: 41
They're basically scanning your site for pages/sites you link to, and displaying data to their subscribers; it's likely not much benefit to you or your site, but to advertisers or your competitors it could be an SEO marketing tool.

It won't affect your search engine ranking anywhere to ban them, and may even speed up your site by not having them wasting your bandwidth for their service that you'll likely never use.


I am a Web Development Contractor, I do not work for UBBCentral. I have provided free User to User Support since the beginning of these support forums.
Do you need Forum Install or Upgrade Services?
Forums: A Gardeners Forum, Scouters World
UBB.threads: UBBWiki, UBB Styles, UBB.Sitemaps
Longtime Supporter & Resident Post-A-Holic
VNC Web Services: Code Modifications, Upgrades, Styling, Coding Services, Disaster Recovery, and more!
Joined: Dec 2003
Posts: 6,222
Likes: 32
Ruben Offline OP
OP Offline
Joined: Dec 2003
Posts: 6,222
Likes: 32
That's what I found out from my research.

Thanks.


Blue Man Group
There is no such thing as stupid questions. Just stupid answers

Link Copied to Clipboard
ShoutChat Box
Comment Guidelines: Do post respectful and insightful comments. Don't flame, hate, spam.
Recent Topics
Quick Help on issue changing forum styles
by Mors - 09/29/2020 7:59 PM
Individual Post Count Adjustments?
by FREAK - 09/25/2020 11:17 PM
Like unlike bug?
by Morgan - 09/24/2020 5:03 PM
looking to renew :)
by Mors - 09/23/2020 7:52 PM
test
by Mors - 09/23/2020 7:51 PM
Who's Online Now
1 members (JAISP), 75 guests, and 57 robots.
Key: Admin, Global Mod, Mod
Random Gallery Image
Latest Gallery Images
gallery test
gallery test
by Mors, September 23
Los Angeles
Los Angeles
by isaac, August 6
3D Creations
3D Creations
by JAISP, December 30
Artistic structures
Artistic structures
by isaac, August 29
Powered by UBB.threads™ PHP Forum Software 7.7.5
(Snapshot build 20200925)