I have a site that is just crawled to death by AhrefsBot.
Every day when I view WOL at least 300 minimum by the same bot.
I looked them up
They do respect robots.txt.
But it appears it is a bot for marketing research only
Does this bot do any good for me?
Seems to me it does not help for search engine indexing for visitor traffic.
This site has the bulk of the bots as AhrefsBot..
Last edited by Ruben; 04/27/2019 12:47 PM. Reason: Added comment.