My 2 cents I would block SemrushBot AhrefsBot They don't give you anything as far as a user search,They provide statistical data to subscribers only. and they are over whelming your site. According to the web they both honor the robots.txt file. Or block them in cloudflare.
|