ms

Do you Disallow Ahrefs/Semrush and similar bots?

by @ms (3058), 3 weeks ago

Tricky question SEOs.

Do you Disallow those bots (which according to their docs respect robots.txt)? If yes, why? If not, why?

For example, to prevent your competition from getting data about your site/business...

83 Views
0 Upvotes
3 Replies
3 Users
Sort replies:
binayjha
by @binayjha (2050), 3 weeks ago

No, I have never blocked those bots. I never felt any harm in letting them give such SEO data.

ms
by @ms (3058), 2 weeks ago

I personally use Ahrefs so I don't block their bot, however I do block Semrush as I don't use it. Why allow it to consume my server resources just to give my competitors extra data about my website..

Gamerseo
by @Gamerseo (975), 2 weeks ago

We also never block any bots.

Join the forum to unlock true power of SEO community

You're welcome to become part of SEO Forum community. Register for free, learn and contribute.

Log In Sign up