Do you Disallow Ahrefs/Semrush and similar bots?

by @ms (4012), 1 year ago

Tricky question SEOs.

Do you Disallow those bots (which according to their docs respect robots.txt)? If yes, why? If not, why?

For example, to prevent your competition from getting data about your site/business...

3 Replies
3 Users
Sort replies:
by @binayjha (4999), 1 year ago

No, I have never blocked those bots. I never felt any harm in letting them give such SEO data.

by @ms (4012), 1 year ago

I personally use Ahrefs so I don't block their bot, however I do block Semrush as I don't use it. Why allow it to consume my server resources just to give my competitors extra data about my website..

by @Gamerseo (1330), 1 year ago

We also never block any bots.

Join the forum to unlock true power of SEO community

You're welcome to become part of SEO Forum community. Register for free, learn and contribute.

Log In Sign up