Do you Disallow Ahrefs/Semrush and similar bots?
by @ms (3855), 1 year ago
Tricky question SEOs.
Do you Disallow those bots (which according to their docs respect robots.txt)? If yes, why? If not, why?
For example, to prevent your competition from getting data about your site/business...
by @Gamerseo (1330), 1 year ago
We also never block any bots.
by @ms (3855), 1 year ago
I personally use Ahrefs so I don't block their bot, however I do block Semrush as I don't use it. Why allow it to consume my server resources just to give my competitors extra data about my website..
by @binayjha (4699), 1 year ago
No, I have never blocked those bots. I never felt any harm in letting them give such SEO data.
