llms.txt - Do you use it on your site?
Personally I am not using it, as neither OpenAI, Google, or Anthropic honor it as of now. They still rely mostly on robots.txt, but I found out Semrush already implemented this check into their Site Audit feature and is now showing it (if missing) as a medium severity issue (under Warnings).
Curious what you think about this proposed standard and its chances becoming new standard for web.
They will keep relying on robots.txt. The llms.txt file is just directive to AI bots which URL should be looked upon and what can be left. Searchengineland has stated, > OpenAI, Anthropic, Perplexity, and other leading AI companies have started referencing LLMS.txt when crawling sites
Here is the reference link,
https://searchengineland.com/llms-txt-isnt-robots-txt-its-a-treasure-map-for-ai-456586We should use it as I am finding. Above all, there is no harm in using it.
Definitely no harm in having one tiny extra TXT file in your public root, just curious if anyone seen anything happening differently after publishing the file.