Robots.txt & Sitemap.xml not working

It is important to have a robots.txt file on your website because it is a file where crawlers look for the content available on your website. You should make sure that the robots.txt file is accessible by the crawlers the same can be checked with the google's official tool.

If you use any CMS like wordpress or joomla, then by installing plugin you can generate the sitemap and robots.txt. On the other side, if you have a website without CMS or developed using any core language then you need to manually generate the same.

The same has happened with me because one of my article on best kodi addons was not accessible by the crawlers.

If your robots.txt and sitemap.xml files are not working as expected, there could be several reasons for this issue. Here are a few common troubleshooting steps you can take:

Verify the file location: Make sure that both the robots.txt and sitemap.xml files are placed in the root directory of your website. The root directory is the main folder where your website's files are stored.

Check file permissions: Ensure that the file permissions are set correctly to allow access to these files. The recommended permissions for robots.txt and sitemap.xml are typically 644, which means the owner has read and write permissions, and others have read-only permissions.

Confirm the correct file names: Double-check that the file names are spelled correctly, including proper capitalization. The file should be named "robots.txt" and "sitemap.xml" exactly.

Hi, Use different SEO auditing tools or validators to verify if your sitemap and robots.txt files are being recognized. Some tools may have different ways of identifying and detecting these files, so testing with multiple tools can give you a clearer picture of the issue.