Search Console - Url not indexed due to Robot.txt but still getting crawled.
by @seodump27 (112), 4 weeks ago
My Search Console displaying Url not indexed due to Robot.txt but still getting crawled. So, How to stop crwaling for a specific url even though the website is not getting indexed?
by @binayjha (4999), 4 weeks ago
The web page might be crawled before you applied no index property in robots.txt file. You can simply remove the crawled URL from Google's cache using the GSC and then it should not be crawled again as per the instruction in the robots.txt file.
