Hi @binayjha

Thanks for your reply, but I believe this tool only "filters" these pages/URLs from the results for 6 months. However, it might be worth attempting, as I don't think we can noindex these site search results/

Hi there,

I have an ecommerce client who launched their online store without a robot.txt file to block the crawling of filters and site search. Due to this, they have thousands "Site Search" URLs indexed. I want to noindex these results before blocking crawling in the rpbots.txt file.

Does anyone know how you can do this with Magento? Or other suggestions on how to get these results removed before adding it to the robots.txt file?