Not automatic removal
Google may show a known URL without crawling it. Crawl blocking is not index removal.
It tells compliant crawlers which URLs they may access.
Google may show a known URL without crawling it. Crawl blocking is not index removal.
Private areas need access control. Disallow is public and cannot stop non-compliant clients.
Check image and stylesheet access. A crawler must be able to see a noindex directive to process it.
The explainer is checked against these institutional and industry sources. Links open the original material.