Robots.txt Validator
Your workspaceProcessed on your device
A few things to know.
Checks up to 500 KiB of UTF-8 text. Recognizes core exclusion rules, sitemap declarations and Crawl-delay syntax. The path test is a local simulation based on the parsed rules and exact product token. It cannot check deployed HTTP responses, provider-specific crawler aliases or indexing.
- Paste robots.txt or open a local UTF-8 file.
- To test a decision, supply a crawler product token and a path or absolute URL. Include the query string when a rule depends on it.
- Validate the file. Check the original line numbers and matched rule, then correct the text or download the JSON report.