robots.txt Validator
Validate a robots.txt file online — directive syntax, rules outside agent groups, duplicates, sitemap URLs, Google's 500 KB limit. Paste content or fetch a live file.
- line 7Crawl-delay is ignored by Google (only Bing/Yandex honour it).
Need to build one instead? Open the robots.txt Generator.
About robots.txt Validator
Paste your robots.txt and get instant feedback: malformed directives, rules outside any user-agent group, duplicate rules and sitemaps, non-absolute sitemap URLs, unsupported noindex directives, full-site Disallow: / and Google's 500 KB processing limit — each issue flagged with severity and line number.
You can also fetch a live file: browser fetch stays fully local (but CORS blocks most sites), while the server fetch mode retrieves the URL through our server with the trade-off clearly disclosed. Parsing itself always runs in your browser.