Robots.txt Generator
Generate a robots.txt file online — user-agent rules, disallow paths, crawl-delay and sitemap directive, ready to download.
| User-agent | Disallow paths (comma-separated) | |
|---|---|---|
robots.txt
User-agent: *
Disallow:
Want to check an existing file? Open the robots.txt Validator.
About Robots.txt Generator
Build a valid robots.txt visually: add user-agent blocks with disallow paths, set a crawl-delay, append your sitemap URL, then copy or download the file.
Presets cover the common cases — allow-all, block-all and blocking an admin area — so you can start from a working baseline.