SEO

Robots.txt Generator

Create a correct robots.txt by adding user-agent rules, allow and disallow paths, crawl delay and a sitemap reference — with inline validation.

Examples

Input

Disallow /admin

Output

User-agent: *
Disallow: /admin
Sitemap: https://site.com/sitemap.xml

Choose user agents

Target all crawlers or specific bots.

Add rules

List the paths to allow or disallow.

Copy

Copy the generated robots.txt to your site root.

Per-user-agent rules

Sitemap reference

Crawl-delay support

Valid output

Does robots.txt hide pages from Google?

No. It controls crawling, not indexing. To keep a page out of results, use a noindex meta tag or HTTP header.

Share feedback or ask a question about this tool.

Comments are stored to a moderated queue via the API.