thaitools.techPractical tools for life in Thailand
English interface

robots.txt generator

Generate user-agent, allow, disallow and sitemap directives.

ReviewedModerate-impact
Saves this tool's name on this device only (up to 20). No form values.
Only visible result text is included. Form fields are not read. Review sensitive values before sharing.

Processed locally. Review generated code before deployment.

Enter values, then process to see the result.

How to use it

  1. Paste or enter input within the displayed limit for robots.txt generator.
  2. Generate user-agent, allow, disallow and sitemap directives.
  3. Validate the output against the documented format and limitations before production use.

Important limitation

robots.txt is not access control and disallowed URLs may still be discovered.

The interactive form is fully localized in English.

Reviewed 2026-09-16; next review 2027-03-16.

Core calculations, cited sources and main test cases have been reviewed within the stated scope.

Sources

RFC 9309 — Robots Exclusion ProtocolIETF / RFC Editor · opens in a new tabGenerates user-agent, allow, disallow and sitemap directives.Source checked 2026-09-16
Traceable review record

Review history

Site changelog

This is the recorded review history available in the repository, not an inferred release history. Dates are added only when evidence is recorded.

  1. Current review snapshot recorded

    Core calculations, cited sources and main test cases have been reviewed within the stated scope.

    Method version: review-2026-09-16

Related guides

Check a web page from metadata through sitemap publicationUnderstand the meaning, scope and sources before using the result.

Related tools

JSON formatter and syntax validatorFormat or minify JSON locally while preserving original numeric tokens.UTF-8 Base64 text converterEncode or decode standard Base64 text locally, including Thai and emoji.YAML formatter and validatorParse and format YAML 1.2 while checking duplicate keys.XML formatter and validatorValidate and format well-formed XML with DOCTYPE blocked.