llms.txt: A Cautious Implementation and Test Guide
llms.txt is a voluntary proposal for a Markdown file that points to useful site material. It is not a web standard, access-control file, ranking signal, or documented guarantee of assistant use.
Decide whether the experiment is worth maintaining
Read the proposal, identify the audience for the file, and assign an owner. If your canonical documentation is incomplete or stale, fix that first. The file should point to public material rather than introduce a second, conflicting source of truth.
Create a minimal file
- Use the organization or project name as the main heading.
- Add a short, factual summary.
- Group links to canonical documentation with concise descriptions.
- Include only public URLs you are willing to maintain.
- Do not place secrets, private endpoints, or unsupported marketing claims in the file.
Validate deployment
Serve the file at the intended path with a successful response and text content type. Check links, redirects, encoding, cache behavior, and production deployment. Add a maintenance check for broken or non-canonical links.
Evaluate separately from outcomes
Record crawler requests to the file if available and explicitly permitted by your privacy policy. A request does not show that content was used. Any assistant citation study still requires fixed prompts and retained responses before and after deployment.
Limits and next step
Treat llms.txt as a low-cost publishing experiment only when you can maintain it. Do not report implementation, fetches, or technical validity as evidence of recommendations or citations.