An llms.txt file is a plain-text Markdown file at the root of your website that gives AI tools a short, curated list of your most useful pages. It is a proposal, not a standard, and the evidence that it improves how often AI tools cite you is weak. It is cheap to make, so it is a fair optional experiment, but it should come after the technical basics.
This guide covers what the file is, what is and isn’t known about whether it works, and how to write one. We also use one ourselves, and we will show you how it is structured.
What is llms.txt?
Jeremy Howard proposed llms.txt in September 2024. The idea is that language models have limited context windows and websites are messy, so a short Markdown file can give an AI tool a clean entry point to the content that matters. The llms.txt proposal describes it as a way to give agents a concise, curated map of a site, with detail behind links they can fetch on demand.
The proposal says use is expected mainly at inference time, meaning when someone asks a question, rather than for model training.
What goes in an llms.txt file?
The proposal defines this structure, in order:
- An H1 with the name of the site or project. This is the only required element.
- A blockquote with a short summary.
- Optional paragraphs or lists of detail, without headings.
- Optional H2 sections that hold lists of links. Each item is a Markdown link, optionally followed by a colon and a note.
- By convention, an H2 section named “Optional” for secondary links that an agent can skip when it needs less context.
The proposal also suggests that pages can have clean Markdown versions available at the same URL with .md added, and describes link tags for discovery. Those parts are optional and add work, so most sites start without them.
llms.txt vs. robots.txt vs. sitemap.xml
| File | Job | Who reads it |
|---|---|---|
| robots.txt | Says which crawlers may access which paths | Search and AI crawlers, before crawling |
| sitemap.xml | Lists all the URLs you want indexed | Search engines |
| llms.txt | Offers a short, curated guide to your best content | Proposed for AI tools and agents at question time |
The proposal itself positions llms.txt as a complement: sitemaps list everything, and robots.txt governs access. llms.txt is a selective summary. It cannot block anything, and it does not replace the other two. If you want to control which AI crawlers can reach your site, that is a robots.txt job, and the AI crawlers publish their own names and rules (for example, OpenAI’s crawler documentation).
Does llms.txt work?
This is the question that matters, and the answer is that nobody has shown that it does.
- Google says it is not needed. Google’s guidance on AI features states that you do not need new machine-readable files, AI text files, or special markup to appear in AI Overviews and AI Mode. See Google Search Central: AI features. In a June 2026 report, Google’s John Mueller called llms.txt “purely speculative for now”, noting that none of the AI systems appear to use it.
- The large study found no effect. An SE Ranking analysis published October 15, 2026 looked at nearly 300,000 domains. About 10.13% had an llms.txt file. The authors found no correlation between having the file and how often a domain was cited by AI models. SE Ranking is an SEO tool vendor, and the authors note that their results are specific to the model and dataset they tested.
- We found no official support from OpenAI or Anthropic. We did not find a statement from either company saying their products read the file. That is an absence in what we found, not proof that nobody uses it.
So the honest summary is: it may help some AI agents that choose to read it, and there is no demonstrated benefit to citations. Anyone who tells you it is a proven ranking factor is going beyond the evidence.
How to create an llms.txt file
- Choose 10 to 30 pages. Pick the pages you most want an AI tool to understand: your product or service pages, your best definitions and guides, pricing, and any documentation.
- Write the H1 and the summary. Use your company name and one or two plain sentences about what you do and for whom. Keep the wording consistent with your site and your external profiles.
- Group the links under H2 headings. Use headings that match how a reader would think about the site, such as Products, Guides, and Company.
- Add a short note to each link. One sentence on what the page covers is enough.
- Put an Optional section at the end for pages that are useful but not essential.
- Save it as
llms.txtat the site root. On a Jekyll site like ours, that means a file at the top level of the repository. - Check that it loads. Open
yourdomain.com/llms.txtin a browser and confirm it returns plain text, not a 404 or an HTML page. - Keep it current. Update it when you publish or retire important pages, otherwise it points to dead links.
A minimal example:
# Example Co
> Example Co makes scheduling software for dental practices.
## Product
- [Scheduling software](https://example.com/product/): Overview, features, and pricing.
## Guides
- [How to reduce no-shows](https://example.com/blog/reduce-no-shows/): A practical guide for front-desk teams.
## Optional
- [Press kit](https://example.com/press/): Logos and company facts.
Our own file as an example
We publish one at fixmyairank.com/llms.txt. It follows the structure above: an H1 with the company name, a blockquote summary of what we do, a short paragraph about who runs the site, and then H2 sections for the report, the glossary, the blog, and the company page. Each entry is a link with a one-line note.
We keep it in step with the site. When we merged overlapping pages earlier this year, we removed the retired URLs and updated the titles. We include it for completeness and as a low-cost experiment. We are not claiming it has produced a measurable result.
Common mistakes
- Listing every page. The point is to be selective. A sitemap already lists everything.
- Leaving dead or redirected links. If you merge or retire pages, update the file.
- Contradicting your site. The company description should match your homepage and profiles.
- Treating it as a block list. It cannot stop AI tools from using your content. Use robots.txt for access control.
- Expecting a lift. Nothing in the evidence supports promising one.
How to tell whether it matters for you
Check your server or CDN logs for requests to /llms.txt and see which user agents make them. If nothing reads the file, you have your answer for now. Then watch your AI-referral traffic and run a fixed set of buyer questions each month, as described in how to track your brand’s AI visibility. Any change in citations is likely to come from other work, so log what you change and when.
What to do before llms.txt
If AI tools cannot reach or understand your site, a summary file will not fix it. Start with the basics in how to optimize for AI search: allow the crawlers you want, get key pages indexed, and keep your entity signals and schema markup consistent. If you want a baseline first, why your website isn’t showing up in AI answers is the audit.