What Is llms.txt and Should You Add One?
llms.txt is a proposed plain-text file, written in Markdown and placed at the root of a website, that gives large language models a short, clean map of the site's most useful pages.
Jeremy Howard of Answer.AI proposed it in September 2024. It is a suggestion for AI tools to read, with no official standards body behind it.
What goes in the file
The proposal asks for a simple layout:
- An H1 with the name of the site or project.
- A one-paragraph summary in a blockquote.
- Optional notes on how to use the information.
- H2 sections, each with a list of links and a short note on what each page covers.
- An "Optional" section for pages a tool can skip when it is short on space.
An example for a service business
# Bright Smile Dental Clinic > Family dental clinic in Model Town, Ludhiana. Open Monday to Saturday. > Treatments: cleaning, fillings, root canal, braces, implants. ## Services - [Root canal treatment](https://example.com/root-canal/): cost, steps, recovery - [Dental implants](https://example.com/implants/): who it suits, price range ## About - [Our dentists](https://example.com/team/): qualifications and registration numbers - [Contact and hours](https://example.com/contact/) ## Optional - [Blog](https://example.com/blog/)
Who reads it
The proposal's own site lists OpenAI, Anthropic and Google Gemini among those publishing llms.txt files for their developer docs, and some coding assistants read them.
Google representatives said in 2025 that Google Search does not use it. None of the big AI search engines has publicly confirmed using it to decide what to cite.
Should you add one?
It takes about 20 minutes and does no harm, so it is a cheap bet. Don't expect a ranking or citation change from it. If you add one, check your server logs after a month to see which bots actually requested it. That tells you more than any opinion online.
Common confusions
llms.txt doesn't block or allow AI crawlers. That job belongs to robots.txt.
llms-full.txt is a related idea: one file with the full text of your key pages, instead of links.
A starter llms.txt, line by line
A small file teaches the format faster than a description can.
This example is invented for this page, for a fictional dental clinic, and every line is explained under it.
# Bright Smile Dental > Dental clinic in Model Town, Ludhiana. Implants, braces and family dentistry. ## Services - [Implants](https://example.com/implants/): process, timeline and prices - [Braces](https://example.com/braces/): options for adults and teens ## Guides - [Cost guide](https://example.com/dental-costs-ludhiana/): dated prices for common treatments
A real file swaps in the site’s own pages and keeps the shape.
llms.txt against its neighbours
Three text files live at the root of many sites, and they get confused with each other.
Each one talks to a different reader.
| File | Main reader | What it says |
|---|---|---|
| robots.txt | Crawlers | Which parts of the site may or may not be crawled |
| sitemap.xml | Search engines | The full list of URLs the site wants indexed |
| llms.txt | AI assistants | A short, human-written map of the pages that matter most |
llms-full.txt, the bigger sibling
The plain llms.txt is an index: names, links and one-line notes. A companion file, llms-full.txt, goes further and carries the full text of the key pages in markdown. Some AI tools read the index, follow it to the full file, and work from complete text instead of crawling page by page.
llms.txt
Short. A map of your important pages with notes on what each holds. Cheap to keep current.
llms-full.txt
Long. The actual content of core pages in one file. Powerful for tools, heavier to maintain.
Start with the index
Add the full file only when a tool you care about asks for it, or when your core pages change rarely.
Keeping the file current
- Tie it to publishing. When a key page is added or retired, the file changes in the same sitting.
- Review quarterly. Check every link still resolves and every note still describes its page.
- Watch the logs. Server logs show which AI crawlers fetch the file. A file nobody fetches still costs little to keep.
- Keep notes honest. A note that oversells its page teaches crawlers to distrust the whole file.
What crawlers actually do with it
| Reader | Typical behaviour | What that asks of you |
|---|---|---|
| AI search crawlers | May fetch the file while mapping a site | Accurate links and notes |
| AI coding and agent tools | Read it directly when a user points them at your site | Clear structure, plain markdown |
| Classic search engines | No announced use in ranking | Keep your SEO basics; this file is an extra |
The common mistake. Listing every page on the site. The file works because it is selective. Ten to thirty important pages, each with a truthful note, beat a sitemap copy with hundreds of links.
Questions people ask
Do ChatGPT and other assistants actually read llms.txt?
Reading is voluntary and uneven today. Some tools and agents fetch the file, others ignore it, and no major engine promises citations for having one. The file costs little, so the open question is about benefit, not risk.
Where does llms.txt sit on a site?
At the root, beside robots.txt, at yourdomain.com/llms.txt, written in Markdown. A fuller companion file, llms-full.txt, can carry complete page text for tools that want it.
Should a small business add one?
Yes, once the site’s main pages are in good shape. The file takes under an hour to write well, and writing it forces a useful decision: which ten pages best represent the business.
Can llms.txt hurt SEO?
No mechanism makes it harmful. It is a plain text file that crawlers may read or ignore. Keep it accurate, since anything in it can be quoted back to your customers by an assistant.