Vibeking

llms.txt: what it is, what it does and what it doesn't

By Vibeking · Published 30 September 2026 · Facts checked on 30 September 2026

llms.txt is a text file in the root of your website that gives AI agents a short overview of your most important pages. It is a proposal from 2024, not an official standard.

Google Search does not use it, and no major AI search service has said publicly that it reads the llms.txt of other sites. It costs little and does not harm your discoverability in Google, but do not expect any extra visibility from it.

What is llms.txt?

Jeremy Howard of Answer.AI proposed llms.txt on 3 September 2024. The file is located at /llms.txt and is written in Markdown, so that a language model can read it directly. According to the specification, the aim is to provide information that helps agents use a website.

The difference from robots.txt is important. robots.txt indicates which bots may fetch your site; an llms.txt is intended for the moment an agent needs information while it is helping someone. OpenAI, Anthropic and Perplexity control their crawlers' access through robots.txt, not through llms.txt.

Version 2 of the specification was published on 10 August 2026. It specifies that an llms.txt in a subfolder applies to the pages below it and that the most specific file takes precedence. Version 2 also describes links through which a page refers to a Markdown version of itself and to the llms.txt that goes with it.

What does an llms.txt look like?

Only an H1 with the name of your site or business is required. After that come, in this order and all optional: a blockquote with a short summary and the key information, paragraphs or lists with more explanation (without headings) and sections with an H2 heading and a list of links. Each line in such a list is a Markdown link in the form [name](url), optionally followed by a colon and a short note.

By convention, you use a section with the heading Optional for links an agent can skip when space is limited; since version 2, that heading no longer has any special technical meaning. An example for a fictional bakery:

# Example Bakery

> Artisan bakery in Utrecht with two shops and delivery to businesses.

Open Tuesday to Saturday. Orders for businesses until 16.00 the day before.

## Services

- [Delivery for businesses](https://www.example.com/bedrijven): lunch and pastries for offices in Utrecht
- [Ordering cakes](https://www.example.com/taarten): at least 48 hours in advance

## About us

- [Contact and opening hours](https://www.example.com/contact)

## Optional

- [Vacancies](https://www.example.com/vacatures)

Who reads llms.txt?

Not Google Search. In its guide to optimising for AI search features (updated 10 July 2026), Google writes that such files neither help nor harm your visibility in Google Search, because Google Search ignores them. That also applies to AI Overviews and AI Mode.

AI companies do publish it, but do not say they use it. OpenAI, Anthropic, Perplexity and Google have an llms.txt for their own developer documentation. We found no official statement from these companies or from Microsoft that their assistants use the llms.txt of other sites when answering questions.

The figures point the same way. In May 2026, Ahrefs examined 137,210 domains: 28% had a valid llms.txt, and 97% of those files received no requests at all. Ahrefs itself calls that 28% an upper bound, because its customers are more technical than average. Of the requests that did come in, 1.1% came from AI search bots such as those of ChatGPT, Perplexity and Claude, and fetching a file is not the same as using it.

Across roughly 300,000 domains, SE Ranking found no relationship between having an llms.txt and the number of times a site was cited in AI answers. It is correlational research and therefore not hard evidence, but it fits the same picture.

So is an llms.txt worth it?

For most SME websites it is a small job with an uncertain effect. Get the basics right first: not blocking AI search bots in robots.txt, pages that can be indexed and content that clearly answers your visitor's question.

An llms.txt makes the most sense if you have documentation, an API or a lot of product information that agents and coding tools use. Jeremy Howard, the author of the specification, writes that coding agents use llms.txt reliably, but gives no figures for this.

Mistakes to avoid

  • An HTML page at /llms.txt. If your site redirects that address to the homepage or an error page, there is no valid file. Ahrefs counted such pages as invalid.
  • Dead links. An agent that follows a link and gets an error gains nothing from your overview.
  • A complete sitemap. Version 2 of the specification asks for a file that stays small enough to fit in the context of a language model.
  • Instructions to AI. Ahrefs advises against including anything that looks like a command, because of the risk of prompt injection.
  • Thinking it controls access. If you want to keep a crawler out, you do that in robots.txt. According to OpenAI and Perplexity, fetches that a user starts themselves, such as ChatGPT-User and Perplexity-User, do not always follow robots.txt.

John Mueller of Google wrote in July 2025 that a noindex header on llms.txt can make sense, so that the file itself does not appear in the search results. With Yoast SEO for WordPress you create the file in the free plugin; Wix creates it automatically for sites with a paid plan and their own domain, but stops updating it as soon as you edit it yourself.

How to check it

Open yourdomain.com/llms.txt in your browser: you should see plain text that starts with a # and your company name. Then click through the links to check them.

The free Vibeking SEO check does this automatically: it checks whether there is a text file at /llms.txt (an HTML page does not count) and tests up to 100 links in it.

Sources

How is your site doing?

The free Vibeking SEO check runs 30 checks and 7 AI checks on your site and turns every finding into a fix task with guidance.

Run the free SEO check

More in the knowledge base

How does your website score?

Run the free check: 30 checks and 7 AI checks, with a fix task for every finding. No payment details needed.