Lesson 6.12
llms.txt: what it is, and what Google says about it
Published . Sources checked .
llms.txt is a proposed file that a site can publish to give language models a tidy summary of itself. This lesson covers where the proposal came from, what Google Search now says about it in unusually plain words, what other platforms document, and what this site’s own queued experiment can and cannot settle.
- You will be able to
- Say what
llms.txtis proposed to do, quote Google’s current position on it, and decide whether to publish one for reasons that are actually yours. - Before this lesson
- Lesson 6.11, AI crawler access.
Concept
The proposal is public and has an author. It says: “We propose adding a /llms.txt markdown file to websites to provide LLM-friendly content.” Evidence: Expert / industry
The problem it describes is that pages are built for people. Navigation, adverts and layout surround the content, and the proposal argues that “Agents are best served by concise, expert-level information gathered in a single, accessible location.” Evidence: Expert / industry
It was proposed by Jeremy Howard on 3 September 2024 and has been revised since, with the site showing modifications through 10 August 2026.
On when it would be used: “Our expectation was that llms.txt would mainly be useful for inference rather than training, and that is how it has been used, though training runs could take advantage of the information too.” Evidence: Expert / industry
Note the word expectation. That sentence is the proposal describing its own intent and its author’s understanding of uptake. It is not a platform stating what that platform does.
What Google Search says
Google’s position is now written out plainly, and it is worth quoting in full rather than paraphrasing.
“You don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn’t use them.” Evidence: Documented by Google
“It’s completely fine if you decide to create and maintain LLMS.txt files (or other similar files) for other services or systems that use these files.” Evidence: Documented by Google
“Doing so will neither harm nor help your site’s visibility or rankings in Google Search, as Google Search ignores them.” Evidence: Documented by Google
Three separate statements, and it is worth keeping them apart. Google Search does not use the file. Publishing one is fine. It changes nothing either way in Google Search.
That is narrower than either of the two positions you will usually see argued. It is not “llms.txt is useless”, and this lesson does not say that: Google is speaking for Google Search, not for every system that might read a file. It is also not “llms.txt helps AI search”, because for Google’s own AI features the same sentence applies.
What other systems document
Here the evidence thins out considerably, and the honest answer is a short one.
Chrome ships a Lighthouse audit for the file. Chrome’s documentation says “Lighthouse flags the pages if a server error occurs when attempting to retrieve the llms.txtfile”, and explains the rationale as: “Without this file, agents may spend more time crawling the site to understand its high-level structure and primary content.” Evidence: Documented by Chrome
Two things about that. Chrome is a different product from Google Search, and the Chrome page does not mention Google Search anywhere. And the audit checks that the file does not error, treating a missing file as not applicable, since publishing one is optional. An audit that checks a file parses is not evidence that any model reads it.
Beyond that: reviewing the crawler and platform documentation covered in lesson 6.11, no documented use of a site’s llms.txt was found in OpenAI’s, Anthropic’s or Perplexity’s own documentation as re-read on 20 September 2026.
That is an absence of documentation. It is not proof that no system reads the file. Those are different statements and this lesson does not turn the first into the second.
How to decide
- Do not publish one expecting Google Search to respond. Google says it ignores the file, and says so about its AI features too.
- Do not expect a ranking change in either direction. Google’s sentence covers both directions explicitly.
- Publish one if a system you care about documents using it. That is the reason the proposal gives, and Google explicitly says doing so is fine.
- If you publish one, keep its contents accurate and current.
- Do not let it replace the page. The content still has to be on the page for everything covered in lessons 6.1 to 6.7.
Open Knowledge Format, briefly
The Open Knowledge Format comes up alongside llms.txt and is a different thing, from a different part of Google, for a different job. It was announced on the Google Cloud blog on 13 June 2026 as “an open specification that formalizes the LLM-wiki pattern into a portable, interoperable format”, describing knowledge as “a directory of markdown files with YAML frontmatter”, and Google Cloud updated its Knowledge Catalog “to be able to ingest Open Knowledge Format and serve it to our agents”. Evidence: Documented by Google Cloud
The problem it addresses is organisational: table schemas, metric definitions, runbooks and similar internal knowledge scattered across systems. Google Search is not mentioned anywhere on that page, and no Search Central page mentions OKF. It is not an SEO file, it is not a variant of llms.txt, and nothing here claims any connection between it and Google Search, rankings or AI Overviews.
What we changed on this site
Nothing. This site does not publish an llms.txt, and does not implement OKF.
What happened
Evidence: Observation
| Item | State |
|---|---|
llms.txt on this site | Not published |
| OKF on this site | Not implemented |
Queued experiment, llms.txt presence versus AI citation rate | Not run |
| Result | Not recorded |
The experiment is listed as queued. It has not been designed in detail, it has not run, and there is no result to report.
What this experiment could not settle even if it ran
Worth stating before it runs rather than after.
Measuring “presence versus citation rate” needs a reliable citation count, and lesson 6.11 records that retrieval is not observable with this site’s current stack. Answer sampling by hand would be the fallback, and lesson 6.2 sets out why single runs are anecdotes.
A single site adding one file is also not a controlled trial. Anything that changed afterwards would have many candidate causes, on a site that is actively publishing new lessons every day.
So if the experiment runs, its honest output is a dated observation with a sample size, not a finding about whether llms.txt works.
What we learned
The documentation is clearer than the discourse. Google’s three sentences settle the Google Search question completely, in both directions, and most of the argument about llms.txt is conducted as though they had not been written.
What they do not settle is whether any other system reads the file, and on that the operators’ own pages are silent rather than negative. Silence is where this lesson stops.
Terms used
- llms.txt
- A proposed markdown file giving language models a concise summary of a site. A proposal, not a standard any platform documents using.
- Inference
- A model answering now, possibly retrieving as it goes.
- Training
- Building a model from collected data beforehand.
- Open Knowledge Format
- A Google Cloud specification for organisational knowledge, unrelated to Google Search.