/llms.txt is a Markdown index that directs an agent to selected documentation URLs. Put one project name in an H1, follow it with a short blockquote, then group document links under H2 headings. /llms-full.txt is an optional companion for systems that need the document bodies in one response; publishing either file does not feed a model automatically. Source: llms.txt proposal
The file format
The August 2026 revision of the /llms.txt proposal permits a file at the site root or within a path such as /docs/llms.txt. A file within /docs/ covers that path, and the proposal says an agent should prefer the most specific applicable file. Only the H1 is required by the format; the blockquote, explanatory text, and H2 file lists are permitted in that order. Source: format specification
Each file list entry needs a Markdown link. A colon and a short description can follow its URL. The ## Optional heading marks a deterministic cutoff for retrieval pipelines, agents, or chunking scripts configured to treat that section as lower priority. Under a context-window budget or strict token threshold, those implementations can drop its links before links in the core sections. The published v2 proposal does not require every client to apply that rule. Source: v2 changes
Copyable /llms.txt template
Replace example.com and every sample product fact before deployment. Publish the linked Markdown files as public documents whose content matches the current product release.
# Example Cloud API
> Example Cloud API provides event ingestion and query endpoints for application telemetry.
The API base URL is https://api.example.com/v1.
Documentation version: 2026-09.
Authentication: Send a bearer token in the Authorization header.
Use the versioned API reference for endpoint behavior and limits.
## Start here
- [Quickstart](https://example.com/docs/quickstart.md): Create a token and send the first event.
- [Authentication](https://example.com/docs/authentication.md): Token scopes, rotation, and failure responses.
## API reference
- [Event ingestion](https://example.com/docs/api/events.md): Request schema, response codes, and limits.
- [Query API](https://example.com/docs/api/queries.md): Filters, pagination, and result fields.
- [Errors](https://example.com/docs/api/errors.md): Error codes and retry rules.
## Optional
- [Changelog](https://example.com/docs/changelog.md): Dated API changes.
The index should describe the product as it exists. Link to stable, specific pages and keep each description distinct enough for a retriever to select a document without fetching the entire corpus. The proposal expects an agent to inspect the index, choose relevant links, and fetch those resources when needed. Source: proposal
What /llms-full.txt does
/llms-full.txt is a convenient name for a compiled Markdown corpus. Unlike /llms.txt, it has no required grammar in the published proposal. Define its boundaries for your own RAG system: which documents it includes, how each source is identified, and when the bundle is regenerated. Source: proposal
A single bundle saves link resolution and multiple HTTP requests during a controlled import. It can also waste context on unrelated endpoints, duplicate text across versions, and make source attribution harder. For query-time retrieval, the compact index plus individually addressable documents gives the retriever smaller units to select.
Copyable /llms-full.txt template
This example uses source URLs as section boundaries. They are a publishing convention for this bundle, not reserved /llms.txt syntax.
# Example Cloud API: Complete Documentation
Documentation version: 2026-09
Generated: YYYY-MM-DD
Canonical documentation: https://example.com/docs/
API base URL: https://api.example.com/v1
## Source: https://example.com/docs/quickstart.md
# Quickstart
[Insert the complete, current quickstart text here.]
## Source: https://example.com/docs/authentication.md
# Authentication
[Insert the complete, current authentication text here.]
## Source: https://example.com/docs/api/events.md
# Event ingestion
[Insert the complete endpoint specification, limits, and examples here.]
## Source: https://example.com/docs/api/queries.md
# Query API
[Insert the complete query specification here.]
## Source: https://example.com/docs/api/errors.md
# Errors
[Insert the complete error and retry reference here.]
Generate the bundle from the same source files as the linked pages. Preserve code fences, tables, version labels, and source URLs during compilation. If the corpus exceeds the context or import limits of a target system, split it by product or API version instead of truncating the last sections without notice.
How retrieval clients encounter the files
An agent that implements the proposal can request /llms.txt, parse the H1 and H2 sections, then fetch a relevant Markdown link. A custom RAG pipeline can follow the same path and chunk the fetched documents with their source URLs attached. The proposal also recommends advertising page Markdown with rel="alternate" and its covering index with rel="describedby" in HTML or HTTP Link headers. Source: proposal
Provider behavior varies. OpenAI documents OAI-SearchBot for ChatGPT search, GPTBot for potential model training, and ChatGPT-User for certain user initiated fetches. Those roles do not establish automatic /llms.txt ingestion. Anthropic likewise documents separate bots for model development, search, and user requests; neither provider promises that publishing a Markdown index will put its contents into an answer. OpenAI crawlers · Anthropic crawlers
The shorthand "Googlebot reads HTML; AI engines read Markdown" obscures the delivery path. Google states that its Search features, including generative AI features, do not use llms.txt files; ordinary crawlable pages still matter. Markdown is useful when an agent or ingestion job elects to retrieve it, not a replacement for the public HTML documentation. Source: Google Search Central
Deployment and validation
Serve /llms.txt, /llms-full.txt, and their linked documents over HTTPS with successful HTTP responses. Check that a missing path returns a real 404 rather than an HTML application shell with a 200 status. Verify that a CDN, authentication layer, or bot challenge does not substitute a login page for the Markdown body.
A deployment check can start with the response headers and first lines:
curl -i https://example.com/llms.txt
curl -i https://example.com/llms-full.txt
curl -fsS https://example.com/llms.txt | head -n 12
Parse every Markdown link in /llms.txt, fetch its target, and flag redirects to sign-in pages, empty bodies, broken links, and content that belongs to an older release. Check robots.txt and bot access separately. OpenAI's guidance identifies crawler access as a factor in ChatGPT search eligibility; an index file cannot override a blocked destination page. OpenAI crawlers · Publisher FAQ
GEOall's platform scanner checks the presence and validity of these files as an AI readiness signal. The useful pass condition is narrower than "AI visibility": a readable index, working document links, and a full bundle that matches its source pages when one is published. Track actual bot requests and test source retrieval in the target system before attributing citations or answer changes to the files.
Discussion
No comments yet — start the discussion.