agentsready
Discoverability Easy

How to write llms.txt: a guide with an example

What llms.txt is, why AI agents need it, a minimal working format example, right vs wrong, common mistakes, and how to verify.

Updated:

What it is

llms.txt is a Markdown file at /llms.txt, proposed as a standard for guiding large language models. Unlike robots.txt (which says what may be crawled), llms.txt gives the LLM a curated map of your site: what the project is, which pages matter, and why.

Why it matters for AI agents

An LLM has a limited context window — it can’t “read your whole site.” llms.txt concentrates the essence into one short file: a brief description plus links to key pages with explanations. That raises the chance an agent pulls your important content into an answer rather than a random page.

Minimal working example

# Example Inc.

> Analytics platform for e-commerce: dashboards, reports, API.

## Documentation

- [Quickstart](https://example.com/docs/quickstart): set up in 5 minutes
- [API Reference](https://example.com/docs/api): full REST API description

## Product

- [Pricing](https://example.com/pricing): plans and prices
- [Security](https://example.com/security): SOC 2, encryption

## Optional

- [Blog](https://example.com/blog): case studies and updates

The ## Optional section is what can be skipped when context is tight.

Right vs wrong

RightWrong
One H1 + a short > blockquote overviewNo description, straight to a link list
Absolute URLs + an explanation after :Bare links with no context
Only key pages (10-30)A dump of every site URL — meaning is lost
Served at /llms.txt, text/plain or text/markdownAn HTML page instead of plain markdown

Common mistakes

  • Dumping the whole sitemap into llms.txt — the file should be curated, not complete.
  • No short overview (> ...) — the model can’t tell what the site is.
  • Links with no explanations — the format’s core value is lost.
  • Served as HTML (SPA) — not an llms.txt to an agent.
  • Confusing it with llms-full.txt (the extended variant) — start with llms.txt.

How to verify

A scan checks that /llms.txt exists and is well-formed. Manually:

curl -s https://example.com/llms.txt | head -30

Make sure the file starts with # Title, has a > overview, and links with explanations.

Sources