Discoverability · Essential · Critical impact
/llms.txt published and well-structured
The most widely adopted contract for making a site legible to agents. In the curated corpus we measured, 46% of sites published one and the average quality score was 37 out of 100 — adoption is not the same as usefulness.
- Essential
- Critical impact
- Draft standard
| Property | Value |
|---|---|
| Check id | llms-txt |
| Dimension | Discoverability (20%) |
| Tier | Essential — pool 80 pts |
| Weight inside tier | 3 |
| Impact | Critical |
| Maturity | Draft — A published specification that is not yet ratified. |
| Conditional | No |
| Publishing skill | llms-txt |
| Standards | llmstxt.org |
What the scanner requests
-
/llms.txt
What is measured
The scanner fetches https://<domain>/llms.txt (the exact path this check probes) and parses the markdown against the llmstxt.org contract. It then requests up to 6 same-origin links the file names, beyond the paths the scan already requests. A link on another origin is not requested, and a link that gets no answer is not measured. Pass = a 2xx response served as text/plain or text/markdown, opening with an H1, followed by a blockquote summary and at least one H2 section listing markdown links written as name. No requested same-origin link may answer a status other than 2xx, and the pass states how many links were requested. Partial = the file is served with another content type, has no H1, has an H1 after other lines, has no H2 link list, has no blockquote, or names a requested same-origin link that answered a status other than 2xx. Fail = a non-2xx status for the file, or an HTML document returned for the path. A request for the file that gets no answer is not measured.
Reproduce it yourself
You do not have to take our word for any of this. Run the same measurement against your own domain:
DOMAIN=example.com
curl -sS "https://$DOMAIN/llms.txt" | head -25
The fix
This is the artifact to ship. It is the complete change, not a description of one.
# Example Inc
> Example Inc runs a payments API for marketplaces. This file lists the pages an agent needs, with a short note on each.
Example Inc is US-based; support is at support@example.com.
API access uses OAuth 2.0 tokens, see the authentication guide below.
## Docs
- [Quickstart](https://example.com/docs/quickstart.md): Install the CLI and make a first API call
- [API reference](https://example.com/docs/api.md): Every endpoint, with request and response examples
- [Authentication](https://example.com/docs/auth.md): OAuth 2.0 flows and token lifetimes
## Company
- [Pricing](https://example.com/pricing.md): Plans, limits and overage rates
- [Status](https://example.com/status): Live service status and incident history
## Optional
- [Changelog](https://example.com/changelog.md): Release notes since 2024
Standards referenced
-
llmstxt.org
Related checks
| Check | Tier | Impact | Same skill |
|---|---|---|---|
reachable — Homepage open to agent access | Essential | Critical | — |
robots-txt — robots.txt with agent directives | Essential | High | — |
sitemap-xml — sitemap.xml valid and current | Essential | High | — |
link-headers — Discovery via Link response headers | Emerging | Medium | — |
dns-aid — DNS-AID discovery record | Emerging | Medium | — |
canonical — Canonical and hreflang discipline | Recommended | Medium | — |
llms-full — /llms-full.txt expanded context | Recommended | Low | — |
agents-md — AGENTS.md with the required sections | Recommended | High | — |