# Signed agents: how Web Bot Auth works and what it does not prove > Web Bot Auth verifies which key signed an HTTP request. It proves who sent the traffic — not what that agent is allowed to do on your site. - Source: /learn/web-bot-auth-explained - Updated: 2026-10-08 - Reading time: 7 minutes - Tags: Web Bot Auth, agent identity, HTTP signatures --- A signed request is not a permission slip. Web Bot Auth gives a site a way to check that an HTTP request was signed by the holder of a particular key — a genuinely useful fact, and a much smaller one than most deployments assume. ## The foundation is a general-purpose RFC, not an agent protocol [HTTP Message Signatures, RFC 9421](https://www.rfc-editor.org/rfc/rfc9421.html), is a published Standards Track RFC that defines how to sign a chosen subset of an HTTP message and how to carry the result in two fields: `Signature-Input` and `Signature`. It deliberately stops short of being a complete security story. Section 1.4 requires anyone applying it to define a profile: which components must be covered, which algorithms are acceptable, how key material is located, and what a verifier does when something is missing. Web Bot Auth is such a profile, currently being standardised by the [IETF's Web Bot Auth working group](https://datatracker.ietf.org/wg/webbotauth/about/) in the Web and Internet Transport area. The mechanisms are RFC 9421's; the policy is the profile's. That split matters when you read vendor documentation. When a CDN says "Web Bot Auth supported", it is describing its own profile — its required components, its tolerated parameters, its key discovery path. Two verifiers can both claim support and disagree about whether a given request is valid. ## A key pair, and a directory that publishes it The agent holds an Ed25519 private key. Its public half is published as a JWK — `kty: "OKP"`, `crv: "Ed25519"`, a base64url-encoded `x` — inside a JWK Set at `/.well-known/http-message-signatures-directory`. The path is fixed, so a verifier that has never seen the agent before knows where to look; the domain in the URL is the trust anchor. Whoever controls that HTTPS origin controls the keys, so the signature moves the question from "is this string honest?" to "is this the domain I think it is?". The directory response is itself signed. It carries `Signature-Input` and `Signature` under a separate `tag`, with the content type `application/http-message-signatures-directory+json`. A key directory is therefore not merely a data file: the document and the keys inside it are bound together cryptographically. ## What a signed request looks like Three headers. `Signature-Input` declares the covered components and the parameters; `Signature` carries the signature bytes; `Signature-Agent` points at the key directory. Abridged from Cloudflare's documentation, with long values shortened: ```http Signature-Agent: "https://signature-agent.test" Signature-Input: sig2=("@authority" "signature-agent") ;created=1735689600 ;keyid="poqkLGiymh_W0uP6PZFw-dvez3QJT5SolqXBCW38r0U" ;alg="ed25519" ;expires=1735693200 ;nonce="e8N7S2MF…" ;tag="web-bot-auth" Signature: sig2=:jdq0SqOwHdyHr9+r5jw3iYZH6aNGKijYp/EstF4RQTQdi5N5YYKrD+mCT…: ``` `Signature-Input` is a structured-field dictionary: each entry names a signature label, lists the components it covers in order, then the parameters. The verifier rebuilds the signature base from exactly that list, so the list is not a hint — it is the definition of what the signature protects. `@authority` is a derived component covering the target host; `signature-agent` is the header itself, and it must be in the list. A verifier that accepted a signature whose covered components excluded the host would be verifying a signature over almost nothing. ## Three details that do the security work **`keyid` is a thumbprint, not a chosen name.** It is the base64url-encoded SHA-256 thumbprint of the JWK, computed per [RFC 7638](https://www.rfc-editor.org/rfc/rfc7638.html): only the required members, serialised as canonical JSON with no whitespace, ordered lexicographically. For an OKP key those members are `crv`, `kty` and `x`, as [RFC 8037's worked example](https://www.rfc-editor.org/rfc/rfc8037.html) spells out. Nobody picks a memorable identifier, and nobody can relabel someone else's key. Rotation is legible too: a new key is a new `keyid`, which is exactly what a verifier needs to notice one. **`tag` binds a signature to its purpose.** RFC 9421 gives `tag` its own security consideration about collisions between applications. Web Bot Auth uses one tag for signed requests and a different one for key directories, so a signature lifted from one context cannot be presented in another that demands a different tag. This is a check a verifier has to perform; if it does not, the binding is decorative. **A short expiry limits replay.** `created` and `expires` are signature parameters, so they sit inside the signed material: an attacker cannot extend a captured signature's lifetime without invalidating it. Cloudflare recommends short windows — a minute is often sufficient — and documents something worth repeating. It recommends the `nonce` parameter but does not currently validate it and keeps no database of seen nonces, so expiry is the replay control in that implementation. The specification provides the nonce; the deployment decides whether it means anything. That is a general lesson here: read the verifier's documentation, not the specification's capabilities. ## Identity is not authorisation What a successful verification proves is narrow and precise: at the moment of the check, the holder of the private key corresponding to the published public key produced a signature over the components the verifier required, within the accepted time window. That is all. "This request was signed by that key" is a statement about a key and a message. It does not prove that the operator is well behaved, or that the action the agent is about to take is permitted. The working group's own charter puts two relevant items explicitly out of scope: authenticating the end user of a participating client or agent, and defining a vocabulary for agent intents. The standard will not answer "what may this agent do here?" — not because the draft is young, but because that question is a policy decision belonging to the site. So keep the code paths separate. A valid signature is an input to an authorisation decision, never a substitute for one. If a request that verifies falls straight through to "allow", you have built an expensive user-agent allowlist with extra cryptography. Authorisation needs its own artifact: a declared default, per-action allow and deny rules, and an explicit human-approval path for actions that carry consequences. That is what [the agent permission model check](/checks/agent-permissions) measures, and it is deliberately a different document from the key directory — a key directory answers "who"; a permission model answers "what". ## User-agent strings were never an identity mechanism A `User-Agent` header is a client-supplied string. Nothing in HTTP authenticates it; it is as forgeable as any other request field, and the only cost of forging it is typing. Cloudflare's IP-validation documentation rejects user-agent patterns that match generic client libraries — `python-requests`, `Go-http-client`, `node-fetch` and similar — precisely because such a string says nothing about who is sending the traffic. The working group charter names user-agent strings alongside IP allowlists and shared API keys as "current solutions" with significant limitations in security, scalability and manageability. None of that makes user-agent handling worthless. It is a declaration tool: [named AI crawler rules in robots.txt](/checks/ai-rules) are how a site states its preferences, and [an agent identity page](/checks/bot-page) is how a site explains whom it welcomes and how a claimed identity is checked. But a block-or-allow rule keyed on a self-declared string is a speed bump, not a verification, and it should never be described to a security reviewer as authentication. ## Maturity: a draft, deployed, and inconsistent Web Bot Auth is not an RFC. The profile and the key-directory mechanism are Internet-Drafts in an active working group, so implementations vary and will keep varying until the documents settle. Cloudflare runs the mechanism in production as a verification method for its Verified bots programme, and its documentation is reassuringly specific about the seams: only Ed25519 keys are accepted; a `Signature-Agent` value must be an `https://` URL, must be enclosed in double quotes, must be covered by the signature, and the dictionary form used in later drafts is rejected; `@query-params` and `@status` are unsupported; the `sf`, `bs`, `key`, `req` and `name` component parameters cause verification to fail. The practical consequence: "does Web Bot Auth work?" is not a well-formed question. The well-formed version is "does this verifier accept this signed request, and what does it do with the result?" Test against the verifier you depend on, keep profile assumptions in configuration rather than compiled in, and treat a published key directory as a separately observable fact — which is what [our Web Bot Auth check](/checks/webbotauth) measures: the endpoint exists, parses as a JWK Set, and contains a usable Ed25519 verification key with a resolvable `kid`. It does not measure whether any intermediary validates signatures correctly, or whether a site's verifier enforces coverage. Those are outside what a scanner can see, and [the methodology](/methodology) says so rather than implying otherwise. ## Why verify instead of blocking by IP range IP allowlisting works until infrastructure moves. A published IP list has to be re-fetched, re-reviewed and re-trusted, and it assumes the operator's egress addresses are stable and belong only to them — an assumption Cloudflare's [IP validation method](https://developers.cloudflare.com/bots/reference/bot-verification/ip-validation/) states outright. Reverse DNS has the same shape. Both approaches describe where traffic comes from. A signed request describes who constructed it, and that survives a migration between clouds, a new egress region or a peering change. The allow decision can also be scoped to the request rather than the network path, because the signature covers components such as `@authority`: a signature captured for one host does not verify against another. The cost is real: you are now running credential verification — structured-field parsing, canonicalisation, key caching and rotation, clock skew, replay windows — and the failure modes are quiet, because a verifier that is too strict drops legitimate traffic and one that is too lax accepts forgeries. None of it is free. ## Transitive trust is the unsolved part An agent reaching your site is often not operated by the company that built it: a platform runs automation on behalf of many end users, and the party holding the key is the platform. Cloudflare names this chain — site owner, agent operator, end user — "transitive trust", and [labels a verified agent](https://developers.cloudflare.com/bots/concepts/bot/verified-bots/) as Direct (one narrow operator, its own infrastructure) or Intermediary (a service many end users can drive). It is experimenting with carrying operator information and a content-use commitment through intermediaries using the `Forwarded` header, a mechanism its documentation calls experimental and subject to change. Read that before writing policy. A signature identifies the operator. It tells you nothing about the person on whose behalf the request was made, and the working group's charter explicitly excludes end-user authentication from its scope. If you allowlist an intermediary, you have allowlisted every end user it will ever front. That is an authorisation decision wearing an identity decision's clothes, and it is the most common way a sound mechanism gets used badly. A signed request is evidence about a key, produced just now, over a message you can reconstruct. Treat it as exactly that: one input to a policy you still have to write.