AgenticWebScore
Scan

Agent readiness tools

HTTP cache checker

Enter a domain to test the cache headers of its homepage and a conditional request. The report opens on those, and on what a missing page answers.

Free. No account and no email address. A site that blocks every automated client is reported as not measured, not as a low score.

The HTTP cache checker runs the full scan of 60 checks, not one request. The five exemplar scans of 20 October 2026 sent 29 to 89 requests each and took 3 to 8 seconds. A scan that reaches its 60 s deadline is reported as not measured.

A stored result: vercel.com, measured on 20 October 2026. Its report opens on the same panel yours will.

What is checked

The report opens on 3 checks of rubric v0.9. Each is described here in the rubric's own words.

HTTP caching of the homepage

Cache-Control says how long the homepage may be reused. A validator lets a crawler ask whether it changed, and a 304 answer skips the body.

http-caching is a scored check. Web standards: 70% of the score. A check in it can hold a level.

The scanner reads the Cache-Control, ETag and Last-Modified headers of the homepage response ($home in the probe map). When the homepage sends an ETag or a Last-Modified date, the scanner requests the URL the homepage answered at once more, with If-None-Match set to the ETag and If-Modified-Since set to the date, each when it was sent, and reads at most 16 KB of the answer. A validator longer than 1,024 characters, or with a line break, is not sent. Pass = Cache-Control and a validator, and the conditional request answered 304 Not Modified. Partial = a validator, and the conditional request answered a status other than 304 that is not listed under Not measured; a validator and a 304 with no Cache-Control; or Cache-Control with no validator, and then no conditional request is made. Fail = none of the three headers. Cache-Control: private and no-store are policies like any other, and these rules apply to them: no-store with no validator is partial, because a crawler then requests the whole page each time. Not measured = a homepage that answered a status other than 2xx; a validator the scan did not send; or a conditional request that was not made, got no answer, or answered 401, 403, 407, 429 or a 5xx status, or 406, which refuses the scan's Accept header.

Run the same measurement against your own site. Set the origin on the first line:

HOMEPAGE=https://example.com/
HEADERS=$(curl -sSL -o /dev/null -D - -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$HOMEPAGE" | tr -d '\r')
printf '%s\n' "$HEADERS" | grep -iE '^(cache-control|etag|last-modified):'
ETAG=$(printf '%s\n' "$HEADERS" | sed -n 's/^[Ee][Tt][Aa][Gg]: *//p' | tail -1)
DATE=$(printf '%s\n' "$HEADERS" | sed -n 's/^[Ll][Aa][Ss][Tt]-[Mm][Oo][Dd][Ii][Ff][Ii][Ee][Dd]: *//p' | tail -1)
curl -sSL -o /dev/null -w 'status=%{http_code}\n' -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' -H "If-None-Match: $ETAG" -H "If-Modified-Since: $DATE" "$HOMEPAGE"

Read the http-caching check page for its sources and the change to make.

A real 404 for a missing page

A URL that does not exist answers 404 or 410 (RFC 9110). A 200 there makes every mistyped link look like a page.

not-found-status is a scored check. Web standards: 70% of the score. A check in it can hold a level.

The scanner requests one path that does not exist on the measured origin: /aws-probe- followed by 12 random hexadecimal characters, new for every scan, with the Accept header of a page. It follows redirects on the same site only, and the request is kept out of the paths found, so a soft 404 never counts as a page the site serves. Pass = 404 or 410, after any redirect. Partial = a redirect to the homepage that answers 2xx, which Google reads as a soft 404; a redirect to another site, which is not followed; or another 4xx status. Fail = a 2xx answer at the path, or after a redirect to a page other than the homepage: a soft 404. Not measured = 401, 403, 407, 429 or a 5xx status, 406, which refuses the scan's Accept header, or no answer.

Run the same measurement against your own site. Set the origin on the first line:

ORIGIN=https://example.com
MISSING="/aws-probe-$(head -c 6 /dev/urandom | od -An -tx1 | tr -d ' \n')"
curl -sS -o /dev/null -w 'status=%{http_code} location=%{redirect_url}\n' -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$ORIGIN$MISSING"

Read the not-found-status check page for its sources and the change to make.

Structured errors for JSON requests

An agent that asks for JSON and gets an HTML error page has to read a page meant for people. RFC 9457 defines a JSON error format.

structured-errors is a scored check. Agent conventions: 30% of the score. A convention never holds a level.

The scanner requests the path of not-found-status a second time, with Accept: application/json, application/problem+json;q=0.9, */*;q=0.1, and reads the status, the Content-Type and the body of the answer. Present = an error status (4xx or 5xx) with a JSON type, application/problem+json (RFC 9457), application/json or another +json type, and a body that parses as a JSON object or is cut at the read limit. Partial = an error status with a JSON body served with another type or with none, or a JSON array served as JSON. Served with an error = an error status with a JSON type whose body does not parse as a JSON object. Not used = a status below 400, such as a 2xx, whatever the body, which tells a JSON client the path exists; or an HTML page, another type, or no body. Not measured = 406, which refuses the scan's Accept header; 429; 401, 403, 407 or a 5xx status with a body that is not JSON; a redirect to another site, which is not followed; or no answer. Present earns the check's points in the agent-conventions part, and Partial half of them.

Run the same measurement against your own site. Set the origin on the first line:

ORIGIN=https://example.com
MISSING="/aws-probe-$(head -c 6 /dev/urandom | od -An -tx1 | tr -d ' \n')"
curl -sS -D - -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: application/json, application/problem+json;q=0.9, */*;q=0.1' "$ORIGIN$MISSING" | head -20

Read the structured-errors check page for its sources and the change to make.

Questions about the HTTP cache checker

Who reads these headers?

Googlebot sends If-None-Match and If-Modified-Since. Every HTTP cache reads Cache-Control. Google reports a missing page that answers 200 as a soft 404. Every HTTP client reads the status first.

What does the scan request for a missing page?

The scanner requests one path that does not exist on the measured origin: /aws-probe- followed by 12 random hexadecimal characters, new for every scan, with the Accept header of a page. The scanner requests the path of not-found-status a second time, with Accept: application/json, application/problem+json;q=0.9, */*;q=0.1, and reads the status, the Content-Type and the body of the answer.

How do these checks count in the score?

http-caching and not-found-status are in the web-standards part: 70% of the score, and the one part whose checks hold a level. structured-errors is in the agent-conventions part: 30% of the score, and none of its checks holds a level. The methodology says what each state earns.

Why does it run the full scan?

The form starts the same scan as /scan, so the result is a scan record with a fingerprint you can verify. The report opens on http-caching, not-found-status, and structured-errors, and the full report follows. A stored record younger than 6 hours opens instead of a new scan.

Read next

Related checks in the rubric:

The methodology is the scoring contract for all 60 checks, including the rule for a site that is not measured.