rubric v0.9 · scanned
github.com
AI crawler checker
Your AI crawler result
The AI crawler checker puts these checks first. The full report below lists every check, these included.
-
Present Adopted AI crawler policy in robots.txt — 6 of 18 AI crawler tokens have their own robots.txt group: Bytespider disallowed, GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot and anthropic-ai partly disallowed. The other 12 follow the * group (165 of the file's 501 rules).
Observation
Bytespider: own group, 1 rule, disallowed · GPTBot: own group, 165 rules, partly disallowed · OAI-SearchBot: own group, 165 rules, partly disallowed · ClaudeBot: own group, 165 rules, partly disallowed · PerplexityBot: own group, 165 rules, partly disallowed · anthropic-ai: own group, 165 rules, partly disallowed · * group: 165 rules, partly disallowed, followed by 12 AI crawler tokens · robots.txt: 6 groups, 501 rules
A crawler follows the robots.txt group that names it, otherwise the * group. A named group states the site's rules for that crawler.
Sources: RFC 9309, 2.2.1 (IETF), Google common crawlers (Google Search Central), OpenAI bots (OpenAI)
- Read today by
- AI crawlers that document robots.txt tokens.
- Google Search
- None
-
Passed Adopted robots.txt crawl policy — /robots.txt has 6 groups and 501 rules across all groups, and 1 Sitemap line.
Observation
HTTP 200 · text/plain · 14.2 KB · # If you would like to crawl GitHub contact us via https://support.github.com?tags=dotcom-robots # We also provide an extensive API: https:…
Crawlers read robots.txt before they fetch a page (RFC 9309). Its Sitemap line tells them where the URL list is.
Sources: RFC 9309 (IETF, 2022-09), Google robots.txt (Google Search Central)
- Read today by
- Google, Bing, and the AI crawlers that document robots.txt support.
- Google Search
- Direct
-
Not used worth 3.7 points Draft Content-Signal preferences — No Content-Signal line in robots.txt or the homepage headers. Under the policy, no signal states no preference.
Observation
HTTP 200 · text/plain · 14.2 KB
Content-Signal states search, ai-input and ai-train preferences in robots.txt. No crawler documents honouring it.
Sources: Cloudflare: Content Signals Policy (Cloudflare, 2025-09), IETF AIPREF (IETF)
- Read today by
- Cloudflare's scanner scores it. The IETF AIPREF drafts are the standards track.
- Google Search
- None
Fix
Content-Signal: search=<choose: yes or no>, ai-input=<choose: yes or no>, ai-train=<choose: yes or no>Note
Add this line to the site's own User-agent: * group in /robots.txt, below its User-agent lines. Each value is the site owner's choice.
Verify with
ORIGIN=https://github.com curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/plain, */*;q=0.5' "$ORIGIN/robots.txt" | grep -i -B3 '^content-signal:' curl -sSL -o /dev/null -D - -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$ORIGIN/" | grep -i '^content-signal:' -
Passed Adopted Readable homepage — HTTP 200 text/html, 559 KB, 1091 words of visible text.
Observation
HTTP 200 · text/html · 559 KB
The homepage is the first URL every agent requests. An error or an empty page gives it nothing to read.
Sources: RFC 9110 (IETF), Google: 301 redirects (Google Search Central), Bing Webmaster Guidelines, §7: 301 and 302 redirects (Microsoft Bing)
- Read today by
- Every crawler and agent.
- Google Search
- Direct
Verdict
L2 · Crawlable
Web standards 79 · Agent conventions 25
Usable by agents: No action surface found
Held at L2: L3 Machine-readable needs jsonld to pass or be partial. Read the level rules
Readable: HTTP 200 at https://github.com/, 1,091 words in the HTML. 5 to fix, 3 optional choices, 7 agent conventions not used, 1 agent convention to correct, and 27 out of scope.
How this was measured
Measured once, from 1 vantage point, at https://github.com. The scanner reads the homepage and at most 129 URLs, runs no JavaScript, and calls no API or MCP tool it finds.
Not a ranking prediction, not AI visibility, not a certification.
We sell no consulting. No ads, no sponsored listings, and we never contact the sites that are scanned. Scanning is free today.
Defects 5
Web-standards checks that fail or are partial, and optional files served with an error. Each lists what fixing it is worth.
-
Fails worth 9.1 points Adopted JSON-LD structured data — No structured data on the homepage: 0 JSON-LD blocks, 0 Microdata items, and 0 RDFa items.
Observation
JSON-LD: 0 blocks, 0 parsed · schema.org @context: no · Microdata: 0 items · RDFa: 0 items · top-level types: none
Structured data names the entity a page describes. Google reads JSON-LD, Microdata and RDFa alike.
Sources: Google: intro to structured data (Google Search Central), schema.org (Schema.org), Bing Webmaster Guidelines, §14: structured data (Microsoft Bing)
- Read today by
- Google rich results; any schema.org reader.
- Google Search
- Direct
Fix
<script type="application/ld+json"> { "@context": "https://schema.org", "@type": "Organization", "name": "GitHub · Change is constant. GitHub keeps you ahead. · GitHub · GitHub", "url": "https://github.com/" } </script>Note
Fill
namewith the name the organization uses, and keep each block valid JSON: one syntax error, such as a trailing comma, hides the whole block.Verify with
HOMEPAGE=https://github.com/ curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' -o /tmp/aws-home.html "$HOMEPAGE"; perl -0777 -ne 'my $j = () = m{<script[^>]*application/ld\+json}gi; my $m = () = m{\bitemtype=["\x27]?https?://schema\.org/}gi; my $r = () = m{\btypeof=}gi; print "JSON-LD blocks: $j, Microdata itemtypes: $m, RDFa typeof: $r\n"' /tmp/aws-home.html; perl -0777 -ne 'while (m{<script[^>]*application/ld\+json[^>]*>(.*?)</script>}gsi) { print "$1\n" }' /tmp/aws-home.html | jq -c '[(.. | objects | select(has("@type")) | {type: .["@type"], name: (.name // .headline // null)})][:10]' -
Partial worth 3 points Adopted Title and description — Title "GitHub · Change is constant. GitHub keeps you ahead. · GitHub · GitHub" (70 characters) and a description of 186 characters. The first H1 "The future of building happens together" shares 0 of 4 words with the title, brand names aside. Microsoft's guide to AI search answers (October 2025) asks for an H1 that matches the title or closely reflects it.
Observation
title (70 characters): GitHub · Change is constant. GitHub keeps you ahead. · GitHub · GitHub · description (186 characters): Join the world's most widely adopted, AI-powered developer platform where millions of developers, businesses, and the largest open source community build software that advances humanity. · first H1: The future of building happens together
The title and the description say what a page is about. Google uses them for the title link and the snippet.
Sources: Google: title links (Google Search Central), Google: snippets (Google Search Central), Bing Webmaster Guidelines, §13: titles, descriptions and headings (Microsoft Bing), Microsoft: optimizing content for AI search answers (Microsoft Advertising, 2025-10)
- Read today by
- Google, Bing, and link previews.
- Google Search
- Direct
Note
Give the first H1 and the title the same subject. Either can change, and brand names do not count.
Verify with
HOMEPAGE=https://github.com/ curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$HOMEPAGE" | perl -0777 -CS -ne 'if (m{<title[^>]*>(.*?)</title>}si) { my $t=$1; $t=~s/\s+/ /g; $t=~s/^ | $//g; print "title (", length($t), "): $t\n" } else { print "title: none\n" } if (m{<meta\b[^>]*\bname=["\x27]?description["\x27]?[^>]*>}i) { my ($c)=$& =~ /\bcontent=["\x27]([^"\x27]*)/i; $c //= ""; print "description (", length($c), "): $c\n" } else { print "description: none\n" }' -
Fails worth 2 points Adopted API catalog — No API catalog linkset was served: /.well-known/api-catalog answered HTTP 404.
An API catalog lists a publisher's APIs in one document at a fixed address (RFC 9727).
Sources: RFC 9727 (IETF, 2025-06)
- Read today by
- API clients that follow RFC 9727.
- Google Search
- None
Fix
{ "linkset": [ { "anchor": "<choose: URL that identifies the API>", "service-desc": [ { "href": "<choose: URL of the API description>" } ] } ] }Note
Serve it as Content-Type application/linkset+json with profile="https://www.rfc-editor.org/info/rfc9727" and keep the anchors and hrefs absolute URLs that resolve. A target only counts once it answers: point service-desc at a document the scanner already requests, such as /openapi.json, and make it return 200, because an href that is never fetched leaves the catalog partial.
Verify with
ORIGIN=https://github.com curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: application/linkset+json, application/json;q=0.9, */*;q=0.5' "$ORIGIN/.well-known/api-catalog" | jq '.linkset[] | {anchor, "service-desc": .["service-desc"], status: .status}' -
Partial worth 0.5 points Adopted Service status and changelog feed — A status page answered at /status (HTTP 200), but the homepage links no status or changelog feed.
A status page and a changelog feed say whether the service is up and what changed.
Sources: RFC 9727 (status relation) (IETF), RFC 4287 (IETF)
- Read today by
- Integrators and monitoring tools.
- Google Search
- None
Fix
<link rel="alternate" type="application/atom+xml" title="Changelog" href="https://github.com<choose: path of the changelog feed>">Note
A link only counts if it resolves: a feed that 404s is served with an error, and one whose newest entry is months old is stale. The scan reads a status page on the scanned origin, such as /status; a status page on another host is not requested, so it is reported as not measured.
Verify with
ORIGIN=https://github.com curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$ORIGIN/" | grep -oiE '<link[^>]+(atom|rss)[^>]*>|href="[^"]*(status|changelog)[^"]*"' | sort -u for p in /status /changelog; do printf '%s -> ' "$p"; curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' -o /dev/null -w '%{http_code} %{content_type}\n' "$ORIGIN$p"; done -
optional file, served with an error 0 points Experimental ARD resource discovery manifest — ARD manifest at /.well-known/ai-catalog.json lists 1 entry of which 0 are complete and 0 carry both url and data, which the specification forbids.
An ARD manifest lists a site's agent resources. Lighthouse does not count a missing one against a site.
Sources: Lighthouse agentic browsing (Chrome for Developers)
- Read today by
- Lighthouse (ard-schema audit).
- Google Search
- None
Note
Give each entry an identifier, a displayName, a type, and exactly one of url or data.
Verify with
ORIGIN=https://github.com curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$ORIGIN/" | grep -oiE '<link[^>]+rel="?ard"?[^>]*>' for p in /.well-known/ard.json /.well-known/agent-resources.json /.well-known/ai-catalog.json /ard.json; do printf '%s -> ' "$p"; curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: application/json, application/linkset+json;q=0.9, */*;q=0.5' "$ORIGIN$p" | jq -c '[.entries[]? | {identifier,displayName,type,url}]' 2>/dev/null || echo "missing or not JSON"; done
Optional choices 4
Show 4 Hide 4 in Optional choices
Decisions only you can make. None earns points or holds a level.
-
Not used 0 points Adopted Snippet and AI grounding directives — Not used: the homepage has no noarchive, nocache, nosnippet or max-snippet:0 directive, and no data-nosnippet attribute, so Bing applies none of the limits they set on snippets and Copilot answers.
Observation
no robots meta tag for robots, bingbot or googlebot, no X-Robots-Tag header · data-nosnippet: 0 elements
noarchive, nocache, nosnippet and data-nosnippet limit how Bing shows a page and how Copilot uses it. Each is the site owner's choice.
Sources: Bing Webmaster Guidelines, §10: NOARCHIVE, NOCACHE and NOSNIPPET (Microsoft Bing), Bing: robots meta tags and attributes that Bing supports (Microsoft Bing), Google: robots meta tag specifications (Google Search Central)
- Read today by
- Bing reads all five, and Copilot follows noarchive and nocache. Google reads nosnippet, max-snippet and data-nosnippet.
- Google Search
- Direct
Note
Each directive is your choice. noarchive keeps the page out of Copilot answers, nocache lets Copilot use only the URL, the title and the snippet, and nosnippet or data-nosnippet may limit the quality of what Copilot cites.
Verify with
HOMEPAGE=https://github.com/ curl -sSL -o /dev/null -D - -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$HOMEPAGE" | tr -d '\r' | grep -i '^x-robots-tag:' curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$HOMEPAGE" | perl -0777 -ne 'my $h = /^(.*?)<\/head>/si ? $1 : substr($_, 0, 8192); while ($h =~ m{<meta\b[^>]*>}gi) { my $t = $&; next unless $t =~ /\bname=["\x27]?(robots|bingbot|googlebot)["\x27]?[\s>\/]/i; my $n = lc $1; my ($c) = $t =~ /\bcontent=["\x27]([^"\x27]*)/i; print "meta $n: ", ($c // ""), "\n" } my $body = $_; $body =~ s{<!--.*?-->}{}gs; $body =~ s{<(script|style|template|title|textarea|noscript)\b[^>]*>.*?</\1\s*>}{}gsi; my $n = 0; while ($body =~ m{<[a-z][a-z0-9-]*\b((?:[^>"\x27]|"[^"]*"|\x27[^\x27]*\x27)*)>}gi) { (my $t = $1) =~ s{"[^"]*"|\x27[^\x27]*\x27}{}g; $n++ if $t =~ /(?:^|\s)data-nosnippet(?=[\s=\/]|$)/i } print "data-nosnippet: $n\n"' -
Not used 0 points Experimental AI usage policy page — Not published: /ai-policy, /ai-usage-policy, /legal/ai-policy and /yapay-zeka-politikasi answered HTTP 404; /ai answered HTTP 200 with a page that is not named an AI policy. No homepage link names an AI policy. Checked because robots.txt disallows / for Bytespider.
Observation
/ai-policy: HTTP 404 · text/html · first 16 KB read; /ai-usage-policy: HTTP 404 · text/html · first 16 KB read; /ai: HTTP 200 · text/html · 249 KB read; /legal/ai-policy: HTTP 404 · text/html · first 16 KB read; /yapay-zeka-politikasi: HTTP 404 · text/html · first 16 KB read
A page that explains in prose the AI restrictions the site states. No standard defines it.
Sources: RFC 9309 (IETF)
- Read today by
- People reading the site's terms.
- Google Search
- None
Note
Publish a page that explains the AI restriction robots.txt or Content-Signal states, and link it from the homepage with the words AI policy. No standard defines the page or its content.
Verify with
ORIGIN=https://github.com curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$ORIGIN/" | grep -Eio '<a[^>]+>[^<]*' | grep -Ei 'ai[-_ ]?(usage[-_ ]?)?polic|yapay[-_ ]?zeka' | head -3 for P in /ai-policy /ai-usage-policy /ai /legal/ai-policy /yapay-zeka-politikasi; do curl -sSL -o /dev/null -w "$P HTTP %{http_code} %{content_type}\n" -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$ORIGIN$P"; done -
Not used 0 points Adopted Breadcrumb markup on an inner page — The inner page /features/ai/github-app has no BreadcrumbList in its JSON-LD, Microdata, or RDFa.
Observation
https://github.com/features/ai/github-app · HTTP 200 · text/html · BreadcrumbList: 0
BreadcrumbList states where an inner page sits in the site. A homepage has no trail.
Sources: Google: breadcrumb (Google Search Central), Google blog (Google Search Central, 2025-01)
- Read today by
- Google shows breadcrumbs in desktop results.
- Google Search
- Direct
Fix
<script type="application/ld+json"> { "@context": "https://schema.org", "@type": "BreadcrumbList", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "GitHub · Change is constant. GitHub keeps you ahead. · GitHub · GitHub", "item": "https://github.com/" }, { "@type": "ListItem", "position": 2, "name": "<choose: name of this section>", "item": "https://github.com/features" }, { "@type": "ListItem", "position": 3, "name": "<choose: name of this section>", "item": "https://github.com/features/ai" }, { "@type": "ListItem", "position": 4, "name": "GitHub Copilot app · GitHub", "item": "https://github.com/features/ai/github-app" } ] } </script>Note
Replace each <choose: …> with the name or the URL the site uses for that section, and drop a level that has no page. Show the same trail as links on the page.
Verify with
PAGE=https://github.com/features/ai/github-app curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' -o /tmp/aws-page.html "$PAGE"; perl -0777 -ne 'while (m{<script[^>]*application/ld\+json[^>]*>(.*?)</script>}gsi) { print "$1\n" }' /tmp/aws-page.html | jq -r '.. | objects | select((.["@type"]? | if type == "array" then .[] else . end) == "BreadcrumbList") | .itemListElement[] | [(.position | tostring), (.name // .item.name? // "NO NAME"), ((.item["@id"]? // .item) // "NO URL" | tostring)] | @tsv'; grep -c 'schema.org/BreadcrumbList' /tmp/aws-page.html -
Present 0 points Adopted security.txt (RFC 9116) — security.txt at /.well-known/security.txt declares 1 Contact and an Expires 2026-11-19T17:05:55z in the future. Not scored: its Policy URL is on bounty.github.com, which this scan does not request.
Observation
Contact: https://hackerone.com/github Acknowledgments: https://hackerone.com/github/hacktivity Preferred-Languages: en Canonical: https://github.com/.well-known/security.txt Policy: https://bounty.github.com Hiring: https://github.careers Expires: 2026-11-19T…
security.txt tells a researcher where to report a vulnerability (RFC 9116). It has no effect on search.
Sources: RFC 9116 (IETF, 2022-04)
- Read today by
- Security researchers and disclosure tools.
- Google Search
- None
Agent conventions 10
Show 10 Hide 10 in Agent conventions
The scored ones make up 30% of the score, and the unscored ones earn no points. None has an effect on Google Search today. Each scored row not in use lists what using it is worth.
-
Not used worth 3.7 points Draft Content-Signal preferences — No Content-Signal line in robots.txt or the homepage headers. Under the policy, no signal states no preference.
Observation
HTTP 200 · text/plain · 14.2 KB
Content-Signal states search, ai-input and ai-train preferences in robots.txt. No crawler documents honouring it.
Sources: Cloudflare: Content Signals Policy (Cloudflare, 2025-09), IETF AIPREF (IETF)
- Read today by
- Cloudflare's scanner scores it. The IETF AIPREF drafts are the standards track.
- Google Search
- None
Fix
Content-Signal: search=<choose: yes or no>, ai-input=<choose: yes or no>, ai-train=<choose: yes or no>Note
Add this line to the site's own User-agent: * group in /robots.txt, below its User-agent lines. Each value is the site owner's choice.
Verify with
ORIGIN=https://github.com curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/plain, */*;q=0.5' "$ORIGIN/robots.txt" | grep -i -B3 '^content-signal:' curl -sSL -o /dev/null -D - -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$ORIGIN/" | grep -i '^content-signal:' -
Not used worth 3.7 points Adopted Discovery via Link response headers — Not served: the homepage response has no Link header naming /llms.txt.
Observation
HTTP 200 · text/html · 559 KB
A Link header (RFC 8288) points a client at a resource before it parses the HTML.
Sources: RFC 8288 (IETF), IANA link relations (IANA)
- Read today by
- Clients that follow RFC 8288 and RFC 9727 links.
- Google Search
- None
Fix
Link: <https://github.com/llms.txt>; rel="describedby"Note
Name only documents the site serves. Several links go in one comma-separated Link header, and absolute URLs need no base to resolve (RFC 8288).
Verify with
HOMEPAGE=https://github.com/ curl -sSL -o /dev/null -D - -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$HOMEPAGE" | grep -i '^link:' -
Served with an error worth 3.7 points Draft /llms-full.txt expanded context — /llms-full.txt is a list of links, not page text: 117 link lines of 142 lines, 2778 words in 28 KB.
Observation
HTTP 200 · text/plain · 28 KB · # GitHub > GitHub is a developer platform for building, shipping, and maintaining software. It provides cloud-based Git…
A convention that puts a site's text in one file. The llms.txt proposal does not define it.
Sources: llmstxt.org (llmstxt.org), Mintlify (Mintlify)
- Read today by
- Some documentation tools.
- Google Search
- None
Note
Tools that write /llms.txt can also write /llms-full.txt from the same pages: the text of each page, under its URL. Link it from /llms.txt.
Verify with
ORIGIN=https://github.com curl -sSL -o /dev/null -w 'status=%{http_code} type=%{content_type} bytes=%{size_download}\n' -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/plain, text/markdown;q=0.9, */*;q=0.5' "$ORIGIN/llms-full.txt" curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/plain, text/markdown;q=0.9, */*;q=0.5' "$ORIGIN/llms.txt" | grep -i 'llms-full' -
Not used worth 3.7 points Experimental Markdown content negotiation — The homepage answered `Accept: text/markdown, text/html;q=0.5` with HTML, which is a valid response.
Observation
HTTP 200 · text/html · Vary: X-PJAX, X-PJAX-Container, Turbo-Visit, Turbo-Frame, X-Requested-With, X-GitHub-Client-Version, Accept-Language, Sec-Fetch-Site,Accept-Encoding, Accept, X-Requested-With · markdown body: no
Some coding agents ask for text/markdown. Google Search does not use markdown.
Sources: RFC 7763 (IETF), Cloudflare: Markdown for Agents (Cloudflare), Google: AI optimization guide (Google Search Central)
- Read today by
- Claude Code, OpenCode and Cursor send Accept: text/markdown by default (Cloudflare, 2026-04).
- Google Search
- None
Fix
Answer a request whose Accept header prefers text/markdown with a markdown version of the same page, at the same URL. - On a CDN that converts HTML to markdown, turn that conversion on for the site. - On a CMS or a static build, publish one .md file per page, and serve it when Accept prefers text/markdown. Send Content-Type: text/markdown; charset=utf-8 with the markdown. Send Vary: Accept with both the markdown and the HTML, so a shared cache keeps them apart.Note
Send
Vary: Acceptwith the HTML response too, so a shared cache never serves one representation in place of the other.Verify with
HOMEPAGE=https://github.com/ curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/markdown, text/html;q=0.5' -D - -o /tmp/aws-md.body "$HOMEPAGE" | grep -iE '^(HTTP/|content-type|vary)'; head -c 300 /tmp/aws-md.body; echo -
Not used worth 3.7 points Adopted Structured errors for JSON requests — Not used: /aws-probe-0cbf9a5e6978, asked for JSON, answered HTTP 404 with an HTML page, not a JSON error.
Observation
HTTP 404 · text/html · first 16 KB read
An agent that asks for JSON and gets an HTML error page has to read a page meant for people. RFC 9457 defines a JSON error format.
Sources: RFC 9457: Problem Details for HTTP APIs (IETF, 2023-07), RFC 9110, 12.5.1: Accept (IETF)
- Read today by
- API clients and agents that send Accept: application/json. No search engine reads it.
- Google Search
- None
Fix
{ "type": "about:blank", "title": "Not Found", "status": 404, "detail": "Nothing is published at this address." }Note
Send this with Content-Type: application/problem+json and the status 404 when a request for a missing path asks for JSON. Browsers can keep the HTML page (RFC 9457).
Verify with
ORIGIN=https://github.com MISSING="/aws-probe-$(head -c 6 /dev/urandom | od -An -tx1 | tr -d ' \n')" curl -sS -D - -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: application/json, application/problem+json;q=0.9, */*;q=0.1' "$ORIGIN$MISSING" | head -20 -
Not used worth 3.7 points Experimental WebMCP browser tool declaration — Not used: the homepage has 2 forms, and none declares a WebMCP tool.
WebMCP lets a page register tools for an in-browser agent. It is a W3C Community Group draft.
Sources: WebMCP draft (W3C Web Machine Learning Community Group), Chrome: WebMCP (Chrome for Developers)
- Read today by
- Chrome, behind an origin trial. Lighthouse audits it.
- Google Search
- None
Fix
// WebMCP (W3C Community Group draft): register a tool an in-browser agent can call. if ('modelContext' in navigator) { navigator.modelContext.registerTool({ name: 'search_site', description: 'Search this site and return the matching pages.', inputSchema: { type: 'object', properties: { query: { type: 'string', description: 'The words to search for' } }, required: ['query'], }, async execute({ query }) { // Call the site's own search here and return its results as text. return { content: [{ type: 'text', text: 'Results for ' + query }] }; }, }); }Note
WebMCP is a W3C Community Group draft, not on the W3C Standards Track, and Chrome exposes it behind an origin trial. The draft defines the in-page API only, so there is no manifest file to publish.
Verify with
HOMEPAGE=https://github.com/ curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$HOMEPAGE" | grep -oE 'registerTool\(|<form[^>]*toolname[^>]*>' | sort | uniq -c -
Present Adopted AI crawler policy in robots.txt — 6 of 18 AI crawler tokens have their own robots.txt group: Bytespider disallowed, GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot and anthropic-ai partly disallowed. The other 12 follow the * group (165 of the file's 501 rules).
Observation
Bytespider: own group, 1 rule, disallowed · GPTBot: own group, 165 rules, partly disallowed · OAI-SearchBot: own group, 165 rules, partly disallowed · ClaudeBot: own group, 165 rules, partly disallowed · PerplexityBot: own group, 165 rules, partly disallowed · anthropic-ai: own group, 165 rules, partly disallowed · * group: 165 rules, partly disallowed, followed by 12 AI crawler tokens · robots.txt: 6 groups, 501 rules
A crawler follows the robots.txt group that names it, otherwise the * group. A named group states the site's rules for that crawler.
Sources: RFC 9309, 2.2.1 (IETF), Google common crawlers (Google Search Central), OpenAI bots (OpenAI)
- Read today by
- AI crawlers that document robots.txt tokens.
- Google Search
- None
-
Present Draft /llms.txt published and well-structured — /llms.txt opens with H1 "GitHub", with 12 H2 link sections and 117 links, 28 KB. The scan requested 1 link it names, and it answered 2xx. Not requested: 116 links on another origin.
Observation
HTTP 200 · text/plain · 28 KB · headings: GitHub | How to use | Building with GitHub, for coding agents and automation | GitHub Copilot | Authentication | Get st…
A 2024 proposal for a markdown index of a site. Google Search does not use it.
Sources: llmstxt.org (llmstxt.org), Google: AI optimization guide (Google Search Central), Lighthouse llms.txt (Chrome for Developers)
- Read today by
- Some documentation tools and coding agents. Lighthouse audits it.
- Google Search
- None
-
Not used 0 points Draft Agent Skills declaration — Not served: no agent skills index with a non-empty skills array was found as JSON (/.well-known/agent-skills/index.json 404 application/json; /.well-known/agent-skills.json 404 application/json; /.well-known/skills.json 404 application/json).
An agent skills index lists skill packages for agent runtimes. It is a draft on GitHub.
Sources: Agent Skills Discovery RFC (Cloudflare)
- Read today by
- Agent runtimes that read the draft.
- Google Search
- None
Fix
{ "skills": [ { "name": "<choose: skill name>", "type": "skill-md", "description": "<choose: what the skill does>", "url": "https://github.com/.well-known/agent-skills/<choose: skill name>/SKILL.md", "digest": "sha256:<choose: SHA-256 of the SKILL.md bytes, in hex>" } ] }Note
An early convention rather than a ratified specification, and it has already renamed its path once. The digest is the SHA-256 of the exact bytes served, so recompute it whenever SKILL.md changes. An artifact on another host is reported as declared, not observed, because this scan requests no other host.
Verify with
ORIGIN=https://github.com for p in /.well-known/agent-skills/index.json /.well-known/agent-skills.json /.well-known/skills.json; do printf '%s -> ' "$p"; curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: application/json, application/linkset+json;q=0.9, */*;q=0.5' "$ORIGIN$p" | jq -c '{skills:[.skills[]? | {name,type,url,digest}]}' 2>/dev/null || echo "missing or not JSON"; done -
Not used 0 points Experimental DNS-AID discovery record — Not published: 0 SVCB records and 0 TXT records resolve at _index._agents.github.com.
Observation
DNS-AID names queried for SVCB and TXT records: _index._agents.github.com
DNS-AID publishes agents in DNS. It is an individual Internet-Draft with no IETF standing.
Sources: draft-mozleywilliams-dnsop-dnsaid (IETF)
- Read today by
- Cloudflare's scanner.
- Google Search
- None
Fix
; DNS-AID organization entry point (draft-mozleywilliams-dnsop-dnsaid) _index._agents.github.com. IN SVCB 1 <choose: host name that serves the agent index, ending in a dot> alpn=<choose: ALPN of the agent endpoint>Note
DNS-AID is an individual Internet-Draft with no IETF standing. TYPE64 is the SVCB type by number, which older dig builds also answer.
Verify with
DOMAIN=github.com dig +short TYPE64 _index._agents.$DOMAIN; dig +short TXT _index._agents.$DOMAIN
Not measured 1
Show 1 Hide 1 in Not measured
The scanner could not observe these. Unscored checks earn no points, so the score and confidence are unchanged.
-
Not measured 0 points Draft MCP server card — Not measured: /.well-known/ai-catalog.json lists an MCP server card at https://api.githubcopilot.com/mcp/server-card, which this scan did not request.
An MCP server card describes a remote MCP server. It is an optional extension (SEP-2127), not core MCP.
Sources: SEP-2127 (Model Context Protocol, 2026-10), discovery.md (Model Context Protocol)
- Read today by
- MCP clients that implement SEP-2127; Cloudflare's scanner.
- Google Search
- None
Verify with
ORIGIN=https://github.com for p in /.well-known/ai-catalog.json /.well-known/mcp/server-card.json /.well-known/mcp.json /mcp.json; do printf '%s -> ' "$p"; curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: application/json, application/linkset+json;q=0.9, */*;q=0.5' "$ORIGIN$p" | jq -c '{name,version,description,remotes,entries}' 2>/dev/null || echo "missing or not JSON"; done
Out of scope for this site 27
Show 27 Hide 27 in Out of scope for this site
The scan did not ask this site for these checks. Each says why. If one should apply, the error is ours, not yours.
Sites with an API or agent surface
The scan found an MCP server. A check that reads another kind of surface is not asked.
-
Out of scope 0 points Draft A2A agent card — Not scored: this check applies to a site that runs an A2A agent, and the scan found none.
An A2A agent card describes an agent the site runs. An A2A server publishes it.
Sources: A2A specification (A2A Project)
- Read today by
- A2A clients.
- Google Search
- None
-
Out of scope 0 points Draft Cryptographically signed agent card — Not scored: this check applies to a site that runs an A2A agent, and the scan found none.
An A2A card may be signed with JWS. Signing is optional in the specification.
Sources: A2A specification (A2A Project), RFC 7515 (IETF)
- Read today by
- A2A clients that verify cards.
- Google Search
- None
-
Out of scope 0 points Draft /auth.md agent authentication guide — Not scored: this check applies to a site that uses OAuth, and the scan found none.
auth.md is one vendor's convention (WorkOS) for telling an agent how to authenticate.
Sources: WorkOS auth.md (WorkOS)
- Read today by
- Agents built on WorkOS.
- Google Search
- None
-
Out of scope Adopted MCP server answers and lists its tools — Not scored: /.well-known/ai-catalog.json lists a server card at https://api.githubcopilot.com/mcp/server-card, off the scanned origin, which this scan does not request. /mcp answered HTTP 200, which does not signal an MCP server.
An agent given the address of an MCP server can call its tools. The scan opens one session to see that the server answers and what it offers.
Sources: MCP 2025-11-25: lifecycle (Model Context Protocol, 2025-11), MCP 2025-11-25: transports (Model Context Protocol, 2025-11), MCP 2025-11-25: tools (Model Context Protocol, 2025-11), Agentic AI Foundation: MCP joins the Linux Foundation (Linux Foundation, 2025-12)
- Read today by
- MCP clients: agent apps, coding agents, and chat products that connect to MCP servers.
- Google Search
- None
-
Out of scope 0 points Draft MPP discovery in OpenAPI (x-payment-info) — Not scored: no OpenAPI document was found, so x-payment-info is not asked of this site.
An operation with x-payment-info tells an agent that the call is paid, and states the intent, the payment method, and the amount. MPP is a draft, and charging agents is a business decision.
Sources: MPP (mpp.dev), MPP: multi-method discovery (mpp.dev), draft-ryan-httpauth-payment (IETF (individual draft)), isitagentready.com: MPP skill (isitagentready.com)
- Read today by
- Agents that pay with MPP; isitagentready.com.
- Google Search
- None
-
Out of scope Adopted OAuth discovery documents — Not scored: this check applies to a site that uses OAuth, and the scan found none.
-
Out of scope Adopted OpenAPI definition — Not scored: this check applies to a site that runs an HTTP API, and the scan found none.
An OpenAPI document describes an HTTP API so a client can call it without guessing.
Sources: OpenAPI 3.1 (OpenAPI Initiative), RFC 8631 (IETF)
- Read today by
- API clients and code generators.
- Google Search
- None
-
Out of scope 0 points Experimental Rate limits and quotas documented — Not scored: this check applies to a site that runs an HTTP API, and the scan found none.
Declared limits let a client pace its calls. RateLimit headers are an IETF draft.
Sources: draft-ietf-httpapi-ratelimit-headers (IETF)
- Read today by
- API clients.
- Google Search
- None
-
Out of scope 0 points Experimental Sandbox or test environment documented — Not scored: this check applies to a site that runs an HTTP API, and the scan found none.
A documented sandbox lets a client test calls without touching production data.
Sources: OpenAPI 3.1 (servers) (OpenAPI Initiative)
- Read today by
- API integrators.
- Google Search
- None
-
Out of scope 0 points Draft Web Bot Auth key directory — Not scored: this check applies to a site that runs an A2A agent, and the scan found none.
The operator of a signing agent publishes its keys here. A site that only serves content needs none.
Sources: draft-ietf-webbotauth-httpsig-protocol (IETF, 2026-09), Cloudflare: Web Bot Auth (Cloudflare)
- Read today by
- Sites and CDNs that verify signed agents.
- Google Search
- None
Sites with a commercial signal
The homepage shows no commercial signal, so commerce checks are not asked of this site.
-
Out of scope 0 points Draft ACP discovery (/.well-known/acp.json) — Not scored: the homepage shows no commercial signal, so commerce checks are not asked of this site.
An ACP discovery document at /.well-known/acp.json tells an agent where a business's checkout API is and which services it offers. The RFC behind it is a proposal.
Sources: ACP discovery RFC (status: Proposal) (Agentic Commerce Protocol), isitagentready.com: ACP skill (isitagentready.com)
- Read today by
- Agents that implement ACP checkout; isitagentready.com.
- Google Search
- None
-
Out of scope 0 points Draft Agent purchase policy — Not scored: the homepage shows no commercial signal, so commerce checks are not asked of this site.
Whether agents may buy, and with what approval, is a policy the site states.
Sources: ACP (Agentic Commerce Protocol)
- Read today by
- Agent checkouts.
- Google Search
- None
-
Out of scope Adopted Inventory and price freshness — Not scored: the homepage shows no commercial signal, so commerce checks are not asked of this site.
availability says whether an item is in stock. A past priceValidUntil can hide a product snippet.
Sources: Google: product snippet (Google Search Central)
- Read today by
- Google product snippets.
- Google Search
- Direct
-
Out of scope 0 points Draft Agent payment protocol (UCP profile or x402) — Not scored: the homepage shows no commercial signal, so commerce checks are not asked of this site.
-
Out of scope Adopted Product/catalog schema — Not scored: the homepage shows no commercial signal, so commerce checks are not asked of this site.
Product and Offer markup on product pages feeds merchant listings.
Sources: Google: merchant listing (Google Search Central), Google: product (Google Search Central)
- Read today by
- Google merchant listings and shopping agents.
- Google Search
- Direct
-
Out of scope Adopted Return policy machine-readable — Not scored: the homepage shows no commercial signal, so commerce checks are not asked of this site.
A MerchantReturnPolicy states return terms in markup, also under the Organization.
Sources: Google: return policy (Google Search Central)
- Read today by
- Google merchant listings.
- Google Search
- Direct
-
Out of scope Adopted Seller identity declared — Not scored: the homepage shows no commercial signal, so commerce checks are not asked of this site.
On a marketplace, the seller named on an offer is who the buyer contracts with.
Sources: Google: merchant listing (Google Search Central)
- Read today by
- Google merchant listings.
- Google Search
- Direct
-
Out of scope Adopted Shipping cost and delivery declared — Not scored: the homepage shows no commercial signal, so commerce checks are not asked of this site.
shippingDetails lets a client compute the delivered price.
Sources: Google: merchant listing (Google Search Central)
- Read today by
- Google merchant listings.
- Google Search
- Direct
-
Out of scope 0 points Draft UCP manifest (/.well-known/ucp) — Not scored: the homepage shows no commercial signal, so commerce checks are not asked of this site.
A UCP profile at /.well-known/ucp declares commerce capabilities. UCP is a draft.
Sources: UCP (Universal Commerce Protocol), Shopify profiles (Shopify)
- Read today by
- UCP agents; Shopify stores serve it.
- Google Search
- None
Every site
-
Out of scope 0 points Draft Machine-readable booking action — Not scored: this check applies to services sites, and the site type could not be confirmed (best candidates: api, saas). Declare the type to have it measured.
A ReserveAction declares a booking target in markup. No search engine documents a feature that reads it.
Sources: schema.org/ReserveAction (Schema.org)
- Read today by
- None documented.
- Google Search
- None
-
Out of scope Adopted Structured contact information — Not scored: the homepage markup declares no Organization-type node.
A ContactPoint in the Organization markup makes the contact route machine-readable.
Sources: Google: Organization structured data (Google Search Central)
- Read today by
- Google lists contactPoint as a recommended Organization property.
- Google Search
- Indirect
-
Out of scope 0 points Adopted Documents in machine-readable formats — Not scored: this check applies to institution sites, and the site type could not be confirmed (best candidates: api, saas). Declare the type to have it measured.
Tagged PDF or an HTML version can be read by screen readers and by machines.
Sources: GOV.UK: HTML not PDF (Government Digital Service), ISO 14289 (PDF/UA) (ISO)
- Read today by
- Screen readers; GOV.UK publishes HTML first.
- Google Search
- None
-
Out of scope 0 points Adopted FAQ markup for visible questions — Not scored: no heading on the homepage names a FAQ section, and the homepage carries no FAQPage markup.
Observation
visible questions under a FAQ heading: 0
FAQPage marks up questions that are visible on the page. Google stopped showing FAQ rich results on 7 May 2026.
Sources: Google Search updates (Google Search Central), Google: structured data policies (Google Search Central)
- Read today by
- Bing names FAQ markup as an example of labelled content.
- Google Search
- None
-
Out of scope 0 points Draft Reuse and licensing terms declared — Not scored: this check applies to publisher or institution sites, and the site type could not be confirmed (best candidates: api, saas). Declare the type to have it measured.
TDMRep reserves text and data mining rights in a machine-readable form, as EU DSM Article 4(3) allows.
Sources: TDMRep CG report (W3C), Directive (EU) 2019/790 (European Union)
- Read today by
- Text and data mining tools that implement TDMRep.
- Google Search
- None
-
Out of scope Adopted Structured pricing information — Not scored: no Offer or PriceSpecification in the homepage markup states a price, and no pricing page states a complete one.
A price in markup is read without parsing prose. An Offer with a price and no currency is ambiguous.
Sources: schema.org/Offer (Schema.org), Google: product snippet (Google Search Central)
- Read today by
- Clients that read schema.org Offer. Google reads an Offer only inside a Product, which product-schema measures.
- Google Search
- None
-
Out of scope 0 points Adopted Content provenance declared — Not scored: this check applies to publisher or institution sites, and the site type could not be confirmed (best candidates: api, saas). Declare the type to have it measured.
Article markup with an author and dates lets a client attribute and date a source.
Sources: Google: Article structured data (Google Search Central)
- Read today by
- Google article features.
- Google Search
- Direct
-
Out of scope Adopted Service/product scope machine-readable — Not scored: the homepage markup declares no Service-type node.
Service markup states what is offered, by whom and where. An incomplete node leaves a client to guess.
Sources: schema.org/Service (Schema.org)
- Read today by
- Clients that read schema.org Service. Google has no Service rich result.
- Google Search
- None
Passed 13
Show 13 Hide 13 in Passed
Scored checks that pass, the heaviest first.
-
Passed Adopted Content that needs no JavaScript — 7396 characters of text are in the HTML before any script runs.
Observation
static text: 7396 characters · inside <noscript>: 0 characters · markup: 572359 characters
Several AI crawlers do not run JavaScript. Text that appears only after scripts run is invisible to them.
Sources: Vercel: the rise of the AI crawler (Vercel), Google: JavaScript SEO basics (Google Search Central), Bing Webmaster Guidelines, §8: rendering (Microsoft Bing)
- Read today by
- GPTBot and ClaudeBot fetch HTML without rendering; Googlebot renders.
- Google Search
- Indirect
-
Passed Adopted Readable homepage — HTTP 200 text/html, 559 KB, 1091 words of visible text.
Observation
HTTP 200 · text/html · 559 KB
The homepage is the first URL every agent requests. An error or an empty page gives it nothing to read.
Sources: RFC 9110 (IETF), Google: 301 redirects (Google Search Central), Bing Webmaster Guidelines, §7: 301 and 302 redirects (Microsoft Bing)
- Read today by
- Every crawler and agent.
- Google Search
- Direct
-
Passed Adopted Semantic heading structure — 1 H1, 9 H2s, and 16 H3s in an outline of 26 headings, with no skipped level.
Observation
outline: H2 H1 H2 H2 H2 H3 H3 H3 H3 H3 H2 H3 H2 H3 H3 H3 H3 H3 H2 H2 H2 H3 H3 H3 H3 H3
Headings outline the page for screen readers and for any client that splits text by section.
Sources: W3C WAI headings tutorial (W3C), Google SEO starter guide (Google Search Central), Bing Webmaster Guidelines, §13: titles, descriptions and headings (Microsoft Bing)
- Read today by
- Screen readers; clients that chunk HTML by heading.
- Google Search
- Indirect
-
Passed Adopted Homepage indexable: no noindex directive — No noindex: the homepage has no robots meta tag for robots, bingbot or googlebot, and no X-Robots-Tag header.
Observation
no robots meta tag for robots, bingbot or googlebot, no X-Robots-Tag header
A noindex in a meta tag or in a response header keeps the page out of search results. Bing keeps it out of Copilot too.
Sources: Bing Webmaster Guidelines, §10: NOINDEX and the other robots directives (Microsoft Bing), Bing: robots meta tags and attributes that Bing supports (Microsoft Bing), Google: block indexing with noindex (Google Search Central), Google: robots meta tag specifications (Google Search Central)
- Read today by
- Google and Bing. Bing applies the same directives to Copilot answers.
- Google Search
- Direct
-
Passed Adopted robots.txt crawl policy — /robots.txt has 6 groups and 501 rules across all groups, and 1 Sitemap line.
Observation
HTTP 200 · text/plain · 14.2 KB · # If you would like to crawl GitHub contact us via https://support.github.com?tags=dotcom-robots # We also provide an extensive API: https:…
Crawlers read robots.txt before they fetch a page (RFC 9309). Its Sitemap line tells them where the URL list is.
Sources: RFC 9309 (IETF, 2022-09), Google robots.txt (Google Search Central)
- Read today by
- Google, Bing, and the AI crawlers that document robots.txt support.
- Google Search
- Direct
-
Passed Adopted XML sitemap — Sitemap https://github.com/sitemap_marketing.xml parses and lists 376 URLs, none with <lastmod>, which is optional.
Observation
HTTP 200 · application/xml · 31.7 KB · root <urlset>
A sitemap lists the URLs a site wants crawled. Google says a small, well-linked site may not need one.
Sources: Google: build a sitemap (Google Search Central), sitemaps.org (sitemaps.org), Bing Webmaster Guidelines, §3: lastmod (Microsoft Bing)
- Read today by
- Google and Bing.
- Google Search
- Direct
-
Passed Adopted Canonical URL — Canonical https://github.com/ matches the page reached.
Observation
<link rel="canonical" href="https://github.com">
A canonical link names the one URL a page is known by. Google treats it and redirects as canonical signals.
Sources: Google: consolidate duplicate URLs (Google Search Central), RFC 6596 (IETF), Bing Webmaster Guidelines, §6: duplicate URLs and canonicals (Microsoft Bing)
- Read today by
- Google and Bing.
- Google Search
- Direct
-
Passed Adopted Documentation reachable — Documentation is reachable at /docs (standard path) and returns HTTP 200 text/html with 620 readable words.
Observation
tried: /docs 200 | /documentation 200 | /developers 200 | /gelistiriciler 200
Documentation linked as service-doc (RFC 8631), or from the homepage, is how a client learns the API.
Sources: RFC 8631 (IETF)
- Read today by
- Developers and agents integrating the API.
- Google Search
- None
-
Passed Adopted HTTP caching of the homepage — The homepage sends Cache-Control "max-age=0, private, must-revalidate" and ETag, and a request with If-None-Match was answered HTTP 304 Not Modified.
Observation
Cache-Control: max-age=0, private, must-revalidate · ETag: W/"76b51e6294203f0db816b8b32e600faa" · Last-Modified: (none) · conditional GET with If-None-Match: HTTP 304
Cache-Control says how long the homepage may be reused. A validator lets a crawler ask whether it changed, and a 304 answer skips the body.
Sources: RFC 9111: HTTP Caching (IETF, 2022-06), RFC 9110, 13: conditional requests (IETF), Google: HTTP caching for crawlers (Google Search Central, 2024-12), Bing Webmaster Guidelines, §3: ETag and lastmod (Microsoft Bing)
- Read today by
- Googlebot sends If-None-Match and If-Modified-Since. Every HTTP cache reads Cache-Control.
- Google Search
- Direct
-
Passed Adopted Language declaration and multilingual access — html lang="en" with 8 hreflang alternates (en-us, pt-br, es, ja, ko-kr, fr-fr), each with an absolute href, and one refers to this page.
Observation
html lang="en" · alternates: en-us -> https://github.com, pt-br -> https://github.com?locale=pt-br, es -> https://github.com?locale=es-419, ja -> https://github.com?locale=ja, ko-kr -> https://github.com?locale=ko-kr, fr-fr -> https://github.com?locale=fr-fr, de-de -> https://github.com?locale=de-de, x-default -> https://github.com/ · content-language: (none)
The lang attribute names the language of the text. WCAG 3.1.1 requires it at level A. Google reads hreflang, not lang.
Sources: WCAG 2.2, 3.1.1 (W3C), Google: managing multi-regional sites (Google Search Central), Google: localized versions (Google Search Central)
- Read today by
- Screen readers and translation tools. Google ignores the lang attribute and reads hreflang.
- Google Search
- Indirect
-
Passed Adopted A real 404 for a missing page — /aws-probe-0cbf9a5e6978, a path that does not exist, answered HTTP 404.
Observation
HTTP 404 · text/html · first 16 KB read
A URL that does not exist answers 404 or 410 (RFC 9110). A 200 there makes every mistyped link look like a page.
Sources: RFC 9110, 15.5.5: 404 Not Found (IETF), Google: HTTP status codes and soft 404s (Google Search Central), Bing Webmaster Guidelines, §9: 404 for missing pages (Microsoft Bing)
- Read today by
- Google reports a missing page that answers 200 as a soft 404. Every HTTP client reads the status first.
- Google Search
- Direct
-
Passed Adopted Per-page content separation (no SPA fallback) — The homepage and /about have different titles ("GitHub · Change is constant. GitHub kee…" and "About GitHub · GitHub") and different main text (12% word overlap).
Observation
compared / with /about · titles "GitHub · Change is constant. GitHub keeps you ahe…" / "About GitHub · GitHub" · skeleton 67% · main-text word overlap 12% · main text 2137 characters
A route that answers every URL with the same page makes every URL look alike to a crawler.
Sources: Google: HTTP errors and soft 404s (Google Search Central)
- Read today by
- Google flags such pages as soft 404s.
- Google Search
- Direct
-
Passed Adopted Responsive markup — viewport content "width=device-width" sets width=device-width and leaves zoom on.
Observation
viewport meta: width=device-width
Google indexes the smartphone rendering of a page. A viewport that blocks zoom fails WCAG 1.4.4.
Sources: Google: mobile-first indexing (Google Search Central), WCAG 2.2, 1.4.4 (W3C)
- Read today by
- Browsers; Google's smartphone crawler renders the page with it. No Google feature reads the tag itself.
- Google Search
- Indirect
Hand this report to your coding agent
The prompt lists 5 checks to fix. Each has the value the scan measured and a command that repeats the measurement. It adds the change to make where the scan could build one. After them it lists 6 agent conventions, at lower priority. The prompt is served at https://agenticwebscore.com/aws/github.com/prompt.md. The command below prints it, so you can paste the output into your agent.
curl -s https://agenticwebscore.com/aws/github.com/prompt.md - prompt.md The prompt Paste this into your own agent
- agent-brief.md Agent brief The prompt plus every finding
- bundle.md Bundle Status and brief in one document
- status.md Status Every check with its measurement
This scan cannot see which AI answers cite the site. Bing Webmaster Tools reports citations in Copilot and in Bing's AI summaries under AI Performance. Google Search Console includes AI Overviews and AI Mode in its Performance report totals, without a separate view (Google's guide to AI features). Both need a verified owner of the site.
Badge
Show this score on your own site. Press Get the badge, paste its snippet into your homepage, and press Check my badge. The directory then lists your site as a verified listing.
Ship a fix, then re-scan
Measured 8 days ago. Nothing re-scans a site on its own. A scan runs once, when somebody asks for it — and that somebody need not be you. Anyone can request a scan of any public host, so this page may be the first you hear of it. After you ship a fix, press Re-scan now. A check that now passes adds its points to the score.
Why a re-scan can differ
The current rubric evaluates 3 of its 60 checks against a clock: security-txt, status-page, and inventory-freshness. They can change with no change to the site. A response slower than 12 s is reported as not measured, so the same bytes served slowly can move the score and the fingerprint.
Site profile could not be confirmed
Because we could not confirm the site profile, no profile-specific check was scored and none was turned into a recommendation. 3 profile-specific checks were reported as out of scope, not as failures. A missed optimisation is cheaper than an instruction that wastes your time. Declare the site profile and re-scan to have them measured.
- · Two profiles are too close to separate: api (4) and saas (4), margin 0 < 2.
- · Profile-scoped checks were not scored rather than guessed at.
Signal scores
- API platform 4
- SaaS 4
- Marketplace 3
- Services 2
- Publisher 1
Modules
- Every site: always in scope.
- Sites with an API or agent surface : in scope.
- · /.well-known/ai-catalog.json lists an MCP server card
- Sites with a commercial signal : out of scope. The homepage shows no commercial signal.
Scan record
- Target
- https://github.com
- Measured at
- https://github.com/
- Scanned
- Requests
- 51
- Duration
- 4.0s
- Confidence
- 100% · high
- Rubric
- rubric v0.9
Fingerprint
7784eb45842e… verify this record
full hash
7784eb45842ec9cd499e72255c8b0285eee23c65c91d26ec2e4786ae365aa243
Page as an agent receives it
- Words
- 1091
- Est. tokens
- 1473
- Markup
- 559.0 KB
- Language
- en
“GitHub · Change is constant. GitHub keeps you ahead. · GitHub · GitHub”
By dimension
Each score is the share of the in-scope web-standards points a dimension earned. A dash means the dimension has no web-standards check in scope. The agent conventions are scored apart, as their own part.
-
Discoverability
100- Pass
- 6
- Partial
- 0
- Fails
- 0
- Not counted
- 0
Unscored: 0 in use, 1 not used
-
Content
63.6- Pass
- 5
- Partial
- 1
- Fails
- 1
- Not counted
- 3
Unscored: 0 in use, 1 not used
-
Crawler access
100- Pass
- 1
- Partial
- 0
- Fails
- 0
- Not counted
- 0
Unscored: 1 in use, 2 not used
-
API and agent
50- Pass
- 1
- Partial
- 1
- Fails
- 1
- Not counted
- 3
Unscored: 0 in use, 1 not used, 1 with an error
-
Commerce
—- Pass
- 0
- Partial
- 0
- Fails
- 0
- Not counted
- 5
Unscored: 0 in use, 0 not used
Request transcript 51 requests
Every fetch this scan record logged, in order. A fetch that followed redirects shows how many redirect requests it made.
48 fetches, 3 redirect requests.
-
/
200 text/html 559.0 KB 134ms
-
/robots.txt
200 text/plain 14.2 KB 208ms
-
/sitemap.xml
406 application/xml 0 B 206ms
-
/llms.txt
200 text/plain 28.0 KB 228ms
-
/.well-known/security.txt
200 text/plain 269 B 338ms
-
/security.txt
404 text/plain 9 B 239ms
-
/about
200 text/html 291.8 KB 27ms
-
/contact
200 +3 redirects text/html 12.0 KB 772ms
-
/pricing
200 text/html 689.8 KB 363ms
-
/docs
200 text/html 242.2 KB 506ms
-
/developers
200 text/html 159.9 KB 386ms
-
/.well-known/api-catalog
404 text/html 16.0 KB 321ms
-
/openapi.json
410 application/json 394 B 274ms
-
/.well-known/openapi.json
404 application/json 21 B 223ms
-
/.well-known/mcp/server-card.json
404 application/json 21 B 131ms
-
/.well-known/ai-catalog.json
200 application/ai-catalog+json 187 B 220ms
-
/.well-known/agent-card.json
404 application/json 21 B 223ms
-
/.well-known/oauth-protected-resource
404 text/html 16.0 KB 379ms
-
/.well-known/oauth-authorization-server
404 text/html 16.0 KB 266ms
-
/.well-known/openid-configuration
404 text/html 16.0 KB 286ms
-
/llms-full.txt
200 text/plain 28.0 KB 179ms
-
/ai-policy
404 text/html 16.0 KB 322ms
-
/ai-usage-policy
404 text/html 16.0 KB 322ms
-
/ai
200 text/html 248.5 KB 553ms
-
/legal/ai-policy
404 text/html 16.0 KB 298ms
-
/yapay-zeka-politikasi
404 text/html 16.0 KB 268ms
-
/.well-known/mcp.json
404 application/json 21 B 192ms
-
/mcp.json
200 application/json 30.9 KB 352ms
-
/mcp
200 text/html 255.8 KB 450ms
-
/.well-known/agent-skills/index.json
404 application/json 21 B 137ms
-
/.well-known/agent-skills.json
404 application/json 21 B 183ms
-
/.well-known/skills.json
404 application/json 21 B 195ms
-
/.well-known/ard.json
404 application/json 21 B 197ms
-
/.well-known/agent-resources.json
404 application/json 21 B 169ms
-
/ard.json
410 application/json 394 B 196ms
-
/documentation
200 text/html 181.7 KB 308ms
-
/gelistiriciler
200 text/html 159.7 KB 363ms
-
/status
200 text/html 46 B 188ms
-
/changelog
404 text/html 16.0 KB 337ms
-
/releases
404 text/html 16.0 KB 271ms
-
/whats-new
200 text/html 180.0 KB 419ms
-
/sitemap_marketing.xml
200 application/xml 31.7 KB 210ms
-
/features/ai/github-app
200 text/html 336.4 KB 272ms
-
/github/github-mcp-server
200 text/html 811.0 KB 677ms
-
/
200 text/html 559.0 KB 358ms
-
/
304 text/html 0 B 43ms
-
/aws-probe-0cbf9a5e6978
404 text/html 16.0 KB 275ms
-
/aws-probe-0cbf9a5e6978
404 text/html 16.0 KB 512ms