AgenticWebScore
Scan

rubric v0.9 · scanned

github.com

AI crawler checker

Your AI crawler result

The AI crawler checker puts these checks first. The full report below lists every check, these included.

  • Present Adopted AI crawler policy in robots.txt — 6 of 18 AI crawler tokens have their own robots.txt group: Bytespider disallowed, GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot and anthropic-ai partly disallowed. The other 12 follow the * group (165 of the file's 501 rules).

    Observation

    Bytespider: own group, 1 rule, disallowed · GPTBot: own group, 165 rules, partly disallowed · OAI-SearchBot: own group, 165 rules, partly disallowed · ClaudeBot: own group, 165 rules, partly disallowed · PerplexityBot: own group, 165 rules, partly disallowed · anthropic-ai: own group, 165 rules, partly disallowed · * group: 165 rules, partly disallowed, followed by 12 AI crawler tokens · robots.txt: 6 groups, 501 rules

    A crawler follows the robots.txt group that names it, otherwise the * group. A named group states the site's rules for that crawler.

    Sources: RFC 9309, 2.2.1 (IETF), Google common crawlers (Google Search Central), OpenAI bots (OpenAI)

    Read today by
    AI crawlers that document robots.txt tokens.
    Google Search
    None

    Read the check page

  • Passed Adopted robots.txt crawl policy — /robots.txt has 6 groups and 501 rules across all groups, and 1 Sitemap line.

    Observation

    HTTP 200 · text/plain · 14.2 KB · # If you would like to crawl GitHub contact us via https://support.github.com?tags=dotcom-robots # We also provide an extensive API: https:…

    Crawlers read robots.txt before they fetch a page (RFC 9309). Its Sitemap line tells them where the URL list is.

    Sources: RFC 9309 (IETF, 2022-09), Google robots.txt (Google Search Central)

    Read today by
    Google, Bing, and the AI crawlers that document robots.txt support.
    Google Search
    Direct

    Read the check page

  • Not used worth 3.7 points Draft Content-Signal preferences — No Content-Signal line in robots.txt or the homepage headers. Under the policy, no signal states no preference.

    Observation

    HTTP 200 · text/plain · 14.2 KB

    Content-Signal states search, ai-input and ai-train preferences in robots.txt. No crawler documents honouring it.

    Sources: Cloudflare: Content Signals Policy (Cloudflare, 2025-09), IETF AIPREF (IETF)

    Read today by
    Cloudflare's scanner scores it. The IETF AIPREF drafts are the standards track.
    Google Search
    None

    Fix

    Content-Signal: search=<choose: yes or no>, ai-input=<choose: yes or no>, ai-train=<choose: yes or no>
    

    Note

    Add this line to the site's own User-agent: * group in /robots.txt, below its User-agent lines. Each value is the site owner's choice.

    Verify with

    ORIGIN=https://github.com
    curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/plain, */*;q=0.5' "$ORIGIN/robots.txt" | grep -i -B3 '^content-signal:'
    curl -sSL -o /dev/null -D - -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$ORIGIN/" | grep -i '^content-signal:'

    Read the check page

  • Passed Adopted Readable homepage — HTTP 200 text/html, 559 KB, 1091 words of visible text.

    Observation

    HTTP 200 · text/html · 559 KB

    The homepage is the first URL every agent requests. An error or an empty page gives it nothing to read.

    Sources: RFC 9110 (IETF), Google: 301 redirects (Google Search Central), Bing Webmaster Guidelines, §7: 301 and 302 redirects (Microsoft Bing)

    Read today by
    Every crawler and agent.
    Google Search
    Direct

    Read the check page

Read the full report Open the AI crawler checker

Verdict

62.8
out of 100

L2 · Crawlable

Web standards 79 · Agent conventions 25

Usable by agents: No action surface found

Held at L2: L3 Machine-readable needs jsonld to pass or be partial. Read the level rules

Readable: HTTP 200 at https://github.com/, 1,091 words in the HTML. 5 to fix, 3 optional choices, 7 agent conventions not used, 1 agent convention to correct, and 27 out of scope.

How this was measured

Measured once, from 1 vantage point, at https://github.com. The scanner reads the homepage and at most 129 URLs, runs no JavaScript, and calls no API or MCP tool it finds.

Not a ranking prediction, not AI visibility, not a certification.

We sell no consulting. No ads, no sponsored listings, and we never contact the sites that are scanned. Scanning is free today.

Defects 5

Web-standards checks that fail or are partial, and optional files served with an error. Each lists what fixing it is worth.

  • Fails worth 9.1 points Adopted JSON-LD structured data — No structured data on the homepage: 0 JSON-LD blocks, 0 Microdata items, and 0 RDFa items.

    Observation

    JSON-LD: 0 blocks, 0 parsed · schema.org @context: no · Microdata: 0 items · RDFa: 0 items · top-level types: none

    Structured data names the entity a page describes. Google reads JSON-LD, Microdata and RDFa alike.

    Sources: Google: intro to structured data (Google Search Central), schema.org (Schema.org), Bing Webmaster Guidelines, §14: structured data (Microsoft Bing)

    Read today by
    Google rich results; any schema.org reader.
    Google Search
    Direct

    Fix

    <script type="application/ld+json">
    {
      "@context": "https://schema.org",
      "@type": "Organization",
      "name": "GitHub · Change is constant. GitHub keeps you ahead. · GitHub · GitHub",
      "url": "https://github.com/"
    }
    </script>

    Note

    Fill name with the name the organization uses, and keep each block valid JSON: one syntax error, such as a trailing comma, hides the whole block.

    Verify with

    HOMEPAGE=https://github.com/
    curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' -o /tmp/aws-home.html "$HOMEPAGE"; perl -0777 -ne 'my $j = () = m{<script[^>]*application/ld\+json}gi; my $m = () = m{\bitemtype=["\x27]?https?://schema\.org/}gi; my $r = () = m{\btypeof=}gi; print "JSON-LD blocks: $j, Microdata itemtypes: $m, RDFa typeof: $r\n"' /tmp/aws-home.html; perl -0777 -ne 'while (m{<script[^>]*application/ld\+json[^>]*>(.*?)</script>}gsi) { print "$1\n" }' /tmp/aws-home.html | jq -c '[(.. | objects | select(has("@type")) | {type: .["@type"], name: (.name // .headline // null)})][:10]'

    Read the check page

  • Partial worth 3 points Adopted Title and description — Title "GitHub · Change is constant. GitHub keeps you ahead. · GitHub · GitHub" (70 characters) and a description of 186 characters. The first H1 "The future of building happens together" shares 0 of 4 words with the title, brand names aside. Microsoft's guide to AI search answers (October 2025) asks for an H1 that matches the title or closely reflects it.

    Observation

    title (70 characters): GitHub · Change is constant. GitHub keeps you ahead. · GitHub · GitHub · description (186 characters): Join the world's most widely adopted, AI-powered developer platform where millions of developers, businesses, and the largest open source community build software that advances humanity. · first H1: The future of building happens together

    The title and the description say what a page is about. Google uses them for the title link and the snippet.

    Sources: Google: title links (Google Search Central), Google: snippets (Google Search Central), Bing Webmaster Guidelines, §13: titles, descriptions and headings (Microsoft Bing), Microsoft: optimizing content for AI search answers (Microsoft Advertising, 2025-10)

    Read today by
    Google, Bing, and link previews.
    Google Search
    Direct

    Note

    Give the first H1 and the title the same subject. Either can change, and brand names do not count.

    Verify with

    HOMEPAGE=https://github.com/
    curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$HOMEPAGE" | perl -0777 -CS -ne 'if (m{<title[^>]*>(.*?)</title>}si) { my $t=$1; $t=~s/\s+/ /g; $t=~s/^ | $//g; print "title (", length($t), "): $t\n" } else { print "title: none\n" } if (m{<meta\b[^>]*\bname=["\x27]?description["\x27]?[^>]*>}i) { my ($c)=$& =~ /\bcontent=["\x27]([^"\x27]*)/i; $c //= ""; print "description (", length($c), "): $c\n" } else { print "description: none\n" }'

    Read the check page

  • Fails worth 2 points Adopted API catalog — No API catalog linkset was served: /.well-known/api-catalog answered HTTP 404.

    An API catalog lists a publisher's APIs in one document at a fixed address (RFC 9727).

    Sources: RFC 9727 (IETF, 2025-06)

    Read today by
    API clients that follow RFC 9727.
    Google Search
    None

    Fix

    {
      "linkset": [
        {
          "anchor": "<choose: URL that identifies the API>",
          "service-desc": [
            {
              "href": "<choose: URL of the API description>"
            }
          ]
        }
      ]
    }

    Note

    Serve it as Content-Type application/linkset+json with profile="https://www.rfc-editor.org/info/rfc9727" and keep the anchors and hrefs absolute URLs that resolve. A target only counts once it answers: point service-desc at a document the scanner already requests, such as /openapi.json, and make it return 200, because an href that is never fetched leaves the catalog partial.

    Verify with

    ORIGIN=https://github.com
    curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: application/linkset+json, application/json;q=0.9, */*;q=0.5' "$ORIGIN/.well-known/api-catalog" | jq '.linkset[] | {anchor, "service-desc": .["service-desc"], status: .status}'

    Read the check page

  • Partial worth 0.5 points Adopted Service status and changelog feed — A status page answered at /status (HTTP 200), but the homepage links no status or changelog feed.

    A status page and a changelog feed say whether the service is up and what changed.

    Sources: RFC 9727 (status relation) (IETF), RFC 4287 (IETF)

    Read today by
    Integrators and monitoring tools.
    Google Search
    None

    Fix

    <link rel="alternate" type="application/atom+xml" title="Changelog" href="https://github.com<choose: path of the changelog feed>">

    Note

    A link only counts if it resolves: a feed that 404s is served with an error, and one whose newest entry is months old is stale. The scan reads a status page on the scanned origin, such as /status; a status page on another host is not requested, so it is reported as not measured.

    Verify with

    ORIGIN=https://github.com
    curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$ORIGIN/" | grep -oiE '<link[^>]+(atom|rss)[^>]*>|href="[^"]*(status|changelog)[^"]*"' | sort -u
    for p in /status /changelog; do printf '%s -> ' "$p"; curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' -o /dev/null -w '%{http_code} %{content_type}\n' "$ORIGIN$p"; done

    Read the check page

  • optional file, served with an error 0 points Experimental ARD resource discovery manifest — ARD manifest at /.well-known/ai-catalog.json lists 1 entry of which 0 are complete and 0 carry both url and data, which the specification forbids.

    An ARD manifest lists a site's agent resources. Lighthouse does not count a missing one against a site.

    Sources: Lighthouse agentic browsing (Chrome for Developers)

    Read today by
    Lighthouse (ard-schema audit).
    Google Search
    None

    Note

    Give each entry an identifier, a displayName, a type, and exactly one of url or data.

    Verify with

    ORIGIN=https://github.com
    curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$ORIGIN/" | grep -oiE '<link[^>]+rel="?ard"?[^>]*>'
    for p in /.well-known/ard.json /.well-known/agent-resources.json /.well-known/ai-catalog.json /ard.json; do printf '%s -> ' "$p"; curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: application/json, application/linkset+json;q=0.9, */*;q=0.5' "$ORIGIN$p" | jq -c '[.entries[]? | {identifier,displayName,type,url}]' 2>/dev/null || echo "missing or not JSON"; done

    Read the check page

Optional choices 4

Show 4 in Optional choices

Decisions only you can make. None earns points or holds a level.

  • Not used 0 points Adopted Snippet and AI grounding directives — Not used: the homepage has no noarchive, nocache, nosnippet or max-snippet:0 directive, and no data-nosnippet attribute, so Bing applies none of the limits they set on snippets and Copilot answers.

    Observation

    no robots meta tag for robots, bingbot or googlebot, no X-Robots-Tag header · data-nosnippet: 0 elements

    noarchive, nocache, nosnippet and data-nosnippet limit how Bing shows a page and how Copilot uses it. Each is the site owner's choice.

    Sources: Bing Webmaster Guidelines, §10: NOARCHIVE, NOCACHE and NOSNIPPET (Microsoft Bing), Bing: robots meta tags and attributes that Bing supports (Microsoft Bing), Google: robots meta tag specifications (Google Search Central)

    Read today by
    Bing reads all five, and Copilot follows noarchive and nocache. Google reads nosnippet, max-snippet and data-nosnippet.
    Google Search
    Direct

    Note

    Each directive is your choice. noarchive keeps the page out of Copilot answers, nocache lets Copilot use only the URL, the title and the snippet, and nosnippet or data-nosnippet may limit the quality of what Copilot cites.

    Verify with

    HOMEPAGE=https://github.com/
    curl -sSL -o /dev/null -D - -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$HOMEPAGE" | tr -d '\r' | grep -i '^x-robots-tag:'
    curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$HOMEPAGE" | perl -0777 -ne 'my $h = /^(.*?)<\/head>/si ? $1 : substr($_, 0, 8192); while ($h =~ m{<meta\b[^>]*>}gi) { my $t = $&; next unless $t =~ /\bname=["\x27]?(robots|bingbot|googlebot)["\x27]?[\s>\/]/i; my $n = lc $1; my ($c) = $t =~ /\bcontent=["\x27]([^"\x27]*)/i; print "meta $n: ", ($c // ""), "\n" } my $body = $_; $body =~ s{<!--.*?-->}{}gs; $body =~ s{<(script|style|template|title|textarea|noscript)\b[^>]*>.*?</\1\s*>}{}gsi; my $n = 0; while ($body =~ m{<[a-z][a-z0-9-]*\b((?:[^>"\x27]|"[^"]*"|\x27[^\x27]*\x27)*)>}gi) { (my $t = $1) =~ s{"[^"]*"|\x27[^\x27]*\x27}{}g; $n++ if $t =~ /(?:^|\s)data-nosnippet(?=[\s=\/]|$)/i } print "data-nosnippet: $n\n"'

    Read the check page

  • Not used 0 points Experimental AI usage policy page — Not published: /ai-policy, /ai-usage-policy, /legal/ai-policy and /yapay-zeka-politikasi answered HTTP 404; /ai answered HTTP 200 with a page that is not named an AI policy. No homepage link names an AI policy. Checked because robots.txt disallows / for Bytespider.

    Observation

    /ai-policy: HTTP 404 · text/html · first 16 KB read; /ai-usage-policy: HTTP 404 · text/html · first 16 KB read; /ai: HTTP 200 · text/html · 249 KB read; /legal/ai-policy: HTTP 404 · text/html · first 16 KB read; /yapay-zeka-politikasi: HTTP 404 · text/html · first 16 KB read

    A page that explains in prose the AI restrictions the site states. No standard defines it.

    Sources: RFC 9309 (IETF)

    Read today by
    People reading the site's terms.
    Google Search
    None

    Note

    Publish a page that explains the AI restriction robots.txt or Content-Signal states, and link it from the homepage with the words AI policy. No standard defines the page or its content.

    Verify with

    ORIGIN=https://github.com
    curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$ORIGIN/" | grep -Eio '<a[^>]+>[^<]*' | grep -Ei 'ai[-_ ]?(usage[-_ ]?)?polic|yapay[-_ ]?zeka' | head -3
    for P in /ai-policy /ai-usage-policy /ai /legal/ai-policy /yapay-zeka-politikasi; do curl -sSL -o /dev/null -w "$P HTTP %{http_code} %{content_type}\n" -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$ORIGIN$P"; done

    Read the check page

  • Not used 0 points Adopted Breadcrumb markup on an inner page — The inner page /features/ai/github-app has no BreadcrumbList in its JSON-LD, Microdata, or RDFa.

    Observation

    https://github.com/features/ai/github-app · HTTP 200 · text/html · BreadcrumbList: 0

    BreadcrumbList states where an inner page sits in the site. A homepage has no trail.

    Sources: Google: breadcrumb (Google Search Central), Google blog (Google Search Central, 2025-01)

    Read today by
    Google shows breadcrumbs in desktop results.
    Google Search
    Direct

    Fix

    <script type="application/ld+json">
    {
      "@context": "https://schema.org",
      "@type": "BreadcrumbList",
      "itemListElement": [
        {
          "@type": "ListItem",
          "position": 1,
          "name": "GitHub · Change is constant. GitHub keeps you ahead. · GitHub · GitHub",
          "item": "https://github.com/"
        },
        {
          "@type": "ListItem",
          "position": 2,
          "name": "<choose: name of this section>",
          "item": "https://github.com/features"
        },
        {
          "@type": "ListItem",
          "position": 3,
          "name": "<choose: name of this section>",
          "item": "https://github.com/features/ai"
        },
        {
          "@type": "ListItem",
          "position": 4,
          "name": "GitHub Copilot app · GitHub",
          "item": "https://github.com/features/ai/github-app"
        }
      ]
    }
    </script>

    Note

    Replace each <choose: …> with the name or the URL the site uses for that section, and drop a level that has no page. Show the same trail as links on the page.

    Verify with

    PAGE=https://github.com/features/ai/github-app
    curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' -o /tmp/aws-page.html "$PAGE"; perl -0777 -ne 'while (m{<script[^>]*application/ld\+json[^>]*>(.*?)</script>}gsi) { print "$1\n" }' /tmp/aws-page.html | jq -r '.. | objects | select((.["@type"]? | if type == "array" then .[] else . end) == "BreadcrumbList") | .itemListElement[] | [(.position | tostring), (.name // .item.name? // "NO NAME"), ((.item["@id"]? // .item) // "NO URL" | tostring)] | @tsv'; grep -c 'schema.org/BreadcrumbList' /tmp/aws-page.html

    Read the check page

  • Present 0 points Adopted security.txt (RFC 9116) — security.txt at /.well-known/security.txt declares 1 Contact and an Expires 2026-11-19T17:05:55z in the future. Not scored: its Policy URL is on bounty.github.com, which this scan does not request.

    Observation

    Contact: https://hackerone.com/github Acknowledgments: https://hackerone.com/github/hacktivity Preferred-Languages: en Canonical: https://github.com/.well-known/security.txt Policy: https://bounty.github.com Hiring: https://github.careers Expires: 2026-11-19T…

    security.txt tells a researcher where to report a vulnerability (RFC 9116). It has no effect on search.

    Sources: RFC 9116 (IETF, 2022-04)

    Read today by
    Security researchers and disclosure tools.
    Google Search
    None

    Read the check page

Agent conventions 10

Show 10 in Agent conventions

The scored ones make up 30% of the score, and the unscored ones earn no points. None has an effect on Google Search today. Each scored row not in use lists what using it is worth.

  • Not used worth 3.7 points Draft Content-Signal preferences — No Content-Signal line in robots.txt or the homepage headers. Under the policy, no signal states no preference.

    Observation

    HTTP 200 · text/plain · 14.2 KB

    Content-Signal states search, ai-input and ai-train preferences in robots.txt. No crawler documents honouring it.

    Sources: Cloudflare: Content Signals Policy (Cloudflare, 2025-09), IETF AIPREF (IETF)

    Read today by
    Cloudflare's scanner scores it. The IETF AIPREF drafts are the standards track.
    Google Search
    None

    Fix

    Content-Signal: search=<choose: yes or no>, ai-input=<choose: yes or no>, ai-train=<choose: yes or no>
    

    Note

    Add this line to the site's own User-agent: * group in /robots.txt, below its User-agent lines. Each value is the site owner's choice.

    Verify with

    ORIGIN=https://github.com
    curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/plain, */*;q=0.5' "$ORIGIN/robots.txt" | grep -i -B3 '^content-signal:'
    curl -sSL -o /dev/null -D - -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$ORIGIN/" | grep -i '^content-signal:'

    Read the check page

  • Not used worth 3.7 points Adopted Discovery via Link response headers — Not served: the homepage response has no Link header naming /llms.txt.

    Observation

    HTTP 200 · text/html · 559 KB

    A Link header (RFC 8288) points a client at a resource before it parses the HTML.

    Sources: RFC 8288 (IETF), IANA link relations (IANA)

    Read today by
    Clients that follow RFC 8288 and RFC 9727 links.
    Google Search
    None

    Fix

    Link: <https://github.com/llms.txt>; rel="describedby"
    

    Note

    Name only documents the site serves. Several links go in one comma-separated Link header, and absolute URLs need no base to resolve (RFC 8288).

    Verify with

    HOMEPAGE=https://github.com/
    curl -sSL -o /dev/null -D - -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$HOMEPAGE" | grep -i '^link:'

    Read the check page

  • Served with an error worth 3.7 points Draft /llms-full.txt expanded context — /llms-full.txt is a list of links, not page text: 117 link lines of 142 lines, 2778 words in 28 KB.

    Observation

    HTTP 200 · text/plain · 28 KB · # GitHub > GitHub is a developer platform for building, shipping, and maintaining software. It provides cloud-based Git…

    A convention that puts a site's text in one file. The llms.txt proposal does not define it.

    Sources: llmstxt.org (llmstxt.org), Mintlify (Mintlify)

    Read today by
    Some documentation tools.
    Google Search
    None

    Note

    Tools that write /llms.txt can also write /llms-full.txt from the same pages: the text of each page, under its URL. Link it from /llms.txt.

    Verify with

    ORIGIN=https://github.com
    curl -sSL -o /dev/null -w 'status=%{http_code} type=%{content_type} bytes=%{size_download}\n' -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/plain, text/markdown;q=0.9, */*;q=0.5' "$ORIGIN/llms-full.txt"
    curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/plain, text/markdown;q=0.9, */*;q=0.5' "$ORIGIN/llms.txt" | grep -i 'llms-full'

    Read the check page

  • Not used worth 3.7 points Experimental Markdown content negotiation — The homepage answered `Accept: text/markdown, text/html;q=0.5` with HTML, which is a valid response.

    Observation

    HTTP 200 · text/html · Vary: X-PJAX, X-PJAX-Container, Turbo-Visit, Turbo-Frame, X-Requested-With, X-GitHub-Client-Version, Accept-Language, Sec-Fetch-Site,Accept-Encoding, Accept, X-Requested-With · markdown body: no

    Some coding agents ask for text/markdown. Google Search does not use markdown.

    Sources: RFC 7763 (IETF), Cloudflare: Markdown for Agents (Cloudflare), Google: AI optimization guide (Google Search Central)

    Read today by
    Claude Code, OpenCode and Cursor send Accept: text/markdown by default (Cloudflare, 2026-04).
    Google Search
    None

    Fix

    Answer a request whose Accept header prefers text/markdown with a markdown version of the same page, at the same URL.
    - On a CDN that converts HTML to markdown, turn that conversion on for the site.
    - On a CMS or a static build, publish one .md file per page, and serve it when Accept prefers text/markdown.
    Send Content-Type: text/markdown; charset=utf-8 with the markdown.
    Send Vary: Accept with both the markdown and the HTML, so a shared cache keeps them apart.

    Note

    Send Vary: Accept with the HTML response too, so a shared cache never serves one representation in place of the other.

    Verify with

    HOMEPAGE=https://github.com/
    curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/markdown, text/html;q=0.5' -D - -o /tmp/aws-md.body "$HOMEPAGE" | grep -iE '^(HTTP/|content-type|vary)'; head -c 300 /tmp/aws-md.body; echo

    Read the check page

  • Not used worth 3.7 points Adopted Structured errors for JSON requests — Not used: /aws-probe-0cbf9a5e6978, asked for JSON, answered HTTP 404 with an HTML page, not a JSON error.

    Observation

    HTTP 404 · text/html · first 16 KB read

    An agent that asks for JSON and gets an HTML error page has to read a page meant for people. RFC 9457 defines a JSON error format.

    Sources: RFC 9457: Problem Details for HTTP APIs (IETF, 2023-07), RFC 9110, 12.5.1: Accept (IETF)

    Read today by
    API clients and agents that send Accept: application/json. No search engine reads it.
    Google Search
    None

    Fix

    {
      "type": "about:blank",
      "title": "Not Found",
      "status": 404,
      "detail": "Nothing is published at this address."
    }
    

    Note

    Send this with Content-Type: application/problem+json and the status 404 when a request for a missing path asks for JSON. Browsers can keep the HTML page (RFC 9457).

    Verify with

    ORIGIN=https://github.com
    MISSING="/aws-probe-$(head -c 6 /dev/urandom | od -An -tx1 | tr -d ' \n')"
    curl -sS -D - -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: application/json, application/problem+json;q=0.9, */*;q=0.1' "$ORIGIN$MISSING" | head -20

    Read the check page

  • Not used worth 3.7 points Experimental WebMCP browser tool declaration — Not used: the homepage has 2 forms, and none declares a WebMCP tool.

    WebMCP lets a page register tools for an in-browser agent. It is a W3C Community Group draft.

    Sources: WebMCP draft (W3C Web Machine Learning Community Group), Chrome: WebMCP (Chrome for Developers)

    Read today by
    Chrome, behind an origin trial. Lighthouse audits it.
    Google Search
    None

    Fix

    // WebMCP (W3C Community Group draft): register a tool an in-browser agent can call.
    if ('modelContext' in navigator) {
      navigator.modelContext.registerTool({
        name: 'search_site',
        description: 'Search this site and return the matching pages.',
        inputSchema: {
          type: 'object',
          properties: { query: { type: 'string', description: 'The words to search for' } },
          required: ['query'],
        },
        async execute({ query }) {
          // Call the site's own search here and return its results as text.
          return { content: [{ type: 'text', text: 'Results for ' + query }] };
        },
      });
    }

    Note

    WebMCP is a W3C Community Group draft, not on the W3C Standards Track, and Chrome exposes it behind an origin trial. The draft defines the in-page API only, so there is no manifest file to publish.

    Verify with

    HOMEPAGE=https://github.com/
    curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$HOMEPAGE" | grep -oE 'registerTool\(|<form[^>]*toolname[^>]*>' | sort | uniq -c

    Read the check page

  • Present Adopted AI crawler policy in robots.txt — 6 of 18 AI crawler tokens have their own robots.txt group: Bytespider disallowed, GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot and anthropic-ai partly disallowed. The other 12 follow the * group (165 of the file's 501 rules).

    Observation

    Bytespider: own group, 1 rule, disallowed · GPTBot: own group, 165 rules, partly disallowed · OAI-SearchBot: own group, 165 rules, partly disallowed · ClaudeBot: own group, 165 rules, partly disallowed · PerplexityBot: own group, 165 rules, partly disallowed · anthropic-ai: own group, 165 rules, partly disallowed · * group: 165 rules, partly disallowed, followed by 12 AI crawler tokens · robots.txt: 6 groups, 501 rules

    A crawler follows the robots.txt group that names it, otherwise the * group. A named group states the site's rules for that crawler.

    Sources: RFC 9309, 2.2.1 (IETF), Google common crawlers (Google Search Central), OpenAI bots (OpenAI)

    Read today by
    AI crawlers that document robots.txt tokens.
    Google Search
    None

    Read the check page

  • Present Draft /llms.txt published and well-structured — /llms.txt opens with H1 "GitHub", with 12 H2 link sections and 117 links, 28 KB. The scan requested 1 link it names, and it answered 2xx. Not requested: 116 links on another origin.

    Observation

    HTTP 200 · text/plain · 28 KB · headings: GitHub | How to use | Building with GitHub, for coding agents and automation | GitHub Copilot | Authentication | Get st…

    A 2024 proposal for a markdown index of a site. Google Search does not use it.

    Sources: llmstxt.org (llmstxt.org), Google: AI optimization guide (Google Search Central), Lighthouse llms.txt (Chrome for Developers)

    Read today by
    Some documentation tools and coding agents. Lighthouse audits it.
    Google Search
    None

    Read the check page

  • Not used 0 points Draft Agent Skills declaration — Not served: no agent skills index with a non-empty skills array was found as JSON (/.well-known/agent-skills/index.json 404 application/json; /.well-known/agent-skills.json 404 application/json; /.well-known/skills.json 404 application/json).

    An agent skills index lists skill packages for agent runtimes. It is a draft on GitHub.

    Sources: Agent Skills Discovery RFC (Cloudflare)

    Read today by
    Agent runtimes that read the draft.
    Google Search
    None

    Fix

    {
      "skills": [
        {
          "name": "<choose: skill name>",
          "type": "skill-md",
          "description": "<choose: what the skill does>",
          "url": "https://github.com/.well-known/agent-skills/<choose: skill name>/SKILL.md",
          "digest": "sha256:<choose: SHA-256 of the SKILL.md bytes, in hex>"
        }
      ]
    }

    Note

    An early convention rather than a ratified specification, and it has already renamed its path once. The digest is the SHA-256 of the exact bytes served, so recompute it whenever SKILL.md changes. An artifact on another host is reported as declared, not observed, because this scan requests no other host.

    Verify with

    ORIGIN=https://github.com
    for p in /.well-known/agent-skills/index.json /.well-known/agent-skills.json /.well-known/skills.json; do printf '%s -> ' "$p"; curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: application/json, application/linkset+json;q=0.9, */*;q=0.5' "$ORIGIN$p" | jq -c '{skills:[.skills[]? | {name,type,url,digest}]}' 2>/dev/null || echo "missing or not JSON"; done

    Read the check page

  • Not used 0 points Experimental DNS-AID discovery record — Not published: 0 SVCB records and 0 TXT records resolve at _index._agents.github.com.

    Observation

    DNS-AID names queried for SVCB and TXT records: _index._agents.github.com

    DNS-AID publishes agents in DNS. It is an individual Internet-Draft with no IETF standing.

    Sources: draft-mozleywilliams-dnsop-dnsaid (IETF)

    Read today by
    Cloudflare's scanner.
    Google Search
    None

    Fix

    ; DNS-AID organization entry point (draft-mozleywilliams-dnsop-dnsaid)
    _index._agents.github.com. IN SVCB 1 <choose: host name that serves the agent index, ending in a dot> alpn=<choose: ALPN of the agent endpoint>
    

    Note

    DNS-AID is an individual Internet-Draft with no IETF standing. TYPE64 is the SVCB type by number, which older dig builds also answer.

    Verify with

    DOMAIN=github.com
    dig +short TYPE64 _index._agents.$DOMAIN; dig +short TXT _index._agents.$DOMAIN

    Read the check page

Not measured 1

Show 1 in Not measured

The scanner could not observe these. Unscored checks earn no points, so the score and confidence are unchanged.

  • Not measured 0 points Draft MCP server card — Not measured: /.well-known/ai-catalog.json lists an MCP server card at https://api.githubcopilot.com/mcp/server-card, which this scan did not request.

    An MCP server card describes a remote MCP server. It is an optional extension (SEP-2127), not core MCP.

    Sources: SEP-2127 (Model Context Protocol, 2026-10), discovery.md (Model Context Protocol)

    Read today by
    MCP clients that implement SEP-2127; Cloudflare's scanner.
    Google Search
    None

    Verify with

    ORIGIN=https://github.com
    for p in /.well-known/ai-catalog.json /.well-known/mcp/server-card.json /.well-known/mcp.json /mcp.json; do printf '%s -> ' "$p"; curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: application/json, application/linkset+json;q=0.9, */*;q=0.5' "$ORIGIN$p" | jq -c '{name,version,description,remotes,entries}' 2>/dev/null || echo "missing or not JSON"; done

    Read the check page

Out of scope for this site 27

Show 27 in Out of scope for this site

The scan did not ask this site for these checks. Each says why. If one should apply, the error is ours, not yours.

Sites with an API or agent surface

The scan found an MCP server. A check that reads another kind of surface is not asked.

  • Out of scope 0 points Draft A2A agent card — Not scored: this check applies to a site that runs an A2A agent, and the scan found none.

    An A2A agent card describes an agent the site runs. An A2A server publishes it.

    Sources: A2A specification (A2A Project)

    Read today by
    A2A clients.
    Google Search
    None

    Read the check page

  • Out of scope 0 points Draft Cryptographically signed agent card — Not scored: this check applies to a site that runs an A2A agent, and the scan found none.

    An A2A card may be signed with JWS. Signing is optional in the specification.

    Sources: A2A specification (A2A Project), RFC 7515 (IETF)

    Read today by
    A2A clients that verify cards.
    Google Search
    None

    Read the check page

  • Out of scope 0 points Draft /auth.md agent authentication guide — Not scored: this check applies to a site that uses OAuth, and the scan found none.

    auth.md is one vendor's convention (WorkOS) for telling an agent how to authenticate.

    Sources: WorkOS auth.md (WorkOS)

    Read today by
    Agents built on WorkOS.
    Google Search
    None

    Read the check page

  • Out of scope Adopted MCP server answers and lists its tools — Not scored: /.well-known/ai-catalog.json lists a server card at https://api.githubcopilot.com/mcp/server-card, off the scanned origin, which this scan does not request. /mcp answered HTTP 200, which does not signal an MCP server.

    An agent given the address of an MCP server can call its tools. The scan opens one session to see that the server answers and what it offers.

    Sources: MCP 2025-11-25: lifecycle (Model Context Protocol, 2025-11), MCP 2025-11-25: transports (Model Context Protocol, 2025-11), MCP 2025-11-25: tools (Model Context Protocol, 2025-11), Agentic AI Foundation: MCP joins the Linux Foundation (Linux Foundation, 2025-12)

    Read today by
    MCP clients: agent apps, coding agents, and chat products that connect to MCP servers.
    Google Search
    None

    Read the check page

  • Out of scope 0 points Draft MPP discovery in OpenAPI (x-payment-info) — Not scored: no OpenAPI document was found, so x-payment-info is not asked of this site.

    An operation with x-payment-info tells an agent that the call is paid, and states the intent, the payment method, and the amount. MPP is a draft, and charging agents is a business decision.

    Sources: MPP (mpp.dev), MPP: multi-method discovery (mpp.dev), draft-ryan-httpauth-payment (IETF (individual draft)), isitagentready.com: MPP skill (isitagentready.com)

    Read today by
    Agents that pay with MPP; isitagentready.com.
    Google Search
    None

    Read the check page

  • Out of scope Adopted OAuth discovery documents — Not scored: this check applies to a site that uses OAuth, and the scan found none.

    OAuth metadata tells a client where and how to authorise against the API.

    Sources: RFC 8414 (IETF), RFC 9728 (IETF)

    Read today by
    OAuth clients; MCP clients use RFC 9728.
    Google Search
    None

    Read the check page

  • Out of scope Adopted OpenAPI definition — Not scored: this check applies to a site that runs an HTTP API, and the scan found none.

    An OpenAPI document describes an HTTP API so a client can call it without guessing.

    Sources: OpenAPI 3.1 (OpenAPI Initiative), RFC 8631 (IETF)

    Read today by
    API clients and code generators.
    Google Search
    None

    Read the check page

  • Out of scope 0 points Experimental Rate limits and quotas documented — Not scored: this check applies to a site that runs an HTTP API, and the scan found none.

    Declared limits let a client pace its calls. RateLimit headers are an IETF draft.

    Sources: draft-ietf-httpapi-ratelimit-headers (IETF)

    Read today by
    API clients.
    Google Search
    None

    Read the check page

  • Out of scope 0 points Experimental Sandbox or test environment documented — Not scored: this check applies to a site that runs an HTTP API, and the scan found none.

    A documented sandbox lets a client test calls without touching production data.

    Sources: OpenAPI 3.1 (servers) (OpenAPI Initiative)

    Read today by
    API integrators.
    Google Search
    None

    Read the check page

  • Out of scope 0 points Draft Web Bot Auth key directory — Not scored: this check applies to a site that runs an A2A agent, and the scan found none.

    The operator of a signing agent publishes its keys here. A site that only serves content needs none.

    Sources: draft-ietf-webbotauth-httpsig-protocol (IETF, 2026-09), Cloudflare: Web Bot Auth (Cloudflare)

    Read today by
    Sites and CDNs that verify signed agents.
    Google Search
    None

    Read the check page

Sites with a commercial signal

The homepage shows no commercial signal, so commerce checks are not asked of this site.

  • Out of scope 0 points Draft ACP discovery (/.well-known/acp.json) — Not scored: the homepage shows no commercial signal, so commerce checks are not asked of this site.

    An ACP discovery document at /.well-known/acp.json tells an agent where a business's checkout API is and which services it offers. The RFC behind it is a proposal.

    Sources: ACP discovery RFC (status: Proposal) (Agentic Commerce Protocol), isitagentready.com: ACP skill (isitagentready.com)

    Read today by
    Agents that implement ACP checkout; isitagentready.com.
    Google Search
    None

    Read the check page

  • Out of scope 0 points Draft Agent purchase policy — Not scored: the homepage shows no commercial signal, so commerce checks are not asked of this site.

    Whether agents may buy, and with what approval, is a policy the site states.

    Sources: ACP (Agentic Commerce Protocol)

    Read today by
    Agent checkouts.
    Google Search
    None

    Read the check page

  • Out of scope Adopted Inventory and price freshness — Not scored: the homepage shows no commercial signal, so commerce checks are not asked of this site.

    availability says whether an item is in stock. A past priceValidUntil can hide a product snippet.

    Sources: Google: product snippet (Google Search Central)

    Read today by
    Google product snippets.
    Google Search
    Direct

    Read the check page

  • Out of scope 0 points Draft Agent payment protocol (UCP profile or x402) — Not scored: the homepage shows no commercial signal, so commerce checks are not asked of this site.

    Agent payment protocols are early. Accepting one is a business decision. This check reads UCP and x402. ACP and MPP have checks of their own.

    Sources: UCP (Universal Commerce Protocol), x402 (x402)

    Read today by
    Agent checkouts that implement UCP or x402.
    Google Search
    None

    Read the check page

  • Out of scope Adopted Product/catalog schema — Not scored: the homepage shows no commercial signal, so commerce checks are not asked of this site.

    Product and Offer markup on product pages feeds merchant listings.

    Sources: Google: merchant listing (Google Search Central), Google: product (Google Search Central)

    Read today by
    Google merchant listings and shopping agents.
    Google Search
    Direct

    Read the check page

  • Out of scope Adopted Return policy machine-readable — Not scored: the homepage shows no commercial signal, so commerce checks are not asked of this site.

    A MerchantReturnPolicy states return terms in markup, also under the Organization.

    Sources: Google: return policy (Google Search Central)

    Read today by
    Google merchant listings.
    Google Search
    Direct

    Read the check page

  • Out of scope Adopted Seller identity declared — Not scored: the homepage shows no commercial signal, so commerce checks are not asked of this site.

    On a marketplace, the seller named on an offer is who the buyer contracts with.

    Sources: Google: merchant listing (Google Search Central)

    Read today by
    Google merchant listings.
    Google Search
    Direct

    Read the check page

  • Out of scope Adopted Shipping cost and delivery declared — Not scored: the homepage shows no commercial signal, so commerce checks are not asked of this site.

    shippingDetails lets a client compute the delivered price.

    Sources: Google: merchant listing (Google Search Central)

    Read today by
    Google merchant listings.
    Google Search
    Direct

    Read the check page

  • Out of scope 0 points Draft UCP manifest (/.well-known/ucp) — Not scored: the homepage shows no commercial signal, so commerce checks are not asked of this site.

    A UCP profile at /.well-known/ucp declares commerce capabilities. UCP is a draft.

    Sources: UCP (Universal Commerce Protocol), Shopify profiles (Shopify)

    Read today by
    UCP agents; Shopify stores serve it.
    Google Search
    None

    Read the check page

Every site

  • Out of scope 0 points Draft Machine-readable booking action — Not scored: this check applies to services sites, and the site type could not be confirmed (best candidates: api, saas). Declare the type to have it measured.

    A ReserveAction declares a booking target in markup. No search engine documents a feature that reads it.

    Sources: schema.org/ReserveAction (Schema.org)

    Read today by
    None documented.
    Google Search
    None

    Read the check page

  • Out of scope Adopted Structured contact information — Not scored: the homepage markup declares no Organization-type node.

    A ContactPoint in the Organization markup makes the contact route machine-readable.

    Sources: Google: Organization structured data (Google Search Central)

    Read today by
    Google lists contactPoint as a recommended Organization property.
    Google Search
    Indirect

    Read the check page

  • Out of scope 0 points Adopted Documents in machine-readable formats — Not scored: this check applies to institution sites, and the site type could not be confirmed (best candidates: api, saas). Declare the type to have it measured.

    Tagged PDF or an HTML version can be read by screen readers and by machines.

    Sources: GOV.UK: HTML not PDF (Government Digital Service), ISO 14289 (PDF/UA) (ISO)

    Read today by
    Screen readers; GOV.UK publishes HTML first.
    Google Search
    None

    Read the check page

  • Out of scope 0 points Adopted FAQ markup for visible questions — Not scored: no heading on the homepage names a FAQ section, and the homepage carries no FAQPage markup.

    Observation

    visible questions under a FAQ heading: 0

    FAQPage marks up questions that are visible on the page. Google stopped showing FAQ rich results on 7 May 2026.

    Sources: Google Search updates (Google Search Central), Google: structured data policies (Google Search Central)

    Read today by
    Bing names FAQ markup as an example of labelled content.
    Google Search
    None

    Read the check page

  • Out of scope 0 points Draft Reuse and licensing terms declared — Not scored: this check applies to publisher or institution sites, and the site type could not be confirmed (best candidates: api, saas). Declare the type to have it measured.

    TDMRep reserves text and data mining rights in a machine-readable form, as EU DSM Article 4(3) allows.

    Sources: TDMRep CG report (W3C), Directive (EU) 2019/790 (European Union)

    Read today by
    Text and data mining tools that implement TDMRep.
    Google Search
    None

    Read the check page

  • Out of scope Adopted Structured pricing information — Not scored: no Offer or PriceSpecification in the homepage markup states a price, and no pricing page states a complete one.

    A price in markup is read without parsing prose. An Offer with a price and no currency is ambiguous.

    Sources: schema.org/Offer (Schema.org), Google: product snippet (Google Search Central)

    Read today by
    Clients that read schema.org Offer. Google reads an Offer only inside a Product, which product-schema measures.
    Google Search
    None

    Read the check page

  • Out of scope 0 points Adopted Content provenance declared — Not scored: this check applies to publisher or institution sites, and the site type could not be confirmed (best candidates: api, saas). Declare the type to have it measured.

    Article markup with an author and dates lets a client attribute and date a source.

    Sources: Google: Article structured data (Google Search Central)

    Read today by
    Google article features.
    Google Search
    Direct

    Read the check page

  • Out of scope Adopted Service/product scope machine-readable — Not scored: the homepage markup declares no Service-type node.

    Service markup states what is offered, by whom and where. An incomplete node leaves a client to guess.

    Sources: schema.org/Service (Schema.org)

    Read today by
    Clients that read schema.org Service. Google has no Service rich result.
    Google Search
    None

    Read the check page

Passed 13

Show 13 in Passed

Scored checks that pass, the heaviest first.

  • Passed Adopted Content that needs no JavaScript — 7396 characters of text are in the HTML before any script runs.

    Observation

    static text: 7396 characters · inside <noscript>: 0 characters · markup: 572359 characters

    Several AI crawlers do not run JavaScript. Text that appears only after scripts run is invisible to them.

    Sources: Vercel: the rise of the AI crawler (Vercel), Google: JavaScript SEO basics (Google Search Central), Bing Webmaster Guidelines, §8: rendering (Microsoft Bing)

    Read today by
    GPTBot and ClaudeBot fetch HTML without rendering; Googlebot renders.
    Google Search
    Indirect

    Read the check page

  • Passed Adopted Readable homepage — HTTP 200 text/html, 559 KB, 1091 words of visible text.

    Observation

    HTTP 200 · text/html · 559 KB

    The homepage is the first URL every agent requests. An error or an empty page gives it nothing to read.

    Sources: RFC 9110 (IETF), Google: 301 redirects (Google Search Central), Bing Webmaster Guidelines, §7: 301 and 302 redirects (Microsoft Bing)

    Read today by
    Every crawler and agent.
    Google Search
    Direct

    Read the check page

  • Passed Adopted Semantic heading structure — 1 H1, 9 H2s, and 16 H3s in an outline of 26 headings, with no skipped level.

    Observation

    outline: H2 H1 H2 H2 H2 H3 H3 H3 H3 H3 H2 H3 H2 H3 H3 H3 H3 H3 H2 H2 H2 H3 H3 H3 H3 H3

    Headings outline the page for screen readers and for any client that splits text by section.

    Sources: W3C WAI headings tutorial (W3C), Google SEO starter guide (Google Search Central), Bing Webmaster Guidelines, §13: titles, descriptions and headings (Microsoft Bing)

    Read today by
    Screen readers; clients that chunk HTML by heading.
    Google Search
    Indirect

    Read the check page

  • Passed Adopted Homepage indexable: no noindex directive — No noindex: the homepage has no robots meta tag for robots, bingbot or googlebot, and no X-Robots-Tag header.

    Observation

    no robots meta tag for robots, bingbot or googlebot, no X-Robots-Tag header

    A noindex in a meta tag or in a response header keeps the page out of search results. Bing keeps it out of Copilot too.

    Sources: Bing Webmaster Guidelines, §10: NOINDEX and the other robots directives (Microsoft Bing), Bing: robots meta tags and attributes that Bing supports (Microsoft Bing), Google: block indexing with noindex (Google Search Central), Google: robots meta tag specifications (Google Search Central)

    Read today by
    Google and Bing. Bing applies the same directives to Copilot answers.
    Google Search
    Direct

    Read the check page

  • Passed Adopted robots.txt crawl policy — /robots.txt has 6 groups and 501 rules across all groups, and 1 Sitemap line.

    Observation

    HTTP 200 · text/plain · 14.2 KB · # If you would like to crawl GitHub contact us via https://support.github.com?tags=dotcom-robots # We also provide an extensive API: https:…

    Crawlers read robots.txt before they fetch a page (RFC 9309). Its Sitemap line tells them where the URL list is.

    Sources: RFC 9309 (IETF, 2022-09), Google robots.txt (Google Search Central)

    Read today by
    Google, Bing, and the AI crawlers that document robots.txt support.
    Google Search
    Direct

    Read the check page

  • Passed Adopted XML sitemap — Sitemap https://github.com/sitemap_marketing.xml parses and lists 376 URLs, none with <lastmod>, which is optional.

    Observation

    HTTP 200 · application/xml · 31.7 KB · root <urlset>

    A sitemap lists the URLs a site wants crawled. Google says a small, well-linked site may not need one.

    Sources: Google: build a sitemap (Google Search Central), sitemaps.org (sitemaps.org), Bing Webmaster Guidelines, §3: lastmod (Microsoft Bing)

    Read today by
    Google and Bing.
    Google Search
    Direct

    Read the check page

  • Passed Adopted Canonical URL — Canonical https://github.com/ matches the page reached.

    Observation

    <link rel="canonical" href="https://github.com">

    A canonical link names the one URL a page is known by. Google treats it and redirects as canonical signals.

    Sources: Google: consolidate duplicate URLs (Google Search Central), RFC 6596 (IETF), Bing Webmaster Guidelines, §6: duplicate URLs and canonicals (Microsoft Bing)

    Read today by
    Google and Bing.
    Google Search
    Direct

    Read the check page

  • Passed Adopted Documentation reachable — Documentation is reachable at /docs (standard path) and returns HTTP 200 text/html with 620 readable words.

    Observation

    tried: /docs 200 | /documentation 200 | /developers 200 | /gelistiriciler 200

    Documentation linked as service-doc (RFC 8631), or from the homepage, is how a client learns the API.

    Sources: RFC 8631 (IETF)

    Read today by
    Developers and agents integrating the API.
    Google Search
    None

    Read the check page

  • Passed Adopted HTTP caching of the homepage — The homepage sends Cache-Control "max-age=0, private, must-revalidate" and ETag, and a request with If-None-Match was answered HTTP 304 Not Modified.

    Observation

    Cache-Control: max-age=0, private, must-revalidate · ETag: W/"76b51e6294203f0db816b8b32e600faa" · Last-Modified: (none) · conditional GET with If-None-Match: HTTP 304

    Cache-Control says how long the homepage may be reused. A validator lets a crawler ask whether it changed, and a 304 answer skips the body.

    Sources: RFC 9111: HTTP Caching (IETF, 2022-06), RFC 9110, 13: conditional requests (IETF), Google: HTTP caching for crawlers (Google Search Central, 2024-12), Bing Webmaster Guidelines, §3: ETag and lastmod (Microsoft Bing)

    Read today by
    Googlebot sends If-None-Match and If-Modified-Since. Every HTTP cache reads Cache-Control.
    Google Search
    Direct

    Read the check page

  • Passed Adopted Language declaration and multilingual access — html lang="en" with 8 hreflang alternates (en-us, pt-br, es, ja, ko-kr, fr-fr), each with an absolute href, and one refers to this page.

    Observation

    html lang="en" · alternates: en-us -> https://github.com, pt-br -> https://github.com?locale=pt-br, es -> https://github.com?locale=es-419, ja -> https://github.com?locale=ja, ko-kr -> https://github.com?locale=ko-kr, fr-fr -> https://github.com?locale=fr-fr, de-de -> https://github.com?locale=de-de, x-default -> https://github.com/ · content-language: (none)

    The lang attribute names the language of the text. WCAG 3.1.1 requires it at level A. Google reads hreflang, not lang.

    Sources: WCAG 2.2, 3.1.1 (W3C), Google: managing multi-regional sites (Google Search Central), Google: localized versions (Google Search Central)

    Read today by
    Screen readers and translation tools. Google ignores the lang attribute and reads hreflang.
    Google Search
    Indirect

    Read the check page

  • Passed Adopted A real 404 for a missing page — /aws-probe-0cbf9a5e6978, a path that does not exist, answered HTTP 404.

    Observation

    HTTP 404 · text/html · first 16 KB read

    A URL that does not exist answers 404 or 410 (RFC 9110). A 200 there makes every mistyped link look like a page.

    Sources: RFC 9110, 15.5.5: 404 Not Found (IETF), Google: HTTP status codes and soft 404s (Google Search Central), Bing Webmaster Guidelines, §9: 404 for missing pages (Microsoft Bing)

    Read today by
    Google reports a missing page that answers 200 as a soft 404. Every HTTP client reads the status first.
    Google Search
    Direct

    Read the check page

  • Passed Adopted Per-page content separation (no SPA fallback) — The homepage and /about have different titles ("GitHub · Change is constant. GitHub kee…" and "About GitHub · GitHub") and different main text (12% word overlap).

    Observation

    compared / with /about · titles "GitHub · Change is constant. GitHub keeps you ahe…" / "About GitHub · GitHub" · skeleton 67% · main-text word overlap 12% · main text 2137 characters

    A route that answers every URL with the same page makes every URL look alike to a crawler.

    Sources: Google: HTTP errors and soft 404s (Google Search Central)

    Read today by
    Google flags such pages as soft 404s.
    Google Search
    Direct

    Read the check page

  • Passed Adopted Responsive markup — viewport content "width=device-width" sets width=device-width and leaves zoom on.

    Observation

    viewport meta: width=device-width

    Google indexes the smartphone rendering of a page. A viewport that blocks zoom fails WCAG 1.4.4.

    Sources: Google: mobile-first indexing (Google Search Central), WCAG 2.2, 1.4.4 (W3C)

    Read today by
    Browsers; Google's smartphone crawler renders the page with it. No Google feature reads the tag itself.
    Google Search
    Indirect

    Read the check page

Hand this report to your coding agent

The prompt lists 5 checks to fix. Each has the value the scan measured and a command that repeats the measurement. It adds the change to make where the scan could build one. After them it lists 6 agent conventions, at lower priority. The prompt is served at https://agenticwebscore.com/aws/github.com/prompt.md. The command below prints it, so you can paste the output into your agent.

curl -s https://agenticwebscore.com/aws/github.com/prompt.md

This scan cannot see which AI answers cite the site. Bing Webmaster Tools reports citations in Copilot and in Bing's AI summaries under AI Performance. Google Search Console includes AI Overviews and AI Mode in its Performance report totals, without a separate view (Google's guide to AI features). Both need a verified owner of the site.

Badge

Show this score on your own site. Press Get the badge, paste its snippet into your homepage, and press Check my badge. The directory then lists your site as a verified listing.

Agent readiness score for github.com

Read how listing works

Ship a fix, then re-scan

Measured 8 days ago. Nothing re-scans a site on its own. A scan runs once, when somebody asks for it — and that somebody need not be you. Anyone can request a scan of any public host, so this page may be the first you hear of it. After you ship a fix, press Re-scan now. A check that now passes adds its points to the score.

Why a re-scan can differ

The current rubric evaluates 3 of its 60 checks against a clock: security-txt, status-page, and inventory-freshness. They can change with no change to the site. A response slower than 12 s is reported as not measured, so the same bytes served slowly can move the score and the fingerprint.

Site profile could not be confirmed

Because we could not confirm the site profile, no profile-specific check was scored and none was turned into a recommendation. 3 profile-specific checks were reported as out of scope, not as failures. A missed optimisation is cheaper than an instruction that wastes your time. Declare the site profile and re-scan to have them measured.

  • · Two profiles are too close to separate: api (4) and saas (4), margin 0 < 2.
  • · Profile-scoped checks were not scored rather than guessed at.

Signal scores

  • API platform 4
  • SaaS 4
  • Marketplace 3
  • Services 2
  • Publisher 1

Modules

  • Every site: always in scope.
  • Sites with an API or agent surface : in scope.
    • · /.well-known/ai-catalog.json lists an MCP server card
  • Sites with a commercial signal : out of scope. The homepage shows no commercial signal.

Scan record

Target
https://github.com
Measured at
https://github.com/
Scanned
Requests
51
Duration
4.0s
Confidence
100% · high
Rubric
rubric v0.9

Fingerprint

7784eb45842e… verify this record

full hash

7784eb45842ec9cd499e72255c8b0285eee23c65c91d26ec2e4786ae365aa243

Page as an agent receives it

Words
1091
Est. tokens
1473
Markup
559.0 KB
Language
en

“GitHub · Change is constant. GitHub keeps you ahead. · GitHub · GitHub”

By dimension

Each score is the share of the in-scope web-standards points a dimension earned. A dash means the dimension has no web-standards check in scope. The agent conventions are scored apart, as their own part.

  • Discoverability

    100
    Pass
    6
    Partial
    0
    Fails
    0
    Not counted
    0

    Unscored: 0 in use, 1 not used

  • Content

    63.6
    Pass
    5
    Partial
    1
    Fails
    1
    Not counted
    3

    Unscored: 0 in use, 1 not used

  • Crawler access

    100
    Pass
    1
    Partial
    0
    Fails
    0
    Not counted
    0

    Unscored: 1 in use, 2 not used

  • API and agent

    50
    Pass
    1
    Partial
    1
    Fails
    1
    Not counted
    3

    Unscored: 0 in use, 1 not used, 1 with an error

  • Commerce

    —
    Pass
    0
    Partial
    0
    Fails
    0
    Not counted
    5

    Unscored: 0 in use, 0 not used

Request transcript 51 requests

Every fetch this scan record logged, in order. A fetch that followed redirects shows how many redirect requests it made.

48 fetches, 3 redirect requests.

  1. /

    200 559.0 KB 134ms

  2. /robots.txt

    200 14.2 KB 208ms

  3. /sitemap.xml

    406 0 B 206ms

  4. /llms.txt

    200 28.0 KB 228ms

  5. /.well-known/security.txt

    200 269 B 338ms

  6. /security.txt

    404 9 B 239ms

  7. /about

    200 291.8 KB 27ms

  8. /contact

    200 +3 redirects 12.0 KB 772ms

  9. /pricing

    200 689.8 KB 363ms

  10. /docs

    200 242.2 KB 506ms

  11. /developers

    200 159.9 KB 386ms

  12. /.well-known/api-catalog

    404 16.0 KB 321ms

  13. /openapi.json

    410 394 B 274ms

  14. /.well-known/openapi.json

    404 21 B 223ms

  15. /.well-known/mcp/server-card.json

    404 21 B 131ms

  16. /.well-known/ai-catalog.json

    200 187 B 220ms

  17. /.well-known/agent-card.json

    404 21 B 223ms

  18. /.well-known/oauth-protected-resource

    404 16.0 KB 379ms

  19. /.well-known/oauth-authorization-server

    404 16.0 KB 266ms

  20. /.well-known/openid-configuration

    404 16.0 KB 286ms

  21. /llms-full.txt

    200 28.0 KB 179ms

  22. /ai-policy

    404 16.0 KB 322ms

  23. /ai-usage-policy

    404 16.0 KB 322ms

  24. /ai

    200 248.5 KB 553ms

  25. /legal/ai-policy

    404 16.0 KB 298ms

  26. /yapay-zeka-politikasi

    404 16.0 KB 268ms

  27. /.well-known/mcp.json

    404 21 B 192ms

  28. /mcp.json

    200 30.9 KB 352ms

  29. /mcp

    200 255.8 KB 450ms

  30. /.well-known/agent-skills/index.json

    404 21 B 137ms

  31. /.well-known/agent-skills.json

    404 21 B 183ms

  32. /.well-known/skills.json

    404 21 B 195ms

  33. /.well-known/ard.json

    404 21 B 197ms

  34. /.well-known/agent-resources.json

    404 21 B 169ms

  35. /ard.json

    410 394 B 196ms

  36. /documentation

    200 181.7 KB 308ms

  37. /gelistiriciler

    200 159.7 KB 363ms

  38. /status

    200 46 B 188ms

  39. /changelog

    404 16.0 KB 337ms

  40. /releases

    404 16.0 KB 271ms

  41. /whats-new

    200 180.0 KB 419ms

  42. /sitemap_marketing.xml

    200 31.7 KB 210ms

  43. /features/ai/github-app

    200 336.4 KB 272ms

  44. /github/github-mcp-server

    200 811.0 KB 677ms

  45. /

    200 559.0 KB 358ms

  46. /

    304 0 B 43ms

  47. /aws-probe-0cbf9a5e6978

    404 16.0 KB 275ms

  48. /aws-probe-0cbf9a5e6978

    404 16.0 KB 512ms