Snippet and AI grounding directives

Sources

• Bing Webmaster Guidelines, §10: NOARCHIVE, NOCACHE and NOSNIPPET, Microsoft Bing
• Bing: robots meta tags and attributes that Bing supports, Microsoft Bing
• Google: robots meta tag specifications, Google Search Central

What is measured

The scanner reads the homepage response it already holds ($home in the probe map), so the check makes no request of its own. It reads the robots directives of <meta name="robots">, <meta name="bingbot"> and <meta name="googlebot"> in the document head and of the X-Robots-Tag header, the way indexing-directives reads them, and counts the elements that carry a data-nosnippet attribute, read as an attribute name of a tag and never inside another attribute's value, outside comments and the bodies of scripts, styles, templates, titles, textareas and noscript. Present = noarchive, nocache, nosnippet or max-snippet:0 in any of them, or at least 1 element with data-nosnippet. The detail names each one, where it was found, and its effect. Under Bing Webmaster Guidelines §10, noarchive keeps the page out of Copilot answers and grounding results; nocache lets Copilot use only the URL, the title and the snippet; nosnippet and data-nosnippet keep Bing from showing a snippet and may limit the quality of Copilot's citations. Under Bing's page on the robots meta tags it supports, max-snippet:0 means Bing shows no text snippet, and with both noarchive and nocache Bing applies nocache. A directive found only for googlebot is named as one Bing does not read, and noarchive or nocache for googlebot as one Google does not use either. Not used = none of them. Not measured = a homepage that answered a status other than 2xx, or did not answer, and a directive found on a homepage with fewer than 25 words of visible text, which may be an interstitial such as a bot challenge. Neither state is a failure, and the check earns no points.

Reproduce it yourself

You do not have to take our word for any of this. Run the same measurement against your own domain:

              HOMEPAGE=https://example.com/
curl -sSL -o /dev/null -D - -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$HOMEPAGE" | tr -d '\r' | grep -i '^x-robots-tag:'
curl -sSL -A 'AgenticWebScore/0.3 (+https://agenticwebscore.com/bot)' -H 'Accept-Language: en' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.7' "$HOMEPAGE" | perl -0777 -ne 'my $h = /^(.*?)<\/head>/si ? $1 : substr($_, 0, 8192); while ($h =~ m{<meta\b[^>]*>}gi) { my $t = $&; next unless $t =~ /\bname=["\x27]?(robots|bingbot|googlebot)["\x27]?[\s>\/]/i; my $n = lc $1; my ($c) = $t =~ /\bcontent=["\x27]([^"\x27]*)/i; print "meta $n: ", ($c // ""), "\n" } my $body = $_; $body =~ s{<!--.*?-->}{}gs; $body =~ s{<(script|style|template|title|textarea|noscript)\b[^>]*>.*?</\1\s*>}{}gsi; my $n = 0; while ($body =~ m{<[a-z][a-z0-9-]*\b((?:[^>"\x27]|"[^"]*"|\x27[^\x27]*\x27)*)>}gi) { (my $t = $1) =~ s{"[^"]*"|\x27[^\x27]*\x27}{}g; $n++ if $t =~ /(?:^|\s)data-nosnippet(?=[\s=\/]|$)/i } print "data-nosnippet: $n\n"'
            

The fix

This page shows no artifact for this check, because the change depends on something only the site owner can supply. The note below says what to do.

Related checks