Agent readiness: the 71 was a false negative

Cloudflare's isitagentready.com scan of astrarad.com, read on Sat Aug 15, 2026. The reported score was 71, "Level 1 Basic Web Presence", with four failures. Three of the four are a defect in the scanner. This page proves it, and closes the fourth as a decision rather than a chore.

What it is

The forensic read on a scan that looked alarming and was not. Scenario: Rafael runs the agent readiness scan, sees 71 and "Level 1", and reasonably concludes the site's crawl rules are broken. Before anyone edits robots.txt, this page shows what a clean re-scan actually returns, why the failing run failed, and what the single genuine gap is worth. Companion to seo-external-reports-remediation-2026-08-09.html.

Level 5, "Agent-Native" 13 of 14 scored checks pass. The site needs no change. The only real gap, DNS-AID, is being declined on purpose.

The two scans, four minutes apart

The reported run and a clean re-scan through the same scanner's API with cache bypass, run four minutes later against the identical unchanged site:

Check	Reported run	Clean re-scan	Real?
robots.txt	could not fetch	pass	Scanner defect
AI bot rules in robots.txt	cannot check	pass	Scanner defect
Content Signals in robots.txt	cannot check	pass	Scanner defect
Web Bot Auth	fail, 0/2 category	neutral	Not scored either way
DNS-AID	fail	fail	Genuine
Sitemap, Link headers, Markdown negotiation	pass	pass	Fine
API catalog, OAuth, OIDC, auth.md, MCP card, A2A card, Agent Skills, WebMCP	7/7	pass	Fine

What the report said

Level 1, Basic Web Presence
Discoverability 2/4
Bot Access Control 0/2
10 of 14 scored checks passing

What the site actually is

Level 5, Agent-Native
Discoverability 3/4
Bot Access Control 2/2
13 of 14 scored checks passing

Why one flake cost twenty points

The evidence chain

1. The origin is healthy. /robots.txt serves HTTP 200, text/plain; charset=UTF-8, 1496 bytes, in 45 to 140 ms, with Cache-Control: no-store. It already carries the Content-Signal line and all nine Disallow rules. Measured repeatedly, from a normal host, before and after the failing scan.

2. The three robots checks share a single fetch. robots.txt is fetched once, outside any per-check timer, and all three checks are pure parses of that one body. That is why every one of them reports Completed in 0ms, on success exactly as on failure. The 0ms is not a symptom of the failure, it is the normal reading.

3. The error string identifies the caller, not the origin. The operation was aborted is workerd's exception for a bare AbortController.abort() with no reason given. It is raised inside Cloudflare's own scanner Worker and carries no information whatsoever about what our server did.

It is specifically none of the plausible origin-side stories:

Not the Workers subrequest cap, which raises Too many subrequests.
Not AbortSignal.timeout(), which appends due to timeout
Not a rate limit, which would return 429
Not the CSP header, which is a browser directive and is ignored by fetch()
Not compression and not IPv6: astrarad.com has no AAAA record at all

4. The arithmetic confirms it. The scanner scores passed / (checks minus neutral). That denominator is 14, because Web Bot Auth is neutral. The reported 71 is exactly 10/14. Restore the three robots checks and it is 13/14, or 93. The reported number landing precisely on 10/14 is independent confirmation that the only thing wrong was that single fetch.

5. The level collapse follows. robots.txt gates Level 1 and the other two gate Level 2, so one aborted fetch drops the ladder from 5 to 1. "Level 1 Basic Web Presence" was an artifact of the cascade, not a description of the site.

The one real gap: DNS-AID, declined

DNS-AID records genuinely are absent. _index._agents, _a2a._agents and _mcp._agents under astrarad.com all return NXDOMAIN for SVCB, HTTPS and TXT. The decision is to leave it that way.

Why not to publish them

The spec is not settled. draft-mozleywilliams-dnsop-dnsaid is an individual submission, not adopted by the dnsop working group, and expires 28 Nov 2026.
Nothing in it is registered. None of its SvcParamKeys (cap, bap, well-known) and neither ALPN identifier (mcp, a2a) are IANA-assigned. Real records have to use private-use keyNNNNN numbering that any other implementation is free to number differently. "DNS-AID compatible" today means compatible with one reference implementation.
The scanner asks for a shape the spec deleted. Draft -02 removed _a2a._agents and _mcp._agents entirely, keeping only _index._agents. The scanner still probes all three because the old shape is what the launch material shows. So publishing what the current spec says may not turn the check green, and publishing what turns it green means publishing a deprecated shape.
It cannot express our endpoint anyway. SVCB carries host and port only. The path /mcp is not representable in the record; a client still has to fetch a descriptor to learn it.
The registrar cannot do it. GoDaddy's DNS editor has no SVCB or HTTPS row, so this would mean delegating _agents to another provider or moving the whole zone, which is production DNS movement in exchange for a checkbox.

DNSSEC is not the blocker people assume. Draft -02 downgraded it from a requirement to a SHOULD and states DNS-AID is deployable without it. The unsigned zone costs nothing here.

Revisit when dnsop adopts the draft, or the record shape stabilises at -03 or later, or a customer or partner actually asks for it. Until one of those happens this is roughly 7 points on one vendor's scanner in exchange for permanent DNS complexity in an already-deprecated shape.

Two things that stay exactly as they are

Web Bot Auth stays a 404

/.well-known/http-message-signatures-directory is where an AI bot operator publishes the public keys it signs its outbound crawler requests with, so origins can verify the crawler is who it says. AstraRad is the origin, not the crawler. Publishing a key directory would be a claim to operate a signing bot fleet that does not exist.

It is scored neutral, so leaving it 404 costs nothing. That neutral status is exactly what makes the 14-check denominator work, and therefore what makes the 71 equal 10/14.

This is the honesty line the rest of the site already holds. Every other passing check is backed by something real: a working MCP server, 6 WebMCP tools, 4 published agent skills, live OAuth and OIDC discovery, a real API catalog, genuine markdown twins. A key directory would be the first file on the site describing a capability nobody has, and unlike a decorative meta tag it is machine-actionable: a verifier could try to validate signatures against keys we never sign with. Same rule that keeps prices off the public site, keeps physician names out of the schema, and dropped the empty sameAs rather than publishing a meaningless one.

robots.txt gets no named AI crawler groups

The scan's advice is to add explicit User-agent: GPTBot, Claude-Web, Google-Extended entries. Following it would make the file worse.

Under RFC 9309 a crawler selects the single most specific matching user-agent group and obeys only that group. Groups do not merge and do not inherit. The moment User-agent: GPTBot exists, GPTBot stops seeing the User-agent: * group, and with it loses all nine Disallow rules covering /portal, /reader, /admin, /sales, /login, /api, /django-admin, /kit and /book-a-call, plus the Content-Signal line, which is only honoured inside the group carrying it.

Staying safe would mean restating all ten lines inside every named group and keeping N+1 copies in sync forever. One missed edit silently opens a private route to the largest AI crawler on the internet. There are no points to win: both checks already pass. AstraRad blocks no AI crawler and wants all of them indexing and grounding on the marketing pages, so there is no per-crawler policy to encode in the first place. The file's own comments already say this.

How to re-scan properly

The public UI caches. The API with noCache is the only trustworthy read:

curl -s -X POST https://isitagentready.com/api/scan \
  -H 'Content-Type: application/json' \
  -d '{"url":"https://astrarad.com","noCache":true}' \
| jq -r '"level=\(.level) \(.levelName)",
         (.checks|to_entries[]|.key as $c|.value|to_entries[]|"\($c)/\(.key): \(.value.status)")'

Expect level=5 Agent-Native, every check pass except dnsAid failing, and webBotAuth plus the five commerce checks neutral. ~/.claude/skills/agent-ready/scan.sh wraps the same call.

The standing rule

A robots.txt failure reported at 0ms with "The operation was aborted" is a scanner-side abort. Re-run the scan. Do not edit robots.txt, netlify.toml headers or cache directives in response to a client-side AbortError raised inside somebody else's Worker. If the origin needs checking, check it directly:

curl -sS -o /dev/null -D - -w 'code=%{http_code} time=%{time_total}\n' https://astrarad.com/robots.txt

Verified 15 Aug 2026 against the live site and the scanner API. Sources: draft-mozleywilliams-dnsop-dnsaid, RFC 9460, RFC 9309, IANA SVCB parameter registry, Cloudflare Workers limits.

One caveat kept honest: the claim that GoDaddy's DNS editor offers no SVCB row is high confidence from documentation rather than observed in the panel. It changes nothing, since the decision is to skip DNS-AID regardless. Worth a one minute check in the panel only if that decision is ever reopened.