Most systems that read the web for an AI agent will, sooner or later, make something up. They fetch a page, summarize it, and hand back an answer that sounds right and isn't. The BotVisibility Answer Gateway is built so that it can't — not as a matter of prompt discipline, but as a matter of construction. This is how, and here is the proof.
The guarantee, stated precisely
Every answer the gateway returns is backed by verbatim excerpts from the page it fetched. An excerpt is not a paraphrase and not a summary — it is a character-for-character span of the source, and a deterministic verifier rejects it if it isn't. The composed one-sentence answer is held to a stricter bar still: every content word in it must be covered by those verified excerpts, and its negation polarity must match the source, or the answer is withheld and only the excerpts are returned. If the page doesn't contain the answer, the gateway returns not_found. It never infers, and it never fills a gap with training data.
This is a narrow, honest guarantee, and the honesty is the point. The gateway can still miss an answer that is on the page — a recall limit. What it will not do is invent one. Fabrication and recall are different failure modes, and only one of them costs you trust.
The verifier is the trust layer, not the model
The language model's only job is to propose candidate excerpts and a short answer. Nothing it says is trusted. A deterministic verifier — about a hundred and twenty lines of string logic, no model involved — then checks every proposal: each excerpt must be a verbatim substring of the exact source lines it cites, after normalizing whitespace and quotation marks; every content token of the composed answer must appear in the union of verified excerpts; and the answer's negation must agree with the source, so a page that says an API does support retries can never be turned into an answer that says it does not. Anything that fails is dropped. On a failure the gateway retries exactly once, telling the model precisely what went wrong, and if that also fails it returns not_found. Two model calls, maximum, ever.
The consequence is that grounding is an engineering property of the verifier, not a behavior we hope the model exhibits on a good day. That distinction is what makes the number below publishable.
The proof harness: a golden set that blocks the deploy
Accuracy claims are worthless without a falsifiable test, so the gateway ships with one. The golden set is 40 curated cases across roughly fifteen documentation and content sites: 28 questions whose answers are genuinely on the page, 8 traps whose answers are genuinely not on the page, 2 pages that are already so token-lean the right move is to fetch them directly, and 2 that block AI access by policy. Every expected fragment was verified against the live extractor's real output during curation — not guessed — so the harness measures the system, not our optimism.
The scoring uses trust semantics, not vibes. The grounded rate asks: of the answers we actually gave, how many were on-topic and backed by verbatim source spans? A false positive is the only real sin — answering a trap, i.e. returning something for a page that doesn't contain it. The deploy gate blocks on a grounded rate below 99% or a single false positive. A miss on an answerable case (a false not_found) is logged as a recall warning, never a block — because declining to answer is not a trust failure.
| Case type | Count | Result |
|---|---|---|
| Answerable questions | 28 | 100% grounded & on-topic |
| Not-in-source traps | 8 | 0 answered — zero fabrications |
| Already-lean pages | 2 | Correctly told the agent to fetch directly |
| Policy-blocked | 2 | Correctly refused (unreachable) |
The result, stable across repeated runs against the live service: 100% grounded rate, zero false positives, median latency around two seconds. Not one answer to a trap. The composed answer is withheld and only excerpts returned whenever coverage isn't total — the system would rather show you the source than assert a sentence it can't fully back.
What the bakeoff found — and why it's the opposite of what we expected
Because the verifier is the trust layer, the model doesn't need to be careful — it needs to be a good reader. That inverts the usual instinct to reach for the biggest model. We ran the same 40 cases against three: Claude Haiku 4.5, Sonnet 4.5, and Sonnet 5.
| Model | Grounded rate | Answered (of 28) | Median latency | Relative cost |
|---|---|---|---|---|
| Haiku 4.5 | 100% | 28 | ~2.3s | 1× |
| Sonnet 5 | ~100% | 28 | ~2.4s | 2–3× |
| Sonnet 4.5 | 100% | 23 | ~3.6s | 3× |
The cheapest model won, and not narrowly. Haiku matched the best recall, matched grounding quality, ran at comparable latency, and cost two to three times less. Sonnet 4.5 was actually the worst on the axis that matters — it declined five cases Haiku answered and verified correctly, because it holds out for a span it's fully confident in while the verifier would have caught anything it got wrong anyway. The build spec predicted a pricier model would have better recall on messy pages. The measurement said the reverse. With a deterministic safety net under it, the right extractor is the eager one, not the cautious one.
“I don't know” is a feature
A scraper always answers. The gateway reports what actually happened. In our measurement panel, crates.io came back not_found — it is a JavaScript-rendered single-page app whose raw HTML contains nothing to ground an answer, so the gateway said so rather than guess. help.shopify.com came back unreachable — access blocked, and honored without a second, quieter attempt under a different user-agent. Neither is a failure. Both are the system refusing to manufacture certainty it doesn't have.
Why this is the foundation for the Agent Tax Index
The Agent Tax Index aggregates the gateway's receipts into a public record of what the web costs AI agents to read — raw tokens at the source versus tokens actually delivered, per domain, from live requests. That record is only as trustworthy as the claim that each receipt measures a real, grounded answer and not a fabricated one. This report is that claim, made falsifiable. The Index doesn't rest on a promise that the reader is honest; it rests on a test anyone can run.
Reproduce it
The golden set and the runner are in the repository. Point the harness at any deployment and it fires all 40 cases, scores them with the trust semantics above, and exits non-zero if the grounded rate drops below 99% or any answer to a trap slips through. Swapping one environment variable reruns it as the model bakeoff. The number in this report is not a marketing figure; it is the output of a command, and the command is yours to run.