Operational article · published

Check AI Output for Unsupported Claims

Audit AI output claim by claim: map consequential statements to evidence, record support status, recompute derived claims, and block unsupported wording.

Reviewed 2026-09-21 · National guidance, Austin proof
01

The task and the failure mode

Built for: Engineers, product owners, analysts, and content teams operating AI-assisted workflows where generated factual, numerical, comparative, policy, or recommendation statements can influence a customer, employee, or downstream business decision.

An unsupported-claim check is a semantic evidence test, not a citation-presence check. A response can contain a real URL, valid JSON, or a correctly shaped tool result and still make a statement the evidence does not support. The release unit should be the consequential claim: the exact subject, predicate, qualifiers, numbers, time period, geography, population, and causal or comparative wording that a reader could act on. Each such claim needs an explicit support state and a disposition before the output ships.

Frame

Decision brief

Split consequential output into atomic, checkable claims. Give each claim an ID, exact text span, claim type, risk level, and the source or calculation it is supposed to rely on. Do not let one paragraph hide several independently testable assertions.

Judge support against the exact wording, not the topic. A citation that mentions the same company, product, study, or metric does not pass a stronger claim. Check scope, date, population, units, qualifiers, comparison basis, and whether the source actually entails the statement.

Make unsupported states operational. Ship supported claims as written; narrow or qualify partially supported claims; recompute derived claims; remove contradicted or unfound claims; and escalate high-consequence unresolved claims. Recheck the edited output rather than assuming the revision is safe.

Ask

Questions to answer before changing the system

  1. 01Which statements would materially change a user or business decision if they were wrong?
  2. 02What evidence is admissible for each claim type, and what freshness or version cutoff applies?
  3. 03Does the evidence support the exact subject, predicate, magnitude, qualifier, time period, geography, and comparison in the output?
  4. 04Can every numerical or derived statement be reproduced from preserved inputs and a visible calculation rule?
  5. 05How does the system distinguish no support found, retrieval failed, sources conflict, and a claim that is outside the available evidence?
02

Workflow

  1. 01Freeze the evaluation context: output text, model and prompt version, retrieval or tool results, source set, timestamps, and any policy or business rules that constrain what may be stated.
  2. 02Atomize the output into consequential claims. Record claim_id, exact span, claim_type, risk, and intended evidence dependency. Keep non-factual style or preference statements separate from factual support checks.
  3. 03Attach candidate evidence to each claim. Preserve source identity, date or version, URL or record ID, the exact supporting passage or field, and any scope limitations needed to interpret it.
  4. 04Assign a support state using the exact wording: SUPPORTED, PARTIAL, CONTRADICTED, NOT_FOUND, or NOT_APPLICABLE. A topical match is not enough; modifiers such as always, best, caused, doubled, in 2026, or for all customers require support at that same strength and scope.
  5. 05Recompute derived claims independently. For percentages, rankings, deltas, totals, or comparisons, preserve the source inputs and calculation rule, then verify the output value and units rather than trusting a cited number nearby.
  6. 06Apply a disposition: PASS, NARROW, RECOMPUTE, REMOVE, or ESCALATE. Regenerate or edit the affected sentence, then rerun the claim check on the final text so the repair itself does not introduce a new unsupported statement.
  7. 07Run regression evals with known supported, partially supported, contradicted, missing-evidence, conflicting-source, stale-source, and adversarial examples. Include human expert review for consequential cases and keep failures as future regression fixtures.
03

Evidence to retain

  • The exact released output plus model, prompt, retrieval, and tool versions needed to reproduce it.
  • A claim ledger with claim_id, exact text, claim type, risk, evidence IDs, support state, disposition, reviewer or evaluator, and review timestamp.
  • Source snapshots or durable record identifiers with the passages or fields used for support, plus source dates and version boundaries.
  • Calculation receipts for derived claims, including inputs, formula or rule, units, rounding, and recomputed result.
  • Conflict records when sources disagree, including which source was preferred, why, or why the claim remained unresolved.
  • Escalation receipts for high-consequence unresolved claims rather than a silent fallback to model confidence.
  • Eval cases and regression results showing how the checker behaves on supported and deliberately unsupported examples.
Sample

Worked unsupported-claim review: five decisions

Citation exists, wording is stronger than the source
Output says “The change doubles conversion.” The source says only that conversion may improve and contains no measured lift. Mark the quantified claim PARTIAL or NOT_FOUND and remove the number and causal certainty unless direct evidence exists.
Right metric, wrong scope
One internal pilot reports 30% lower handling time, but the output says all customers cut handling time by 30%. Mark PARTIAL and rewrite to the measured pilot scope or remove the generalized claim.
Inputs are cited, arithmetic is wrong
The cited inputs are 80 and 100, but the output reports a 30% decrease. Recompute the derived claim: the decrease is 20%. Mark the original value CONTRADICTED, correct it, and recheck the surrounding wording.
Sources conflict
Current policy documentation and an older cached page disagree. Do not choose the convenient source. Mark the claim unresolved, determine source authority and version, or escalate before making the policy statement.
Retrieval failed
The output says “No evidence exists,” but the retrieval job timed out. Retrieval failure is not evidence absence. Report the state as unknown, retry the evidence step, or escalate.
04

Unsupported-claim release gate

  • Consequential output has been split into atomic claims rather than reviewed only as paragraphs.
  • Each factual or derived claim has an admissible evidence dependency or is explicitly marked unknown or unresolved.
  • Citation presence is never treated as proof that the cited source entails the wording.
  • Dates, units, populations, geographies, comparisons, and strong modifiers are checked at the same scope as the claim.
  • Numerical and derived claims are independently recomputed from preserved inputs.
  • Retrieval failure, no evidence found, conflicting evidence, and source staleness are represented as distinct states.
  • PARTIAL, CONTRADICTED, and NOT_FOUND claims cannot silently pass through the release path unchanged.
  • Any edited or regenerated sentence is checked again after disposition.
  • High-consequence unresolved claims have a named escalation path and retained evidence packet.
  • Regression evals include typical, edge, adversarial, and deliberately unsupported cases.
Measure

What to measure—and what it does not prove

  • Primary quality measure: released consequential claims that pass the defined support and disposition gate, reported by claim type and risk tier rather than as one opaque quality score.
  • Seed known unsupported and partially supported claims into evals and track false-pass rate separately from false-block rate so a stricter checker is not mistaken for a better checker by default.
  • Track unresolved and escalated claims by reason: missing evidence, stale evidence, conflicting sources, failed retrieval, or unsupported derived calculation.
  • Track review latency and human workload separately from factual support quality; faster review does not prove better grounding.
05

Boundaries and caveats

Structured Outputs can constrain an API response to a schema, but schema adherence is a formatting and contract property. It does not establish that each value is factually supported by the underlying evidence.

Automated graders can share blind spots with the systems they judge. Use deliberately constructed counterexamples and human expert review for consequential domains instead of treating one model score as ground truth.

A source can itself be stale, wrong, ambiguous, or too narrow. Claim support and source quality are separate judgments and both need provenance.

For legal, medical, financial, safety, employment, or other high-consequence decisions, claim checking does not replace qualified human review or the organization’s applicable policy and compliance controls.

06

Primary sources

  1. OpenAI API: Structured model outputsdevelopers.openai.com
  2. OpenAI API: Evaluation best practicesdevelopers.openai.com
  3. OpenAI API: Function callingdevelopers.openai.com
  4. NIST: AI Risk Management Frameworkwww.nist.gov
Next

Start with one bounded case

Choose one recent production output and split it into consequential claims. For each claim, record the exact wording, evidence dependency, supporting passage or field, support state, and disposition. Recompute derived numbers, repair any PARTIAL, CONTRADICTED, or NOT_FOUND claim, then rerun the check on the final output and save the case as a regression fixture.