TL;DRif you keep one rule from this field note decide what evidence would prove the result before accepting an AI’s claim that the work is done.
AI is very good at sounding finished.
It can report that it found the root cause, updated the relevant rules, handled the edge cases, and verified the result. The summary may be clear, specific, and completely sincere.
That still does not make it true.
A completion report is a claim. So is “implemented,” “fixed,” “verified,” and “all tests pass.”
The useful question is not whether the explanation sounds convincing. It is whether the evidence could have caught the original failure.
We changed how our AI operating system defines “done” after one nearly perfect result showed us exactly why that distinction matters.
The Work Looked Finished
We had been improving the part of our AI operating system that routes requests.
Its job is to understand what someone wants, determine where the work belongs, respect safety boundaries, and send the task to the right destination.
One subtle rule needed clarification:
If a user names the wrong tool or surface, should the system reject the request—or redirect the legitimate work to the safe place that actually owns it?
The updated routing logic looked sound. Its explanations were reasonable. The safety boundary appeared intact.
We were close to treating the work as finished.
Then we ran a blind evaluation.
The Test That Said Otherwise
The evaluation contained 26 requests covering different destinations, permissions, and boundaries. The expected answers were withheld so the router could not simply mirror them.
It passed 25.
The remaining failure looked responsible on the surface.
A request named the wrong—and unsafe—execution surface for a legitimate piece of work. The system correctly recognized that the named surface should not perform the action, so it refused.
That protected the boundary.
It also failed the task.
A safe destination already existed. The router should have redirected the work there. Refusal was appropriate only if the requested connection or action should not exist anywhere.
The system’s explanation sounded careful and safety-conscious. Without the blind test, we could easily have accepted it.
The work was not done.
The Receipt
We clarified the rule:
Naming the wrong surface does not change who owns the work. Redirect to the safe owner when one exists. Refuse only when the requested connection or action itself must not exist.
Then we reran the same evaluation against the same 26 cases.
Before the change:
25 passed out of 26.
After the change:
26 passed out of 26.
That was the receipt.
The system did not get credit because it described the change confidently. It got credit because the exact check that exposed the failure no longer failed.
Why the Check Had to Be Capable of Failing
Imagine that we had changed the routing rule and run a different set of easy tests afterward.
They might all have passed.
That would prove that the system could handle those easy cases. It would not prove that the original boundary failure was gone.
A useful receipt must be capable of turning red.
For a bug, that usually means reproducing the exact symptom before the fix and repeating the same observation afterward.
For a router, it means rerunning the same blind cases.
For a document, it means rendering and inspecting the actual pages—not merely checking that the file opens.
For a spreadsheet, it means recalculating and reconciling the numbers—not merely confirming that no formula shows an obvious error.
The receipt has to operate at the same layer as the claim.
Match the Receipt to the Claim
If the claim is that a bug is fixed
The exact symptom fails before the change and passes afterward.
If the claim is that a router chooses correctly
The same blind routing cases are graded before and after the rule change.
If the claim is that a spreadsheet is correct
The totals are recalculated, reconciled, and checked against representative records.
If the claim is that a page looks right
The page is inspected in a browser at the relevant screen sizes.
If the claim is that a document is ready
The rendered pages are inspected for layout and content.
If the claim is that research supports a conclusion
Current primary sources are checked to confirm that they directly support the claim.
If the claim is that a request was fulfilled
The result is compared separately against the original requirements.
A generic green check is not enough.
A passing code test does not prove that a page looks right. A screenshot does not prove that stored data is correct. A citation list does not prove that its sources support the conclusion.
The evidence must match what “done” is supposed to mean.
Separate a Diagnosis From a Hypothesis
Sometimes the exact failure cannot be reproduced immediately.
It may be intermittent, tied to a specific device, or dependent on information the AI cannot access.
That does not give the system permission to turn its first plausible explanation into a confirmed diagnosis.
An honest progress report separates:
- what was directly observed;
- what remains uncertain;
- the strongest current hypotheses;
- and the smallest additional test that could distinguish between them.
Instrumentation is progress. A confirmed diagnosis is a different claim.
Review Quality and Request Fit Separately
Technically excellent work can still solve the wrong problem.
Our routing change needed two different reviews.
Implementation quality
Did the new rule remain safe, consistent, and testable?
Request fidelity
Did it actually deliver the required behavior: route legitimate work to the safe owner instead of refusing unnecessarily?
Passing only the first review could leave the system safe but helpless.
Passing only the second could make it useful but unsafe.
The work was complete only when it satisfied both.
A Copyable Evidence Receipt
Use this on one AI-assisted task:
Claim:
What exactly is the AI claiming is complete or correct?
Starting observation:
What repeatable check demonstrates the current problem?
Weakest assumption:
What is most likely to make the proposed change wrong?
Change:
What was modified, and what was intentionally left unchanged?
Repeated observation:
What happened when the exact starting check was run again?
Quality review:
Is the result correct, maintainable, tested, and safe?
Request-fidelity review:
Does it deliver the requested outcome without changing the target?
Residual risk:
What remains uncertain, untested, or dependent on human approval?The receipt does not need to be elaborate.
It can be one focused test, a rendered page, a reconciled total, a before-and-after screenshot, or a set of blind cases with an objective score.
It just needs to be capable of catching the thing the AI claims it fixed.
“Done” Became Something We Could Inspect
Our AI operating system still produces completion summaries. They remain useful for explaining what happened.
We just no longer treat the summary as the proof.
The proof is the observation that exposed the failure, the change that addressed it, and the same observation passing afterward.
That is how 25 out of 26 became 26 out of 26—and how “done” became something we could inspect instead of something we had to trust.
What receipt could catch the original failure in the AI task you repeat most?