Claimed, Checked, Proven: three levels of evidence for agent actions

An agent calls a tool. The tool returns success. The agent tells you it’s done. Most of the time that is the end of the story. The trouble is the rest of the time, and from the outside the two look identical.

“Success” can rest on three very different kinds of evidence. We give each a name and put it next to every result, so a person reading the record knows how far to trust it. Each check carries one of three levels and says no more than its evidence: Claimed, Checked or Proven.¹

Success isn’t one thing

Take one action: an agent sets order 42 to shipped. Here are three things you might know afterwards:

1. The SQL tool returned UPDATE 1 with no error.
2. Reading order 42 back shows shipped, written by that transaction.
3. Postgres’s change log for that transaction shows exactly one changed row: order 42, status from paid to shipped.

Each is stronger than the one before. Each catches failures the one before misses. In the 301-row run, statements 1 and 2 were both true. Only 3 showed that 300 other orders had changed too. In our demo the agent said 1 row; Postgres changed 301.³

Claimed: the tool said so

Claimed is the tool’s own report: it returned success, and where a schema exists, the output matched it.¹ Every tool can reach it, because it costs nothing extra.

✓ executed · postgres.query · 10 ms
  action   act_01m4x828md88x3g70vjv9pqn8x
  check    Claimed · Matches · The tool reported success.

For a read, Claimed is usually enough. Nothing changed, and if the answer is wrong the agent’s next step tends to show it. For a write it’s thin: it tells you the tool thinks the write happened, which is exactly what the agent already told you.

What it catches: errors, timeouts, malformed responses. What it misses: everything the tool doesn’t know or doesn’t say.

Checked: we read it back

Checked means asking the target, after the tool reported success, whether the write is really there.¹ For a REST API tool with a read-back configured, that’s a read of the record the write named, compared field by field. For a Postgres single-row update by primary key, it’s the row itself: whether it was last written by this action’s transaction, and whether each column the statement set holds the value it set.¹

             ✓ Checked  Postgres shows row id = 42 in public.orders was last written by this action's transaction.

What it catches: the write that silently didn’t happen, the write another process overwrote a moment later, a trigger that changed the value you set into something else. When the read-back disagrees, the record says Doesn’t match with a field diff.¹

What it misses: anything outside the record you asked about. A read-back of order 42 confirms order 42. It can’t tell you about orders 43 to 342. And it trusts the target’s answer: a target that’s wrong on reads as well as writes can’t be caught by reading it back.

Proven: the database says what committed

Proven compares what the statement named with the rows Postgres committed in the action’s transaction, read from Postgres’s own change log.¹¹ The change log is the stream Postgres writes for replication, so it holds every committed row change, including the ones a trigger made inside the same transaction.

  check      Proven · Doesn't match · postgres.pgoutput
             ✓ Claimed  The tool reported success.
             ✓ Checked  Postgres shows row id = 42 in public.orders was last written by this action's transaction.
             ✕ Proven   Postgres changed 301 rows in public.orders. The agent reported 1.
             ✕ Proven   300 changed rows in public.orders are outside the rows the statement named (id = 42).
             · Proven   public.orders has triggers (orders_ship_open_batch). Their writes to watched tables are checked; writes elsewhere aren't visible.

What it catches: extra rows, missing rows, the wrong values, writes by triggers, rules and functions you didn’t know were there. What it misses: tables outside the watched set (the last line says so), and anything a compromised Postgres server or superuser does. It’s Postgres only, and it needs logical replication turned on.

One answer, never stronger than its evidence

A record carries one level and one result. The rule that allowed the action says which level it needs; in a policy file that’s the verify: line. The result is then one of three:

• Matches: the required level was reached cleanly, and no level found a contradiction.
• Doesn’t match: some level found a contradiction. A Doesn’t match at Checked isn’t hidden by a pass at Claimed.
• Can’t check: no level contradicted the tool, but the required evidence wasn’t there. A Postgres write that needs Proven, on a database without logical replication, is Can’t check, never Matches.¹

Only one function in Deedglass decides a Matches result, and it’s tested against an independent oracle over 1,536 cases: every combination of evidence classes at every level and every required level.² That matters more than it sounds. The easiest bug in a system like this is a code path that says “fine” when it should say “I don’t know”.

Which level to ask for

A practical default:

The last row is the honest one. Some actions can’t be checked afterwards: an email, a payment to an outside bank, a message in someone else’s Slack. For those, the evidence has to come before: a rule that limits them, a person who approves them, and a record of both.

What no level tells you

A level says whether what happened matched what was asked. It doesn’t say whether the request was a good idea. An agent that was talked into a bad refund by a crafted support ticket gets Matches, because the refund that ran is the refund it asked for. That’s what rules and approvals are for: they decide what may run, before it runs. The levels report what did.

To watch all three levels on a real Postgres in your browser, open the playground. To see which of your agent’s tables could reach Proven today, run the free scan, npx deedglass scan; it lists them, and the grade is explained here.