Skip to content
MirrorFirm

Why are mismatch tasks a good test for business agents?

Answers the common buyer question of why reconciliation work, rather than simpler lookups, exposes weaknesses in agents that handle structured business data.

**Why not just test lookups and summaries?**

Lookups show whether an agent can fetch a field. Mismatch tasks show whether it can decide which fields matter, in what order, and what to do with the result.

**What does a mismatch task exercise that other tasks do not?**

- **Hypothesis testing.** The agent must propose a cause and then look for evidence that would rule it out. - **Source authority.** Often one record is correct and the other is stale. The agent must know which, and the policy usually says so. - **Action limits.** Fixes sit behind permissions, such as who may post an adjustment or reverse a payment. - **Escalation.** Some cases should not be fixed by the agent at all, and stopping is the right answer.

**Is there one right answer?**

When a case is generated, its root cause, correct fix and acceptable alternatives can be recorded alongside it. That makes automated scoring possible without relying only on a model grader.

**Does this replace real-world testing?**

No. Generated cases are synthetic and informed by how businesses work. They suit controlled, repeatable evaluation. A buyer should still inspect how well the cases match their own processes and check for disclosure risk before relying on them.

Mismatch tasks separate agents that retrieve data from agents that investigate, apply policy and stop when they should.

Audience:
Evaluation teams
Content type:
FAQ
Buyer question:
What it is and why it matters

Content release: r-a220eb6e099b243b

More on Reconciliation mismatch