How is a mismatch task different from a record-matching benchmark?
Matching benchmarks ask whether two records refer to the same thing. A mismatch task begins after that, requiring the agent to explain a difference and act on it with consequences.
**Q: Matching benchmarks already exist. Why build an environment around a mismatch?**
A matching benchmark asks whether two records refer to the same thing. A mismatch task starts after that: the records are known to relate, they disagree, and the agent must explain the difference and act on it.
**Q: What does the agent do that matching does not test?**
- Chooses which source is authoritative under a stated policy. - Distinguishes a genuine error from an expected difference such as timing, currency rounding or a tolerance band. - Decides whether to correct, reverse, query or escalate. - Writes down the reason so a reviewer can follow it.
**Q: Where does state come in?**
Every corrective action changes the environment. A credit note alters an account balance, a reopened case alters a queue, and a query alters who owns the next step. Tasks can therefore be scored on what the world looks like afterwards, not only on a predicted label.
**Q: Does an environment need many record types for this?**
Not necessarily. A narrow pair, such as purchase orders and supplier invoices in a retail operations team, can carry a full set of mismatch causes. Breadth of causes matters more than breadth of schemas.
**Q: Is the data real?**
No. A release is generated. It is shaped by observation of how teams work, and a buyer should still check the output for grounding and disclosure risk.
Mismatch tasks test diagnosis, policy-bound action and restraint, which label-prediction benchmarks on record pairs do not measure.
- Audience:
- Environment builders
- Content type:
- FAQ
- Buyer question:
- What it is and why it matters
Content release: r-a220eb6e099b243b
Related questions
- How does a mismatch appear in the systems an agent connects to?
- How do I tell if a process is a good fit for mismatch training?
- What does a billing mismatch caused by a transposed meter read look like?
- Why is a mismatch harder for an agent than a normal lookup task?
- What is a record mismatch in a business, and why does it matter for agent evaluation?
- What does a settlement-versus-ledger break look like as an environment task?