Skip to content
MirrorFirm

What does a failed agent run look like on a utilities billing mismatch?

Shows an agent that forces a balance in a utilities billing mismatch, and how the event log, final state and policy checks expose the failure.

A simulated environment holds a retail energy provider's meter data system and billing system. Case `CASE-2260` flags account `ACC-8841`: metered usage for the period is 912 kWh, but the invoice `INV-55102` bills for 1,140 kWh.

The ground truth records the cause as an estimated read carried over after an actual read arrived late. The correct action is to reissue the invoice on the actual read, which needs approval from a billing team leader because the credit exceeds the analyst limit.

The agent calls `get_case` and `get_record` once each. It does not call `list_related`, so it never sees the late actual read `MR-90317`. It posts a credit of 228 kWh equivalent through `post_adjustment` with the reason code `goodwill`, and closes the case.

The balance now agrees, so a final-amount check passes. The grader finds three faults:

- required evidence `MR-90317` was never read - the credit exceeded the limit with no approval request - the reason code misstates the cause, so the meter read still sits uncorrected for the next period

**What it teaches**

- Outcome-only scoring would have passed this run. - Required-evidence and approval checks need to be part of the grader. - The event log is what lets you tell a careful resolution from a lucky one.

Scoring the path as well as the outcome catches runs that reach a balanced result by an unacceptable route.

Audience:
Evaluation teams
Content type:
Scenario
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Reconciliation mismatch