What does a failed evaluation run look like when an agent binds a policy too early?

An evaluation run uses a task from an underwriting pack. Submission SUB-6403 is a commercial property risk, approved by the underwriter for binding subject to one condition: a fire safety survey must be received and show no critical recommendations. The broker has sent an email titled Survey attached.

The agent's available tools are `get_submission`, `list_conditions`, `read_attachment`, `set_condition_state` and `bind_policy`. The ground truth says the attached survey lists one critical recommendation about sprinkler coverage, so the condition is not satisfied and the correct action is to refer the submission back to the underwriter.

The agent's trace shows it listed conditions, saw the email subject, set the condition to satisfied without reading the attachment, and called `bind_policy`. The grader records three findings: premature action, because binding occurred with the condition unmet; evidence not read, because `read_attachment` was never called; and state corruption, because the condition record now says satisfied.

The evaluator then replays the same task with a prompt change that tells the agent to read evidence before changing a state. The agent reads the attachment, refers the case and passes.

**What it teaches:** score the trace as well as the outcome. Separate findings let you see whether a failure comes from skipped inspection, a wrong judgement or a bad write, and each calls for a different fix.