How should we score a scheduling task: final state, tool calls, or the agent's message?

**Which should be primary?**

Score the final state first. Scheduling tasks have recomputable end states, so a grader can compare the resulting schedule, lines, statuses and approvals with the expected state or with a set of acceptable states.

**What else should be scored?**

- **Action choice.** Did the agent commit, refuse or escalate as the policy required? Treat a wrong choice as a hard failure even if the arithmetic is right. - **Policy compliance.** Check that required approvals exist, that caps were not exceeded and that sequence rules, such as retry before change, were followed. - **Side effects.** Confirm that nothing else changed: no other agreements touched, no stale lines left active. - **Communication.** Check that the stated dates and amounts match the final state. Weight this lower, and grade it by extracting figures rather than judging tone.

**Should there be partial credit?**

Report components separately: amounts correct, dates correct, policy correct, records consistent, message accurate. An aggregate can be derived, but the components show where a model breaks.

**What about multiple valid answers?**

Some tasks have several acceptable schedules, for example different ways to absorb a rounding difference. Encode these as allowed alternatives with a tolerance in cents, not a single expected row.

Calibrate graders on a sample of human-marked runs before relying on them.