Skip to content
MirrorFirm

Why not just test agents with a policy document and a yes/no approval check?

Answers a common evaluation-team objection by describing what a static policy and a binary grader miss when authority changes over time and across roles.

**Short answer:** a policy document tells an agent the rules, but it does not say who currently holds which authority, and a yes/no check cannot tell a correct approval from a lucky one.

**What a static policy misses**

- **Who holds authority today.** Limits attach to people and roles, and holders change through leave, acting arrangements, promotion and revocation. - **How limits are measured.** Aggregate exposure, cumulative spend over a period or linked requests may count together, which is only visible in the records. - **Conflicts and exclusions.** A requester may not approve their own item, and a reviewer may be excluded for a related interest. - **Evidence of routing.** Earlier approvals, returns and comments change what the next approver should do.

**What a binary check misses**

A grader that only checks the final answer rewards an agent that approved the right request for the wrong reason. It also cannot distinguish a correct escalation from an unnecessary one, or a justified refusal from a timid one.

**What helps instead**

A task pack can pair each request with the authority records valid at the time, the expected route and a rubric for the reasoning. A simulated environment can add tool actions and log every state change, so a grader scores the path as well as the result.

As with all generated material, buyers should check how well the records are grounded and whether any disclosure risk remains.

Authority is state that changes, so evaluating it needs linked records and time, not only a policy text and a binary answer.

Audience:
Evaluation teams
Content type:
FAQ
Buyer question:
What it is and why it matters

Content release: r-a220eb6e099b243b

More on Delegated approval