Skip to content
MirrorFirm

How do I inspect the quality of generated authority rules and approval cases?

Quality inspection covers internal consistency, boundary coverage, plausibility to your own approvers and grounding or disclosure risk, with a small human review of sampled cases.

Quality for this workflow is mostly about whether the cases are internally correct and whether they test the right boundaries. Four inspections are worth doing on any sample.

**Internal consistency.** Pick a handful of requests and trace each one by hand. Does the matching rule exist? Is the stated expected approver actually authorised on the decision date? Are delegation windows and role-holder periods free of gaps and overlaps that nobody intended?

**Boundary coverage.** Look at the amounts. A useful set has cases below, exactly at and just above each limit, plus cases where risk flags move the request into a higher tier. If every amount sits comfortably inside a band, the pack will not catch off-by-one logic.

**Plausibility.** Ask someone who approves things in your organisation to read ten cases. They should be able to say what they would do and whether the expected answer is sensible. Cases they find odd are worth discussing, because they may be edge cases or generation errors.

**Grounding and disclosure.** The output is generated, and that alone does not settle whether it is safe to use. Ask how structure and wording were derived, and check for any figure, phrase or sequence that resembles material you hold.

Finally, check that answer keys are separable from inputs and that the cases vary enough that an agent cannot succeed by memorising a single matrix.

A short, structured review of consistency, boundaries and plausibility tells you whether generated approval material is worth building an agent evaluation on.

Audience:
Agent developers
Content type:
Explanation
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Delegated approval