Skip to content
MirrorFirm

How do I check that generated approvals are consistent with their rules?

Recompute who held authority at each decision's timestamp from the tables, then compare with the actor across valid decisions, labelled invalid ones and unlabelled edge cases.

The first quality test is mechanical: for every decision in a sample, resolve who held authority at that timestamp and compare it with the actor.

That means joining the decision to the role assignment valid at that time, any delegation in force, the applicable authority rule and the limit utilisation before the decision. A well-formed release should let you do this with ordinary joins and no guesswork.

Run three passes:

- **Valid decisions.** Each should be explainable by exactly one rule, with amount and risk inside its bounds. If two rules could apply, the release should say which takes precedence. - **Labelled invalid decisions.** Each should break at least one stated condition, and the label should name which. Check the label against your own resolution, not just the generator's. - **Unlabelled edge cases.** Look for amounts exactly at a limit, delegations that end on the day of decision, and approvals made before a role assignment began. Boundary handling should be consistent.

Then read a handful of cases end to end as a person would. Do the request narratives match the amounts? Do escalation reasons make sense for the role? Do timestamps run in plausible business hours and sequences?

A release that passes the mechanical tests but reads implausibly can still teach an agent odd habits. One that reads well but fails joins will break your grader.

Because everything is generated, also review grounding: which assumptions about the organisation reflect how businesses operate and which were filled in by the generator. Ask MirrorFirm to describe these for your sample.

If authority can be recomputed with ordinary joins and your result matches the labels, the release is mechanically sound enough to build graders on.

Audience:
Environment builders
Content type:
Explanation
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Delegated approval