Which paired tests reveal whether an agent understands policy applicability?

Use paired episodes to test whether changing one relevant fact changes the agent's action for the right reason.

- Move the governing event across the effective boundary. - Change the operating unit covered by the rule. - Replace an approved revision with an unapproved draft. - Remove evidence required for a permitted transition. - Change only irrelevant wording in a case note. - Rename record identifiers without changing their links.

Each pair should have a stated relationship between its expected outcomes. A change to applicability may require a different decision; a cosmetic change should preserve the decision. These expectations make the test more informative than presenting a collection of unrelated difficult cases.

MirrorFirm could generate paired episodes and private annotations explaining the decisive difference. Builders can compare the agent's selected actions, evidence requests and terminal state across the pair.

Avoid changing several causal conditions at once. If both policy authority and case evidence change, a different outcome becomes hard to attribute. Paired evaluation is most useful when the environment preserves everything except the variable under inspection and records enough of the action trace to explain the response.