Skip to content
MirrorFirm

How can buyers tell whether an agent follows policy instead of superficial cues?

Paired cases change one decision-relevant fact while preserving surrounding context, revealing whether an agent responds to the condition that actually controls follow-up.

An agent can appear competent by learning that old invoices usually need reminders. A stronger evaluation checks whether its action changes when the policy-relevant evidence changes, even though the invoice age remains identical.

Use paired cases with a controlled difference. One case can contain an authorised contact restriction and the other an expired restriction. The expected action changes because the restriction changes. A separate pair can alter an irrelevant presentation detail, such as the ordering of account notes, while keeping the expected action stable.

MirrorFirm could generate these pairs with a manifest describing the changed field, the governing rule and the expected effect on the decision. Keep that manifest outside the agent's observation surface so it supports scoring without revealing the answer.

Evaluate both kinds of behaviour: responsiveness to meaningful changes and resistance to irrelevant ones. Review the agent's cited evidence as well as its final action. A correct outcome reached through an unrelated rationale may fail when the same policy condition appears in a different business context.

Measure sensitivity to controlling evidence and stability against irrelevant changes within the same evaluation design.

Audience:
Agent developers
Content type:
Explanation
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Overdue follow up