Skip to content
MirrorFirm

How can a simulated environment expose the reasoning behind a customer change?

A simulated environment can separate observations from underlying state so evaluators can test how agents discover dependencies before choosing consequential actions.

A useful environment can separate its complete business state from the records available through each tool. The evaluator holds the dependency graph, while the agent discovers relevant information through actions such as `get_customer`, `list_active_services` and `read_change_policy`.

Represent an observation with fields such as `record_id`, `record_type`, `visible_fields` and `retrieved_at`. Keep evaluator-only relationships in a separate state object. This allows a test to establish whether the agent had enough information when it acted, rather than judging only the final value.

The environment can expose a customer profile immediately while requiring another lookup to find an associated service. An evaluation hook can record which facts were available before the first write. It should allow valid alternative discovery paths: a search result and a service listing may supply equivalent evidence.

MirrorFirm could scope generated environments around this distinction between available state and observed state. Evaluation teams can then compare policies for gathering information, including whether an agent stops searching once it has sufficient evidence. Resetting both business state and tool visibility keeps repeated runs comparable.

Use controlled visibility to distinguish informed decisions from lucky edits.

Audience:
Evaluation teams
Content type:
Integration
Buyer question:
What it is and why it matters

Content release: r-a220eb6e099b243b

More on Customer change handling