Skip to content
MirrorFirm

How can evaluation hooks catch consequences that occur after the conversation ends?

Virtual time and scheduled event hooks let evaluators inspect delayed consequences, including whether an accepted referral actually changes downstream account activity as intended.

Add a virtual clock and an explicit event queue to the environment. A referral episode may finish before the next reminder, queue dispatch or account review occurs. Evaluating only at conversation closure misses whether the agent's actions changed those scheduled behaviours.

Generated event records can include `event_id`, `scheduled_at`, `event_type`, `target_id`, `guard_condition` and `execution_status`. A reminder event should inspect current account state when it executes, according to the configured policy, rather than rely solely on the state when it was scheduled.

Expose an evaluator-only action such as `advance_time` that processes due events in a documented order. Capture emitted notices and queue changes in an event trace. The acting agent should not receive private expectations attached to those events.

MirrorFirm could generate initial schedules and expected invariants for bounded time windows. A hook can test whether prohibited reminders remain suppressed while a relevant hold is active and whether unrelated permitted activity continues.

Reset must restore the clock, event queue and execution history together. Otherwise, a second run may inherit consumed events and appear to pass simply because the consequence was never tested again.

Evaluate downstream effects by advancing simulated time through pending events.

Audience:
Environment builders
Content type:
Integration
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Hardship referral