How should an evaluation judge whether complaint actions happened on time?
Timing checks need explicit calendars, clock rules and pause conditions so evaluators can distinguish a missed obligation from an incorrectly calculated deadline.
A timeliness score is only meaningful when the task defines the clock. Evaluation teams should establish the starting event, applicable calendar, time zone and any permitted pause conditions. These belong in the supplied task rules, not in assumptions made by the scoring code.
Separate acknowledgement timing from investigation and escalation timing. They can start from different events and follow different rules. A request for missing evidence should pause a clock only when the configured policy permits it, with the required reason and supporting event recorded.
Boundary cases are especially useful. A complaint received outside business hours can test when counting begins. A pause applied after a deadline has already expired can test whether the agent incorrectly uses later events to excuse an earlier breach. Calendar transitions can expose inconsistent treatment of local dates and elapsed time.
MirrorFirm could generate cases with independently checkable deadline calculations and expected due states. Evaluators should compare both the agent's calculated deadline and its actual action time. An agent that acts promptly using the wrong calculation may still fail on another case, so accidental compliance should not conceal faulty timing logic.
Make deadline reasoning reproducible before using timeliness as a success criterion.
- Audience:
- Evaluation teams
- Content type:
- Explanation
- Buyer question:
- How to evaluate it
Content release: r-a220eb6e099b243b
Related questions
- What quality checks should developers run before training on generated complaint material?
- How can a task test escalation when an earlier case note is misleading?
- How should evaluation separate a valid complaint decision from a polished answer?
- Which environment hooks make complaint agent evaluations reproducible?
- How can complaint evidence test an agent's resistance to embedded instructions?
- How can evaluation hooks expose retry and concurrency failures?