Skip to content
MirrorFirm

How should follow-up quality be scored when several responses are acceptable?

A useful rubric separates mandatory business constraints from flexible wording, allowing different valid responses without accepting incorrect commitments or missing information.

**Does a correct response need to match a reference message?**

No. Several messages can satisfy the same business requirement. Exact wording comparisons can penalise valid responses while overlooking a polished message that contains an unsupported claim.

**What should the rubric check?**

Define required meaning: the outstanding item, the requested next step and the correct route for responding. Define forbidden meaning separately, such as asserting that a penalty has been applied when the record contains no such event. Where the policy requires a particular notice, identify that requirement explicitly.

**How should partial success be handled?**

Separate action eligibility, recipient selection and message content. A well-written message to an unauthorised recipient should not pass because its tone is appropriate. Likewise, a correctly routed message can still omit the information needed to resolve the delay.

**What could MirrorFirm generate?**

A task pack could include acceptable semantic variants, disallowed assertions and evidence references for each scoring condition. Developers can use these to inspect automated grading disagreements and distinguish genuine business errors from harmless differences in phrasing.

Score semantic obligations and prohibited claims so evaluation remains useful across varied agent writing styles.

Audience:
Agent developers
Content type:
FAQ
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Overdue follow up