Skip to content
MirrorFirm

How can resellers keep evaluation answers out of customer-visible task inputs?

Evaluation packaging should separate model-visible evidence from scoring material and inspect accidental clues introduced by filenames, metadata and derived labels.

**What counts as answer leakage in a handover pack?**

Leakage occurs when the task input exposes information the model is supposed to infer. A field such as `expected_receiving_team` is an obvious case. A filename containing the rejection reason can create the same problem even if the record body is clean.

**Why is this a reseller concern?**

Repackaging can accidentally merge evaluator metadata into customer-visible rows. Catalogue previews, export transformations and convenient flattened tables can reveal solution fields that were originally separate from the evidence.

**What should buyers inspect?**

Ask for a field-level visibility contract that distinguishes task inputs, evaluator-only material and descriptive metadata. Inspect the actual exported files, including paths and partition names, rather than relying solely on schema descriptions.

**How can the separation be tested?**

Run an evaluation using only metadata and another with the substantive case evidence removed. Strong performance under those conditions warrants investigation into shortcuts. A reseller should also check whether related cases cross training and evaluation splits, allowing a customer model to recognise a previously encountered solution instead of reasoning through the handover.

Preserve evaluation meaning when converting generated packs into marketplace products or customer-specific exports.

Audience:
Licensed resellers
Content type:
FAQ
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Cross-team handover