Skip to content
MirrorFirm

How can a buyer judge the quality of generated intake data?

Quality comes from internal consistency, policy fidelity, exception coverage and a review of grounding and disclosure risk, all of which a buyer can check on a sample independently.

Quality in generated intake material is about whether it behaves like a working process, not whether individual records look plausible.

Start with **internal consistency**. Every foreign key should resolve, statuses should follow permitted transitions, timestamps should be ordered, and a document should belong to the party named on its case. These checks can be run by script across a sample.

Next, check **policy fidelity**. Take a sample of cases, apply the stated completeness and routing rules by hand or by code, and confirm the expected outcome follows. If the expected outcomes depend on rules not written down, an agent cannot learn or be graded fairly.

Then look at **exception coverage**. Work out how many cases are clean and how many contain duplicates, conflicts or missing items, and ask whether the mix suits your purpose. A set that is almost all clean cases flatters weak agents.

Finally, review **grounding and disclosure risk**. The material is generated, informed by direct observation of how real businesses operate, but a buyer should read samples for names, identifiers or phrasing that look like real people or organisations, and ask how the generation process addresses that.

None of these checks needs MirrorFirm's help to run once you hold a sample, which is the point: the evidence should come from your own inspection.

Four checks you can run yourself on a sample give evidence of quality without relying on claims from the supplier.

Audience:
Agent developers
Content type:
Explanation
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Application intake