Skip to content
MirrorFirm

How does allocation data fit into a reseller's ingest and listing pipeline?

A release can arrive as JSONL and Parquet files with a manifest, so a reseller's ingest, preview and entitlement steps can run without custom handling per release.

Allocation data fits a reseller pipeline when it arrives as plain, inspectable files with a manifest you can validate at ingest.

A release could be delivered as JSONL for event-like records (bank lines, remittances, allocation decisions) and Parquet for larger tables (invoices, accounts, payers). Each file would carry a schema version and a stable identifier column, so your catalogue can join tables without custom code.

Fields worth asking for in the manifest:

- `release_id` and `schema_version` - `generator_version` and a generation date - `tables`: file name, format, row count and checksum - `licence_scope`: the onward-licence terms you agreed, stored as text you can display - `label_files`: which files hold answer labels, kept apart from inputs

Keeping labels separate lets you publish an inputs-only preview and supply labels after a buyer signs up. For task packs, ask for a task file with `task_id`, `inputs` references, `policy_ids` and a scoring hook name. For simulated environments, ask how state is reset between runs and how the reset is invoked.

Scope the exact formats in a pilot. The aim is that your ingest, preview and entitlement steps need no changes per release.

Ask for a validated manifest and separate label files, so ingest, public previews and customer entitlements need no changes per release.

Audience:
Licensed resellers
Content type:
Integration
Buyer question:
What it is and why it matters

Content release: r-a220eb6e099b243b

More on Payment allocation