Skip to content
MirrorFirm

How should a sample be delivered, versioned and load-checked in my pipeline?

Integration detail on release versioning, manifests, schema changes and load-time validation, so a team can ingest successive booking samples without surprises.

Treat each delivery as a versioned artefact that your pipeline can validate.

**Versioning**

Each release carries a version identifier and a schema version. Changes to column names, types or key formats should be listed in a short change note, so your loader can be updated deliberately.

**Manifest**

`manifest.json` records file names, hashes, schema versions and the generation seed. Check the hashes on arrival and store the manifest beside the data.

**Load-time validation**

- every foreign key resolves across tables - identifiers match the agreed prefix style, for example `APP-`, `FAC-` and `COL-` - every task references an existing snapshot and policy - every task has an expected state or an allowed exception outcome

**Isolation**

Load samples into a separate namespace or bucket from anything used for production tooling, until the licence and your own review are complete.

**Iteration**

Keep feedback from each review in a simple log: task id, defect, proposed change. When the next version arrives, rerun the same checks and compare. Ask for the seed so a corrected task can be regenerated without disturbing the others.

Confirm the delivery method and format during scoping.

Version the schema, validate on load and keep seeds with the data, so each iteration of a sample can be compared with the last.

Audience:
Environment builders
Content type:
Integration
Buyer question:
Scope and getting started

Content release: r-a220eb6e099b243b

More on Loan booking