Lifecycle of a data product
A Dataset moves from draft to selling to retired by its Offers; its records are refreshed, corrected and kept, and nothing is purged.
A Dataset moves from draft to selling to retired by its Offers. Its records live longer than any Offer: each is refreshed, corrected and kept, and every past value still reads back.
The product
The Dataset is draft until a published Offer sells it. Offer.publish and the Live gate refuse a Dataset with any Source that is not licensed, collected within its terms, or synthetic. It is retired when its last Offer is withdrawn, and each Customer keeps what its license let it store.
Each record
| Moment | What happens | Read |
|---|---|---|
| Read from its Source | It arrives as a Resource, with mirroredAt set | Sources and refresh |
| Refreshed | A changed value keeps the one it replaced | Dataset refresh, status planned |
| Corrected | A value that was wrong when served is fixed, with its reason | Corrections, status planned |
| Read as it stood | Any earlier value returns with at | As-of reads |
| Deleted | It leaves every list and keeps its history | Thing.restore brings it back |
A takedown request removes a person from the next delivery, and each redistribute buyer gets the removal in its next file. Handle a takedown has the steps.
Its Sources
A Source can lapse: a feed's terms end, or a wrapped Source's Integration needs re-approval. Integration.refresh renews its token before it expires, and a refused refresh moves it to needsReauth. Records already read stay, each with the time it was last refreshed.
Versions
The Dataset's definition is versioned like any other: its Noun, its Sources, its coverage and its refresh. A Version is a build of the definition, not of the records, which change on their own schedule.