Experiment

An Experiment is a test of a Hypothesis that varies copy or behavior through Flags and judges the Variants by a primary Metric. It ends in a decision: ship, kill, or iterate.

PropertiesCopy Anchor Link

ObjectCopy Anchor Link

{
  "$id": "https://api.sb/ledgerlark.com/experiments/experiment_1",
  "$type": "Experiment",
  "variants": {},
  "decision": "ship"
}

RelationshipsCopy Anchor Link

• An Experiment assigns with many Flags: Flags split units across an Experiment’s Variants and log each assignment with its propensity. assignsWith, inverse assigns.
• An Experiment measures many Metrics: one primary Metric judges an Experiment’s Variants; guardrail Metrics stop it. measures, inverse measuredBy.
• An Experiment offers many Variants: the Variants the Experiment can assign, the control among them. offers, inverse offeredBy.
• An Experiment has many Readouts: the Experiment the result belongs to; the learn step drafts the next Variant from it. readouts, inverse reads.
• An Experiment tests one Hypothesis: each Experiment settles one Hypothesis; many Experiments test one Hypothesis in turn. tests, inverse testedBy.

LifecycleCopy Anchor Link

VerbsCopy Anchor Link

1. createCreates an Experiment.
   POST /{namespace}/experimentsdoExperiment.created
2. getReturns one Experiment by id.
   GET /{namespace}/experiments/{id}readNo Event
3. listReturns Experiments, newest first, one page at a time.
   GET /{namespace}/experimentsreadNo Event
4. concludeConclude ends an Experiment with a decision to ship, kill, or iterate, recorded as a Decision.
   POST /{namespace}/experiments/{experiment}/concludedoExperiment.concluded
5. startStart opens an Experiment, splits traffic across its variants, and begins measuring its primary and guardrail Metrics.
   POST /{namespace}/experiments/{experiment}/startdoExperiment.started

An Experiment is kept forever and never edited: it has no update or delete.

PermissionsCopy Anchor Link

Which Access level may call each Verb. One-way Verbs wait their delay window.

Call itCopy Anchor Link

Start the latest experiment.

ExampleCopy Anchor Link

An Experiment tests whether a $0 tier raises paid conversion. A Flag shows the tier to half of new Visitors; paid conversion at 30 days decides, and the decision reads ship.