Skip to content
StartupBuilderby Startups.Studio
Code
Changelog
Project structure

Experiment file

An Experiment file holds one Experiment: the Hypothesis, what it varies, the unit, and the Metrics that judge it.

PropertyValue
Pathexperiments/[experiment].mdx
HoldsExperiment
Kindfile
Leadersdb-first, git-first
TeamProduct Team
Written byChange.merge
Compiled byStartupBuilder

Example

experiments/longer-trial.mdx in a Startup repository. Closewise, a sample business, declares a Close Noun.

experiments/longer-trial.mdx
---
hypothesis: A 30-day trial turns more agencies into paying Customers than a 14-day trial, because a trial that spans a month end shows one full Close.
varies: settings.trial
arms: { control: 14d, month: 30d }
unit: visitor
metric: outside-paying-customers
guardrails: [month-3-retention]
window: 45d
minN: 150
---
A 14-day trial ends before most agencies see their first Close.

Props

The frontmatter keys the file takes. The body is its prose.

PropTypeRequiredDescription
hypothesistextrequiredThe belief under test, stated so the primary Metric can prove it wrong.
variesstringrequiredWhat it varies: a file path such as prompts/support-reply, or a Setting such as settings.trial.
armsjsonEach arm’s value, keyed by arm; control is the current value. File arms come from @arm siblings instead.
unitstringWho is assigned: visitor, customer, contact, conversation or request.
splitjsonEach arm’s share of units. Even by default.
metricMetricrequiredThe primary Metric that decides the winner.
guardrailslist of MetricMetrics that stop the Experiment the moment one breaches.
windowstringHow long it runs before its Readout is read, such as 14d.
minNintegerThe fewest exposed units per arm before a verdict.

Generates

StartupBuilder compiles it into these records, Verbs and routes.

KindWhat
RecordsExperiment, Hypothesis, Variant, Flag
VerbsHypothesis.test, Experiment.start, Flag.evaluate
RoutesNone.