# Experimentation

URL: /workflows/processes/experimentation · Id: `process:experimentation`

> Experimentation tests one change against a control, with the threshold fixed before the start.

Experimentation tests one change against a control, with the threshold fixed before the start.

## Outcome
| Part | This Process |
| --- | --- |
| Outcome | Each Experiment ends at its fixed sample in one recorded Decision: ship, iterate or drop. |
| Reads | [Hypothesis](/business/nouns/hypothesis) |
| Leaves | [Experiment](/business/nouns/experiment), [Flag](/business/nouns/flag), [Decision](/business/nouns/decision) |
| Runs in | Each Startup |

## Accountable
[Marketing Lead](/agents/marketing/marketing-lead), [Marketing Team](/agents/marketing).

## Workflows
| Workflow | Runs | Team |
| --- | --- | --- |
| [Run an experiment](/workflows/processes/experimentation/run-experiment) | on `Experiment.created` | [Marketing Team](/agents/marketing) |

## Who does each step

| Workflow | Step | Role | Calls | Approval |
| --- | --- | --- | --- | --- |
| [Run an experiment](/workflows/processes/experimentation/run-experiment) | 1. State the Hypothesis as a Metric, a baseline, and a threshold. | [Analyst](/agents/marketing/analyst) | [Hypothesis.create](/business/nouns/hypothesis/create) | None |
| [Run an experiment](/workflows/processes/experimentation/run-experiment) | 2. Fix the sample size before the start. | [Analyst](/agents/marketing/analyst) | [Experiment.create](/business/nouns/experiment/create) | None |
| [Run an experiment](/workflows/processes/experimentation/run-experiment) | 3. Split traffic between a control arm and a test arm with a Flag. | [Analyst](/agents/marketing/analyst) | [Flag.create](/business/nouns/flag/create) | None |
| [Run an experiment](/workflows/processes/experimentation/run-experiment) | 4. Read the result only at the fixed sample. | [Analyst](/agents/marketing/analyst) | [Readout.create](/business/nouns/readout/create) | None |
| [Run an experiment](/workflows/processes/experimentation/run-experiment) | 5. Record the Decision: ship, iterate, or drop. | [Analyst](/agents/marketing/analyst) | [Experiment.conclude](/business/nouns/experiment/conclude) | None |

## Events
| Direction | When | What happens |
| --- | --- | --- |
| In | On `Experiment.created` | Starts [Run an experiment](/workflows/processes/experimentation/run-experiment) |
| Out | `Hypothesis.created` | Raised by step 1 of [Run an experiment](/workflows/processes/experimentation/run-experiment) |
| Out | `Experiment.created` | Raised by step 2 of [Run an experiment](/workflows/processes/experimentation/run-experiment) |
| Out | `Flag.created` | Raised by step 3 of [Run an experiment](/workflows/processes/experimentation/run-experiment) |
| Out | `Readout.created` | Raised by step 4 of [Run an experiment](/workflows/processes/experimentation/run-experiment) |
| Out | [`Experiment.concluded`](/business/events/Experiment.concluded) | Raised by step 5 of [Run an experiment](/workflows/processes/experimentation/run-experiment) |

## Plays
- [Smoke test](/workflows/plays/smoke-test)
- [Paid search](/workflows/plays/paid-search)
- [Paid social](/workflows/plays/paid-social)
- [Retargeting](/workflows/plays/retargeting)
- [Sponsorships](/workflows/plays/sponsorships)
- [Video ladder](/workflows/plays/video-ladder)
- [Promote winners](/workflows/plays/promote-winners)
- [Reserve drop](/workflows/plays/reserve-drop)
- [Creator probe](/workflows/plays/creator-probe)
- [Creator ads](/workflows/plays/creator-ads)

## Controls
No Control names this Process or a Workflow of it.

## Settings
| Setting | Type | Default | Say | Set with |
| --- | --- | --- | --- | --- |
| `experiments.minSample` | `integer` | `400` | Read no test before 400 visitors per arm | [Setting.update](/business/nouns/setting/update) |

## Failure
A failed step retries; when retries run out, the Action goes to the [Marketing Lead](/agents/marketing/marketing-lead).

## Loop
| Step | Value |
| --- | --- |
| Hypothesis | A challenger drafted from the last Readout beats its control more often than a challenger drafted fresh. |
| Measure | [Weekly verdicts](/agents/founding#weekly-verdicts) over 28d, at least 8 observations |
| Learn | The learn step reads the Readout, records Evidence and a Decision, and drafts the next Variant. |

## Code

**Ask**

```text
Turn on experimentation
```

**API**

```bash
curl -X POST https://api.sb/leaseline.app/processes/process_1/enable \
  -H "Authorization: Bearer $STARTUP_BUILDER_API_KEY"
```

**SDK**

```ts
import { $ } from 'startup-builder'

await $.Process.enable({ process: 'process_1' })
```

**CLI**

```bash
npx startup-builder do Process.enable --json '{"process":"process_1"}'
```

**MCP**

```json
{
  "name": "do",
  "arguments": {
    "code": "return await $.Process.enable({ process: 'process_1' })"
  }
}
```

## Enable
On by default for every Product Template. [Process.pause](/business/nouns/process/pause) stops new Actions; Actions in progress finish.

**Ask**

```text
Read no test before 400 visitors per arm
```

**API**

```bash
curl -X PATCH https://api.sb/leaseline.app/settings/setting_1 \
  -H "Authorization: Bearer $STARTUP_BUILDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"value":400,"key":"experiments.minSample"}'
```

**SDK**

```ts
import { $ } from 'startup-builder'

await $.Setting.update({ id: 'setting_1', value: 400, key: 'experiments.minSample' })
```

**CLI**

```bash
npx startup-builder do Setting.update --json '{"id":"setting_1","value":400,"key":"experiments.minSample"}'
```

**MCP**

```json
{
  "name": "do",
  "arguments": {
    "code": "return await $.Setting.update({ id: 'setting_1', value: 400, key: 'experiments.minSample' })"
  }
}
```

## Example
A lesson-booking app tests a new signup page against the current one. It fixes 2,000 [Visitors](/business/nouns/visitor) per arm and a 10% lift threshold before the start. At the fixed sample the new page converts 14% better, so the [Decision](/business/nouns/decision) records ship.