# Experimentation

URL: /business/experimentation · Id: `page:business/experimentation`

> How a Startup learns from every run without fooling itself.

Experimentation is the resting state of a [Startup](/business/nouns/startup), not a program someone starts. Every run carries a [Hypothesis](/business/nouns/hypothesis) in and ends in [Evidence](/business/nouns/evidence) and a [Decision](/business/nouns/decision).

## Every run is a loop
Every [Play](/business/nouns/play) declares a loop, and so does every [Process](/business/nouns/process) whose output reaches a buyer. The loop names the Hypothesis, the [Metric](/business/nouns/metric) that settles it, a window and a minimum sample.

```ts
loop: {
  hypothesis: 'Refundable deposits opened for 72 hours after one creator post cross the Building entry check.',
  measure: { metric: 'committed-icp-members', window: '3d', minN: 100 },
}
```

The learn step closes every loop, and no definition removes it. A Process with nothing to learn, such as a tax filing, declares no loop and cites the floor row or [Control](/business/nouns/control) it serves.

## Every loop ends in a Decision
The learn step reads the [Readout](/business/nouns/readout), records Evidence, and writes one Decision: ship, kill or iterate. At the scope of the whole Startup, the Decision is persevere or pivot.

On iterate, the learn step drafts the next [Variant](/business/nouns/variant) from the last one. Most Decisions are recorded and never reach a person. A missed target, a guardrail breach, a one-way ship and a pivot reach the [Founder](/agents/founding/founder).

## Every agent choice is logged
When an [Agent](/business/nouns/agent) chooses among options, such as a send hour or a headline, the choice is randomized. Its [Exposure](/business/nouns/exposure) records every option and the probability each one had.

That log is what later estimates read. Offline replay scores a new Variant against past choices before anyone sees it.

## Small traffic, read honestly
A Startup with 40 [Visitors](/business/nouns/visitor) a week cannot settle a split test in any useful time. It reaches a verdict three ways:

- **Ask the [Portfolio](/business/nouns/portfolio) first.** A Prior lookup reads what sibling Startups with the same [Product Template](/business/nouns/product-template), [Motion](/business/nouns/motion) and state already learned.
- **Pool across siblings.** Siblings testing the same value read as strata of one [Experiment](/business/nouns/experiment).
- **Climb the ladder.** The [test ladder](/business/test-ladder) settles in two weeks what a split test settles in a quarter.

Every number prints how far it can be read:

```text
Visitor to deposit · 37 of 200 exposures · readable in 9 days
Visitor to deposit · pooled with 14 siblings · verdict in 6 days
```

A Startup never ships a high-stakes winner on a number that cannot be read.

## Never randomized
Some paths hold for every unit, and no Experiment varies them. The agent never varies its own yardstick, and a guardrail stops an Experiment without waiting for its window.

| Fixed for every unit | Rule | The question, tested instead |
| --- | --- | --- |
| <a id="offer-price"></a>Offer price | Offer price is the same for every buyer of one Offer in one market at the same moment. | Test a Price through a sibling Offer to new Customers, a geo cluster, or a weekly switchback. |
| <a id="legal-floor"></a>Legal floor | Legal floor rows hold for every unit: no Variant drops a notice, a label, a check or a filing. | Never tested: it holds. |
| <a id="consent"></a>Consent | Consent is asked the same way of every Person: no Variant pre-ticks, hides or delays the choice. | Never tested: it holds. |
| <a id="ai-disclosure"></a>AI disclosure | AI disclosure labels every Agent Message, call and Site the same way in every Variant. | Never tested: it holds. |
| <a id="identity"></a>Identity | Identity checks run the same for every Person: no Variant skips or softens one to lift conversion. | Never tested: it holds. |
| <a id="refund-rights"></a>Refund rights | Refund rights, renewal terms and the cancellation path are the same for every Customer of an Offer. | Never tested: it holds. |
| <a id="protected-attributes"></a>Protected attributes | Protected attributes never select, target or exclude a unit, and no Variant differs by them. | Never tested: it holds. |
| <a id="metric-definitions"></a>Metric definitions | Metric definitions, the OMTM and its line are set one level up, so no agent varies what it is scored on. | Never tested: it holds. |
| <a id="evaluators"></a>Evaluators | Evaluators that grade an agent stay fixed while it runs; a new Evaluator passes its own Evals first. | Never tested: it holds. |
| <a id="guardrails"></a>Guardrails | Guardrails stop an Experiment the moment one breaches, in every Variant, without waiting for a window. | Never tested: it holds. |
| <a id="one-way-windows"></a>One-way windows | One-way windows apply in every Variant: an Experiment never shortens the delay before a one-way Action ships. | Never tested: it holds. |

## Next
From Live, the [experimentation engine](/code/builders/startup/experimentation) keeps a challenger on every path and runs the same loop across the Portfolio.