# Test two onboarding emails

URL: /workflows/guides/test-two-onboarding-emails · Id: `guide:test-two-onboarding-emails`

> Send two welcome emails to half the signups each, with the rule for picking the winner fixed before it starts.

Half of new signups get each welcome email. The [Experiment](/business/nouns/experiment) fixes how the winner is picked before the first email goes out.

Needs: [Onboarding](/workflows/processes/onboarding)

**Before it runs**

- You leave the chat: Never: it all happens in the chat.

## What you get
- 2 welcome emails, each sent to half the new signups (Message)
- 1 test with its rule for picking a winner set before it starts (Experiment)
- 1 result at 400 signups per version, with the winner (Decision)

## Your agent does
**For agents**

- Guide: `guide:test-two-onboarding-emails`, status `live`. Page: [/workflows/guides/test-two-onboarding-emails.md](/workflows/guides/test-two-onboarding-emails.md).
- Approval: none. Run it when the founder asks.
- Leaves the chat: never.
- Needs first: [Onboarding](/workflows/processes/onboarding).
- Run: call the steps in order, each through the API, SDK, CLI or MCP tab below; a later step passes the `$id` an earlier one returns. Each value marked example is the sample business’s: pass the founder’s own.

| Step | Verb | Inputs (type, required, this guide’s value) | Returns |
| --- | --- | --- | --- |
| 1 | [`Experiment.create`](/business/nouns/experiment/create) (`verb:Experiment.create`) | `variants` json, required: example `[{"name":"control","note":"The current welcome email.","measure":"Signups who import a lease within 7 days."},{"name":"short","note":"Three lines and one button: import your first lease."}]` | `experiment` Experiment |
| 2 | [`Experiment.start`](/business/nouns/experiment/start) (`verb:Experiment.start`) | `experiment` Experiment (an id), required: the `$id` step 1 returns | `experiment` Experiment |

Returns: the last call answers with `experiment` and its `action`, whose `status` is `done`, `pending` (waiting on an Approval or a Worker), `failed`, `canceled` or `refused`:

```json
{
  "api": "https://api.sb/leaseline.app",
  "experiment": {
    "$id": "https://api.sb/leaseline.app/experiments/experiment_1",
    "$type": "Experiment",
    "variants": [
      {
        "name": "control",
        "note": "The current welcome email.",
        "measure": "Signups who import a lease within 7 days."
      },
      {
        "name": "short",
        "note": "Three lines and one button: import your first lease."
      }
    ]
  },
  "action": {
    "$id": "https://api.sb/leaseline.app/actions/action_1",
    "operation": "Experiment.start",
    "status": "done"
  },
  "links": {
    "self": "https://api.sb/leaseline.app/actions/action_1"
  }
}
```

Next: offer “Stop the test early if either arm halves activation”, or “Test the subject line only”.

Calls: `Experiment.create` → `Experiment.start`

Ask:

```text
Test two onboarding emails.
Which welcome email works better?
Try a shorter welcome email on half the signups.
```

API:

```bash
# 1. Experiment.create
EXPERIMENT=$(curl -X POST https://api.sb/leaseline.app/experiments \
  -H "Authorization: Bearer $STARTUP_BUILDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"variants":[{"name":"control","note":"The current welcome email.","measure":"Signups who import a lease within 7 days."},{"name":"short","note":"Three lines and one button: import your first lease."}]}' | jq -r '.experiment["$id"] | split("/") | last')

# 2. Experiment.start
curl -X POST https://api.sb/leaseline.app/experiments/$EXPERIMENT/start \
  -H "Authorization: Bearer $STARTUP_BUILDER_API_KEY"
```

SDK:

```ts
import { $ } from 'startup-builder'

const experiment = await $.Experiment.create({ variants: [{ name: 'control', note: 'The current welcome email.', measure: 'Signups who import a lease within 7 days.' }, { name: 'short', note: 'Three lines and one button: import your first lease.' }] })
await $.Experiment.start({ experiment: experiment.$id })
```

CLI:

```bash
EXPERIMENT=$(npx startup-builder do Experiment.create --json '{"variants":[{"name":"control","note":"The current welcome email.","measure":"Signups who import a lease within 7 days."},{"name":"short","note":"Three lines and one button: import your first lease."}]}' | jq -r '.experiment["$id"] | split("/") | last')
npx startup-builder do Experiment.start --json '{"experiment":"'"$EXPERIMENT"'"}'
```

MCP:

```json
{
  "name": "do",
  "arguments": {
    "code": "const experiment = await $.Experiment.create({ variants: [{ name: 'control', note: 'The current welcome email.', measure: 'Signups who import a lease within 7 days.' }, { name: 'short', note: 'Three lines and one button: import your first lease.' }] })\nreturn await $.Experiment.start({ experiment: experiment.$id })"
  }
}
```

## Change it
- Stop the test early if either arm halves activation.
- Test the subject line only.
- Roll the winner out to everyone.

## Under the hood
- Systems: [Experiments](/systems/experiments), [Sequences](/systems/sequences)
- Nouns: [Experiment](/business/nouns/experiment), [Message](/business/nouns/message), [Decision](/business/nouns/decision)
- Events: [`Experiment.started`](/business/events/Experiment.started), [`Experiment.concluded`](/business/events/Experiment.concluded)

Wait for the numbers. Below a few hundred signups per arm, a test reads noise, so this wakes once the [Startup](/business/nouns/startup) has the volume. Until then, talk to the people who did not activate.