Half of new signups get each welcome email. The Experiment fixes how the winner is picked before the first email goes out.

For developers

Needs Onboarding

What you get

2welcome emails, each sent to half the new signupsMessage
1test with its rule for picking a winner set before it startsExperiment
1result at 400 signups per version, with the winnerDecision

Your agent does

Experiment.create → Experiment.start

You, to your agent

Test two onboarding emails.

Which welcome email works better?

Try a shorter welcome email on half the signups.

# 1. Experiment.create
EXPERIMENT=$(curl -X POST https://api.sb/leaseline.app/experiments \
  -H "Authorization: Bearer $STARTUP_BUILDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"variants":[{"name":"control","note":"The current welcome email.","measure":"Signups who import a lease within 7 days."},{"name":"short","note":"Three lines and one button: import your first lease."}]}' | jq -r '.experiment["$id"] | split("/") | last')

# 2. Experiment.start
curl -X POST https://api.sb/leaseline.app/experiments/$EXPERIMENT/start \
  -H "Authorization: Bearer $STARTUP_BUILDER_API_KEY"

import { $ } from 'startup-builder'

const experiment = await $.Experiment.create({ variants: [{ name: 'control', note: 'The current welcome email.', measure: 'Signups who import a lease within 7 days.' }, { name: 'short', note: 'Three lines and one button: import your first lease.' }] })
await $.Experiment.start({ experiment: experiment.$id })

EXPERIMENT=$(npx startup-builder do Experiment.create --json '{"variants":[{"name":"control","note":"The current welcome email.","measure":"Signups who import a lease within 7 days."},{"name":"short","note":"Three lines and one button: import your first lease."}]}' | jq -r '.experiment["$id"] | split("/") | last')
npx startup-builder do Experiment.start --json '{"experiment":"'"$EXPERIMENT"'"}'

{
  "name": "do",
  "arguments": {
    "code": "const experiment = await $.Experiment.create({ variants: [{ name: 'control', note: 'The current welcome email.', measure: 'Signups who import a lease within 7 days.' }, { name: 'short', note: 'Three lines and one button: import your first lease.' }] })\nreturn await $.Experiment.start({ experiment: experiment.$id })"
  }
}

Change it

Stop the test early if either arm halves activation.
Test the subject line only.
Roll the winner out to everyone.

Under the hood

Systems: Experiments, Sequences
Nouns: Experiment, Message, Decision
Events: Experiment.started, Experiment.concluded

Wait for the numbers. Below a few hundred signups per arm, a test reads noise, so this wakes once the Startup has the volume. Until then, talk to the people who did not activate.

What you getCopy Anchor Link

Your agent doesCopy Anchor Link

Change itCopy Anchor Link

Under the hoodCopy Anchor Link

What you get

Your agent does

Change it

Under the hood