Skip to content

Errors and testing

Every error is an RFC 9457 problem whose type is a page explaining it, and every outcome, failures included, is reproducible on demand in test.

Every error from api.sb is an RFC 9457 problem whose type is the URL of a page that explains it. The test environment reproduces every outcome on demand, failures included.

The problem document

tutorpool.com is a Licensed-human marketplace where parents book certified tutors. Refunding a lesson Order that was never paid fails like this:

JSON
{
  "type": "https://startupbuilder.dev/code/api/errors/refused",
  "title": "Refused",
  "status": 422,
  "detail": "Order order_1 is placed, and refund needs a paid Order.",
  "instance": "https://api.sb/tutorpool.com/actions/action_9",
  "requestId": "req_7Hk2",
  "fix": "Read the record, run the Verb that moves it to an allowed state, then retry.",
  "next": "https://api.sb/tutorpool.com/orders/order_1/pay"
}

type is stable: branch on it, never on title or detail. The error catalog page at that URL gives the cause and the fix. fix says what to change, next is the call that most often resolves it, and requestId finds the request in the request log.

The problem reads the same on every Interface:

  • The SDK throws it as an error with the same fields.
  • The CLI prints it and exits 1 for an ended Action, or 2 for invalid input.
  • MCP returns it as a tool result with isError: true, and an Agent reads fix as its next instruction.

Two kinds of failure

Where a call fails decides what comes back, as Actions explains in full.

A call rejected at the door never becomes an Action. Malformed input, an unknown Verb, a revoked key or a key outside its scope returns a problem and nothing else. Nothing was recorded, so nothing needs undoing.

A call the record rejects is an Action born refused. Each refusal is recorded: an Order in the wrong state, a Limit raise whose Approval is denied, a Mandate that falls short. instance links to it.

Which errors to retry

ErrorStatusRetry
invalid-input, mode-not-allowed400No: correct the input
unauthorized, forbidden401, 403No: use a key that reaches the call
payment-required402After accepting the Offer in next
not-found404No: correct the namespace or id
conflict409After reading the record again
idempotency-mismatch409No: a new input needs a new key
refused, limit-exceeded422, 403After the state or the Limit changes
rate-limited429After Retry-After seconds
not-live503After the first deploy
Any other 5xx500–599Yes, with the same Idempotency-Key

Retry what time fixes; fix what time does not. A retry of a recorded call carries the original idempotency key, so a call that landed the first time never runs twice.

A test plan

The test environment runs every Verb, Permission and Limit as live does, with no real money and no outside messages. Cover each path a Customer can take, the refusals as deliberately as the successes:

Terminal
export STARTUP_BUILDER_API_KEY=sk_test_...

# The happy path: book a lesson, pay, deliver, confirm.
ORDER=$(npx startup-builder do Order.place --json '{"offer":"offer_algebra"}' | jq -r '.order["$id"] | split("/") | last')
npx startup-builder do Order.pay --json '{"order":"'"$ORDER"'"}'

# A refusal: refunding an unpaid Order returns refused.
UNPAID=$(npx startup-builder do Order.place --json '{"offer":"offer_algebra"}' | jq -r '.order["$id"] | split("/") | last')
npx startup-builder do Order.refund --json '{"order":"'"$UNPAID"'","reason":"Test"}'

# A preview: try runs every check and stores nothing.
npx startup-builder do Order.refund --try --json '{"order":"'"$ORDER"'","reason":"Test"}'

npx startup-builder schema Order.refund prints a Verb's input and output contract. A flow that never sees refused is untested exactly where Customers push hardest.

Magic values

In test, every outside effect runs against a simulator. A magic value forces the outcome you want to test:

InputOutcome
Card 4242 4242 4242 4242Succeeds
Card 4000 0000 0000 0002Declines
Card 4000 0000 0000 0259Succeeds, then arrives disputed one day later
Email to bounce@simulator.testBounces, and the address is suppressed
Phone +1 555 0199Undeliverable
Credential number FAIL-0000Verification fails
Payout account 000999999The payout fails and returns

Every other value succeeds. The simulators emit the same Events, in the same order, as the outside services they stand in for.

Test clocks

Renewals, dunning, delay windows and scheduled Workflows all wait on time. A test clock moves time forward in test only:

Terminal
npx startup-builder clock advance --by P30D

Everything due in those 30 days fires in order. The tutorpool monthly plan renews, the declined card enters dunning, and each refund past its delay window runs. live is never affected.

Local receivers

A Watch needs a public URL; a laptop does not have one. listen streams test Events and forwards each one, signed, to a local server:

Terminal
npx startup-builder listen 'Order.*' --forward-to http://localhost:3000/hooks
npx startup-builder trigger Order.disputed

listen prints a signing secret for the session, so the receiver's signature check runs unchanged. trigger runs the Verbs that produce an Event in test, so the delivery is a real Event from a real Action.

Next