Skip to content

Run the CLI in CI

Sign a CI job in with the runner's own OIDC token, run the project's tests against the test environment, and branch on the CLI's exit codes.

A CI job signs in with the token its runner already holds, so the repository stores no key. The CLI sees CI, prints JSON, never prompts, and reports each Action's outcome in its exit code.

Prerequisites

  • A Startup project in a GitHub repository. The examples use leaseline.app, which tracks lease renewals for small property managers at $29 a month.
  • A trust rule on the namespace naming the repository and branches allowed to sign in (keys). A pull request from a fork gets nothing.

Steps

Terminal4 steps
  1. Add a workflow that signs in with OIDC and runs the tests on every pull request:

    .github/workflows/startup.yml
    name: startup
    on: { pull_request: {} }
    permissions: { id-token: write, contents: read }
    jobs:
      test:
        runs-on: ubuntu-latest
        steps:
          - uses: actions/checkout@v4
          - run: npm ci
          - run: npx startup-builder login --oidc --scope leaseline.app
          - run: npm test

    login --oidc exchanges the runner's token for one that lasts 15 minutes. With no --env, every call stays in the test environment.

  2. Let npm test run the project's .test.ts files. Verb tests and Role tests replay recorded model output, so CI makes no model call. Scenario tests walk a test copy of the Startup as a buyer would (tests).

  3. To script a call, rehearse it first. --try checks the input, the Permission and the Limit, and stores nothing:

    npx startup-builder do Offer.publish --json '{"offer":"offer_1"}' --try
  4. Run it with --send and an idempotency key, and branch on the exit code. A rerun job with the same key gets the first Action back instead of running again. --send returns at once, so a write that waits on an Approval exits 3 instead of holding the runner:

    npx startup-builder do Offer.publish --json '{"offer":"offer_1"}' --send --idempotency-key "publish-pro-$GITHUB_SHA"case $? in0) echo "published" ;;3) echo "waiting for an approval" ;;*) exit 1 ;;esac

Result

Every pull request runs the project's tests against the test environment with a token that expires in 15 minutes. Scripted calls rehearse first, never run twice, and report their outcome in the exit code:

Exit codeMeaning
0The Action is done, or the read succeeded.
1The Action is failed, refused or canceled; its problem document names the fix.
2The command or its input is invalid.
3The Action is pending, after --send.

Shipping stays a reviewed Change, and CI never deploys around it. Ship my changes shows the three calls.

Next steps

  • Debugging: read the logs of a failed Action by request id.
  • Environments: test and live, and what crosses between them.
  • CLI spec: every command and flag.