Continuity
Continuity sets recovery objectives per Capability, fails over between upstreams, names a successor signer, and runs restore drills on a cadence.
Outcome
| Part | This Process |
|---|---|
| Outcome | Each Capability meets its recovery objectives in a passing restore drill, and a successor signer is on record. |
| Reads | Recovery Objective, Capability |
| Leaves | Recovery Objective, Issue |
| Runs in | Each Startup |
Accountable
Reliability Engineer, Engineering Team.
Responsible: Reliability Engineer.
Consulted: Engineering Lead, Risk Analyst.
Informed: Founder.
Workflows
| Workflow | Runs | Team |
|---|---|---|
| Drill a restore | every quarter | Engineering Team |
Who does each step
| Workflow | Step | Role | Calls | Approval |
|---|---|---|---|---|
| Drill a restore | 1. List each Capability whose restore drill is due. | Reliability Engineer | RecoveryObjective.list | None |
| Drill a restore | 2. Restore each Capability’s records into an isolated namespace and time it. | Reliability Engineer | RecoveryObjective.drill | None |
| Drill a restore | 3. Fail over to each fallback upstream in test, then back. | Reliability Engineer | Integration.sync | None |
| Drill a restore | 4. Confirm the successor signer is named and reachable. | Reliability Engineer | Role.get | None |
| Drill a restore | 5. Open an Issue for every objective the drill missed. | Reliability Engineer | Issue.create | None |
Events
| Direction | When | What happens |
|---|---|---|
| In | Every quarter | Starts Drill a restore |
| Out | RecoveryObjective.drilled | Raised by step 2 of Drill a restore |
| Out | Integration.synced | Raised by step 3 of Drill a restore |
| Out | Issue.created | Raised by step 5 of Drill a restore |
Plays
No Play names this Process.
Controls
No Control names this Process or a Workflow of it.
Settings
| Setting | Type | Default | Say | Set with |
|---|---|---|---|---|
continuity.drillEvery | string | quarterly | Drill a restore every quarter | Setting.update |
Failure
A failed step retries; when retries run out, the Action goes to the Reliability Engineer.
Loop
| Step | Value |
|---|---|
| Hypothesis | Quarterly restore drills keep every restore within its recovery objectives. |
| Measure | Restore drills passed over 1y, at least 4 observations |
| Learn | The learn step reads the Readout, records Evidence and a Decision, and drafts the next Variant. |
Code
Turn on continuityEnable
On by default for every Product Template. Process.pause stops new Actions; Actions in progress finish.
Drill a restore every quarterExample
Each quarter the Reliability Engineer restores the payments records into an isolated namespace. A restore that misses its 1h objective opens an Incident, and service fails over to a second upstream.