# Incident response

URL: /workflows/processes/incident-response · Id: `process:incident-response`

> Incident response restores service first, fixes the cause second, and prevents the repeat third.

Incident response restores service first, fixes the cause second, and prevents the repeat third.

## Outcome
| Part | This Process |
| --- | --- |
| Outcome | Each Incident is on the status page within five minutes, service is restored, and a postmortem links the Change that prevents a repeat. |
| Reads | [Incident](/business/nouns/incident) |
| Leaves | [Incident](/business/nouns/incident), [Change](/business/nouns/change) |
| On time | Within 4 hours (calendar time); the clock never stops. Late Actions go to the [Engineering Lead](/agents/engineering/engineering-lead). |
| Runs in | Each Startup |

## Accountable
[Reliability Engineer](/agents/engineering/reliability-engineer), [Engineering Team](/agents/engineering).

Responsible: [Reliability Engineer](/agents/engineering/reliability-engineer), [Customer Lead](/agents/customer/customer-lead).

Consulted: [Engineering Lead](/agents/engineering/engineering-lead).

Informed: [Founder](/agents/founding/founder).

## Workflows
| Workflow | Runs | Team |
| --- | --- | --- |
| [Resolve an incident](/workflows/processes/incident-response/resolve-incident) | on `Incident.created` | [Engineering Team](/agents/engineering) |

## Who does each step

| Workflow | Step | Role | Calls | Approval |
| --- | --- | --- | --- | --- |
| [Resolve an incident](/workflows/processes/incident-response/resolve-incident) | 1. Acknowledge the Incident and post it to the status page within 5 minutes. | [Reliability Engineer](/agents/engineering/reliability-engineer) | [Incident.acknowledge](/business/nouns/incident/acknowledge) | None |
| [Resolve an incident](/workflows/processes/incident-response/resolve-incident) | 2. Restore service, reverting the last Change when it is the cause. | [Reliability Engineer](/agents/engineering/reliability-engineer) | [Change.revert](/business/nouns/change/revert) | None |
| [Resolve an incident](/workflows/processes/incident-response/resolve-incident) | 3. Fix the cause in a new Change. | [Reliability Engineer](/agents/engineering/reliability-engineer) | [Change.create](/business/nouns/change/create) | None |
| [Resolve an incident](/workflows/processes/incident-response/resolve-incident) | 4. Write the postmortem and link the Change that prevents a repeat. | [Reliability Engineer](/agents/engineering/reliability-engineer) | [Document.create](/business/nouns/document/create) | None |
| [Resolve an incident](/workflows/processes/incident-response/resolve-incident) | 5. Resolve the Incident and update the status page. | [Reliability Engineer](/agents/engineering/reliability-engineer) | [Incident.resolve](/business/nouns/incident/resolve) | None |

## Events
| Direction | When | What happens |
| --- | --- | --- |
| In | On `Incident.created` | Starts [Resolve an incident](/workflows/processes/incident-response/resolve-incident) |
| Out | [`Incident.acknowledged`](/business/events/Incident.acknowledged) | Raised by step 1 of [Resolve an incident](/workflows/processes/incident-response/resolve-incident) |
| Out | [`Change.reverted`](/business/events/Change.reverted) | Raised by step 2 of [Resolve an incident](/workflows/processes/incident-response/resolve-incident) |
| Out | `Change.created` | Raised by step 3 of [Resolve an incident](/workflows/processes/incident-response/resolve-incident) |
| Out | `Document.created` | Raised by step 4 of [Resolve an incident](/workflows/processes/incident-response/resolve-incident) |
| Out | [`Incident.resolved`](/business/events/Incident.resolved) | Raised by step 5 of [Resolve an incident](/workflows/processes/incident-response/resolve-incident) |

## Plays
No Play names this Process.

## Controls
- [Deploy reconciliation](/systems/trust-and-governance/controls/deploy-reconciliation)
- [Anomaly monitoring](/systems/trust-and-governance/controls/anomaly-monitoring)
- [Incident handling](/systems/trust-and-governance/controls/incident-handling)
- [Revert by query](/systems/trust-and-governance/controls/revert-by-query)
- [Breach notification](/systems/trust-and-governance/controls/breach-notification)
- [Capacity headroom](/systems/trust-and-governance/controls/capacity-headroom)
- [Point-in-time recovery](/systems/trust-and-governance/controls/point-in-time-recovery)

## Settings
| Setting | Type | Default | Say | Set with |
| --- | --- | --- | --- | --- |
| `incidents.statusPostWithin` | `string` | `5m` | Post every incident to the status page within five minutes | [Setting.update](/business/nouns/setting/update) |

## Failure
A failed step retries; when retries run out, the Action goes to the [Engineering Lead](/agents/engineering/engineering-lead).

| Step | On failure |
| --- | --- |
| [Resolve an incident](/workflows/processes/incident-response/resolve-incident) 1. Acknowledge the Incident and post it to the status page within 5 minutes. | Times out after 30 minutes. Goes to the [Engineering Lead](/agents/engineering/engineering-lead). |

## Loop
No loop: the path is never randomized, because it serves [Incident handling](/systems/trust-and-governance/controls/incident-handling).

## Code

**Ask**

```text
Turn on incident response
```

**API**

```bash
curl -X POST https://api.sb/leaseline.app/processes/process_1/enable \
  -H "Authorization: Bearer $STARTUP_BUILDER_API_KEY"
```

**SDK**

```ts
import { $ } from 'startup-builder'

await $.Process.enable({ process: 'process_1' })
```

**CLI**

```bash
npx startup-builder do Process.enable --json '{"process":"process_1"}'
```

**MCP**

```json
{
  "name": "do",
  "arguments": {
    "code": "return await $.Process.enable({ process: 'process_1' })"
  }
}
```

## Enable
On by default for every Product Template. [Process.pause](/business/nouns/process/pause) stops new Actions; Actions in progress finish.

**Ask**

```text
Post every incident to the status page within five minutes
```

**API**

```bash
curl -X PATCH https://api.sb/leaseline.app/settings/setting_1 \
  -H "Authorization: Bearer $STARTUP_BUILDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"value":"5m","key":"incidents.statusPostWithin"}'
```

**SDK**

```ts
import { $ } from 'startup-builder'

await $.Setting.update({ id: 'setting_1', value: '5m', key: 'incidents.statusPostWithin' })
```

**CLI**

```bash
npx startup-builder do Setting.update --json '{"id":"setting_1","value":"5m","key":"incidents.statusPostWithin"}'
```

**MCP**

```json
{
  "name": "do",
  "arguments": {
    "code": "return await $.Setting.update({ id: 'setting_1', value: '5m', key: 'incidents.statusPostWithin' })"
  }
}
```

## Example
Lesson booking fails after a deploy. The [Incident](/business/nouns/incident) is on the status page in 3 minutes and the [Change](/business/nouns/change) reverts in 5. The postmortem names the missing test, and the fixing Change adds it.