# Evals

URL: /agents/evals-system · Id: `system:evals`

> Evals grades outputs with versioned Evaluators, gates every Change before merge, and keeps each Worker’s track record.

Evals grades outputs with versioned [Evaluators](/business/nouns/evaluator), gates every [Change](/business/nouns/change) before it merges, and keeps each [Worker](/business/nouns/worker)’s track record. An Evaluator is code, a model, an [Agent](/business/nouns/agent) or a [Human](/business/nouns/human), and returns a verdict, never an [Approval](/business/nouns/approval).

## Summary
| Part | This System |
| --- | --- |
| Group | Moves to Agents |
| Status | Beta: it works, and its Verbs may still change |
| Shelf | Front: offered from a Startup’s first days |

You can ask:

- “How good are the support agent’s answers?”
- “Grade this agent’s last 50 replies”
- “Block changes that make answers worse”

## What you get
| Part | This System |
| --- | --- |
| Starts | On for every Startup at creation, with nothing to install. |
| Built from | [Functions](/code/platform/capabilities/functions), [AI gateway](/code/platform/ai-gateway), [Event streams](/code/platform/event-streams) |

## Nouns it owns
| Noun | Definition |
| --- | --- |
| [Evaluator](/business/nouns/evaluator) | An Evaluator is a versioned Function that grades an output against a rubric and returns a verdict. |

## Verbs
| Verb | Object | Approval | Event |
| --- | --- | --- | --- |
| [Evaluator.create](/business/nouns/evaluator/create) | [Evaluator](/business/nouns/evaluator) | None | `Evaluator.created` |
| [Action.evaluate](/business/nouns/action/evaluate) | [Action](/business/nouns/action) | None | [`Action.evaluated`](/business/events/Action.evaluated) |

## Events
| Direction | Event | How |
| --- | --- | --- |
| Emits | `Evaluator.created` | By [Evaluator.create](/business/nouns/evaluator/create) |
| Emits | [`Action.evaluated`](/business/events/Action.evaluated) | By [Action.evaluate](/business/nouns/action/evaluate) |

## States
None.

## Who operates it
| Part | This System |
| --- | --- |
| Accountable | [Quality Lead](/agents/quality/quality-lead) |
| Team | [Quality Team](/agents/quality) |
| Seat | A role agent holds it until a Human is seated. |
| Autonomy | `supervised` |
| Also calls its Verbs | [Reviewer](/agents/quality/reviewer), [Tester](/agents/quality/tester) |

## Approvals and limits
None.

## Processes that use it
| Process | How it uses it |
| --- | --- |
| [Verification](/code/verification) | A step calls [Action.evaluate](/business/nouns/action/evaluate) |
| [Hiring](/workflows/processes/hiring) | A step calls [Action.evaluate](/business/nouns/action/evaluate) |

## Rail underneath
None.

## Views and dashboards
| View | Shows | Metrics |
| --- | --- | --- |
| The [Quality Lead](/agents/quality/quality-lead)’s view | The Metrics its accountable Role answers for. | [Escaped defects](/agents/quality#escaped-defects) |

## Access
What each Access level may call here.

| Access level | May call |
| --- | --- |
| `owner` | Every Verb (6) |
| `admin` | Every Verb (6) |
| `member` | Every Verb (6) |
| `viewer` | Read only (2 of 6) |

## Import and export
None.

## Fees and pricing
None.

## Availability and limits
| Part | This System |
| --- | --- |
| Status | Beta: it works, and its Verbs may still change |

## Compliance notes
Controls it carries:

- [Worker competence](/systems/trust-and-governance/controls/worker-competence)
- [Log completeness](/systems/trust-and-governance/controls/log-completeness)
- [Append-only records](/systems/trust-and-governance/controls/append-only-records)
- [Change approval](/systems/trust-and-governance/controls/change-approval)
- [Agent definition evals](/systems/trust-and-governance/controls/agent-definition-evals)

## Related Systems
None.

## Example
An [E2E managed service](/products/templates/managed-service) for customer support rewrites its refund-reply prompt. Evals replays 400 past [Conversations](/business/nouns/conversation) through it; the new prompt scores no worse on accuracy or tone, so the Change merges. The Agent’s track record, which earns it wider [Mandates](/business/nouns/mandate), updates with the result.