Skip to content

by Startups.Studio

Legal & trust Evaluator

Evaluator.update

PATCH /{namespace}/evaluators/{id}

Changes the given properties of an Evaluator.

Noun: Evaluator
Mode: do
Event: Evaluator.updated

You, to your agent

Change the latest evaluator.

curl -X PATCH https://api.sb/petpantry.shop/evaluators/evaluator_1 \
  -H "Authorization: Bearer $STARTUP_BUILDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"integration":"integration_1"}'

import { $ } from 'startup-builder'

const evaluator = await $.Evaluator.update({ id: 'evaluator_1', integration: 'integration_1' })

npx startup-builder do Evaluator.update --json '{"id":"evaluator_1","integration":"integration_1"}'

{
  "name": "do",
  "arguments": {
    "code": "return await $.Evaluator.update({ id: 'evaluator_1', integration: 'integration_1' })"
  }
}

Update changes the given properties of an Evaluator. It is inherited: every Noun has get, list, create, update and delete.

Inputs

Input	Type	Required	Description
`id`	`string`	required	The id of the Evaluator.
`kind`	`string`		One of `code`, `generative`, `agentic`, or `human`.
`cascade`	`string[]`		Kinds tried in order when one declines, such as code, then a Human.
`instructions`	`text`		For a generative or agentic Function, what it does and how.
`model`	`string`		For a generative or agentic Function, the model it runs on.
`tools`	list of Verb		For an agentic Function, the Verbs it may call; Permissions and Limits apply to each.
`maxSteps`	`integer`		For an agentic Function, the most steps one Action may take.
`budget`	`money`		The most one Action may spend; the Mandate’s Limits apply as well.
`role`	Role		For a human Function, the Role its Tasks go to.
`form`	`json`		For a human Function, the form; generated from the Verb’s inputs and outputs when absent.
`sla`	`string`		For a human Function, how long a Task may wait, as an ISO 8601 duration.
`escalateTo`	Role		For a human Function, the Role a Task moves to when the service level passes.
`rubric`	`text`		What it grades against. An agent never varies its own Evaluators. Never randomized.
`integration`	Integration		The Integration whose outside run returns the verdict, such as a CI job; empty when the platform runs it.
`required`	`boolean`		True when `Change.merge` refuses until this Evaluator passes on the Change’s head. Never randomized.

Outputs

Output	Type	Required	Description
`evaluator`	Evaluator	required	The Evaluator.

Effects

Emits Evaluator.updated.
Two-way: try, send, do.

Refuses when

Check	Error	Fix	Then call
The key is valid, but its scopes do not allow this operation.	`forbidden`	Use a key whose scopes cover the operation, or ask an owner of the namespace to grant them.	Evaluator.update

When it waits

Gate	Default
Decision class	Low: two-way, nothing at stake
Delay window	None: it runs once review passes.
Limit	The call Limit of the Permission it runs under.
Over the Limit	It waits `pending` for an Approval from the Founder. Approved, it runs; refused, it ends `refused` with `limit-exceeded`; unanswered by its deadline, it ends `canceled`.

Evaluator.list

Previous Page

Evidence

Next Page