Workflows & steps

Jetty provides all the components required to run an agentic workflow at scale. This page describes how workflows function and the best way to author your own, run them, investigate the outputs, and improve their quality. Machine-level detail — endpoints, payloads, config schemas — lives in the machine instructions.

Runbooks: the agent workflow description

A Jetty runbook is a markdown file that provides an AI agent all the information required to perform a long-running task to a pre-defined standard of quality. While it might be tempting to think of runbooks as just fancy prompts, there's a little more going on under the hood.

Conceptually, a runbook is a structured workflow in four parts:

Setup — the environment the work needs: which agent runtime and model, what sandbox it runs in, which credentials and input files it gets.
Task — the objective and the steps to get there.
Evaluation — how the work gets checked: the criteria that define an acceptable result, and a bounded loop in which the agent fixes what falls short before it's allowed to finish. Evaluation is a step in the workflow, not an afterthought bolted on after the run.
Artifacts — the files the run must produce. The deliverable is declared up front, and a run that ends with nothing to show fails.

That structure is what separates a runbook from the other ways of telling an agent what to do:

	Agent prompt	Agent skill	Runbook
What it captures	A one-off ask	How to do something, reusably	A whole job: setup, task, checks, outputs
Definition of done	Whatever the agent decides	Inherited from whoever invokes it	Explicit criteria in the file
Checking the work	You read the output	None built in	The agent scores its output and iterates until it passes
Output	A reply in a session	Depends on the session	Declared artifact files, persisted on every run
Repeatability	Copy, paste, hope	Consistent method, unmanaged results	Parameterized, deployable, schedulable, comparable run to run

The short version the docs use elsewhere: skills + standards = runbooks. A skill tells an agent how to do something; a runbook adds the definition of done and a way to verify it.

Because the standard of quality travels with the file, a runbook is portable in a way a prompt never is: deploy it once and every run has the same output: from the web app, your agent, the SDK, or a schedule. In each case, the execution occurs in an isolated sandbox, gets checked against the same criteria, and is recorded as a trajectory: a complete, replayable trace of what the agent did. The anatomy of the file itself — frontmatter and body — is covered further down.

The runbook lifecycle

Working with Jetty is a loop: author a runbook, run it, investigate the results, and improve the runbook based on what you find.

Authoring

Using your agent. You can point your agent to Jetty's installation guide:

> Guide me through installing https://www.jetty.io and setting up my first runbook.

If you work in Claude Code, install the Jetty plugin and author without leaving your editor:

claude plugin marketplace add jettyio/jettyio-skills
claude plugin install jetty@jetty

Run /jetty-setup once (token, provider keys, optional trial activation), then /create-runbook. The skill walks a short guided flow: describe the task, pick an evaluation pattern (it defaults to rubric, and auto-selects programmatic when the output is checkable by code — schemas, SQL, tests, format conversion), then scaffolds ./RUNBOOK.md from a template, validates its structure, and pre-registers it as a Jetty task ready for a first real run. It deliberately skips a dry run — the first real run is the dry run.

The same MCP server works from Cursor, VS Code Copilot, Windsurf, Zed, Codex CLI, and Gemini CLI; runbook authoring skills run fully in Claude Code and Codex CLI, and the MCP tools cover the rest.

Web interface. On jetty.io, a collection's task page shows the runbook, its run history, and its routines. You can create a task, edit its runbook markdown, run it, and inspect trajectories without touching the API. The onboarding flow can also run a hosted demo before you create an account, then claim the workspace by email.

Starting from an existing runbook. Starting from a runbook that already works beats a blank file. Browse the public directory at find a runbook — each entry shows the full runbook, example outputs, and how it's evaluated. Fork one into your collection and edit, or have your agent fetch any raw runbook markdown and run it inline — the chat-completions API takes the runbook as the system message. Every public task exposes its workflow through the API, and the instruction inside it is the runbook markdown — every public runbook is also a template.

Running

Three surfaces, one execution path:

From your agent — /jetty run <task> in <collection>. The plugin calls the run-workflow MCP tool, which auto-detects trial-key eligibility.
SDK — await jetty.runAndWait("acme", "triage", { ticket }) submits, polls, and returns the fully-resolved trajectory.
API — an async run endpoint that returns a workflow id immediately, and an OpenAI-compatible chat-completions endpoint, so any OpenAI client can trigger a runbook by switching its base URL. Endpoints and payloads are in the machine instructions.

An async run responds immediately with a workflow id that embeds the trajectory id. From there you can poll the trajectory or stream the run's logs live. A synchronous call that outlasts the edge's patience returns a timeout while the run continues server-side: treat it as “still working”, poll the trajectory, never retry the submission — a retry starts a second run.

New collections get a free trial that's automatically activated. When the trial is active and you haven't configured a provider key, Jetty injects its own key for the run.

Investigating

Every run produces a trajectory: an immutable record of the run's inputs, each step's outputs, its status, duration, token usage, and every file the agent wrote. For runbook runs it also captures the agent's full session log.

In the app: jetty.io/{collection}/{task}/{trajectory_id} — step timeline, agent log, output files.
From your agent: the get-trajectory, list-trajectories, and get-stats MCP tools.
Programmatically: trajectory read, file download, and archive endpoints — see the machine instructions.

Attach labels (key/value/author) to group runs and carry eval results — graded: pass, config: warm, cohort: 2026-08. Labels are the raw material for comparison: filter the trajectory list by label, or pull aggregate stats per task.

Improving

The loop is run → read → edit → re-run, and /optimize-runbook automates it. Point it at a task and it fetches recent trajectories, looks for patterns — criteria that consistently score low, steps that burn iterations, timeouts, divergent agent behavior from ambiguous instructions, errors the evaluation never catches — and proposes edits in exact before/after form, each citing the trajectory evidence behind it. Apply all, choose selectively, or save the analysis as a report; applying bumps the runbook's patch version.

Keep the judge independent of the agent under test. When a weak model grades its own output, it scores its mistakes as successes, and the loop climbs a lie. And once a runbook passes reliably, schedule it as a routine so drift shows up as a failed run instead of a surprise.

What is an AI runbook?

A runbook run is more than a model call. Jetty assembles the full working environment around the agent: model access, tools, context, files, credentials, and network inside an isolated sandbox that exists only for the duration of the run.

Anatomy: frontmatter and body

The file itself has two layers.

Frontmatter declares the setup — which agent runtime, which model and provider, which sandbox snapshot, how long it may run, which secrets it needs, and which files count as the deliverable:

---
version: "1.0.0"
evaluation: rubric            # rubric | programmatic
agent: claude-code
model: anthropic/claude-sonnet-5
model_provider: openrouter
snapshot: python312-uv
timeout_sec: 7200
primary_outputs:
  - report.md
secrets:
  SLACK_TOKEN:
    env: SLACK_TOKEN
    required: true
---

The body is the job itself: an objective, an output manifest (the files the run must produce), a parameters table for {{variable}} substitution, the processing steps, and the evaluation criteria with a bounded iteration loop. The agent scores its own output against the criteria, fixes what falls short, and repeats up to a fixed number of rounds before it's allowed to finish.

When you deploy a runbook, Jetty wraps it in a workflow, and every field the frontmatter declares — agent, model, snapshot, instruction — becomes a run-time parameter. The same deployed runbook can be re-run with a different model, agent, or input without editing the file, which is what makes model and agent sweeps a loop instead of a project. The exact workflow shape and the parameter-resolution rules are in the machine instructions.

AI gateway

Runbooks name a model and a model_provider; Jetty routes accordingly. Six providers are supported: anthropic, openai, google, bedrock, openrouter, and vllm (self-hosted endpoints). OpenRouter acts as the multi-provider gateway — one key, most models, with routing controls available per collection.

Provider keys are configured once per collection (Settings → AI providers) and injected into the sandbox at run time; they never appear in the runbook or the trajectory. If model_provider is omitted, Jetty resolves it: an OpenRouter key wins when the runtime supports it, then Bedrock, then the runtime's native provider. The model slug has to match the gateway — anthropic/claude-sonnet-5 on OpenRouter, claude-sonnet-5 direct — and a mismatched slug is the classic silent failure: the agent exits in seconds having written nothing.

The same endpoint doubles as a plain LLM proxy: a chat-completions call without a runbook attached is passthrough mode, routed across 100+ providers, with every call recorded as a trajectory.

Tools, skills, and MCPs

The agent runtime brings its own tools — file editing, shell, search, web fetch. You choose the runtime with the agent: frontmatter field: claude-code, codex, gemini-cli, opencode, hermes, goose, and pi are registered, each with its own provider compatibility. When in doubt, A/B it — same grader, different agent:.

Runbook tasks can also attach MCP servers to the run — declared as either a local command or a remote HTTP endpoint, with bearer tokens flowing in from run parameters when a server needs auth. MCP support varies by runtime: claude-code, gemini-cli, and opencode wire the config in; codex currently ignores it. The config format is in the machine instructions.

Skills relate to runbooks by composition, not competition: a skill is reusable instructions for how; the runbook wraps it with the definition of done. Skills go inside runbooks; runbooks are what you run.

Providing context

The runbook markdown itself is the agent's instruction — frontmatter included. Three mechanisms parameterize it:

Template variables. {{name}} placeholders substitute from the run's vars (declared with defaults in the runbook's ## Parameters table). Substitution is literal string replacement — declare every variable, and restate parameters imperatively in the body; agents ignore values that only appear in metadata.
Uploaded files. Files attached to a run are mounted in the sandbox and their listing is appended to the instruction automatically.
Prior runs. In multi-step workflows, path expressions pull earlier step outputs into the instruction.

One hard limit: the instruction after substitution must fit in ~120 KB (the instruction is passed to the agent CLI as a single argument). Large corpora belong in uploaded files, not inlined variables.

File management

In: attach files to a run directly, or pre-stage them through the upload API and pass the returned storage paths. Either way files are mounted at /app/assets/ in the sandbox, and zips are auto-extracted.

During: the agent works in /app and must write deliverables to /app/results/. This is a contract, not a convention — a run that ends with an empty results directory fails, which is the backstop against an agent chatting instead of working.

Out: everything under /app/results/ is persisted to collection storage, with the frontmatter's primary_outputs surfaced first. Fetch individual files or the whole run as a zip. Storage is per-collection: Jetty-managed by default, or bring your own GCS, S3, or Cloudflare R2 bucket.

Third-party connections

Credentials come in layers, most-specific wins:

AI provider keys — for model access, per collection.
Environment variables — everything else (databases, Slack, internal APIs), set per collection via Settings or the API; injected into every run's sandbox like a local .env. The decision rule: talking to a model → provider keys; talking to any other service → environment variables.
Per-run secrets — merged over the collection's env at launch and never persisted to the trajectory. Use these for anything a single run needs that shouldn't be stored.
Connected accounts — OAuth-linked subscriptions (ChatGPT/Codex, Nous Portal). A run can execute on your subscription's quota instead of an API key; if the linked credential can't be refreshed, the run fails rather than silently falling back.

Runbooks declare the env vars they need in a secrets: frontmatter block, which is what powers the plugin's check-secrets preflight ({configured, missing, ready}). Secrets are masked in logs and redacted from trajectory output. See credentials & secrets for the full guide.

Accessing the web

Sandboxes have network egress: agents can fetch URLs, call APIs, and install packages. claude-code runs with WebFetch enabled. Runbooks can use Firecrawl without any setup, and browser automation uses the prism-playwright snapshot plus the Playwright MCP server.

Help getting started

/jetty-setup in Claude Code: token, provider keys or trial, first runbook — the target is under three minutes.
Quickstart for the manual path; find a runbook to fork instead of starting blank.
The trial covers your first runs without defining your own keys.
For agents: jetty.io/llms.txt, agent-instructions.md (pasteable onboarding), and MACHINE_CONTEXT.md (deep reference).
The full API surface — tasks, runs, parameters, sweeps, snapshots, webhooks, routines — is in the machine instructions.

Next: the anatomy in depth in runbooks, the full endpoint surface in the machine instructions, or the activity catalog in the step reference.