A deploy runbook that runs as an agent

This example is for a team lead or an operations person whose team deploys software by following a written runbook. You will turn the runbook into an instruction, run it as an agent on your deploy machine, approve each new version before the agent takes it, and answer the agent's questions from your phone.

The situation

Your team deploys by hand, from a checklist in a wiki. People skip steps when they are in a hurry, and nobody remembers to ask the release manager before a production deploy.

You want an agent to do the deploy the same way every time. It must stop when something fails, and it must not touch production until a person says yes. When you change the runbook, the agent should keep the old steps until someone has checked the new ones.

What you'll make

  • An instruction with the runbook's steps, its rules, the release manager as the person who says yes, and the same steps drawn as a workflow.
  • A rule that runs every instruction of an Operations project as an agent on your deploy machine.
  • A setup where every new version waits for an owner's or an admin's approval before the agent takes it.
The runbook's workflow as Preview draws it.

What you need

  • A plan with rules. A rule runs every instruction of a project:

    Plans On Team and Enterprise

  • The right role. Only the organization's owners and admins can make a rule and approve agents and versions. See People and roles.

  • A deploy machine, and someone who can use it. Someone runs one command there, once. If that isn't you, send them the command the editor gives you.

  • About 20 minutes, plus the time it takes to set up the machine.

Steps

  1. Start from the Deploy runbook

    Open the command palette (Ctrl+K, or ⌘K on a Mac) and choose Templates…. Pick the card Deploy runbook (Steps, stops and an approver). A new instruction opens with this text:

    The Deploy runbook starter
    ---
    spec: "1"
    title: Deploy runbook
    parameters:
      environment: { type: enum, values: [staging, production], default: staging, source: workspace }
    ---
    
    # Deploy runbook
    
    You run deployments to ${environment}. Follow the steps in order and stop at the first failure.
    
    ## Steps
    
    1. Check that the build for the commit is green.
    2. Announce the deploy in the team channel.
    3. Deploy, then watch error rates for ten minutes.
    4. Roll back at once if error rates double.
    
    ## Rules
    
    MUST: stop and ask a person when a step fails.
    
    NEVER: deploy on a Friday afternoon or during an incident.
    
    MUST (if you deploy to production): get the go-ahead of :person[the release manager]{ref=#release-manager} before step 3.
    BECAUSE: every change there is visible to customers.
    
    ::person{id=release-manager role=approver}

    It has four steps in order, and three rules. The agent stops and asks a person when a step fails. It never deploys on a Friday afternoon or during an incident. For production, it must get the go-ahead of the release manager before it deploys. The last line names the release manager as the person who approves.

    The word ${environment} is a parameter: a value each agent can get. Here it is staging or production, and staging when nobody says. See Use parameters.

  2. Make it yours

    Change the steps to match your own runbook. Keep one action per step, and say what "failed" means, such as "the build is red" or "error rates double". Rules that start with MUST:, SHOULD: or NEVER: are what the agent must do, should do or must never do. A BECAUSE: line says why. See What an instruction can say.

  3. Add the steps as a workflow

    At the end of the text, add a heading such as The process. On an empty line under it, type / and choose Workflow. The editor puts in a short example. Replace it with this:

    The deploy, as a workflow
    :::workflow{id=deploy dialect="instruction/workflow" title="Deploys a commit after the release manager says yes"}
    name: deploy
    steps:
      start:
        kind: manual
      check:
        kind: agent
        depends_on: [start]
        instruction: Check that the build for the commit is green. Stop if it isn't.
      announce:
        kind: agent
        depends_on: [check]
        instruction: Announce the deploy in the team channel.
      approve:
        kind: human
        depends_on: [announce]
        question: Deploy this commit now?
        to: release-manager
        schema: { type: string, enum: [deploy, stop] }
        timeout: 1h
      route:
        kind: switch
        depends_on: [approve]
        on: "{{ steps.approve.output }}"
        cases: { deploy: deploy, stop: stopped }
      deploy:
        kind: agent
        depends_on: [route]
        instruction: Deploy, then watch error rates for ten minutes. Roll back at once if they double.
      done:
        kind: finish
        depends_on: [deploy]
        output: "{{ steps.deploy.output }}"
      stopped:
        kind: finish
        depends_on: [route]
        status: cancelled
        reason: The release manager stopped the deploy.
    :::

    Preview draws it as boxes with arrows: the check, the announcement, the release manager's decision, then either the deploy or a stop. The decision step asks "Deploy this commit now?", accepts only deploy or stop, and waits one hour at most. If a step names a step that doesn't exist, the editor says This workflow's steps aren't set up correctly.

    Read Who runs the workflow before you count on this step: the drawing documents the process, it doesn't run it.

  4. Put it in a project

    A rule runs the instructions of a project or a folder. Make a project called Operations and move the runbook into it: open the instruction's ⋯ menu, choose Move to…, and pick the project. See Group instructions in projects and folders.

  5. Publish it

    Choose Publish… and publish version 1. An agent always runs a published version, never the draft. See Publish a version.

  6. Make a rule for the project

    Open the ⋯ menu of the Operations project and choose Run every instruction here on…. Fill it in:

    • Where they run: a short name for the machine, such as deploy-server.
    • Open How the agents act. Under When a new version is released, choose Wait for an owner's or an admin's approval. Keep When a new agent is needed at Wait for an owner's or an admin's approval.
    • Under What it would start now, check that the runbook is listed with v1.

    Choose Save and continue. The dialog shows the command to run on the machine. Under The agent to run, pick Claude Code or Codex, then choose Copy the command.

    Every choice is explained in The choices in a rule.

  7. Start it on the deploy machine

    Send the command to whoever looks after the deploy machine, with the page for developers: The runner. They run it there once. From then on, the machine starts the agent and keeps it in step with your rule.

  8. Approve the agent

    Open Settings ▸ Deployments. Under Waiting for approval, find the runbook's agent and choose Approve. Its badge changes to Not started yet, then to Running once the machine reports it.

Who runs the workflow

instruction.md keeps the workflow, checks it, draws it and delivers it to the agent with the rest of the text. It never runs it, and neither does the machine's command. See Who runs a workflow.

So in this setup, two things happen:

  • The agent reads the workflow as part of its instruction. It follows the steps and the rules, like a colleague reading the runbook.
  • The go-ahead comes from the agent's question. When the agent reaches the rule about the release manager, it stops and asks. The question arrives as a card, which you answer. See Answer the agent from your phone.

A program built to run workflows, such as agentd, can run the same block as a real process. It then asks the release manager itself and records the answer. Pressing on the drawing or ticking a box in the editor is never an approval.

Approve each new version

Your rule says Wait for an owner's or an admin's approval for new versions. So when you change the runbook and publish version 2:

  1. The agent keeps running version 1.
  2. In Settings ▸ Deployments, the agent shows the badge v2 waiting for approval, and owners and admins get a notification.
  3. Someone reads the change, then chooses Approve the version. The agent takes version 2.

To compare the two versions before you approve, open the instruction's history: See history and compare versions.

Another way to hold changes back is a version named stable, which moves only when you say. Then set Version to follow to The version named stable. A stable version exists only once someone creates it: see Create stable.

Answer the agent from your phone

The agent sends you two kinds of request:

  • A question, such as "Deploy this commit to production now?". You pick a choice, or type your own answer.
  • A permission, such as "asks to use Bash", with the command it would run. You choose Allow, Allow for this run or Deny.

You get a notification, which opens the agent's chat in the Agents app. The app works in your phone's browser: open instruction.md/agents and sign in. Under the card, the assistant explains the request, how risky it is, and which lines of your runbook decide it. Nothing happens until you press a button.

A card waits 30 minutes, unless a developer gave the rule another time. A permission nobody answered is denied. A question nobody answered is left to the agent, which picks the safest choice and says which one. See When nobody answers.

Only the people who answer for the agent can answer its cards. They are whoever made the rule, whoever runs it on the machine, and the organization's owners and admins. The notification goes to whoever made the rule and whoever runs it. Owners and admins are told only when there is nobody else, but they can always answer. If your release manager is none of these people, they never see the question. Make sure the person who says yes is one of them.

Answering the runbook agent's permission request from a phone.

Check it worked

  • In Settings ▸ Deployments, the agent shows Running and runs v1.
  • On the instruction, the faces in the header include the agent. See See which agents follow an instruction.
  • When you publish version 2, the agent shows v2 waiting for approval and keeps running version 1.

Clean up

  • To hold everything for a while, choose Pause on the rule's row This rule. The agent keeps the version it has, and nothing new starts.
  • To stop the agent, choose Stop on it and confirm. It is stopped for good. To run it again, deploy it again.
  • To remove the rule, choose Delete on it. With Stop its agent under When an instruction leaves this place, its agents stop. The instruction stays as it is.

See Pause, resume or delete a rule.

If it doesn't work

What you see Why What to do
Running every instruction of a place comes with the Team plan Your plan doesn't include rules. Choose See the plans, or deploy the runbook on its own with Share ▸ Deploy to…. See Deploy one instruction.
Only the organization's owners and admins run every instruction of a place You aren't an owner or an admin. Ask one of them to make the rule.
This workflow's steps aren't set up correctly. A step names a step that doesn't exist, or the steps go round in a circle. Fix the steps it points to. See Check your instruction before you publish.
No agents yet under the rule Nobody has run the command on the machine yet. Open the rule's Edit ▸ The command to run, and run it on the machine.
Needs sign-in A tool the agent uses isn't signed in on the machine. Run what the line under the agent says, on that machine.
The release manager never got the question Only the people who answer for the agent see its cards. See Answer the agent from your phone.
Still stuck Something isn't working, or ask us.

Questions

Does the workflow in my runbook run by itself when I publish it?

No. instruction.md and the deploy machine's command never run a workflow. The agent reads it as part of its instruction, and a program built to run workflows, such as agentd, can run it. See Who runs the workflow.

How do I stop the agent taking a new version before someone has checked it?

In the rule, set When a new version is released to Wait for an owner's or an admin's approval. The agent keeps its version until someone chooses Approve the version in Settings ▸ Deployments. See Approve each new version.

Can I answer the agent from my phone?

Yes. The notification opens the agent's chat in the Agents app, which works in a phone's browser. See Answer the agent from your phone.

What happens if nobody answers the agent?

A permission nobody answered is denied after 30 minutes, unless a developer gave the rule another time. A question is left to the agent, which picks the safest choice and says which. See When nobody answers.

Last checked against the product:

Was this page helpful? Report a problem