feat(runbooks): runbook execution engine + pup workflows PoC

[platinummonkey]

Summary

Runbook execution engine + pup workflows command, proposed in #143. Adds two new command groups with no new dependencies.

pup runbooks

YAML runbooks live in ~/.config/pup/runbooks/. Each file defines sequential steps that mix pup commands, shell tools, Datadog Workflow triggers, HTTP calls, and interactive confirm gates.

pup runbooks list [--tag=key:value ...]   # discover by tag
pup runbooks describe <name>              # show steps + vars
pup runbooks run <name> [--arg K=V ...]   # execute
pup runbooks validate <name>              # lint without running
pup runbooks import <path-or-url>         # fetch into runbooks dir

Example runbook (~/.config/pup/runbooks/hello.yaml):

name: hello
description: Test runbook
vars:
  NAME:
    default: world
steps:
  - name: Say hello
    kind: shell
    run: echo "Hello, {{ NAME }}!"
  - name: List monitors
    kind: pup
    run: monitors list --limit=3

Output while running:

runbook: hello  (2 steps)  2026-03-03 00:16:08 UTC
  Test runbook

[1/2] Say hello  (shell)  2026-03-03 00:16:08 UTC
  $ echo "Hello, pup!"
  stdout:
Hello, pup!
  ✓ done  10ms  ·  next: step 2/2 — List monitors (pup)

[2/2] List monitors  (pup)  2026-03-03 00:16:08 UTC
  $ monitors list --limit=3
  stdout:
{ ... }
  ✓ done  320ms  ·  last step

✓ done  hello  2/2 steps  330ms  2026-03-03 00:16:08 UTC

pup workflows

Full CRUD + execution for Datadog Workflow Automation via the typed SDK client:

pup workflows get <workflow-id>
pup workflows create --file=workflow.json
pup workflows update <workflow-id> --file=workflow.json
pup workflows delete <workflow-id>
pup workflows run <workflow-id> [--payload '{"k":"v"}'] [--payload-file=f.json] [--wait] [--timeout=5m]
pup workflows instances list <workflow-id> [--limit=10] [--page=0]
pup workflows instances get <workflow-id> <instance-id>
pup workflows instances cancel <workflow-id> <instance-id>

--wait polls every 2 s until terminal state. Requires DD_API_KEY + DD_APP_KEY.

Step kinds

Kind	What it does
pup	Shells out to the current pup binary with --output json; supports poll: loops
shell	sh -c "..." with template rendering; surfaces stderr even on success
datadog-workflow	POST trigger + auto-poll to terminal state
confirm	Prompts [y/N]; bypassed in agent mode
http	Authenticated requests to DD API paths or external URLs

HTTP step

Supports all methods (GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS), configurable content types for both request and response, file-based bodies, and binary response handling.

- name: Upload CSV
  kind: http
  method: POST
  url: /api/v2/reference-tables/{{ TABLE_ID }}/rows
  body_file: /tmp/rows.csv        # reads raw bytes from file
  content_type: text/csv

- name: Fetch JSON
  kind: http
  method: GET
  url: /api/v1/slo/{{ SLO_ID }}
  # content_type and accept default to application/json

- name: Download binary report
  kind: http
  method: GET
  url: https://internal/reports/{{ ID }}.pdf
  accept: application/pdf
  output_file: /tmp/report-{{ ID }}.pdf   # writes raw bytes to disk

Request fields:

Field	Default	Purpose
method	GET	Any HTTP method
body	—	Inline body template (rendered before sending)
body_file	—	Read body from file (template-rendered path); takes precedence over body
content_type	application/json when body set; application/octet-stream when body_file set	Request Content-Type
accept	application/json	Request Accept header
headers	—	Additional headers; values are template-rendered, merged with template headers

Response decoding:

• output_file set → raw bytes written to disk; step output is "written N bytes to <path>"
• *json* → pretty-printed JSON
• text/*, *yaml*, *csv*, *xml*, *html* → raw UTF-8 string
• Binary without output_file → actionable error suggesting output_file

For DD API paths (starting with /), auth headers are added automatically. External URLs are sent without auth.

Reusable step templates

Common step patterns live in ~/.config/pup/runbooks/_templates/<name>.yaml. Steps reference them with template:; step fields override template fields; headers are merged per-key.

# _templates/dd-get.yaml
kind: http
method: GET
on_failure: warn

steps:
  - name: Fetch SLO
    template: dd-get
    url: /api/v1/slo/{{ SLO_ID }}
    on_failure: fail          # overrides template

Templates support all step fields including the new content_type, accept, body_file, and output_file. The _templates/ directory and any _-prefixed files are excluded from pup runbooks list.

Control flow

• {{ VAR }} and {{ VAR | default: "x" }} template substitution in all string fields
• on_failure: warn | confirm | fail per step
• when: always | on_success to run cleanup steps after failure
• optional: true to swallow errors silently
• capture: VAR_NAME to pipe step stdout into a variable for later steps
• poll: { interval, timeout, until } with conditions: empty, status == X, value < N, decreasing

Reference runbooks

Three annotated examples in docs/examples/runbooks/:

• deploy-service.yaml — SLO check → incident gate → DD Workflow trigger → monitor poll → Slack notify
• incident-triage.yaml — fetch incident → search logs → check monitors → auto-mitigation workflow → shell diagnostics
• maintenance-window.yaml — create downtime (capture ID) → drain → metric poll → confirm → delete downtime

Not in this PoC

• Parallel step execution
• Step retry logic
• Remote runbook registry / sync
• Web UI

Platform support

pup runbooks is native-only, excluded from wasm builds via #[cfg(not(target_arch = "wasm32"))]. pup workflows uses the typed SDK and works on all targets.

Testing

cargo build

mkdir -p ~/.config/pup/runbooks
cat > ~/.config/pup/runbooks/hello.yaml <<'YAML'
name: hello
description: Test runbook
vars:
  NAME:
    default: world
steps:
  - name: Say hello
    kind: shell
    run: echo "Hello, {{ NAME }}!"
YAML

pup runbooks list
pup runbooks validate hello
pup runbooks run hello --arg NAME=pup

All existing tests pass (cargo test, cargo clippy -- -D warnings, cargo fmt --check).

---

Discussion: #143

🤖 Generated with Claude Code