Routine Testing

Routine Testing

Overview

apijack provides three tools for validating routines before running them against a live API:

• routine validate — checks YAML structure and catches problems without executing anything
• routine run --dry-run — resolves variables and prints the command sequence that would execute
• routine test — executes a dedicated spec file that asserts expected API behavior

Use validate first to catch authoring mistakes, dry-run to confirm variable resolution looks correct, and test to verify that the routine produces the expected results against a real environment.

Validating YAML

apijack routine validate <name>

Validate checks the following without making any API calls:

• Valid YAML syntax and parseable structure
• Required top-level fields (name and steps) are present
• Every step has a name
• Every step has a command, forEach, or range field
• No duplicate step names
• No collisions between step names and output aliases

Valid routine

Given a routine at ~/.apijack/routines/create-pet.yaml:

name: create-pet
description: Create a new pet
steps:
  - name: create
    command: pets create-pet
    args:
      --name: "Buddy"
      --species: "dog"
    output: pet

Running validate:

apijack routine validate create-pet

Output:

Routine "create-pet" is valid.

Invalid routine

A routine missing the required name field:

steps:
  - name: create
    command: pets create-pet
    args:
      --name: "Buddy"

Output:

Error: Routine must have a 'name' field

A routine with a duplicate step name:

name: bad-routine
steps:
  - name: create
    command: pets create-pet
    args:
      --name: "Buddy"
  - name: create
    command: pets get
    args:
      --id: "1"

Output:

Validation errors:
  - Duplicate step name: "create" at steps[1] "create"

When validation fails, routine validate exits with a non-zero status code, making it suitable for use in CI pipelines.

Dry-run Mode

apijack routine run <name> --dry-run

Dry-run resolves all variables and prints the fully-resolved command sequence — including any --set overrides — without making API calls. It is useful for checking that variable substitution is working as expected before running against a live environment.

For a routine named create-and-verify-pet with a variable pet_name:

apijack routine run create-and-verify-pet --dry-run --set pet_name="Luna"

Output:

Running routine: create-and-verify-pet — Create a pet and retrieve it

[1] create-pet
[1] create-pet: pets create-pet --name Luna --species dog --age 3
[2] get-pet
[2] get-pet: pets get --id $created.id

Routine completed: 2 run, 0 skipped, 0 failed (0.0s)

Note that $created.id remains unresolved because dry-run does not execute steps or capture outputs — it can only resolve variables defined in the variables: block and overrides passed via --set. References to previous step outputs will appear as-is.

Conditions are evaluated during dry-run. Steps whose condition: expression evaluates to false will be counted as skipped rather than printed.

Routine Test Files

apijack routine test <name>

A routine test file is a companion YAML file that lives alongside the routine and is used to verify behavior against a real API. It follows the same routine YAML format as a regular routine, but the focus is on assert: blocks that validate response fields.

Where test files live

Test files use the folder-based routine layout. A routine named create-pet would be structured as:

~/.apijack/routines/
  create-pet/
    routine.yaml   ← the routine itself
    spec.yaml      ← the test file

The spec.yaml filename is required — routine test looks for it at that path. The routine list command shows (has spec) next to any routine that has a corresponding spec file.

How test files differ from routines

A spec file is a full routine definition. The key difference is intent: where a routine is designed for day-to-day use, a spec file is designed to verify correctness. Steps in a spec file typically:

• Create resources, capture their outputs, and then assert on the response
• Chain create, read, update, and delete operations to cover a full lifecycle
• Clean up after themselves so the test can be run repeatedly

The onStep output from routine test annotates steps that have assertions:

[1/9] create-owner
[2/9] create-pet
[3/9] verify-name (assert)
[4/9] verify-status (assert)
[5/9] adopt
[6/9] verify-adopted (assert)
[7/9] verify-adopted-owner (assert)
[8/9] cleanup-pet
[9/9] cleanup-owner

PASSED: 9 steps run, 0 skipped

If an assertion fails, the run reports FAILED with the number of failed steps and exits with a non-zero status code.

Variable overrides work the same as with routine run:

apijack routine test create-pet --set pet_name="Test Luna"

Example: Testing a Petstore Workflow

The following spec file tests a full pet lifecycle: create an owner, create a pet, verify the initial state, adopt the pet, verify the adopted state, then clean up both records.

Save this as ~/.apijack/routines/pet-lifecycle/spec.yaml:

name: test-pet-lifecycle
description: Verify the full pet create-adopt-delete lifecycle
steps:
  - name: create-owner
    command: owners create
    args:
      --name: "Test Owner"
      --email: "test@example.com"
    output: owner

  - name: create-pet
    command: pets create-pet
    args:
      --name: "Test Pet"
      --species: "cat"
      --age: "2"
    output: pet

  - name: verify-name
    command: pets get
    args:
      --id: "$pet.id"
    assert: "$verify-name.name == Test Pet"

  - name: verify-status
    command: pets get
    args:
      --id: "$pet.id"
    assert: "$verify-status.status == available"

  - name: adopt
    command: pets adopt-pet
    args:
      --id: "$pet.id"
      --ownerId: "$owner.id"

  - name: verify-adopted
    command: pets get
    args:
      --id: "$pet.id"
    assert: "$verify-adopted.status == adopted"

  - name: verify-adopted-owner
    command: pets get
    args:
      --id: "$pet.id"
    assert: "$verify-adopted-owner.ownerId == $owner.id"

  - name: cleanup-pet
    command: pets delete
    args:
      --id: "$pet.id"

  - name: cleanup-owner
    command: owners delete
    args:
      --id: "$owner.id"

Run it:

apijack routine test pet-lifecycle

Each verify step asserts a single condition. If any assertion fails — for example, if the status is not "available" after creation — the test reports the failure and exits with a non-zero status code, stopping further steps.

To run the test against a different variable (for example, a different pet name) without modifying the spec file:

apijack routine test pet-lifecycle --set pet_name="Luna"

See Routine Conditions & Assertions for the full list of assertion operators and other step fields.