axiom flow test#

Run the flow's tests/*.test.yaml suites against the real worker (ADR-200)

Run every tests/<name>.test.yaml file beside flow.yaml as a suite of cases, each executed by compiling a MOCKED dev artifact (ADR-176) and invoking it through the REAL worker — the same compile+invoke path "axiom flow run" uses, never an offline interpreter. That is deliberate: a second evaluator of the graph is a second set of semantics, and the bugs that matter live in the gap between them.

A test file looks like:

cases:
  - name: happy path
    input: {amount: 100, currency: "usd"}
    mocks:                        # optional per-case overlay over flow.yaml's mocks:
      charge_card:
        output: {status: "ok", id: "ch_1"}
    expect:
      status: success             # union: "success" | "failure";
                                  # or name one terminal state exactly:
                                  # "completed" | "failed" | "cancelled" | "compensated"
      output:
        subset: {status: "ok"}    # or "equals: {...}" for an exact match
        fields:                   # value-free shape assertions
          receipt_url: non_empty  # present | absent | non_empty
      nodes:
        executed: [charge_card]
        not_executed: [refund]

  - name: the controller signs the capital order
    input: {amount: 5000, currency: "usd"}
    resume:                       # the human decision a config.hitl hold waits for,
      approve_spend:              # keyed by the PAUSING NODE'S alias
        approved: true
    expect:
      status: success

  - name: declined card routes to on_error
    input: {amount: 100, currency: "usd"}
    mocks:
      charge_card: {error: {code: USER, message: "card_declined"}}
    expect:
      status: failure
      error: {node: charge_card, code: USER}

A case's mock overlay REPLACES flow.yaml's mock for that alias wholesale (never a field-by-field merge) and is never written back to flow.yaml.

"failure" is a UNION over FAILED / CANCELLED / COMPENSATED, so it asserts only that the run did not succeed — a condition a flow can reach by routes that never touch the mechanism under test. When a case exists to prove a particular ending (a human-approval timeout that CANCELS, a compensation that rolls back), name that state instead of the union. Checking an exact name needs the run's recorded terminal state, which the synchronous answer does not carry; if that read does not resolve, the case reports "cannot evaluate" rather than guessing.

Boundary — a green suite proves the GRAPH behaves, not that the deployed app works: it cannot catch routing, secret delivery, cold starts, real-DB constraints, or cross-app descriptor collisions. Keep "axiom flow run" against a live invocation for those. Also out of scope here: the flow's compiled mock artifact is never publishable (ADR-176) — this command never publishes anything.

A "resume:" entry is an ASSERTION, not plumbing. It claims the flow HOLDS for a human at that alias: the case is driven asynchronously, the decision is posted the moment the hold appears, and a declared alias that never holds FAILS the case. That is what stops a flow with no "config.hitl" from passing a signed-path case by running straight through. The value must decode into the paused node's own input message (that is what a resume replaces), and a resume value is recorded against the pause as resumed_by "axiom flow test".

Exits non-zero (and prints a diff naming case / expectation / want / got) the moment any case fails. Requires a prior "axiom login".

axiom flow test                       # ./flow.yaml, every tests/*.test.yaml
axiom flow test my-flow.flow.yaml
axiom flow test --case "happy path"   # run only cases named exactly this
axiom flow test --json                # one JSON report object on stdout

Usage#

axiom flow test [flow.yaml] [flags]

Flags#

See also#

• axiom flow — Author and compile flows (graphs of published nodes)