Running API Tests

Arda has two API suites, and which one you run depends on the component.

Component	Suite	Entry point
`operations`	In-repository TypeScript/vitest, at `api-tests/`	`make -C api-tests verify SUT=local`
`accounts`, and any other component	Bruno collections in the separate `api-test` repository	`make -C api-test test-<component>-<env>`, from the workspace root

The operations suite re-implements and extends the Bruno tests and is what CI’s sanity check runs, so it is the one to use for operations. The Bruno collections still exist and still run; where the two disagree about operations, the in-repository suite is authoritative.

The api-test repository is deprecated for operations only. Operations coverage is no longer added there — it goes to operations/api-tests/, versioned with the backend it exercises. For every other component, accounts above all, api-test remains the live and only route for API-level tests, and new coverage for those components is still written in it.

Both workflows share the same first two steps — build the component and deploy it to the local cluster — and diverge at the point of running the tests.

For Bruno test authoring conventions — how to structure collections, write workflows, handle authentication, and use variables — see api-testing-bruno.md.

Prerequisites

Docker and a local Kubernetes cluster (docker-desktop for the operations flow) are running, with kubectl pointing at it.
The 1Password CLI is signed in. Credentials are resolved from op:// references; nothing is stored in the repository.
The component has make build and make localInstall targets.
For the Bruno flow only: the api-test project is checked out with its dependencies installed (npm install), and api-test/1Password/ is populated.

Part 1 — operations, the in-repository suite

Step 1: Build

# in the operations worktree
make build

Step 2: Deploy, and resolve credentials in the same authorization

make localInstall wraps the deploy in op run --env-file 1Password.env, which holds every secret for the life of that command. Chaining the test-side credential pull into the same shell invocation means one biometric unlock covers both; a separate call is a second prompt.

make localInstall && make -C api-tests secrets

make secrets writes a git-ignored api-tests/.env from the op:// references declared in sut-config.ts. Do not run this unattended — the biometric prompt needs a person.

localInstall restarts the deployment, so wait for the new pod before testing:

kubectl rollout status deployment/operations -n operations --timeout=300s

Step 3: Run the suite

make -C api-tests verify SUT=local ARGS="--category all"

Pass --category all if you mean all the tests. verify defaults to --category sanity, and the categories are a progression: sanity means probe + sanity, which is 10 files out of 46. A default run yields a green that covers roughly a fifth of the suite and looks indistinguishable from a full pass.

Category	Covers
`probe`	`probe`
`sanity`	`probe` + `sanity`
`acceptance`	`probe` + `sanity` + `acceptance`
`functional`, `stress`, `bug`, `extra`	that category alone
`all`	everything

verify is the whole pipeline: it discovers the local NodePort through kubectl, regenerates the OpenAPI proxy types from the running component, type-checks the harness against them, then runs the tests. The generated types are build artifacts rather than a committed snapshot, so a tsc failure straight after regeneration is a genuine contract change in the component’s published surface, not a broken harness.

Step 4: Read the result

The runner writes api-tests/.vitest-report.json. Report numPassedTests, numPendingTests (skipped) and numFailedTests separately — a single headline number that folds skips into passes overstates what was verified.

Expected skips

Skips are normal here and worth naming rather than glossing:

The ui-compat/* scenarios no longer skip. They used to sit behind a UI_COMPAT_ENABLED=1 environment gate; that gate was removed in operations 9.0.1 and each scenario now carries its category in its filename, so the runner selects them like any other test. g-absence-probes is a probe, the kanban / order-queue / card-read / item-grid / business-affiliate / email suites are acceptance, and the item upload-and-print chain is functional. A --category sanity run therefore picks up only the absence probes; --category acceptance picks up the rest bar the upload chain. Nothing needs an environment variable.
Capability-gated tests skip cleanly on a SUT that lacks the capability — printing and upload among them. SUT=local declares Baseline and OrderV2; the orders-v2 demand / inventory / procurement-order surface is local-only, so those suites skip on alpha002-dev and can only be covered against a local deployment.

Part 2 — the Bruno suite

The api-test repository holds the Bruno collections. Most of them cover operations surfaces — item, kanban, business-affiliates, orders, operations-configuration — alongside the auth-api, user-tenant and partitionCheck collections. They remain in place and still run, and their status depends on the component.

For operations they are historical coverage: Part 1 is authoritative, and new operations coverage is written there rather than here. Use them for operations only when working with a collection that has no Part 1 equivalent. For accounts and every other component this suite is the live and only route — this part is where their API-level tests are both run and added.

The steps below apply to any component with collections in that repository. operations is used as the worked example because that is what most of the collections address — not because it is the recommended route for testing operations.

Step 1: Build the Component

From the component’s repository root:

make build

Verify the build exits successfully before continuing. A failed build will produce a stale or absent image and the install step will either fail or run with old code.

For operations:

# In the operations/ directory
make build

Step 2: Install to the Local Cluster

make localInstall

After the install completes, confirm the pod is running and has reached Ready status:

kubectl get pods -n <component-namespace>

For operations:

kubectl get pods -n operations

Wait until the pod shows Running and all containers are ready (e.g., 1/1). If the pod is in ContainerCreating or Pending, give it a few seconds and re-check.

Step 3: Run the API Tests

The api-test Makefile provides targets scoped to each component and environment. For local development, use the local environment.

General pattern:

# In the api-test/ directory
make test-<component>-<env>

For operations against the local cluster:

# In the api-test/ directory
make test-operations-local

If the first run fails immediately after pod startup, this is usually a timing issue — the pod is healthy but not yet fully initializing its request handlers. Wait five seconds and retry.

Available Environments

Environment	When to use
`local`	Local Kubernetes cluster on your machine
`dev`	Shared development environment
`ci`	CI pipeline (typically driven by automated tooling, not manually)

Step 4: Verify Test Results

The test runner writes a JSON results file to api-test/out/. Parse it with the results checker to get a structured pass/fail summary:

# In the api-test/ directory
npx tsx /workspace/instructions/claude/scripts/check-api-results.ts \
  out/<results-file>.json \
  --ignore Upload

For operations-local:

npx tsx /workspace/instructions/claude/scripts/check-api-results.ts \
  out/operations-local.json \
  --ignore Upload

The --ignore Upload flag excludes tests that require S3/LocalStack infrastructure, which is not present in the standard local setup. See api-testing-bruno.md for full details on reading output, adding ignore patterns, and distinguishing regressions from pre-existing failures.

Step 5: Act on Results

All non-ignored tests pass: The component is ready. Produce a summary of the test counts (requests, tests, assertions) and proceed.

Non-ignored failures present:

Review each failure in the script output. It will show the request path, HTTP status, and individual test/assertion errors.
If a failure is clearly pre-existing and unrelated to your current change, note it and add an --ignore pattern for the affected path.
If a failure is a regression introduced by your change, investigate and fix it before continuing.
If a failure looks flaky (e.g., inconsistent HTTP 503 or timeout on a polling endpoint), wait five seconds and re-run the full test suite.

Output File Naming Convention

The Makefile targets write output files according to this convention:

Make Target	JSON Output	HTML Report
`test-<component>-<env>`	`out/<component>-<env>.json`	`out/<component>-index-<env>.html`
`test-<env>`	`out/allresults-<env>.json`	`out/index-<env>.html`
`test/<path>-<env>`	`out/<path>-results.json`	`out/<path>-<env>-index.html`

The HTML report is useful for a visual summary when reviewing results in a browser.

Adding a component to the Bruno suite

Part 2’s workflow applies to any component in the Arda backend stack, provided:

The component has a make build and make localInstall target that produces and deploys a container image to the local cluster.
The api-test project contains a collection folder for that component with workflow tests.
There is a corresponding test-<component>-<env> target in the api-test Makefile.

If you are adding API tests for a new component for the first time, see api-testing-bruno.md for how to structure the collection, and add the corresponding Makefile target in api-test/Makefile.

For operations, add the test to the in-repository suite described in Part 1 instead.

api-testing-bruno.md — Bruno test authoring: collection structure, authentication, variables, workflows, and interpreting test results.
release-lifecycle.md — When to run API tests as part of a multi-repository release sequence.
Arda workspace repository — Project-specific test configurations and CI pipeline definitions.