Operations API Tests — Verification

Verification is largely executable: the same make verify SUT=<name> spine that runs the suite is the primary check that the design holds. This document maps each success criterion to a concrete verification and records current evidence.

Criteria → verification

#	Success criterion	How to verify	Status
1	Self-contained suite, parity with Bruno, green on `local` and `dev`	`make verify SUT=local` and `make verify SUT=alpha002-dev` run the full spine and report the result matrix	Green on both (acceptance); parity coverage in progress
2	One command runs the spine, discovers local URL	Run `make verify SUT=local` with no `SUT_BASE_URL` set — it must print a discovered `http://localhost:<nodePort>` and proceed	Verified (NodePort auto-discovered)
3	Any test under api-key or OAuth; auth-specific tests gate	Run the same category with `--auth api-key` and `--auth oauth`; confirm auth-specific tests skip/`[auth:…]` under the wrong mode	Verified — all four auth×SUT quadrants green
4	Wire generated from live OpenAPI, not committed; drift → compile failure	Confirm `generated/*.ts` are git-ignored; regenerate against a changed contract and observe `tsc` fail (see contract check below)	Verified — un-committed; `tsc` gate wired
5	Four layers a build-enforced DAG, structured for reuse	`make check` runs `dependency-cruiser` (no-circular, layer matrix, shared-visibility, tests-are-leaves) + `tsc` project refs; each area is its own package	Verified — gates green
6	No secret values in repo; capabilities + quarantine govern	Grep the tree for secrets (only `op://` refs present); run a capability-gated and a quarantined test and read the matrix	Verified — refs only; gating + budget exercised

The executable checks

The spine. make verify = resolve base URL → gen-proxies → tsc (contract gate) → vitest run → matrix + quarantine verdict. A non-zero exit means the generated contract failed to compile against the harness, a test failed, or the quarantine budget was exceeded. Run per SUT; the exit code is the verdict.

Auth × SUT matrix. The definitive functional check is that every quadrant is green:

	api-key	OAuth
`local`	✓ (`sandbox0-test-key`)	✓ (self-minted local JWT via Caddy JWKS)
`dev`	✓ (static key)	✓ (Cognito access token)

Reproduce with make verify SUT=<sut> ARGS="--auth <mode> --category acceptance".

Contract / backwards-compatibility check. Because generated/ is regenerated per run and the façade binds it, a contract change that the harness depends on (removed/renamed schema, new required request field) makes tsc fail before any test runs. To demonstrate: point gen-proxies at a SUT whose contract diverges from the harness’s expectations and confirm make verify stops at the compile gate. Additive, backwards-compatible changes correctly do not break the build.

Enforcement gates. make check (hermetic — no SUT) must pass: ESLint, the quarantine validator (ticket present, not expired, within existence budget), and — when proxies are present — tsc -b and dependency-cruiser. A layering violation, an illegal cross-layer import, or a test importing another test is a red build.

Determinism of generation. Run make verify twice against the same SUT; the second run must report every module up-to-date (spec unchanged), proving the canonicalized spec cache is stable (no false regeneration from non-deterministic key ordering).

Reusability (pattern) verification

The design’s central claim — a replicable pattern — is verified structurally: util/main and the client-proxies framework (the HttpClient, TokenProvider glue, and gen-proxies) carry no operations-specific code and are independently publishable; a second component adopts the suite by authoring its own per-module façades and supplying its own sut-config.ts, TokenProvider wiring, and module list. Confirm by inspection that no operations-specific identifier appears in util/main or the non-façade framework code of client-proxies (the per-module façades and generated types are populated per component).

Not yet verified

CI workflow (post-deploy trigger). The GitHub Actions job that runs against the dev partition after a deploy is designed but not built; verification is manual (make verify SUT=alpha002-dev) until then.
Higher categories. Evidence to date covers probe…acceptance; functional/stress/bug grow with coverage.
Local OAuth after key rotation. The local JWT path requires a make localInstall redeploy to serve the current JWKS; verified against the api-key path and against a prior local deployment, to be re-confirmed after redeploy.
Automated major/minor classification. Mapping a contract break to a major/minor CHANGELOG decision is developer judgment today; a spec differ (oasdiff) is the planned automation.