Phase 1 -- External Resources Provisioning -- Verification

The verification strategy for Phase 1 with bidirectional traceability between requirements.md and the test catalogue. Each test is described in actionable terms: inputs, observable outcomes, and pass/fail criteria, suitable for direct translation into Jest cases or operator walk-through scripts.

1. Test categories

Category	Identifier prefix	What it covers	Where it runs
Unit — TypeScript	V-PLAT-NNN, V-CI-00N	Static-typing assertions, content of platform/ constants, drift-check logic with mocked dependencies	npm test in /infrastructure
Connectivity — live integrations	V-EXT-NNN	Real authentication against Postmark, real 1Password resolution. Operator-driven before the drift workflow is live	Operator workstation + first manual workflow_dispatch of the drift workflow
Workflow — GitHub Actions	V-CI-1NN	The drift workflow’s behaviour end-to-end (reachability of the live external state)	GitHub Actions runner, both ad-hoc and scheduled
Operator-runbook walk-through	V-OPS-NNN	The runbook is executable; the documentation pages are well-formed	Operator + make pr-checks in /documentation
Documentation	V-DOC-NNN	Documentation page presence and content	make pr-checks + reviewer pass

2. Traceability table

Each requirement is covered by one or more test identifiers; each test maps back to one or more requirements. Status starts at pending and transitions to passed after the corresponding test runs successfully.

Requirement	Test ID(s)	Status
REQ-EXT-001	V-EXT-001, V-EXT-005	pending
REQ-EXT-002	V-EXT-002, V-EXT-005	pending
REQ-EXT-003	V-EXT-003	pending
REQ-EXT-004	V-EXT-004	pending
REQ-EXT-005	V-EXT-005	pending
REQ-PLAT-001	V-PLAT-001	pending
REQ-PLAT-002	V-PLAT-002	pending
REQ-PLAT-003	V-PLAT-003	pending
REQ-CI-001	V-CI-101	pending
REQ-CI-002	V-CI-001, V-CI-002, V-CI-101, V-CI-102	pending
REQ-CI-003	V-CI-103	pending
REQ-OPS-001	V-OPS-001, V-DOC-003	pending
REQ-OPS-002	V-OPS-001	pending
REQ-OPS-003	V-OPS-001	pending
REQ-OPS-004	V-OPS-002	pending
REQ-DOC-001	V-DOC-001, V-DOC-004	pending
REQ-DOC-002	V-DOC-002, V-DOC-004	pending
REQ-DOC-003	V-DOC-002	pending
REQ-DOC-004	V-DOC-004	pending

3. Test catalogue

V-EXT-001 — PostmarkProd account responds with the captured token

• Inputs: the value held in 1Password at op://Arda-SystemsOAM/Postmark-Prod/credential.
• Procedure: operator (or CI) issues GET https://api.postmarkapp.com/servers?count=1&offset=0 with the header X-Postmark-Account-Token: <value> and a Accept: application/json header. Both count and offset are required by the Postmark Account API; omitting either returns HTTP 422 with ErrorCode 600.
• Pass: response is HTTP 200 with a JSON body containing a Servers array (length may be zero or more).
• Fail: any non-2xx response, or a 2xx without the expected JSON shape.

V-EXT-002 — PostmarkNonProd account responds with the captured token

• Inputs: the value held at op://Arda-SystemsOAM/Postmark-NonProd/credential.
• Procedure: as V-EXT-001 against the same endpoint.
• Pass: as V-EXT-001.
• Fail: as V-EXT-001.

V-EXT-003 — 1Password service-account token resolves a known item

• Inputs: the value held at op://Arda-SystemsOAM/IAC-SCRIPTS Service Account Token/credential.
• Procedure: a small helper script (operator-runnable) configures the 1Password SDK with the service-account token and invokes client.secrets.resolve("op://Arda-SystemsOAM/Postmark-Prod/credential").
• Pass: the call returns a non-empty string.
• Fail: the call throws (auth failure, scope failure, or item-not-found).

V-EXT-004 — All four 1Password items present and shape-valid

• Inputs: the same service-account token.
• Procedure: a script enumerates Postmark-Prod, Postmark-NonProd, IAC-SCRIPTS Service Account Token, and Free Kanban Generator Postmark items and confirms each item exists in the Arda-SystemsOAM vault. For the first three, also confirms the credential field is non-empty. For the Free Kanban Generator Postmark item, only confirms existence (the item is a placeholder; Phase 3 populates fields).
• Pass: all four items exist; the first three have non-empty credential fields.
• Fail: any item missing; any of the first three has empty credential.

V-EXT-005 — Account-token header convention is honoured

• Inputs: any operator script or CI workflow that invokes the Postmark Account API.
• Procedure: review of the source code of any Postmark Account API call shipped under this phase confirms the X-Postmark-Account-Token header is used. No Authorization: Bearer ... header for Account API calls. No header transformation between 1Password and Postmark.
• Pass: confirmed by code review and by the network calls observed in V-EXT-001 and V-EXT-002.
• Fail: any Postmark Account API call that uses a different authentication header.

V-PLAT-001 — platform/postmark-service.ts exports the required surface

• Inputs: the file at infrastructure/src/main/cdk/platform/postmark-service.ts.
• Procedure: a Jest test imports the module and asserts:
  • POSTMARK_PROD_ACCOUNT.name === "PostmarkProd".
  • POSTMARK_PROD_ACCOUNT.credentialReference === "op://Arda-SystemsOAM/Postmark-Prod/credential".
  • POSTMARK_NONPROD_ACCOUNT.name === "PostmarkNonProd".
  • POSTMARK_NONPROD_ACCOUNT.credentialReference === "op://Arda-SystemsOAM/Postmark-NonProd/credential".
  • POSTMARK_ACCOUNT_API_BASE_URL === "https://api.postmarkapp.com".
  • POSTMARK_PLAN === "Platform".
  • POSTMARK_API_SURFACE.freshnessDate is an ISO date string parseable by new Date(...).
  • POSTMARK_API_SURFACE.observationsNotePath is a non-empty string ending in .md.
• Pass: all assertions hold.
• Fail: any assertion fails.

V-PLAT-002 — platform/one-password.ts exports typed item references

• Inputs: the file at infrastructure/src/main/cdk/platform/one-password.ts.
• Procedure: a Jest test imports the module and asserts, for each of POSTMARK_PROD_ITEM, POSTMARK_NONPROD_ITEM, IAC_SCRIPTS_SERVICE_ACCOUNT_ITEM, FREE_KANBAN_POSTMARK_ITEM:
  • vault === OAM_VAULT.
  • title is non-empty.
  • primaryField === "credential".
  • reference === "op://" + OAM_VAULT + "/" + title + "/" + primaryField.
• Pass: all assertions hold for all four items.
• Fail: any assertion fails for any item.

V-PLAT-003 — No inline op:// string literals in infrastructure/src/main/cdk/

• Inputs: the source tree at infrastructure/src/main/cdk/.
• Procedure: a grep-based test that searches for "op://Arda-SystemsOAM/" (literal substring) anywhere except inside platform/one-password.ts and platform/postmark-service.ts. (The exclusion list lives in the test fixture and is reviewed at test-time.)
• Pass: zero matches outside the exclusion list.
• Fail: any match outside the exclusion list (would indicate a downstream consumer bypassing the typed constants).

Note: V-PLAT-003 is forward-looking. It is meaningful once Phase 3 / Phase 4 land consumers; before then, the test’s match set is empty by construction. The test still ships in Phase 1 so consumer phases inherit the constraint.

V-CI-001 — Drift-check unit test: full success path

• Inputs: a mocked 1Password SDK that resolves all op:// references to non-empty strings; a mocked HTTP client that returns 200 with a Servers array for any Postmark Account API call.
• Procedure: a Jest test invokes the drift-check module’s main entry point with the mocks injected. Asserts:
  • The function exits without throwing.
  • Every declared op:// reference in platform/ was queried.
  • Every Postmark account credential resolved was used in exactly one HTTP call.
  • The result reports zero failures.
• Pass: all assertions hold.
• Fail: any assertion fails.

V-CI-002 — Drift-check unit test: 1Password reference fails to resolve

• Inputs: a mocked 1Password SDK that throws for one configured op:// reference (e.g., op://Arda-SystemsOAM/Postmark-NonProd/credential); other references resolve normally.
• Procedure: the drift-check module’s main entry point is invoked. Asserts:
  • The function exits non-zero.
  • The result names the failing reference and the underlying error.
  • No HTTP call is attempted for the Postmark account whose credential failed to resolve.
• Pass: all assertions hold.
• Fail: any assertion fails.

V-CI-003 — Drift-check unit test: Postmark API returns non-2xx

• Inputs: a mocked 1Password SDK that resolves all references; a mocked HTTP client that returns 401 for one configured Postmark account and 200 for the other.
• Procedure: the drift-check module’s main entry point is invoked. Asserts:
  • The function exits non-zero.
  • The result names the failing account, the URL queried, and the response status.
• Pass: all assertions hold.
• Fail: any assertion fails.

V-CI-101 — OP_SERVICE_ACCOUNT_TOKEN provisioned and matched against 1Password

• Inputs: the OP_SERVICE_ACCOUNT_TOKEN GitHub Actions secret.
• Procedure: a CI smoke step in the drift workflow’s first run uses the secret to resolve op://Arda-SystemsOAM/IAC-SCRIPTS Service Account Token/credential and asserts the resolved value matches the secret value byte-for-byte.
• Pass: values match.
• Fail: secret missing, secret resolves to a different value, or 1Password resolution fails.

V-CI-102 — Workflow first run completes successfully

• Inputs: the live external resources after Task 1 sign-off.
• Procedure: an operator triggers the drift workflow via workflow_dispatch. The run completes.
• Pass: workflow exits zero; no GitHub issue is created.
• Fail: workflow exits non-zero; a GitHub issue is created with the run URL.

V-CI-103 — No Postmark token in GitHub Actions secrets

• Inputs: the secrets configured in the Arda-cards/infrastructure repository.
• Procedure: an operator (or gh secret list) enumerates the repository secrets. Asserts:
  • OP_SERVICE_ACCOUNT_TOKEN is present.
  • No secret named POSTMARK_PROD_ACCOUNT_TOKEN or POSTMARK_NONPROD_ACCOUNT_TOKEN (or any case variant) is present.
• Pass: the assertion holds.
• Fail: any Postmark account token secret is found.

V-OPS-001 — Operator-runbook walk-through

• Inputs: the page at current-system/oam/postmark-service/operator-runbook.md.
• Procedure: an operator reads the runbook end to end. Confirms:
  • All steps for REQ-EXT-001 — REQ-EXT-005 are present in execution order.
  • Each step’s preconditions, expected observable outcomes, and failure-recovery notes are present.
  • A troubleshooting table maps the most likely failure modes (Postmark 2FA blocked, account-name conflict, 1Password write fails) to causes and fixes.
  • A sign-off section captures operator name, date, and deviations.
• Pass: all confirmations hold.
• Fail: any element missing or unclear.

V-OPS-002 — No infrastructure-side parser-enforced gate

• Inputs: the source tree at infrastructure/.
• Procedure: a grep-based test searches for any TypeScript import of human-steps or any reference to a HUMAN-STEPS.md file. (The former parser code in scripts/postmark-foundations/src/human-steps.ts is also confirmed deleted.)
• Pass: zero matches; the parser file does not exist.
• Fail: any match remains.

V-DOC-001 — Postmark service overview present and well-formed

• Inputs: the page at current-system/oam/postmark-service/index.md.
• Procedure: make pr-checks in /documentation succeeds, and a reviewer confirms the page covers: account topology, credential storage, OAM model (Postmark Console primary), drift cadence.
• Pass: pr-checks pass; reviewer confirms.
• Fail: pr-checks fail; reviewer flags missing content.

V-DOC-002 — Postmark API observations note present, well-formed, and bounded

• Inputs: the page at current-system/oam/postmark-service/postmark-api-observations.md.
• Procedure: make pr-checks succeeds, and a reviewer confirms the page covers: authentication models for API and Webhooks, error model, idempotency / retry policy, webhook payload shapes, version-pin assumptions. Page length is approximately 3 pages (target; not a hard limit — reviewer judgement).
• Pass: pr-checks pass; reviewer confirms content and brevity.
• Fail: pr-checks fail; reviewer flags missing content or excessive length.

V-DOC-003 — Operator runbook present and well-formed

• Inputs: the page at current-system/oam/postmark-service/operator-runbook.md.
• Procedure: make pr-checks in /documentation succeeds; reviewer applies V-OPS-001 checklist.
• Pass: pr-checks pass and V-OPS-001 passes.
• Fail: pr-checks fail or V-OPS-001 fails.

V-DOC-004 — Cross-link discoverability

• Inputs: the documentation tree under current-system/oam/.
• Procedure: a reviewer navigates from current-system/oam/index.md (or its absence — in that case from current-system/index.md) to the Postmark service section without relying on direct URL knowledge. The link checker (make test-links) confirms no broken internal links to or from the new pages.
• Pass: navigation succeeds; link checker passes.
• Fail: navigation requires a direct URL or any link is broken.

4. Verification of project-level acceptance criteria touched by Phase 1

The project-level acceptance criteria in ../goal.md that Phase 1 contributes to:

Goal acceptance	Phase 1 contribution	Phase 1 verification
AC#7 — Drift detection is live	The drift-detection workflow scaffolded by REQ-CI-002 is the founding instance of the drift convention.	V-CI-001, V-CI-002, V-CI-003, V-CI-101, V-CI-102, V-CI-103.
AC#8 — External references resolve from CI	OP_SERVICE_ACCOUNT_TOKEN provisioned + drift workflow uses it to resolve every op:// reference declared in platform/.	V-CI-001, V-CI-101, V-CI-102.
AC#9 — Operator-runbook documentation exists	The Postmark service overview, API observations note, and operator runbook delivered by Task 5 satisfy this AC.	V-DOC-001, V-DOC-002, V-OPS-001.
AC#10 — Documentation is current	Phase 1’s contribution is the new current-system/oam/postmark-service/ pages. The reconciliation work (existing-page updates) belongs to Phase 5b per ../current-system-retrofit.md.	V-DOC-001, V-DOC-002, V-DOC-003, V-DOC-004.

5. Test-execution surfaces

The 22 tests are heterogeneous and live in different runners. There is no single command that runs all of them; each category has a natural home. The table below is the contract between a test ID and its invocation path.

Category	Test IDs	Runner / surface	Invocation	When it runs
Unit + lint-style	V-PLAT-001, V-PLAT-002, V-PLAT-003, V-CI-001, V-CI-002, V-CI-003, V-OPS-002	Jest in infrastructure	npm test	On every PR; CI required check
Connectivity	V-EXT-001, V-EXT-002, V-EXT-003, V-EXT-004, V-EXT-005, V-CI-101	The drift-check TypeScript module (delivered by spec Task 4; dual-purpose)	Operator: npx ts-node tools/drift-check.ts (DesktopAuth via 1Password). CI: scheduled drift workflow body.	After Task 1 sign-off (operator), and monthly + on-demand (CI). Same code, two contexts.
Workflow first run	V-CI-102	The drift workflow itself (workflow_dispatch)	gh workflow run external-resources-drift.yml -R Arda-cards/infrastructure (or via the GitHub Actions UI)	Once after spec Task 4 lands; before merging the infrastructure PR
Repo-state check	V-CI-103	One-shot CLI	gh secret list -R Arda-cards/infrastructure — assert no POSTMARK_*_TOKEN secret is present	Once during PR review of the infrastructure PR; captured in the operator runbook
Documentation lint	V-DOC-001, V-DOC-002, V-DOC-003, V-DOC-004	make pr-checks in documentation (lint + link check + smoke)	make pr-checks	On every PR touching documentation; CI required check
Reviewer walk-through	V-OPS-001, plus the content-review portion of V-DOC-001 and V-DOC-002	Human reviewer + PR checklist	Reviewer reads the page end-to-end and ticks the checklist items	Once during PR review of the documentation PR

Note on the dual-purpose drift-check script

The connectivity tests (V-EXT-001 — V-EXT-005, V-CI-101) and the drift-workflow body share the same TypeScript module. The module imports the typed constants from platform/postmark-service.ts and platform/one-password.ts, resolves every declared op:// reference via the 1Password SDK, and issues a benign Postmark Account API call against each account. Two invocation paths exercise the same code:

• Operator-local (DesktopAuth): the operator runs npx ts-node tools/drift-check.ts after Task 1 sign-off. Satisfies V-EXT-001 — V-EXT-005.
• CI / scheduled (OP_SERVICE_ACCOUNT_TOKEN): the drift workflow’s main step invokes the same script. Satisfies V-CI-101, V-CI-102. Failure opens an automated GitHub issue.

Failures produce identical structured output regardless of context. Test IDs that name “drift-check” assertions (V-CI-001, V-CI-002, V-CI-003) test this module in isolation with mocked dependencies.

Failure handling

A test fails: pause the affected task, address the failure, re-run. The test surfaces above are independent; a failure in one category does not block the others from running, but a failure must be resolved before the gate that depends on it (per the task-plan acceptance gates).

6. Out of scope of verification

• Tests that exercise downstream consumers of Phase 1’s exports (Phase 3 / Phase 4 stacks, Phase 5b operations module). Their verification documents cover their own consumption.
• Tests that exercise the tools/gha-secret.ts utility. Its existing implementation is assumed; Phase 1 declaratively designs as if the utility is present, and its tests (if any) are out of scope of this project.