Runbook: Postmark External Resource Provisioning

Author: Miguel Pinilla Last Verified: 2026-05-05 Environments: all (PostmarkProd serves prod; PostmarkNonProd serves dev and stage)

This runbook is the canonical record for the manual provisioning steps required by Phase 1 of the email-integration project. It covers requirements REQ-EXT-001 through REQ-EXT-005. The operator walks these steps once during Phase B of the project. Subsequent drift-detection is handled automatically by the monthly GitHub Actions workflow added in Phase 1.

This runbook supersedes the parser-gated operator runbook (HUMAN-STEPS.md) used by the prior Phase-0 implementation. The legacy file is deleted as part of the companion Phase 1 infrastructure PR (task T-C6) when that PR merges. There is no infrastructure-side parser gate; sign-off is a human record only (per REQ-OPS-004).

Purpose and Audience

This runbook is addressed to an operator with all three of the following access grants:

• Postmark Console: owner (or admin) access to the PostmarkProd and PostmarkNonProd accounts.
• 1Password: access to the Arda-SystemsOAM vault via the desktop application with biometric unlock (DesktopAuth).
• GitHub: admin access to the Arda-cards/infrastructure repository (required to read and write repository-level Actions secrets).

No AWS access is required for any step in this runbook. The runbook uses three command-line tools: the 1Password CLI (op) for verifying secret references resolve, the GitHub CLI (gh) for the Actions-secret step, and Node.js / npx ts-node if the optional post-provisioning connectivity check (tools/drift-check.ts) is exercised. All three are listed in the pre-flight checklist below.

---

Pre-flight Checklist

Before executing any step, confirm the following prerequisites are satisfied:

• You are logged in to the Postmark Console at https://account.postmarkapp.com with the Arda owner account.
• You have the 1Password desktop application open and unlocked (biometric unlock).
• The Arda-SystemsOAM vault is visible in the 1Password sidebar.
• You have the 1Password CLI (op) installed and signed in (op account list returns the Arda account). Used to verify each op:// reference resolves at the end of the relevant steps.
• You have gh CLI installed and authenticated: gh auth status returns your GitHub identity with Arda-cards/infrastructure repo scope.
• You have node and npx available if you intend to run the optional local connectivity check (tools/drift-check.ts) after completing provisioning.

---

Step 1 — Confirm or Create the PostmarkProd Account (REQ-EXT-001)

Goal: a Postmark account named PostmarkProd exists, is active, is on the Platform plan, has 2FA enabled on the owner mailbox, and has a valid account-level API token.

1.1 Verify or create the account

1. Sign in to the Postmark Console at https://account.postmarkapp.com/login. After login, the Console home (server list) is at https://account.postmarkapp.com/servers; switch accounts from the top-right account selector.
2. Check whether an account named PostmarkProd already exists.
   • If it exists, click into it and verify:
     • Status is Active.
     • Plan is Platform.
   • If it does not exist, click Create Account, enter name PostmarkProd, and select the Platform plan.

Postmark signup (if starting from scratch): https://account.postmarkapp.com/sign_up. Choose the Platform plan — it is the only plan that supports unlimited servers and domains.

1.2 Enable 2FA on the owner mailbox

1. Open the account settings page at https://account.postmarkapp.com/account for the PostmarkProd account.
2. Confirm that Two-Factor Authentication is enabled. If not, enable it now before proceeding.

1.3 Generate an account-level API token

1. Open the account-level API token page at https://account.postmarkapp.com/account/api_tokens for the PostmarkProd account (account-level token, not server-level).
2. If a token already exists and is captured in 1Password (Step 2 below), skip generation — tokens cannot be retrieved after initial display.
3. If no token exists (or the existing token has been revoked), click Generate Token. Copy the token immediately — it is shown only once.

Expected state after this step: the PostmarkProd account is active, on the Platform plan, 2FA-enabled, and you hold an account-level API token in your clipboard or a secure note.

---

Step 2 — Capture the PostmarkProd Token in 1Password (REQ-EXT-002)

Goal: the 1Password item Postmark-Prod in the Arda-SystemsOAM vault contains the PostmarkProd account-level API token in the credential field.

2.1 Locate or create the Postmark-Prod item

1. Open 1Password, navigate to the Arda-SystemsOAM vault.
2. Search for an item named Postmark-Prod.
   • If the item exists and the credential field contains the current token, confirm the op:// reference resolves (see Step 2.2) and proceed to Step 3.
   • If the item does not exist, create a new Login or API Credential item named exactly Postmark-Prod.

2.2 Populate the required fields

Set the following fields on the Postmark-Prod item:

Field	Value	Type
credential	The account-level API token from Step 1.3	Concealed (password)
accountOwnerEmail	The email address of the Postmark account owner	Text (optional metadata)
plan	Platform	Text (optional metadata)
createdAt	Creation date in ISO format (YYYY-MM-DD)	Text (optional metadata)

The mandatory field is credential. The optional metadata fields are useful for auditing but are not read by any automated process.

2.3 Confirm the op:// reference

The canonical reference for this item is:

op://Arda-SystemsOAM/Postmark-Prod/credential

You can verify resolution locally:

op read "op://Arda-SystemsOAM/Postmark-Prod/credential"

Expected output: the token value (a 40-character alphanumeric string). If you see an error, re-check vault name, item name, and field name for typos.

---

Step 3 — Confirm or Create the PostmarkNonProd Account (REQ-EXT-003)

Goal: a Postmark account named PostmarkNonProd exists, is active, is on the Platform plan, has 2FA enabled, and has a valid account-level API token.

The procedure is identical to Step 1, applied to the PostmarkNonProd account.

3.1 Verify or create the account

1. Return to the Postmark account list (same URL as Step 1.1).
2. Check whether an account named PostmarkNonProd exists.
   • If it exists, confirm Status = Active and Plan = Platform.
   • If it does not exist, click Create Account, name it PostmarkNonProd, select Platform plan.

The PostmarkNonProd account serves the dev and stage partitions. It is safe to send test email from this account without production risk. Any sandbox or delivery restriction that Postmark applies by default (e.g., on-trial sending limits) must be resolved before Phase 3 server provisioning.

3.2 Enable 2FA on the owner mailbox

Same procedure as Step 1.2 for the PostmarkNonProd account.

3.3 Generate an account-level API token

Same procedure as Step 1.3 for the PostmarkNonProd account. If a token is already captured in Postmark-NonProd (Step 4 below), skip generation.

---

Step 4 — Capture the PostmarkNonProd Token and Populate All 1Password Items (REQ-EXT-004)

Goal: all three 1Password items that Phase 1 declares exist in the Arda-SystemsOAM vault with correct fields.

4.1 Create or update Postmark-NonProd

Same fields as Postmark-Prod (Step 2.2), with the PostmarkNonProd token in the credential field.

Canonical reference:

op://Arda-SystemsOAM/Postmark-NonProd/credential

Verify:

op read "op://Arda-SystemsOAM/Postmark-NonProd/credential"

4.2 Confirm IAC-SCRIPTS Service Account Token

This item was created on 2026-04-29 (per Phase 0 record). Confirm it still exists and the credential resolves:

op read "op://Arda-SystemsOAM/IAC-SCRIPTS Service Account Token/credential"

If the item is missing or the token has expired, generate a new 1Password service account scoped read-only to the Arda-SystemsOAM vault and store it under this item name with the credential field.

Open https://my.1password.com/developer-tools/active/service-accounts to view, create, or rotate service accounts. The IAC-SCRIPTS Service Account Token 1Password item must contain a token issued from this page that has read access to the Arda-SystemsOAM vault.

4.3 Confirm all three items

1Password Item	Vault	Field	op:// Reference	Status
Postmark-Prod	Arda-SystemsOAM	credential	op://Arda-SystemsOAM/Postmark-Prod/credential	must be non-empty
Postmark-NonProd	Arda-SystemsOAM	credential	op://Arda-SystemsOAM/Postmark-NonProd/credential	must be non-empty
IAC-SCRIPTS Service Account Token	Arda-SystemsOAM	credential	op://Arda-SystemsOAM/IAC-SCRIPTS Service Account Token/credential	must be non-empty

The drift-detection workflow asserts every op:// reference declared in platform/one-password.ts resolves to a non-empty value (REQ-CI-002). Phase 1 declares only the three items above; the Free Kanban Tool’s Postmark server token is created by Phase 3 in the separate Arda-CorporateOAM vault per DQ-R1-007 and is not part of Phase 1’s typed surface.

Note — per-partition copies (Phase 4 prerequisite). The Arda-SystemsOAM items above (Postmark-Prod, Postmark-NonProd) are the qualified global-utility items, named with the suffix so Arda-SystemsOAM can hold both Postmark accounts without ambiguity. Following the Arda partition-vault convention, each partition’s Arda-{Env}OAM vault also holds an independently stored copy of its relevant Postmark account token under the service-name-only title Postmark (e.g., op://Arda-ProdOAM/Postmark/credential, op://Arda-DevOAM/Postmark/credential). Those per-partition copies are created as part of Phase 4 partition provisioning. platform/postmark-service.ts will need a partition-aware credential accessor (postmarkCredentialOpReference(partition: PartitionId): string) before Phase 4 deploy tooling can consume them — this is filed as a Phase 4 prerequisite.

---

Step 5 — Verify OP_SERVICE_ACCOUNT_TOKEN in GitHub Actions (REQ-EXT-005 / REQ-CI-001)

Goal: the GitHub Actions repository secret OP_SERVICE_ACCOUNT_TOKEN in Arda-cards/infrastructure contains the IAC-SCRIPTS Service Account Token credential so that the drift-detection workflow can authenticate to 1Password in CI.

5.1 Read the service-account token locally

OP_SA_TOKEN=$(op read "op://Arda-SystemsOAM/IAC-SCRIPTS Service Account Token/credential")

5.2 Check whether the secret is already provisioned

gh -R Arda-cards/infrastructure secret list

Look for OP_SERVICE_ACCOUNT_TOKEN in the output. gh secret list does not reveal secret values, only names.

5.3 Provision or rotate the secret

If the secret is absent or stale, provision it using tools/set-gha-repo-secret.sh from the Arda-cards/infrastructure repository:

./tools/set-gha-repo-secret.sh \
  --repo Arda-cards/infrastructure \
  --secret-name OP_SERVICE_ACCOUNT_TOKEN \
  --from-1password "op://Arda-SystemsOAM/IAC-SCRIPTS Service Account Token/credential"

The script does the op read | gh secret set translation for you, plus preflight checks that op and gh are installed and authenticated. To verify everything resolves before writing to GitHub, append --dry-run:

./tools/set-gha-repo-secret.sh \
  --repo Arda-cards/infrastructure \
  --secret-name OP_SERVICE_ACCOUNT_TOKEN \
  --from-1password "op://Arda-SystemsOAM/IAC-SCRIPTS Service Account Token/credential" \
  --dry-run

Alternatively, provision directly via the GitHub CLI without the helper script:

op read "op://Arda-SystemsOAM/IAC-SCRIPTS Service Account Token/credential" | \
  gh -R Arda-cards/infrastructure secret set OP_SERVICE_ACCOUNT_TOKEN

5.4 Verify the secret is accessible

After provisioning, re-run gh secret list and confirm OP_SERVICE_ACCOUNT_TOKEN appears. You can also trigger a workflow_dispatch run of the drift-detection workflow after Phase 1 Task 4 has landed to confirm end-to-end resolution.

gh -R Arda-cards/infrastructure secret list

Expected output: a table row containing OP_SERVICE_ACCOUNT_TOKEN with a recent update timestamp.

---

Post-Provisioning Connectivity Check (Optional but Recommended)

After completing Steps 1—5, run the local connectivity check to confirm all external resources are reachable with the configured credentials:

cd <path-to-infrastructure-worktree>
OP_ACCOUNT_NAME=Arda npx ts-node tools/drift-check.ts

Expected output: a JSON report with all items showing status: "ok" and Postmark API calls returning 2xx. This is the same check the monthly drift workflow runs in CI.

---

Troubleshooting

Symptom	Likely Cause	Resolution
op read returns [ERROR] Item "Postmark-Prod" not found	Item name typo or wrong vault	Verify exact item name (case-sensitive) and vault name in 1Password sidebar
op read returns [ERROR] You are not signed in	1Password desktop app locked	Unlock the desktop app via biometrics, then retry
op read returns [ERROR] Vault "Arda-SystemsOAM" not found	Service account lacks vault access	Re-check service account scoping in the 1Password developer portal; vault must be listed under read-only access
Postmark API returns 401 Unauthorized on connectivity check	Stale or revoked token in credential field	Generate a new account-level token in the Postmark Console and update the 1Password item; the old token cannot be recovered
Postmark API returns 422 or plan-related error	Account is not on Platform plan	Upgrade the account in the Postmark Console: review billing at https://account.postmarkapp.com/account/billing_settings and change the plan at https://account.postmarkapp.com/account/subscription
gh secret list does not show OP_SERVICE_ACCOUNT_TOKEN	Secret was never provisioned, or was deleted	Run Step 5.3 to provision it
gh secret set fails with HTTP 404	gh is not authenticated with sufficient scope	Re-authenticate: gh auth login and confirm repo scope; check your GitHub account has admin access to Arda-cards/infrastructure
tools/drift-check.ts exits non-zero in CI but passes locally	OP_SERVICE_ACCOUNT_TOKEN value in GHA does not match current 1Password item	Rotate the GHA secret: read the current value from 1Password and re-run Step 5.3
1Password service account token returns forbidden on item read	Service account scope excludes Arda-SystemsOAM	In the 1Password developer portal, add read-only access for Arda-SystemsOAM to the service account, then rotate the token and re-provision the GHA secret
Step 1.2 / 3.2 — 2FA toggle not visible at https://account.postmarkapp.com/account	Postmark’s UI exposes 2FA on the user-profile page, not the account-settings page. The status is observable on the account’s user list (e.g., https://account.postmarkapp.com/account/users), but the enable action lives on the user’s own profile, reached via the top-right user-avatar menu in the Console.	Locate the user profile via the avatar menu and follow Postmark’s 2FA setup flow. Status returns to “On” on the user list once enabled. The exact URL is per-account and per-user; record it in this row’s resolution column when found, and update the runbook on the next walkthrough.

---

Post-Merge: First Drift-Workflow Run (T-C5) and GHA Secret Audit (T-C7)

Two operator actions complete Phase 1 once Arda-cards/infrastructure PR #446 merges to main:

T-C5 — First workflow_dispatch run

Before merge, GitHub Actions requires the workflow file to land on the default branch before workflow_dispatch will accept it; pre-merge dispatches return HTTP 404 -- workflow not found on the default branch. Run this once immediately after PR #446 merges:

gh workflow run external-resources-drift.yml -R Arda-cards/infrastructure
gh -R Arda-cards/infrastructure run list --workflow external-resources-drift.yml --limit 1
gh -R Arda-cards/infrastructure run watch <run-id>

Expected: workflow completes successfully. The drift-check step prints a JSON report with passed: 5, failed: 0 and the workflow exits 0. Verifies V-CI-102 (workflow first run completes successfully).

On failure: read the failed step’s logs (gh -R Arda-cards/infrastructure run view <run-id> --log-failed). The most likely first-run failure is the GHA secret being out of sync with the rotated 1Password value — re-run Step 5.3 above to refresh, then re-trigger the workflow.

T-C7 — Confirm no Postmark-token GHA secrets

gh -R Arda-cards/infrastructure secret list | grep -i postmark

Expected: zero matches. The rev1 design resolves Postmark account tokens at runtime via the 1Password SDK using OP_SERVICE_ACCOUNT_TOKEN; no Postmark token is ever stored in GitHub Actions secrets (REQ-CI-003, V-CI-103). If a Postmark-named secret appears, it is leftover from a prior provisioning approach and should be deleted with gh -R Arda-cards/infrastructure secret delete <NAME>.

The 2026-05-05 walkthrough surfaced one such leftover (POSTMARK_NONPROD_ACCOUNT_TOKEN, set 2026-04-30 from the prior Phase-0 era) and deleted it. Future operators should treat any new appearance as a regression to investigate.

---

Looking Ahead: Domain Verification (Phase 3 / Phase 4)

The Postmark accounts you have just provisioned (PostmarkProd, PostmarkNonProd) are usable for API operations today — token resolution, server creation, sender-signature listing, and the drift-detection probe. They are not yet usable for live mail delivery to arbitrary recipients. Postmark restricts outbound delivery on a new account until at least one sending domain is verified under that account; a fresh account is effectively a sandbox.

Domain verification is operator-driven and happens in subsequent phases:

• Phase 3 (Corporate Updates): verify arda.ardamails.com (the Corporate zone parent) under the PostmarkProd account. The Phase 3 Corporate stack emits the required DKIM TXT and Return-Path CNAME records into the arda.ardamails.com zone (via the dns-email-records.ts construct); the operator clicks “Verify” in the Postmark Console and, if needed, submits an account approval request to take PostmarkProd out of sandbox.
• Phase 4 (Runtime Platform Updates): per partition, verify {partition}.ardamails.com (e.g., prod.ardamails.com, dev.ardamails.com) under the appropriate Postmark account. The step-by-step procedure is in Phase 4 Partition Mail Deploy below.

References:

• Postmark help — How do I verify a domain: https://postmarkapp.com/support/article/how-do-i-verify-a-domain
• Postmark Console — Sender Signatures (per account): https://account.postmarkapp.com/signature_domains

---

Phase 4 Partition Mail Deploy

This section is the operator procedure for the per-partition mail surface deploy delivered in Phase 4. Each of the four active Application-Runtime partitions (dev, stage, demo, prod) is deployed by running the same procedure with different positional arguments; the procedure is what the Phase 4 Run-3 operator cascade executes for each cascade entry (see Run-3 project plan and the live execution log).

External dependencies

These must be in place before running the procedure for any partition. Items marked as a one-time dependency are checked once at the start of a cascade run; items marked as per-partition are re-verified for each partition.

Dependency	Scope	How to verify	How to remediate
Phase 1 Postmark provisioning complete	One-time	Steps 1-5 of this runbook signed off	Complete Phase 1
dmarc-reports@arda.cards mailbox exists and is monitored	One-time	Send a test mail; confirm it lands	Google Workspace admin provisions the mailbox
PR #462 (Phase 4 Run-2) merged to Arda-cards/infrastructure main	One-time per Phase-4 cycle	gh -R Arda-cards/infrastructure pr view 462 --json mergedAt returns a non-null mergedAt	Land PR #462; only then can the cascade open
Arda-cards/infrastructure cloned at phase-4/infrastructure-run-3 on branch jmpicnic/email-integration-phase-4-run-3, rebased onto main	One-time per Phase-4 cycle	git -C phase-4/infrastructure-run-3 log --oneline -1 shows the rebased head	git -C phase-4/infrastructure-run-3 fetch origin && git rebase origin/main
1Password vault Arda-{Env}OAM populated with Postmark/credential for the partition being deployed	Per partition	op item get "Postmark" --vault "Arda-{Env}OAM" --format json resolves and the credential field is non-empty	Add the item to the partition’s vault per the workspace 1Password vault convention (Arda-{Dev,Stage,Demo,Prod}OAM per partition; Arda-SystemsOAM for workspace-wide system secrets)
AWS SSO session for the partition’s profile	Per partition	aws sts get-caller-identity --profile <profile> returns the expected 12-digit account ID	aws sso login --profile <profile>
cdk synth for the partition’s stack succeeds	Per partition	bash plan/runs/run-3-operator-cascade/validate-exit.sh <partition> exits 0	Diagnose synth output; fix in the Run-3 PR if a code-level issue surfaces

The four partitions’ bindings (which infrastructure, which Postmark account, which AWS profile, which 1Password vault):

Partition	Infrastructure	AWS profile	1Password vault	Postmark account
dev	Alpha002	Alpha002-Admin	Arda-DevOAM	PostmarkNonProd
stage	Alpha002	Alpha002-Admin	Arda-StageOAM	PostmarkNonProd
demo	Alpha001	Admin-Alpha1	Arda-DemoOAM	PostmarkProd
prod	Alpha001	Admin-Alpha1	Arda-ProdOAM	PostmarkProd

Note on the --profile flag (load-bearing): amm.sh auto-derives its AWS profile as Admin-${infrastructure} when no --profile is passed. That derivation matches the legacy infrastructures (Sandbox / PlatformRoot) but does not match the Phase-4 infrastructures (Alpha002-Admin is suffix-form; Admin-Alpha1 is truncated). Always pass --profile <name> explicitly with the value from the table above when invoking amm.sh for Phase 4 partitions. The flag also propagates into the AWS_PROFILE env var so subsequent CLI calls (region resolution, cdk deploy, etc.) target the same profile.

Cascade order (DQ-R1-021)

The cascade follows the partial order dev → {stage || demo} → prod. The dev partition is the first cascade entry; production (prod) is the last. stage and demo may be executed in either order or interleaved, but both must be verified before opening the prod cascade entry.

After the dev entry verifies and before opening the stage entry, the operator sends a reply to Postmark Compliance ticket #11236089 with the dev.ardamails.com verified-domain evidence (T-O4). This unblocks Sender Signatures on the rest of PostmarkNonProd. If Postmark’s response asks for more evidence, document the additional steps in the execution log; demo and prod are unaffected (different Postmark account).

Per-partition procedure

Substitute <infrastructure> and <partition> for the cascade entry being executed. Examples below use Alpha002 dev; other entries follow the same shape with the bindings from the table above.

Pre-flight (T-O1-{partition})

# 1. AWS SSO login for the partition's profile (skip if a session is already active and unexpired).
aws sso login --profile Alpha002-Admin

# 2. 1Password reference resolves.
op item get "Postmark" --vault "Arda-DevOAM" --format json | jq -r '.fields[] | select(.label=="credential") | .value' | head -c 8
#   Expect: a non-empty token prefix (first 8 chars). If empty, the vault entry is missing or mis-named.

# 3. Pre-merge synth + tests for the partition's stack pass.
bash /Users/jmp/code/arda/projects/email-integration-worktrees/phase-4/documentation/src/content/docs/roadmap/completed/email-integration/4-runtime-platform-updates/plan/runs/run-3-operator-cascade/validate-exit.sh dev
#   Expect: "ALL CHECKS PASSED" with the partition-specific synth check green.

Record results under § <partition> / Pre-flight in the execution log.

Deploy (T-O5-{partition} for dev/stage; T-O6 for demo; T-O7 for prod)

cd /Users/jmp/code/arda/projects/email-integration-worktrees/phase-4/infrastructure-run-3 && \
  ./amm.sh --profile Alpha002-Admin Alpha002 dev

For the other partitions, substitute the profile from the bindings table: --profile Alpha002-Admin for stage, --profile Admin-Alpha1 for both demo and prod.

What this does in sequence:

1. Pre-Deploy CLI runs tools/register-partition-mail-signature.ts against the partition’s bound Postmark account. The script:
   • Resolves the partition’s op://Arda-{Env}OAM/Postmark/credential from 1Password.
   • Lists existing Postmark domains under that account (idempotent — if <partition>.ardamails.com already exists, fetches its DKIM details instead of re-registering).
   • Registers (or fetches) the Sender Signature for <partition>.ardamails.com with ReturnPathDomain: pm-bounces.<partition>.ardamails.com.
   • Writes the active-or-pending DKIM selector, DKIM public key, and Return-Path target into cdk.context.json under the key partitionMail:<infrastructure>:<partition>.
   • Writes the resolved Postmark token to a mktemp-created temp file (mode 0600) for the subsequent cdk deploy to consume via CFN NoEcho parameter.
2. cdk deploy runs against <infrastructure>-<partition>-Email with the token passed in via the NoEcho parameter and --force to ensure CFN propagates rotations. The stack creates:
   • Route 53 hosted zone for <partition>.ardamails.com in the partition’s AWS account.
   • SPF, DMARC, DKIM, and Return-Path CNAME records inside the new zone.
   • Cross-account NS-delegation upstream into ardamails.com (in Platform Root) via the WriteNSRecordsToUpstreamDns custom resource.
   • EmailEncryptionKey SM secret (CFN-native GenerateSecretString).
   • EmailPostmarkAccountToken SM secret (populated from the NoEcho parameter via δ.1).
   • Two STS-assumable IAM roles: EmailDnsProvisioningRole and EmailEncryptionKeyFallbackRole.
   • Six -API- CFN exports for the operations Helm chart to consume.

If cdk deploy fails, CFN rolls back the partial stack and the temp token file is cleaned up. Investigate the cause (CFN events: aws cloudformation describe-stack-events --stack-name <infrastructure>-<partition>-Email --profile <aws-profile> | head -50), address it, and re-run amm.sh for the same partition. Successfully-deployed prior partitions are not affected.

Record amm.sh exit code, the Pre-Deploy CLI outcome (first-time-create vs already-exists), and any anomalies under § <partition> / Deploy in the execution log.

Post-deploy verification (T-O3-{partition})

# Bulk mechanical checks
bash /Users/jmp/code/arda/projects/email-integration-worktrees/phase-4/documentation/src/content/docs/roadmap/completed/email-integration/4-runtime-platform-updates/plan/runs/run-3-operator-cascade/validate-exit.sh dev --post-merge
#   Expect: "ALL CHECKS PASSED". The script verifies:
#     - AWS SSO session, CFN stack status, NS delegation, SPF / DMARC TXT records,
#       both SM secrets, the six -API- CFN exports.

Two operator-driven verifications the script cannot automate:

1. Postmark Console at https://account.postmarkapp.com/signature_domains. Switch to the partition’s bound Postmark account; confirm that <partition>.ardamails.com appears with DKIM marked Verified and Return-Path marked Verified.

   Newly-deployed Sender Domains commonly show “Not Verified” in the Console even after DNS is correct and resolvable. Postmark only re-checks DNS on a periodic schedule (several hours); the Console reflects the last re-check, not live DNS. To force an immediate re-check, call the Postmark API directly:

   TOKEN=$(op read "op://Arda-{Env}OAM/Postmark/credential")
   # find the domain ID
   DOMAIN_ID=$(curl -s -H "X-Postmark-Account-Token: $TOKEN" \
     "https://api.postmarkapp.com/domains?count=500" \
     | jq -r '.Domains[] | select(.Name=="<partition>.ardamails.com") | .ID')
   # trigger DKIM re-check (call once, wait a few seconds, call again to read the result)
   curl -s -X PUT -H "X-Postmark-Account-Token: $TOKEN" \
     "https://api.postmarkapp.com/domains/$DOMAIN_ID/verifyDkim" | jq .
   sleep 3
   curl -s -X PUT -H "X-Postmark-Account-Token: $TOKEN" \
     "https://api.postmarkapp.com/domains/$DOMAIN_ID/verifyDkim" | jq '{DKIMVerified, DKIMHost, DKIMTextValue}'
   # trigger Return-Path re-check (usually verifies on first call)
   curl -s -X PUT -H "X-Postmark-Account-Token: $TOKEN" \
     "https://api.postmarkapp.com/domains/$DOMAIN_ID/verifyReturnPath" | jq '{ReturnPathDomainVerified}'

   The first verifyDkim PUT initiates Postmark’s DNS lookup; the second observes the now-resolved state (Postmark’s check is asynchronous). verifyReturnPath usually returns the verified state on the first call. Refresh the Postmark Console after both APIs return true — the green Verified badges should appear.

2. Final DNS sanity — check the DKIM CNAME and Return-Path CNAME resolve through public DNS:

   dig +short TXT <selector>._domainkey.<partition>.ardamails.com @8.8.8.8
   dig +short CNAME pm-bounces.<partition>.ardamails.com @8.8.8.8

   The <selector> value is the one in cdk.context.json under partitionMail:<infrastructure>:<partition>.dkimSelector. Both should resolve to Postmark-controlled targets.

Record results under § <partition> / Verification in the execution log. Update the per-partition V-check status rows in verification.md (V-OPS-005-{partition}; V-PART-* per-partition rows).

Production deploy confirmation

Before opening the prod cascade entry, the operator records the following in the execution log:

• Both stage and demo cascade entries fully verified.
• Reviewer approval on the Run-3 infra PR’s cdk diff for Alpha001-prod-Email.
• Production-deploy proceeding intentionally — operator signs off here (date + initials in the log).

The prod cascade entry then follows the same procedure as the prior three with --profile Admin-Alpha1 and the Arda-ProdOAM vault. Extra care: confirm cdk diff once more in the worktree before re-running amm.sh (it will report “no changes” if a prior synth populated cdk.context.json correctly; investigate any non-empty diff).

Closeout

When all four partitions are verified and recorded in the execution log:

1. The Run-3 infra PR now contains a single CHANGELOG bullet describing the deployed-system state across all four partitions, the four partitionMail:<infra>:<partition> blocks in cdk.context.json, and any code fixes that emerged during execution.
2. Push the branch, ensure CI is green, request reviewer approval, merge.
3. Run-6 (drift workflow) can open in parallel after the dev cascade entry verified — it does not wait for the cascade to complete.
4. Move the per-partition V-check entries to “complete” in verification.md.

---

Message Stream Discipline

Arda’s mail traffic is transactional only. Every Postmark server configured for Arda — the Free Kanban Tool server today, future per-partition operations servers, future per-tenant servers (Phase 5b) — uses the default Transactional Message Stream. Do not repurpose any of these servers (or their default streams) for marketing, newsletter, or bulk/broadcast sends.

If a future use case requires bulk send (tenant announcements, product updates, periodic digests), provision a separate Broadcast Message Stream on the relevant server per Postmark’s policy (Best Practices for Bulk/Broadcast Sending; How to create and send through message streams). The Broadcast stream is operationally distinct from the Transactional stream — its own throttle, its own suppression list, its own reputation. Co-mingling the two on a single stream both violates Postmark’s policy and risks degrading transactional deliverability for the entire account.

Operator implications:

• When approving a new send-path proposal, confirm the traffic is genuinely transactional (1-to-1, triggered by user action, non-promotional). If it is bulk, the design must specify a dedicated Broadcast stream and a separate operational surface; do not approve “ship it through the existing transactional stream” as a shortcut.
• When investigating a deliverability incident, check the message-stream classification of the offending sends. Broadcast traffic landing on the transactional stream is one of the first hypotheses to rule out.
• Postmark’s account-approval correspondence (e.g., the May 2026 arda-prod approval reply) explicitly lists Broadcast Stream discipline as a deliverability prerequisite — treat it as a standing operational invariant, not a one-time recommendation.

This invariant is also recorded in the project’s cross-cutting design § 7a and is a constraint inherited by Phase 5b’s EmailSender API.

---

Operator Sign-off (REQ-OPS-003)

Record sign-off once all steps are complete and the connectivity check is green. There is no automated gate — this table is the auditable human record. Partial completions are recorded with the partial state in the Deviations column and a forward-pointer in Notes.

Step	Requirement	Operator	Date (YYYY-MM-DD)	Deviations	Notes
1 — PostmarkProd account	REQ-EXT-001	Miguel Pinilla	2026-05-05	None	2FA enabled per Step 1.2; Account-level token generated.
2 — PostmarkProd token in 1Password	REQ-EXT-002	Miguel Pinilla	2026-05-05	None	op://Arda-SystemsOAM/Postmark-Prod/credential resolves; drift-check probe HTTP 200.
3 — PostmarkNonProd account	REQ-EXT-003	Miguel Pinilla	2026-05-05	Partial — 2FA toggle not located in the Postmark Console UI for this account on the date of walkthrough; account otherwise on Platform plan with valid account-level token.	2FA enable URL TBD; deferred. The Postmark-NonProd token resolves and the drift-check probe returns HTTP 200, so Phase 1’s downstream consumers (drift workflow, Phase 4 deploy) are not blocked. Re-run this row when the 2FA toggle is located.
4 — All 1Password items populated	REQ-EXT-004	Miguel Pinilla	2026-05-05	None	All three Phase-1 items (Postmark-Prod, Postmark-NonProd, IAC-SCRIPTS Service Account Token) populated in Arda-SystemsOAM; Free-Kanban-Generator-Postmark-Server is a Phase 3 deliverable in Arda-CorporateOAM per DQ-R1-007.
5 — GHA secret provisioned	REQ-EXT-005 / REQ-CI-001	Miguel Pinilla	2026-05-05	None	OP_SERVICE_ACCOUNT_TOKEN set on Arda-cards/infrastructure via tools/set-gha-repo-secret.sh.
Post-provisioning connectivity check	All above	Miguel Pinilla	2026-05-05	None	npx ts-node tools/drift-check.ts returned 5 passed / 0 failed.
T-C5 — First workflow_dispatch run (post-merge)	REQ-CI-002 / V-CI-102			Deferred — requires PR #446 to merge to main first; GitHub Actions returns HTTP 404 for workflow_dispatch against a workflow file not on the default branch.	Run once immediately after merge; capture run-id and link in Notes.
T-C7 — gh secret list audit (Postmark name match)	REQ-CI-003 / V-CI-103	Miguel Pinilla	2026-05-05	None	Audit returned 1 leftover (POSTMARK_NONPROD_ACCOUNT_TOKEN from the prior Phase-0 era); deleted with gh secret delete. Re-audit returned zero matches.

---

References

• Postmark Service Overview — account topology, credential storage model, OAM model, drift cadence.
• Postmark API Observations — authentication models, error model, retry conventions, version-pin assumptions.
• Encryption-Key Rotation Runbook — rotating a partition’s EmailEncryptionKey SM secret without dropping in-flight tokens.
• Partition Mail Topology — per-partition mail surface reference (sub-zones, IAM roles, CFN exports).
• Phase 1 Requirements — REQ-EXT-001 through REQ-OPS-004 traced here.
• Phase 1 Specification — Task 1 implementation details.
• Phase 4 Run-3 Operator Cascade — run plan for the partition-deploy cascade documented above.
• Phase 4 Run-3 Execution Log — living artefact captured as the cascade is executed.
• Postmark Account API reference: https://postmarkapp.com/developer/api/account-api
• 1Password CLI reference: https://developer.1password.com/docs/cli/reference