Email Integration -- Dependency Analysis

Implementation ordering and verification strategy for the email integration project.

Implementation Order

Resolved Decisions

Both previously-blocking decisions are now resolved:

Decision	Resolution	Notes
DQ-009: Mail root domain	ardamails.com (implementation parametric)	DNS zone creation can proceed
DQ-003: Tenant slug source	From provisioning request (tenantEId, tenantName, tenantSlug); algorithm deferred	Provisioning flow can proceed

Phase 1: Infrastructure

Resources created manually or via CDK. No code dependencies.

Step	Resources	Scope	Blocked by
1.1	PM-1, PM-2, PM-3 (Postmark accounts + Platform plan)	Infrastructure	—
1.2	PM-4, PM-5 (Postmark API tokens)	Infrastructure	1.1
1.3	SM-5..SM-8 (encryption keys)	Partition	—
1.4	SM-1..SM-4 (Postmark account tokens in Secrets Manager)	Partition	1.2
1.5	DNS-Z1 (root zone)	Platform	—
1.6	DNS-R1, DNS-R2 (SPF, DMARC in root zone)	Platform	1.5
1.7	DNS-Z2..DNS-Z5 (partition zones)	Partition	—
1.8	DNS-R3..DNS-R6 (NS delegations in root zone)	Partition	1.5, 1.7
1.9	IAM-1, IAM-2 (DNS provisioning roles)	Infrastructure	1.7

Phase 2: Postmark API Clients

Standalone HTTP client classes wrapping the Postmark REST API. No dependency on the module skeleton, Ktor, persistence, or any Arda framework code. Can be developed and tested as a library against real Postmark sandbox accounts.

Step	What	Depends on
2.1	Postmark Account API client (server CRUD, domain CRUD, webhook CRUD, DNS verification)	Phase 1 (PM-1..PM-5 for tokens)
2.2	Postmark Server API client (send email, bounce management, message search)	Phase 1 (PM-1..PM-5 for tokens)
2.3	Route53 DNS client (create/delete TXT, CNAME, MX records; assumes provisioning role via STS)	Phase 1 (IAM-1/IAM-2, DNS zones)

Phase 3: Module Skeleton

Set up the ShopAccess/Email module within the operations component. No business logic yet.

Step	What	Depends on
3.1	Module bootstrap: application.conf, Flyway migration location, module registration in Main.kt	—
3.2	Helm values: apis.system.shopAccess.email entry for Ingress routing	—
3.3	ESO ExternalSecret entries for email secrets in Helm secrets.yaml	Phase 1 (SM-1..SM-8 exist)
3.4	HOCON configuration entries (non-secret values)	—

Phase 4: EmailConfiguration Service and Persistence

Build the emailConfiguration service with its DataAuthority, persistence, and encryption. Stub Postmark and Route53 initially.

Step	What	Depends on
4.1	EmailConfiguration entity, Flyway migrations (tenant_email_config table)	3.1
4.2	Encryption logic: encrypt/decrypt server token with partition key	3.3 (encryption key available)
4.3	EmailConfigurationService interface + stub implementation (no Postmark, no Route53)	4.1, 4.2
4.4	email-configuration endpoint (routes, request/response mapping)	4.3
4.5	Lifecycle state machine	4.3

Phase 5: EmailJob Service and Persistence

Build the emailJob service with its DataAuthority and lifecycle. Stub ESP initially.

Step	What	Depends on
5.1	EmailJob entity, Flyway migrations (email_job table)	3.1
5.2	EmailJobService interface + stub implementation (no Postmark)	5.1, 4.3 (needs getUnlockedConfiguration())
5.3	email-job endpoint (routes, request/response mapping)	5.2
5.4	Message Events webhook endpoint	5.2
5.5	Lifecycle state machine	5.2

Phase 6: Integration Wiring

Replace stubs with real Postmark and Route53 clients. Wire everything together.

Step	What	Depends on
6.1	Wire emailConfiguration provisioning to Postmark Account API client + Route53 client	4.3, 2.1, 2.3, Phase 1 (IAM roles, DNS zones)
6.2	Wire emailJob sending to Postmark Server API client	5.2, 2.2
6.3	Wire webhook endpoint to receive real Postmark events	5.4, 6.1 (webhook configured on server)
6.4	Trigger-driven bounded DNS verification (provision-success, retry-verification, send-time precondition fail); see DQ-207	6.1
6.5	End-to-end: provision tenant, send email, receive delivery event	6.1, 6.2, 6.3, 6.4

Out of Scope: BFF and Frontend

BFF route definitions and SPA components are deferred to the definition of the Procurement use cases that exercise the email capability. This project delivers the backend ShopAccess/Email module and its infrastructure.

---

Dependency Graph

Critical Path

The longest dependency chain:

Phase 3 (skeleton) → Phase 4 (config) → Phase 5 (job) → Phase 6 (wiring) → E2E test

Three workstreams can run in parallel from the start:

• Infrastructure (Phase 1): Postmark accounts, secrets, DNS zones, IAM roles
• Postmark API clients (Phase 2): standalone HTTP clients, testable against sandbox
• Module skeleton (Phase 3): Ktor module, Helm, ESO, HOCON

Phase 2 needs tokens from Phase 1 for integration testing, but client code can be written before tokens exist (unit-tested with mocks).

---

Verification Strategy

Phase 1: Infrastructure

One-off verification scripts and manual checks. Not ongoing tests.

What	How	Produces
DNS zones exist and delegate correctly	Shell script: dig NS ardamails.com, dig NS dev.ardamails.com, etc. Verify nameservers match expected Route53 zone.	scripts/verify-dns-zones.sh
SPF and DMARC records published	Shell script: dig TXT ardamails.com, dig TXT _dmarc.ardamails.com. Verify expected values.	Part of verify-dns-zones.sh
Secrets exist in Secrets Manager	Shell script: aws secretsmanager get-secret-value --secret-id <name> for each SM-1..SM-8. Verify non-empty.	scripts/verify-secrets.sh (per account)
Postmark accounts functional	Shell script: call Postmark API GET /servers with each account token. Verify 200 response.	scripts/verify-postmark-accounts.sh
IAM roles assumable	Shell script: aws sts assume-role --role-arn <role> from a pod service account context. Verify success.	scripts/verify-iam-roles.sh

Phase 2: Postmark API Clients

Unit tests with mocked HTTP + integration tests against PostmarkNonProd sandbox.

What	How	Produces
Account API client correctness	Unit tests: mock Postmark HTTP responses for server/domain/webhook CRUD, verify request shapes and response parsing.	PostmarkAccountApiClientTest (unit)
Account API client integration	Integration test against PostmarkNonProd: create server, create domain, configure webhook, verify domain, delete domain, delete server. Cleanup on teardown.	PostmarkAccountApiClientIntegrationTest
Server API client correctness	Unit tests: mock Postmark HTTP responses for send, bounce list, message search.	PostmarkServerApiClientTest (unit)
Server API client integration	Integration test against PostmarkNonProd sandbox: send email (accepted but blackholed), query sent messages.	PostmarkServerApiClientIntegrationTest
Route53 DNS client correctness	Unit tests: mock AWS SDK responses for create/delete record sets.	Route53DnsClientTest (unit)
Route53 DNS client integration	Integration test against dev partition zone: create TXT record _test-<timestamp>.dev.ardamails.com, verify with dig, delete. Cleanup on teardown.	Route53DnsClientIntegrationTest

Phase 3: Module Skeleton

Deployment-time checks. Most are validated by the existing component test/deploy pipeline.

What	How	Produces
Module loads without errors	Existing component startup test: add email module to Main.kt, verify no startup exceptions.	Extended existing ApplicationStartupTest
Ingress route reachable	Helm template rendering: helm template produces valid YAML with /v1/shop-access/email/* path.	helm template validation in CI
ESO secrets resolve	Deploy to dev cluster, verify secrets.properties contains email entries. Manual or automated deploy check.	Deploy verification step
HOCON config loads	Component startup test: email config properties are non-null after startup.	Extended existing ApplicationStartupTest

Phase 4: EmailConfiguration Service

Unit tests with containerized Postgres (existing Arda test harness) + endpoint tests.

What	How	Produces
Flyway migrations run	Test with ContainerizedPostgres: migrations apply without error, schema matches expectations.	EmailConfigurationMigrationTest
Entity CRUD	Test with ContainerizedPostgres: create, read, update, query EmailConfiguration entities via DataAuthority.	EmailConfigurationDataAuthorityTest
Encryption round-trip	Unit test: encrypt a token, decrypt it, verify match. Test with wrong key fails.	ServerTokenEncryptionTest
Service: provision with stub (persist-first lifecycle)	Test with ContainerizedPostgres + stub Postmark/Route53 + stub tenantProvisioning L2: provision() INSERTs row in PROVISIONING (DQ-205.j), runs checkAvailability pre-flight (DQ-205.i), executes Postmark-then-Route53 ordering (DQ-205.k), UPDATEs to PENDING_VERIFICATION on success or PROVISIONING_FAILED on partial-progress failure (DQ-205.c). Correct slugs (DQ-206), AES-256-GCM-encrypted token (DQ-202).	EmailConfigurationServiceTest
Service: getUnlockedConfiguration	Test: returns decrypted token for UNLOCKED; for PENDING_VERIFICATION fires off bounded polling kick-off (deduplicated via activePolling, DQ-207.b) and returns PreconditionFailed; for LOCKED / PROVISIONING / PROVISIONING_FAILED / VERIFICATION_FAILED returns PreconditionFailed without kick-off.	Part of EmailConfigurationServiceTest
Service: lifecycle transitions	Test transitions: PROVISIONING → PENDING_VERIFICATION (DQ-205.a), PROVISIONING → PROVISIONING_FAILED, PENDING_VERIFICATION → UNLOCKED (during bounded polling), PENDING_VERIFICATION → PENDING_VERIFICATION (round exhaustion, DQ-207), VERIFICATION_FAILED → PENDING_VERIFICATION (manual retry only; not entered automatically in v1, DQ-207.h), UNLOCKED ↔ LOCKED, deletes from any non-PROVISIONING state run best-effort decommission (DQ-205.d). Invalid transitions return error.	EmailConfigurationLifecycleTest
Endpoint: routes	HTTP test: POST creates config (201), GET returns config (200), PUT retry-verification kicks off bounded polling (200 from PENDING_VERIFICATION or VERIFICATION_FAILED, 409 otherwise; DQ-207.b), PUT lock/unlock (state-machine guards), DELETE invokes best-effort decommission (DQ-205.d).	EmailConfigurationEndpointTest

Phase 5: EmailJob Service

Unit tests with containerized Postgres + endpoint tests including webhook.

What	How	Produces
Flyway migrations run	Test with ContainerizedPostgres: migrations apply, schema correct.	EmailJobMigrationTest
Entity CRUD	Test with ContainerizedPostgres: create, read, update, query EmailJob entities via DataAuthority.	EmailJobDataAuthorityTest
Service: createAndSend with stub	Test: creates job (NEW), calls stub ESP, transitions to QUEUED, stores messageId. With LOCKED config returns error.	EmailJobServiceTest
Service: cancelJob	Test: cancel NEW job → CANCELLED. Cancel QUEUED job → error.	Part of EmailJobServiceTest
Service: resendJob	Test: resend BOUNCED or FAILED job → new job created with NEW status, original retained.	Part of EmailJobServiceTest
Service: handleDeliveryEvent	Test: Delivery event transitions QUEUED/SENT → DELIVERED. Bounce transitions QUEUED/SENT → BOUNCED with diagnostic. SpamComplaint transitions DELIVERED → COMPLAINED. Unknown MessageID logged and ignored.	EmailJobEventHandlerTest
Service: lifecycle transitions	Test all valid transitions including out-of-order (QUEUED → DELIVERED skipping SENT). Invalid transitions rejected.	EmailJobLifecycleTest
Endpoint: email-job routes	HTTP test: POST creates job (201), GET returns status (200), GET details returns full body, PUT resend creates new job (201), DELETE cancels (200 or 409).	EmailJobEndpointTest
Endpoint: webhook	HTTP test: valid Bearer token + valid event → 200, job status updated. Invalid Bearer token → 403. Malformed payload → 400. Unknown MessageID → 200 (idempotent).	PostmarkWebhookEndpointTest

Phase 6: Integration Wiring

Integration tests against real external services (PostmarkNonProd sandbox, dev DNS zone).

What	How	Produces
Provisioning with real Postmark + Route53	Integration test: call provision() with real Postmark Account API + Route53 client. Verify Postmark server created, domain registered, DNS records published, webhook configured. Cleanup: delete server and domain on teardown.	EmailConfigurationProvisioningIntegrationTest
Trigger-driven DNS verification (DQ-207)	Integration test: (a) provision tenant → bounded polling kicks off → wait for verification → assert config transitions to UNLOCKED. (b) Force a PENDING_VERIFICATION row past first round exhaustion (mock domain not yet propagated) → assert row stays in PENDING_VERIFICATION → call getUnlockedConfiguration → assert PreconditionFailed AND new bounded polling round started → eventually transitions to UNLOCKED. (c) Manual /retry-verification while PENDING_VERIFICATION → assert new round is kicked off (deduplicated).	DnsVerificationIntegrationTest
Send with real Postmark	Integration test: provision tenant (UNLOCKED), call createAndSend() with real Postmark Server API. Verify Postmark accepts message (QUEUED). Sandbox drops the email.	EmailJobSendIntegrationTest
Webhook with real Postmark events	Integration test: send email via provisioned config, receive delivery event from Postmark at webhook endpoint, verify job status transitions. May require a live (non-sandbox) Postmark server in demo partition for real delivery events.	PostmarkWebhookIntegrationTest
End-to-end cycle	Integration test: provision tenant → send email → receive delivery event → verify final status. Full cycle against dev or demo partition.	EmailEndToEndTest

Test Infrastructure Requirements

Requirement	Phase	Notes
ContainerizedPostgres (existing)	4, 5	Existing Arda test harness for database tests
MockK / mock HTTP client	2, 4, 5	Unit tests for API clients and services with stub externals
PostmarkNonProd sandbox account	2, 6	Real API calls for integration tests; sandbox drops emails
PostmarkProd demo server	6 (webhook test)	Real delivery events require a live server; demo partition
Dev partition DNS zone	2, 6	Route53 integration tests create/delete test records
Dev partition IAM role	2, 6	Route53 client integration tests assume the provisioning role
Ktor test host (existing)	4, 5	Existing pattern for HTTP endpoint tests