Idempotency Capability

The idempotency capability lets an L3 service safely absorb inbound retries (a caller calling twice for the same operation) and produce stable keys for outbound retry-safe calls (the service calling a downstream system multiple times for the same logical attempt). It ships as a self-contained unit in cards.arda.common.lib.runtime.idempotency, with no schema migration coupled to the library — each consuming service ships its own idempotency_record Flyway migration.

1. Specification

The capability provides three abstractions:

A1 — Deterministic key from natural-key parts. Given a list of natural-key strings and a ConsumerNamespace, produce an IdempotencyKey whose value is a stable hash of the parts. Same parts in the same order produce the same key across invocations, processes, and JVM restarts.

A2 — Caller-supplied key parsing. Given a raw String? (typically from an HTTP header) and a ConsumerNamespace, produce a validated Result<IdempotencyKey?> that returns success-with-null when the input is absent, success with a key when the input is well-formed, and Result.failure(AppError.Invocation.GeneralValidation) when the input is blank, oversize, or contains control characters.

A3 — Begin / commit / fail dispatch. Given an IdempotencyKey and a request payload, the store either lets the caller proceed (FreshAttempt), replays a prior committed result (PriorResult), replays a prior permanent error (PriorError), signals concurrent in-flight processing (InFlight), or rejects a payload-mismatch against a prior key (Mismatch carrying the recorded request). Terminal transitions are commit(result) (persists for replay), failPermanent(error) (persists for replay), or failTransient() (deletes the row so the next begin() is fresh).

The capability has three invariants the L3 service can rely on:

• Same (key, payload) replays exactly once-effective. A successful commit makes every subsequent begin for the same key+payload return PriorResult — the operation does not re-execute.
• Different payload under same key is rejected, not overwritten. Mismatch carries recordedRequest so the consumer can diagnose the divergence; the prior row is never silently replaced.
• Caller owns the transaction. All store methods execute within the caller’s transaction. The factory binds the store to a Connection / Exposed Transaction; the store never opens or closes a transaction (DQ-208).

Two boundary shapes are exposed: a native boundary on kotlinx.serialization.json.JsonElement (RawIdempotencyStore) for callers that need explicit control over the hash projection (e.g. stripping transient fields before hashing), and a typed wrapper (IdempotencyStore<Req, Res>) that captures KSerializer<Req> / KSerializer<Res> once at wiring time. Decode failures at replay time surface as Result.failure(AppError.Internal.IncompatibleState) — bug-class signal that schema drifted without a coordinated drain.

The idempotency_record table is shared across consumers in the same database; the namespace column partitions rows so two consumers’ keyspaces never collide. The Flyway migration that creates the table is not shipped by common-module; each consuming component ships its own migration so the library’s release cadence is decoupled from schema deployment.

2. Functional Elements

The capability decomposes into a core package (key types and outcome variants — no I/O) and a service package (the store interface, the Exposed-backed implementation, and the typed wrapper).

FE-1: IdempotencyKey + ConsumerNamespace

ConsumerNamespace is a @JvmInline value class that enforces a 64-char lowercase/digit/-/_ alphabet at construction. IdempotencyKey is a @ConsistentCopyVisibility data class with an internal constructor — keys cannot be hand-built; they come exclusively from the minter (FE-2) or the parser (FE-3) so format invariants always hold. Value length is bounded at 255 characters.

FE-2: IdempotencyKeyMinter

Bound to a single ConsumerNamespace at construction. mint(parts: List<String>) returns Result<IdempotencyKey>; the value is SHA-256 over a length-prefixed canonical encoding: a 4-byte big-endian part count, then each part encoded as a 4-byte big-endian UTF-8 byte length followed by the UTF-8 bytes. Hex-encoded to 64 characters. Empty parts or a blank part returns Result.failure(AppError.Invocation.GeneralValidation).

FE-3: parseIdempotencyKey

Top-level function parseIdempotencyKey(namespace, rawValue: String?): Result<IdempotencyKey?>. Returns success-with-null for null input. Trims, then rejects blank, oversize (>255 chars), or strings containing control characters (< 0x20 or 0x7F). Otherwise returns the validated key.

FE-4: CanonicalJson

internal object exposing canonicalUtf8(element: JsonElement): ByteArray. Normalises by sorting JsonObject entries by Unicode code-point order recursively; JsonArray order is preserved (array order is semantically meaningful in JSON). Re-serialised without pretty-printing. The store hashes the canonical bytes with SHA-256 to produce the request_hash column.

FE-5: IdempotencyOutcome

Sealed class with five variants — FreshAttempt, InFlight, PriorResult(value), PriorError(error), Mismatch(recordedRequest, recordedHash, submittedHash). Mismatch.equals / hashCode are overridden to compare recordedHash / submittedHash by content rather than reference. The typed wrapper exposes a parallel TypedIdempotencyOutcome<Req, Res> with the same tree.

FE-6: IdempotencyStoreFactory

Stateless factory parameterised by tableName, namespace, and replayWindow. Produces a RawIdempotencyStore bound to a caller-owned transaction context via one of three methods:

• inTransaction(connection) — hard guard: asserts the connection already has an active transaction (auto-commit off, isolation != NONE) and returns Result.failure otherwise.
• inConnection(connection) — accepts any state; the caller is responsible for commit/rollback.
• withTx(tx) — accepts an Exposed Transaction; the most common path in Exposed-based L3 services.

Mirrors the DatabaseBackedMapFactory pattern and honours DQ-208 (the helper never opens or closes a transaction).

FE-7: RawIdempotencyStore and typed wrapper

RawIdempotencyStore is the storage interface, symmetric on JsonElement for both request and result. IdempotencyStore<Req, Res> is the typed wrapper produced by rawStore.typedAs<Req, Res>(json = ...) — an inline extension that resolves KSerializer<Req> and KSerializer<Res> once at wiring time and constructs the internal IdempotencyStoreImpl delegating to the raw store. ExposedRawIdempotencyStore is the internal Exposed/JDBC implementation; it never opens or closes a transaction, operates on a caller-supplied Connection, and dispatches against the idempotency_record row by (namespace, key_value).

FE-8: StoredAppError projection

Internal @Serializable data class capturing classTag, message, and an optional non-recursive cause string. Stored in error_payload for failPermanent. On replay, the typed wrapper reconstructs an AppError subtype from classTag; unknown tags surface as AppError.Internal.Implementation with the original message preserved (operational signal that the consumer’s hierarchy evolved).

3. Behaviors

3.1 Inbound retry — replay on second call

A consumer receives the same logical request twice (e.g. a Postmark webhook delivered twice within the retry window). The first call passes through FreshAttempt and commits; the second sees PriorResult and returns the recorded outcome without re-executing.

3.2 Transition state machine

A begin() call either inserts a fresh in_progress row (FreshAttempt) or short-circuits with a replay outcome. From FreshAttempt, the caller chooses one of three terminal transitions.

3.3 Concurrent in-flight detection

Two threads (or pods) call begin() for the same key concurrently. The unique (namespace, key_value) index makes one INSERT win; the loser’s INSERT collides, the store re-reads the row and finds status = in_progress, and returns InFlight. The loser does not execute the operation; it surfaces the contention to its caller, which typically responds 409 / retry-after.

3.4 Payload mismatch

A second begin() arrives with the same key but a different canonical-hash payload. The store returns Mismatch(recordedRequest, recordedHash, submittedHash). The L3 service must reject the new request (typically 422 with diagnostics) — accepting it would silently overwrite the prior result. recordedRequest is the prior request’s JsonElement as it was first committed, for operational debugging.

3.5 Outbound retry safety via deterministic key

A service calls a downstream system that requires an idempotency token (e.g. Postmark X-Idempotency-Key). The service uses IdempotencyKeyMinter.mint(parts) over the natural-key parts that define the logical attempt (e.g. tenant id, job id, attempt number). Identical inputs yield identical keys; the downstream system deduplicates server-side. The L3 service’s own row optionally co-records the result for in-process replay.

3.6 Expiration

Rows carry expires_at = now + replayWindow. Expiration is enforced lazily via purgeExpired(), not at read time — expired rows continue to replay until they are physically deleted. Consumers must schedule a periodic purgeExpired call; without it the table grows unbounded and expired results continue replaying indefinitely.

3.7 Decode failure on replay (typed wrapper)

The typed wrapper deserializes the stored result_payload against KSerializer<Res> at replay time. If Res has evolved without a coordinated drain (the stored bytes are no longer decodable to the current type), the wrapper returns Result.failure(AppError.Internal.IncompatibleState). This is bug-class, not recoverable application state — the operator must drain or migrate before the consumer can serve traffic again. For Mismatch, recordedRequest decode failures are non-fatal: the mismatch itself is the actionable signal, and a decode failure on the historical request is surfaced through the Mismatch.recordedRequest: Result<Req> field.

4. Verification

Two test layers exist: unit tests for the core types (pure, no DB) and integration tests for the service layer running against a containerized Postgres.

Test scope	What it asserts
ConsumerNamespaceTest	Alphabet, length cap, blank rejection.
IdempotencyKeyTest	Constructor invariants (blank, oversize).
IdempotencyKeyMinterTest	Determinism (same parts → same key), distinctness (different parts / order → different keys), Result.failure for empty parts and blank parts.
IdempotencyKeyParsingTest	null → success-with-null; blank / oversize / control-char → GeneralValidation; trimming; valid value pass-through.
CanonicalJsonTest	Object-key ordering normalised; array order preserved; whitespace stripped; same logical input → byte-identical output.
RawIdempotencyStoreTest	Round-trip of every IdempotencyOutcome variant against real Postgres: FreshAttempt insert, PriorResult after commit, PriorError after failPermanent, Mismatch on hash divergence, InFlight under concurrent INSERT, deletion on failTransient, purgeExpired count.
IdempotencyStoreTypedTest	Typed wrapper round-trips, decode-failure → AppError.Internal.IncompatibleState when stored shape drifts, Mismatch.recordedRequest carrying decoded Req (or surfaced decode failure).
IdempotencyStoreHarness	Containerized-Postgres fixture: spins up the database, applies the test-only V001__idempotency_record.sql migration, hands tests a clean idempotency_record table per scenario.

5. Implementation Artifacts

All paths are relative to Arda-cards/common-module/.

Functional Elements

File	Role
lib/src/main/kotlin/cards/arda/common/lib/runtime/idempotency/core/ConsumerNamespace.kt	@JvmInline value class with alphabet + length invariants.
lib/src/main/kotlin/cards/arda/common/lib/runtime/idempotency/core/IdempotencyKey.kt	data class with internal constructor + length invariant; only the minter and parser produce instances.
lib/src/main/kotlin/cards/arda/common/lib/runtime/idempotency/core/IdempotencyKeyMinter.kt	Deterministic SHA-256 hashing of length-prefixed UTF-8 parts; bound to a single ConsumerNamespace.
lib/src/main/kotlin/cards/arda/common/lib/runtime/idempotency/core/IdempotencyKeyParsing.kt	Top-level parseIdempotencyKey(...) for caller-supplied raw values.
lib/src/main/kotlin/cards/arda/common/lib/runtime/idempotency/core/IdempotencyOutcome.kt	Sealed hierarchy: FreshAttempt, InFlight, PriorResult, PriorError, Mismatch.
lib/src/main/kotlin/cards/arda/common/lib/runtime/idempotency/core/CanonicalJson.kt	internal object producing canonical UTF-8 bytes for hashing.
lib/src/main/kotlin/cards/arda/common/lib/runtime/idempotency/service/RawIdempotencyStore.kt	Native JsonElement-symmetric storage interface.
lib/src/main/kotlin/cards/arda/common/lib/runtime/idempotency/service/IdempotencyStore.kt	Typed wrapper interface IdempotencyStore<Req, Res> + TypedIdempotencyOutcome<Req, Res>.
lib/src/main/kotlin/cards/arda/common/lib/runtime/idempotency/service/IdempotencyStoreImpl.kt	@PublishedApi internal typed wrapper; encodes Req and decodes Res via captured KSerializers.
lib/src/main/kotlin/cards/arda/common/lib/runtime/idempotency/service/TypedAs.kt	inline fun <reified Req, reified Res> RawIdempotencyStore.typedAs(json) — resolves serializers and constructs the impl.
lib/src/main/kotlin/cards/arda/common/lib/runtime/idempotency/service/IdempotencyStoreFactory.kt	Three tx-binding entry points (inTransaction / inConnection / withTx).
lib/src/main/kotlin/cards/arda/common/lib/runtime/idempotency/service/ExposedRawIdempotencyStore.kt	Internal JDBC/Exposed-backed implementation; SQL against idempotency_record; canonical-JSON hashing; StoredAppError projection.

Verification

File	Scope
lib/src/test/kotlin/cards/arda/common/lib/runtime/idempotency/core/ConsumerNamespaceTest.kt	Unit.
lib/src/test/kotlin/cards/arda/common/lib/runtime/idempotency/core/IdempotencyKeyTest.kt	Unit.
lib/src/test/kotlin/cards/arda/common/lib/runtime/idempotency/core/IdempotencyKeyMinterTest.kt	Unit.
lib/src/test/kotlin/cards/arda/common/lib/runtime/idempotency/core/IdempotencyKeyParsingTest.kt	Unit.
lib/src/test/kotlin/cards/arda/common/lib/runtime/idempotency/core/CanonicalJsonTest.kt	Unit.
lib/src/test/kotlin/cards/arda/common/lib/runtime/idempotency/service/RawIdempotencyStoreTest.kt	Integration (ContainerizedPostgres).
lib/src/test/kotlin/cards/arda/common/lib/runtime/idempotency/service/IdempotencyStoreTypedTest.kt	Integration (ContainerizedPostgres).
lib/src/test/kotlin/cards/arda/common/lib/runtime/idempotency/service/IdempotencyStoreHarness.kt	Test fixture + test-only Flyway migration V001__idempotency_record.sql.

Releases

Release	Subject	PR
common-module 11.1.0	Initial idempotency package (core/ + service/, raw + typed APIs).	#184

Consumer responsibilities (not shipped by common-module)

Each consuming service ships its own:

• Flyway migration creating the idempotency_record table in its own database.
• IdempotencyStoreFactory wiring at module-init time with the consumer’s ConsumerNamespace, table name, and replayWindow.
• Scheduled purgeExpired() invocation (cron, timer, or operator-driven).
• typedAs<Req, Res>(json = ...) at wiring time per logical operation, with a stable Json configuration that survives Req schema evolution.

Serialization

TypedIdempotencyOutcome is not currently annotated @Serializable in common-module (tracked in PDEV-768). When an endpoint must serialize it onto the wire, supply a point-of-use KSerializer via the full RequiredBodyMessage(name, kType, kSerializer, typeInfo) constructor. See Endpoint Definition DSL — Body message declarations for the constructor form.

See EmailJobIdempotencyOutcomeSerializer in cards.arda.operations.shopaccess.email.api.rest for the reference implementation. Notes on its wire shape:

• 5 variants under a kind discriminator: Created, PreExisting, OperationInProgress, Mismatch, Failed.
• The Mismatch.recordedRequest field is deliberately omitted from the wire response — clients receive only the kind and minimal metadata. The full recorded request is available for in-process debugging (e.g., operational tooling) but is not exposed to API callers.
• AppError is flat-encoded (message and code fields; no recursive cause chain).

This serializer is a workaround. Delete it and replace with typedAs<Req, Res>(json) once PDEV-768 lands and TypedIdempotencyOutcome becomes directly @Serializable.