Backend Testing

Reference guide for writing reliable backend tests in Kotlin. Covers unit testing patterns with Kotest and MockK, followed by test infrastructure setup for containerized databases and AWS service emulation.

---

Tools and Libraries

Purpose	Library
Test framework	kotlin.test + org.junit.jupiter or kotest (pick one per module)
Assertions	kotlin.test assertions or Kotest matchers; prefer JUnit assertions
Mocking	MockK
Containerized dependencies	testcontainers via ContainerizedPostgres
AWS service emulation	LocalStack via MockAWS

---

Unit Testing

General Practices

• Use meaningful test names that describe the behavior being tested, not the method name.
• Provide assertion messages so failures are self-explanatory during diagnostics.
• Never use println for logging in test code committed to the repository.
• Move common setup code to a Harness object (see The Harness Pattern) to keep tests focused on the behavior under test.

Testing with MockK

Always Use Named Mock Variables

Never pass mockk() inline in constructors. When the mock later requires stubs, a refactor is needed. Always assign mocks to named variables even when no stubs are initially needed.

// Avoid
val service = ItemService.Impl(mockk(), mockk(), itemDb)

// Prefer
val supplierService = mockk<SupplierService>()
val itemDb = mockk<Database>()
val service = ItemService.Impl(supplierService, itemDb)

Reset Mocks Between Tests

Reset all mocks between tests to avoid state leakage. Use clearMocks() or clearAllMocks() in beforeEach.

Never Use Relaxed Mocks for Complex Generics

relaxed = true cannot generate valid default return values for deeply nested generic types like Result<EntityRecord<X, Y>?>. It returns Result.success(Object()), which causes a ClassCastException at runtime.

Do not use relaxed mocks unless the dependency is a low-level service not directly exercised by the code under test. Always provide explicit coEvery stubs for complex generic returns.

Set Up Stubs with Specific Parameter Values

Set up every and coEvery blocks in each test case. Use specific parameter values when known instead of any().

Verify Exact Invocation Counts

Always verify the exact number of invocations using exactly = n where n is the expected count. Use verifyOrder or verifySequence when the order of invocations matters.

Capture Parameters with Slots

Use slot to capture parameters passed to mocks for verification when exact argument values matter.

DBIO Mock Return Syntax

When mocking methods returning DBIO (a suspend lambda), use the lambda form:

// Correct
coEvery { universe.create(any(), any(), any(), any()) } returns { Result.success(record) }

// Incorrect — will not work for DBIO
coEvery { universe.create(any(), any(), any(), any()) } returns Result.success(record)

Simplify Matchers to Avoid Reflection Errors

Mocking the Universe interface’s invoke pattern (universe.create(...)()) can cause KotlinReflectionInternalError. Use any<Type>() matchers with returnsMany instead of complex match<> lambdas.

Testing Services That Resolve References

When a service resolves or links references on create/update — for example, supplier resolution that funnels an ItemSupply through BusinessAffiliateService to find, verify, or create a VENDOR BusinessRole — the collaborator mock must stub every method the resolution path actually calls, not just the happy-path one.

For supplier resolution the full set is detailsFor, businessRolesFor (the eId-match check that the supplied role id is the affiliate’s live VENDOR role), findByNameAndRole, add, and createBusinessRole. Miss one and the test fails deep inside the service rather than at the assertion:

• A relaxed mock returns uncastable defaults for complex generics (Result<EntityRecord<X, Y>?> becomes Result.success(Object())), producing a ClassCastException.
• A strict mock missing a stub throws MockKException at the unstubbed call.

More specific coEvery argument matchers take precedence over any(), so a generic stub composes with a per-test specific one:

// Harness sets a broad default once...
coEvery { baService.detailsFor(any(), any()) } returns Result.success(details)
coEvery { baService.businessRolesFor(any(), any()) } returns Result.success(listOf(vendorRole))

// ...and an individual test overrides the one call it cares about.
coEvery { baService.detailsFor(affiliateEId, any()) } returns Result.success(null)

ItemVendorResolverTest follows this shape: the harness’s setupVendor, setupFindByNameAndRole, and per-test coEvery blocks together cover the resolution path for each branch (STRICT / LAX / PROPAGATE).

Bitemporal Cross-Tenant Isolation Tests

For any tenant-scoped (or parent-scoped) read, prove isolation directly: seed entities in two tenants plus a retired parent, then assert the query returns only the caller-tenant’s live rows — and that the same id under another tenant is never returned.

• CrossItemSupplyUniverseTest seeds supplies under tenant1 and tenant2 (plus a retired parent Item and an item referencing a different role), then asserts findSuppliesBySupplierRole returns only tenant1’s live items for tenant1’s context, and only tenant2’s item for tenant2’s context.
• BusinessRoleUniverseTest (readIncludingRetired is scoped to the parent affiliate ...) covers the parent-scoped equivalent: a role added under affiliate A is invisible to affiliate B’s universe, and remains scoped after retirement (returned with retired = true only to the owning affiliate).

@JvmInline value class Parameters Break coVerify with any()

MockK’s ValueClassAwareCaller has a long-standing bug: coVerify(exactly = N) { mock.fn(any()) } may throw a reflection error or non-deterministically fail when the mocked function takes a @JvmInline value class parameter. The same call with a concrete instance (coVerify { mock.fn(specificId) }) is unaffected.

Workarounds, in order of preference:

1. Prefer structural assertions. When a probe / stub is exhaustive by construction (its when over a sealed return type covers every branch), the no-call invariant is structural, not behavioral — a coVerify(exactly = 0) { ... } adds no extra coverage. Delete the verify.

2. Use a concrete value-class instance.

   coVerify(exactly = 1) { proxy.getSignature(PostmarkSignatureId(42L).getOrThrow()) }

3. Use any<T>() with the concrete type parameter, not the unqualified any():

   coVerify(exactly = 1) { proxy.getSignature(any<PostmarkSignatureId>()) }

   The typed any<T>() works around the reflection path that the unqualified any() triggers.

If any() is unavoidable and the bug persists, capture the parameter with a slot<>() and assert on the slot’s captured value after the call.

The bug is cross-repo (any Arda Kotlin codebase using MockK plus @JvmInline value class); knowing this saves a debugging session every time a refactor introduces a value-class parameter.

Kotest-Specific Notes

Matchers Do Not Accept Message Parameters

shouldBeSuccess() and shouldBeFailure() interpret string arguments as expected values, not assertion messages. To include a diagnostic message, use:

result.isSuccess shouldBe true

Verify Test Coverage Before Writing New Tests

Before creating new tests, verify existing coverage at different layers to avoid duplicating what is already there.

---

The Harness Pattern

All auxiliary methods, functions, and classes that support tests belong in a Harness object rather than scattered inline. This keeps test bodies focused on the behavior under test.

Visibility rules:

• Private to the test file when the harness is not shared outside that file.
• internal to the module when the harness is shared across multiple test files. In this case, define it in a Harness.kt file in the same package as the test code that references it most closely.

private object Harness {
  val supplierService = mockk<SupplierService>()
  val itemDb = mockk<Database>()
  val service by lazy { ItemService.Impl(supplierService, itemDb) }
}

For infrastructure-dependent tests, the harness also owns the container lifecycle — see the Test Infrastructure section below.

---

Test Infrastructure

This section covers containerized dependencies for tests that require a real Postgres instance or AWS service emulation. Both tools are provided by common-module; source is on GitHub.

ContainerizedPostgres

cards.arda.common.lib.testing.persistence.ContainerizedPostgres runs a real Postgres 16 container via testcontainers. There are two factory methods — choose based on your test layer:

Factory	When to use	Example tests
fromTestConfig(config, initFile)	Persistence / Universe layer tests that load an application.conf via ConfigurationProvider. Config values come from the conf file.	BusinessRoleUniverseTest, DbMigrationTest, ComponentTest
fromValues(db, user, pwd, initFile)	Service-layer / E2E tests where the init SQL creates multiple databases and users. Hard-coded values match those in the init script.	SupplyServiceMethodsTest, ItemCsvUpdateE2ETest, DbScopedSequenceServiceTest

Key properties on a cpg instance:

• cpg.dataSource — DataSource (for raw SqlDataSource / Flyway)
• cpg.newExposedDatabase — Database (for Exposed transactions)
• cpg.postgresContainer — the raw PostgreSQLContainer (for jdbcUrl, host, firstMappedPort)

Init SQL Scripts

An init file is a SQL script executed when the container starts, before any test runs. Two flavours exist in the codebase:

1. Simple (testcontainers/init.sql, persistence-test/database/init.sql): Creates tables and extensions in the default database. Used by fromTestConfig tests.
2. Multi-database (kanban-service-test/database/db-init.sql, item-service-test/database/db-init.sql): Uses dblink to create separate databases, users, roles, and grants. Used by fromValues tests where the container DB name is typically "dummy_db" and the init script bootstraps the real application database.

Place init scripts under src/test/resources/<test-suite-name>/database/.

Limitations: production Aurora behavior is not reproduced

ContainerizedPostgres runs a single Postgres instance. Production runs Aurora behind the AWS Advanced JDBC Wrapper (adopted by operations 3.0.0 / common-module 8.4.0), which routes inTransaction(db, readOnly = true) to a reader replica. That routing — and any replication lag between the writer and the reader — is structurally invisible to the test container. Two consequences:

1. Read-after-write paths that race in production cannot be reproduced locally. A test that performs inTransaction(db) { write } followed immediately by inTransaction(db, readOnly = true) { read } will see the write 100 % of the time in ContainerizedPostgres, even when the same code intermittently sees null in production because the reader hasn’t replicated yet. PDEV-672 was exactly this class of bug — the kanban listener’s itemService.getRecord(rId) after the item-create commit succeeded reliably in unit tests and failed intermittently in production.

2. Setting defaultIsolationLevel = SERIALIZABLE does not close the gap. Single-instance SERIALIZABLE uses serializable snapshot isolation on the same connection pool; the writer-to-reader visibility lag has no analogue. A test that exercises SERIALIZABLE is exercising a different concern (predicate-locking conflicts) from the one that bites in Aurora.

The mitigation is structural, not behavioural — write tests that prove the code path does not depend on a post-write read-back at all, rather than tests that try to exhibit the failure. The strongest shape passes the entity directly into the side-effect path without persisting it first:

"addDefaultCardFor persists a KanbanCard from an Item.Entity that was never inserted in the item DB" {
  val (_, kanbanServiceIface) = underTest()
  val kanbanService = kanbanServiceIface as ServiceImpl
  val ephemeralItem = Item.Entity(eId = TypeId(), name = "regression-no-read-back", ...)
  // Deliberately NOT calling `itemService.create(...)` — the item DB never sees it.
  val rs = appCtx.inContext {
    kanbanService.addDefaultCardFor(ephemeralItem, tenantId, clock.next())
  }
  assertTrue(rs.isSuccess, "must build the card from the supplied Item.Entity without reading itemService")
}

If the implementation ever re-introduces an itemService.getRecord(...) lookup on this path, the read returns null and the test fails — a guarantee no environmental flakiness can erode.

Alternative when the read-back is structural and cannot be avoided: mock the producing service’s reads to return Result.success(null) and assert the calling code handles it correctly.

For full background — including the operational surface (the observer dispatcher’s WARN-and-swallow of listener failures, the test that passed by accident under SERIALIZABLE) — see the operations knowledge-base entries:

• observer-must-not-read-back-just-committed-entity.md
• reader-replica-routing-is-invisible-to-tests.md

---

Strategy A — Persistence and Universe Layer Tests (fromTestConfig)

Use this strategy when testing persistence classes (Persistence, Universe, UniverseTable, Component) that work directly with Exposed.

1. Configuration File

Create an application.conf under src/test/resources/<test-suite>/:

dataSource {
  db {
    url = "jdbc:postgres://localhost:5432/DUMMY"   # Replaced by the container at runtime.
    user = "ba_test_user"
    password = "ba_test_password"
  }
  pool { ... }
  initFile = "testcontainers/init.sql"
  flyway {
    locations = ["reference/business-affiliate/database/migrations"]
    adminTable = "business_affiliate_flyway_schema_history"
  }
}

The url is a placeholder — ContainerizedPostgres replaces it with the container’s actual host and port at runtime.

2. Test Harness

private object Harness {
  val cfgProvider = ConfigurationProvider.load("<test-suite>/application.conf")
  val dsConfig: DataSourceConfig by lazy {
    val defaults = cfgProvider.globalDsConfiguration!!
    DataSourceConfig.fromDefaults(defaults).getOrThrow()
  }
  val cpg: ContainerizedPostgres by lazy {
    ContainerizedPostgres.fromTestConfig(dsConfig.db, dsConfig.initFile)
  }
}

3. Lifecycle and Database Access

class MyUniverseTest : StringSpec({
  beforeSpec {
    Harness.cpg.start()
    // Optional: run Flyway migrations after starting the container.
    DbMigration(Harness.dsConfig.flywayConfig!!, Harness.cpg.dataSource.newSqlDataSource())
      .migrate(close = true).onFailure { throw it }
  }
  afterSpec { Harness.cpg.stop() }

  "round-trip test" {
    val db = Harness.cpg.newExposedDatabase
    newSuspendedTransaction(db = db) {
      // ... persistence operations ...
    }
  }
})

Reference files:

• common-module — PersistenceHarness.kt in lib/src/test/.../persistence/bitemporal/
• common-module — ComponentTest.kt in lib/src/test/.../persistence/bitemporal/universe/
• operations — BusinessRoleUniverseTest.kt in src/test/.../reference/businessaffiliates/persistence/ (Universe + Flyway)
• common-module — DbMigrationTest.kt in lib/src/test/.../persistence/rdbms/

---

Strategy B — Service Layer and E2E Tests (fromValues)

Use this strategy when the test requires a fully initialized database with Flyway migrations — typically to test a Service.Impl that owns a Database instance.

1. Init Script

Create a multi-database init script (e.g., <test-suite>/database/db-init.sql):

CREATE EXTENSION IF NOT EXISTS dblink;
CREATE USER item_db_owner WITH PASSWORD 'item_db_owner_password';
CREATE DATABASE item_db CONNECTION_LIMIT = 100;
-- ... grant roles, permissions, set owner ...
ALTER DATABASE item_db OWNER TO item_db_owner;

2. Test Harness

private object Harness {
  val cpg: ContainerizedPostgres by lazy {
    ContainerizedPostgres.fromValues(
      databaseName = "dummy_db",           // Placeholder name for the container's default DB.
      user         = "dummy_user",
      password     = "dummy_pwd",
      initFileName = "<test-suite>/database/db-init.sql"
    )
  }

  fun setup(): TestEnv {
    // Build a PostgresConfig pointing to the real database created by the init script,
    // using the container's dynamic host and port.
    val dbConfig = PostgresConfig(
      user = "item_db_owner",
      password = "item_db_owner_password",
      url = JdbcUri(
        "jdbc:postgresql://${cpg.postgresContainer.host}" +
        ":${cpg.postgresContainer.firstMappedPort}/item_db?LoggerLevel=INFO"
      )
    )
    val ds = DataSource(dbConfig, PoolConfig())
    val db = ds.newDb(FlywayConfig(
      locations = listOf("reference/item/database/migrations"),
      adminTable = "reference_item_flyway_history"
    ))
    // ... build service under test using `db` ...
  }
}

3. Lifecycle

class MyServiceTest : StringSpec({
  beforeSpec { Harness.cpg.start() }
  afterSpec  { Harness.cpg.stop() }

  "creates entity successfully" {
    val env = Harness.setup()
    // ... test using env.service ...
  }
})

Reference files:

• operations — SupplyServiceMethodsTest.kt in src/test/.../reference/item/service/
• operations — ItemCsvUpdateE2ETest.kt in src/test/.../reference/item/service/ (S3 + DB)
• common-module — DbScopedSequenceServiceTest.kt in lib/src/test/.../persistence/sequence/ (standalone harness)

---

General Database Testing Rules

Harness pattern: Always define the cpg instance and setup logic in a Harness object (private or companion), not inline in the test body.

Lazy initialization: Use by lazy for cpg, DataSourceConfig, and ConfigurationProvider to avoid premature initialization.

Lifecycle choices:

• Start in beforeSpec, stop in afterSpec when the database can be shared across all tests in the spec.
• Start in beforeTest, stop in afterTest when test isolation requires a fresh database per test.
• Use beforeTest to clear tables (e.g., table.deleteAll()) if sharing the container but requiring clean state per test.

Connectivity test: Always include a test that verifies the connection is alive:

"confirms database connectivity" {
  val conn = cpg.dataSource.newSqlDataSource().connection
  assertTrue(conn.isValid(5), "Expected a valid database connection")
  conn.close()
}

Transaction isolation: When using TransactionManager directly, set isolation and retry:

TransactionManager.manager.defaultIsolationLevel = Connection.TRANSACTION_SERIALIZABLE
TransactionManager.manager.defaultMaxAttempts = 3

Flyway migration paths: Use the same migration locations defined in main/resources for the module under test to ensure the test schema matches production.

LocalStack S3 flakiness: Add retry and backoff logic around bucket creation during startup for LocalStack S3 tests.

Testcontainers / Ryuk exhaustion: Running many container-backed specs back-to-back can exhaust Docker or the Ryuk resource-reaper, surfacing as spec-launch failures (Before Spec / Ryuk errors) on specs that pass in isolation. During iteration, prefer running targeted spec sets rather than the whole suite. Treat a single clean full ./gradlew build as the authoritative gate, and re-run any suspected-environmental spec in isolation before treating its failure as real.

---

Related Pages

• Mocking Patterns — MockK and Kotest quick-reference patterns
• Dev Workflows — local development setup

---

Copyright: (c) Arda Systems 2025-2026, All rights reserved