Bitemporal and Scoped Universe Design

The Universe framework provides a structured way to manage bitemporal entities, with support for global and scoped (e.g., tenant-based) data segregation.

Universe Classes

Universe<EP, M> Interface

This interface defines the contract for a “universe” of bitemporal entities. A universe provides a set of operations to manage the lifecycle of these entities, including creation, reading, updating, and deletion, while adhering to bitemporal principles.

• Generics:
  • EP: The type of the entity payload, which must implement EntityPayload.
  • M: The type of the payload metadata, which must implement PayloadMetadata.
• Core Operations:
  • create: Creates a new bitemporal entity.
  • read: Reads the current state of a bitemporal entity at a specific TimeCoordinates.
  • readRecord: Reads a specific historical record of a bitemporal entity by its RecordId.
  • findOne: Finds a single entity matching a filter at specific TimeCoordinates.
  • list: Lists entities based on a query, at specific TimeCoordinates.
  • count: Counts entities matching a filter, at specific TimeCoordinates.
  • update: Updates an existing bitemporal entity, creating a new record.
  • delete: Logically deletes a bitemporal entity by creating a new “retired” record.
  • aggregate: Performs aggregation operations on entities with grouping and aggregation functions.
  • history: Retrieves the historical records of an entity within a time range.
• Key Features:
  • All operations are suspend functions, designed for asynchronous execution.
  • Operations return a DBIO (Database Input/Output) action, which encapsulates the database transaction.
  • Bitemporality is managed via TimeCoordinates (effective and recorded time).
  • Validation is performed before create, update, and delete operations.
  • Universal conditions/constraints (like scoping) are applied during read, list, count, update, and delete operations to ensure data isolation or apply global constraints.

AbstractUniverse<EP, M, TBL, PR>

This abstract class provides a skeletal implementation of the Universe interface. It encapsulates common logic for managing bitemporal entities, relying on specialized components for persistence, validation, and universal conditions.

• Generics:
  • EP: EntityPayload type.
  • M: PayloadMetadata type.
  • TBL: The UniverseTable type representing the database table.
  • PR: The BitemporalRecord type representing a row in the table.
• Dependencies:
  • persistence: An instance of Persistence<EP, M, TBL, PR> to handle database interactions (CRUD operations on records).
  • validator: An instance of Validator<EP, M> to validate payloads and operations.
  • universalCondition: An instance of UniversalCondition to apply global filters (e.g., for scoping) to queries.
• Functionality:
  • Implements all methods from the Universe interface.
  • Delegates database operations to the persistence layer via recordCompanion.
  • Invokes the validator before create, update, and delete operations.
  • Applies the universalCondition filter to read, list, count, update, and delete operations to ensure data isolation or apply global constraints.
  • Handles the bitemporal aspects by interacting with the BitemporalEntityClass provided by persistence.recordCompanion.

AbstractScopedUniverse<EP, M, TBL, PR>

This abstract class extends AbstractUniverse and is specifically designed for universes where entities are scoped, typically by a tenant ID. It mandates the use of ScopedMetadata, ScopedTable, and ScopedRecord.

• Generics:
  • EP: EntityPayload type.
  • M: Must be a ScopedMetadata type (which includes tenantId).
  • TBL: Must be a ScopedTable type (which includes a tenantId column).
  • PR: Must be a ScopedRecord type (which manages the tenantId field).
• Behavior:
  • Inherits the core bitemporal entity management logic from AbstractUniverse.
  • Enforces tenant-based scoping through the specialized ScopingValidator and ScopedUniversalCondition. The ScopingValidator ensures that operations are performed within the correct tenant context, and the ScopedUniversalCondition ensures that data access is restricted to the current tenant.

Persistence<EP, M, TBL, PR> Class

The Persistence class is the bridge between the universe abstraction and the database layer. It provides:

• bt: TBL: The table definition for the bitemporal entity.
• recordCompanion: BitemporalEntityClass<EP, M, TBL, PR>: The entity class that handles CRUD operations.
• selfAlias: Alias<TBL>: An alias for the table used in complex queries.

The class is constructed with:

• The table instance
• The record class type
• A factory function to create record instances
• A function to fill record instances

Key Differences Summarized

Feature	AbstractUniverse	AbstractScopedUniverse
Scope Assumption	Generic; can be global or scoped depending on injected Validator and UniversalCondition.	Assumes tenant-based scoping.
Metadata Requirement	PayloadMetadata	ScopedMetadata (must include tenantId)
Table Requirement	UniverseTable	ScopedTable (must include tenantId column)
Record Requirement	BitemporalRecord	ScopedRecord (manages tenantId mapping)
Validator	Validator<EP, M>	ScopingValidator<EP, M> (typically validates tenantId against context)
Universal Condition	UniversalCondition	ScopedUniversalCondition (typically filters by tenantId from context)
Use Cases	Foundation for any universe, including globally unique entities or custom scoping.	Universes where entities are partitioned by tenant, providing out-of-the-box tenant isolation.

In essence, AbstractUniverse provides the common bitemporal framework, while AbstractScopedUniverse specializes it for tenant-scoped data, simplifying the implementation of multi-tenant systems.

Real-World Example: KanbanCardUniverse

The KanbanCardUniverse demonstrates how to extend AbstractScopedUniverse with custom business logic:

class KanbanCardUniverse :
  AbstractScopedUniverse<KanbanCard, KanbanCardMetadata, KANBAN_CARD_TABLE, KanbanCardRecord>(),
  LogEnabled by LogProvider(KanbanCardUniverse::class) {

  override val persistence = KanbanCardRecord.Companion
  override val universalCondition = KanbanCardUniversalCondition
  override val validator = validatorFor(this)

  // Custom aggregation method
  suspend fun summaryByStatus(
    condition: Filter,
    asOf: TimeCoordinates,
    includeRetired: Boolean,
    vararg forStatus: KanbanCardStatus,
  ): DBIO<List<KanbanCardSummary>> =
    aggregate(
      Query(
        Filter.And(listOf(
          condition,
          Filter.In(persistence.bt.status.name, forStatus.toList())
        )),
        Sort(listOf(
          SortEntry(persistence.bt.status.name, SortDirection.ASC),
          SortEntry(persistence.bt.item.name.name, SortDirection.ASC),
          SortEntry(persistence.bt.cardQuantity.unit.name, SortDirection.ASC)
        )),
        paginate = Pagination(0, 500)
      ),
      asOf,
      Aggregation(
        listOf(
          GroupBy.Classifier(persistence.bt.status.name),
          GroupBy.Classifier(persistence.bt.item.name.name),
          GroupBy.Classifier(persistence.bt.item.eId.name),
          GroupBy.Classifier(persistence.bt.cardQuantity.unit.name)
        ),
        listOf(
          GroupBy.Aggregator(
            persistence.bt.cardQuantity.amount.name,
            AggregationType.SUM,
            "quantityAmount"
          )
        )
      ),
      includeRetired = includeRetired,
      rowMapper(asOf)
    ).map { /* Post-processing logic */ }
}

This example shows:

• Custom Methods: summaryByStatus extends the base universe with domain-specific operations.
• Aggregation Usage: Uses the aggregate method for complex queries with grouping and sum operations.
• Row Mapping: Custom result transformation from database rows to business objects.
• Filter Composition: Combines multiple filters for complex queries.

Key Concepts of Bi-temporality

Bi-temporality means tracking two time dimensions for each piece of data:

1. Effective Time (Valid Time): When a fact is true in the real world. This represents the time period during which the information is considered valid.
2. Recorded Time (Transaction Time): When a fact was recorded in the system. This represents the history of how the system’s knowledge of the real world evolved.

How Bi-temporality is Represented in the Code

• TimeCoordinates: This data class is central to the bi-temporal model. It encapsulates both effective and recorded timestamps as Long values (likely representing milliseconds since the epoch).

@Serializable
data class TimeCoordinates(val effective: Long, val recorded: Long) {
    // ... other methods ...
}

• BitemporalRecord: This abstract class, representing a record in the universe, includes bitemporal properties like effectiveAsOf and recordedAsOf.

abstract class BitemporalRecord<EP : EntityPayload, M : PayloadMetadata, TBL : UniverseTable, SELF: BitemporalRecord<EP, M, TBL, SELF>> {
    val rId: EntityID<UUID>
    var eId by table.eId
    var effectiveAsOf by table.effectiveAsOf
    var recordedAsOf by table.recordedAsOf
    var retired by table.retired
    var previous by table.previous
    var author by table.author
    // ... other properties and methods ...
}

• Universe Operations: The AbstractUniverse class extensively uses TimeCoordinates in its operations:
  • create: When a new entity is created, the asOf parameter specifies the effective and recorded time for the new entity’s creation.
  • read: Accepts asOf: TimeCoordinates to query for the entity’s record that was valid at the specified effective time and recorded at or before the specified recorded time.
  • update: Creates a new record representing the updated state, with its own effectiveAsOf and recordedAsOf timestamps, linking to the previous record.
  • delete: Creates a new record marked as retired = true, representing the deletion event with its own timestamps.

Universe Validation

The Universe implementation employs a multi-faceted validation strategy to ensure data integrity, coordinated by the AbstractUniverse and specialized by its components.

1. Payload-Level Validation (EntityPayload.validate)

This initial validation step occurs within the entity payload itself.

• Responsibility: The validate(ctx: ApplicationContext, mutation: Mutation): Result<Unit> method on the EntityPayload interface is responsible for checking the inherent correctness and business rules of the data within the payload. This can include:
  • Format checks (e.g., valid email, URL).
  • Range checks.
  • Required field checks.
  • Internal consistency checks between fields of the payload.
• Contextual Validation: The ApplicationContext and Mutation type (CREATE, UPDATE, DELETE) are passed to allow for context-aware validation if needed.
• Error Reporting: Returns a Result<Unit>, which is Result.success(Unit) if valid, or Result.failure(AppError) if invalid.

2. Universe-Level Validation (Validator interface)

This tier of validation is handled by a component implementing the Validator<EP, M> interface, which is injected into the AbstractUniverse. It performs checks that consider the broader context of the operation within the universe.

• Validator<EP, M> Interface Methods:
  • validateForCreate(ctx: ApplicationContext, payload: EP, metadata: M, asOf: TimeCoordinates, author: String): DBIO<Unit>: Validates before a new entity is created.
  • validateForUpdate(ctx: ApplicationContext, payload: EP, metadata: M, asOf: TimeCoordinates, author: String, idempotency: Idempotency, previous: BitemporalEntity<EP, M>?): DBIO<Unit>: Validates before an existing entity is updated. It receives the previous state of the entity.
  • validateForDelete(ctx: ApplicationContext, candidate: BitemporalEntity<EP, M>, metadata: M, asOf: TimeCoordinates, author: String): DBIO<Unit>: Validates before an entity is logically deleted. It receives the candidate entity to be deleted.
• Responsibilities:
  • Cross-entity validation (e.g., uniqueness constraints beyond the payload itself, referential integrity if not handled by DB).
  • State transition validation (e.g., ensuring an entity can move from its current state to a new state).
  • Authorization-related checks (though primary auth might be handled upstream).
  • For scoped universes, the ScopingValidator (a common implementation of Validator) primarily ensures that the tenantId in the metadata matches the tenant in the ApplicationContext.

3. Integration and Orchestration in AbstractUniverse

AbstractUniverse orchestrates these validation steps:

• crValidate, upValidate, delValidate private methods: These internal methods in AbstractUniverse are responsible for invoking the appropriate Validator methods using the application context.
• Error Handling: If any validation step returns a Result.failure, the operation in AbstractUniverse (create, update, delete) is short-circuited, and the DBIO action will yield the failure.

This layered approach ensures that:

1. The payload itself is internally consistent.
2. The operation is valid within the broader context of the universe and its rules (including scoping).

How to Build a New Universe: Step-by-Step Guide

This guide outlines the process for creating a new bitemporal universe. This typically involves defining your data structures, persistence mechanisms, validation rules, and the universe class itself.

We’ll assume you are creating a scoped universe (entities belong to a tenant), as this is a common pattern. For a global universe, you would use non-scoped base classes like UniverseTable, BitemporalRecord, Validator, UniversalCondition, and AbstractUniverse.

Given:

• YourEntityPayload: The data structure for your entity.
• YourScopedMetadata: The metadata for your entity, including tenantId.

Here are the steps:

Step 1: Define Your Entity Payload

Create a data class for your entity’s payload. It must implement EntityPayload.

@Serializable
data class YourEntityPayload(
    @Serializable(with = UUIDSerializer::class)
    override val eId: EntityId,
    val name: String,
    // ... other fields specific to your entity
) : EntityPayload {
    override fun validate(ctx: ApplicationContext, mutation: Mutation): Result<Unit> {
        if (name.isBlank()) {
            return Result.failure(AppError.ArgumentValidation("name", "Name cannot be blank"))
        }
        // ... other validation logic for your payload
        return Result.success(Unit)
    }
}

Step 2: Define Your Scoped Metadata

Create a data class for your entity’s metadata. It must implement ScopedMetadata.

@Serializable
data class YourScopedMetadata(
    @Serializable(with = UUIDSerializer::class)
    override val tenantId: UUID
    // ... other metadata fields if any
) : ScopedMetadata

Step 3: Define Your Scoped Table

Create an object that extends ScopedTable. Define columns for your payload fields.

object YourScopedTable : ScopedTable(TableConfiguration("YOUR_ENTITY_TABLE_NAME")) {
    // Columns from YourEntityPayload
    val name = varchar("ITEM_NAME", 255)
    // ... other columns for your payload fields

    // Columns from ScopedTable (tenantId, eId, rId, etc.) are inherited.
    // Bitemporal columns (effective_as_of, recorded_as_of, etc.) are also inherited.
}

Step 4: Define Your Scoped Record

Create a class that extends ScopedRecord. This class maps data between your payload/metadata and the database table.

class YourScopedRecord(rId: EntityID<UUID>) :
    ScopedRecord<YourEntityPayload, YourScopedMetadata, YourScopedTable, YourScopedRecord>(rId, YourScopedTable) {

    // Delegate properties to table columns
    var name by YourScopedTable.name
    // ... other delegates for your payload fields

    override fun fillPayload(p: YourEntityPayload) {
        // Called when creating/updating a record from a payload
        payload = p // Store the raw payload
        this.name = p.name
        // ... map other payload fields to record properties
    }

    override fun fillMetadata(m: YourScopedMetadata) {
        // Called when creating/updating a record from metadata
        metadata = m // Store the raw metadata
        this.tenantId = m.tenantId
        // ... map other metadata fields if any
    }

    companion object : Persistence<YourEntityPayload, YourScopedMetadata, YourScopedTable, YourScopedRecord>(
        YourScopedTable,
        YourScopedRecord::class.java,
        ::YourScopedRecord
    ) { }
}

Step 5: Define Your Scoping Validator

Create a validator that extends ScopingValidator. Implement custom validation logic beyond the standard tenant checks if needed.

fun validatorFor(universe: YourUniverse): ScopingValidator<YourEntityPayload, YourScopedMetadata> {
    return object : ScopingValidator<YourEntityPayload, YourScopedMetadata>() {
        override suspend fun validateForCreate(
            ctx: ApplicationContext,
            payload: YourEntityPayload,
            metadata: YourScopedMetadata,
            asOf: TimeCoordinates,
            author: String
        ): DBIO<Unit> = suspend {
            super.validateForCreate(ctx, payload, metadata, asOf, author)().flatMap {
                // Add your custom validation for creation
                if (payload.name.isBlank()) {
                    Result.failure(AppError.ArgumentValidation("name", "Entity name cannot be blank"))
                } else {
                    Result.success(Unit)
                }
            }
        }

        // Override validateForUpdate and validateForDelete as needed
    }
}

Step 6: Define Your Scoped Universal Condition

Create an object that extends ScopedUniversalCondition. Usually, the base implementation is sufficient as it automatically filters by tenantId from the ApplicationContext.

object YourUniversalCondition : ScopedUniversalCondition()

Step 7: Implement Your Scoped Universe

Create a class that extends AbstractScopedUniverse, providing your persistence, validator, and universal condition objects.

class YourUniverse : AbstractScopedUniverse<
    YourEntityPayload,
    YourScopedMetadata,
    YourScopedTable,
    YourScopedRecord
>(), LogEnabled by LogProvider(YourUniverse::class) {

    override val persistence = YourScopedRecord.Companion
    override val validator = validatorFor(this)
    override val universalCondition = YourUniversalCondition

    // Add custom business methods as needed
    suspend fun customBusinessMethod(
        filter: Filter,
        asOf: TimeCoordinates
    ): DBIO<List<YourBusinessObject>> {
        // Implement custom business logic using aggregate, findOne, etc.
        return aggregate(
            Query(filter),
            asOf,
            Aggregation(/* your aggregation */),
            includeRetired = false,
            mapper = { row -> /* your row mapping */ }
        )
    }
}

Step 8: Testing Your Universe

Use the AbstractUniverseTestTemplate for comprehensive testing:

class YourUniverseTest : AbstractUniverseTestTemplate<
    YourEntityPayload,
    YourScopedMetadata,
    YourScopedTable,
    YourScopedRecord,
    YourUniverse
>(
    testName = "YourUniverse",
    appConfigPath = "test-application.conf",
    newUniverse = { YourUniverse() },
    prepareDb = { /* any additional DB setup */ },
    serviceScope = { ServiceScope.Tenant(testTenantId) },
    newPayload = { eId, testCtx, order -> YourEntityPayload(eId, "test-$testCtx-$order") },
    newMetadata = { _, tenantIdStr, _ -> YourScopedMetadata(UUID.fromString(tenantIdStr)) }
) {
    // Add custom tests here
    {
        "Custom business method test" {
            // Test your custom business methods
        }
    }
}

This approach provides:

• Comprehensive Coverage: Tests all CRUD operations, pagination, history, and aggregation.
• Time-Travel Testing: Verifies bitemporal behavior.
• Extensibility: Easy to add custom tests for business-specific methods.

---

Copyright: © Arda Systems 2025, All rights reserved

Comments