Skip to content

Data authority endpoint implementation

Data Authority Endpoint Implementation

This document describes the process of implementing a new Data Authority endpoint for managing a specific type of entity within the Arda platform. The Data Authority pattern provides a standardized way to handle Create, Read, Update, Delete, and Query (CRUDQ) operations for bitemporal entities.

Overview of the Data Authority Pattern

The Data Authority pattern consists of three main layers:

  1. Endpoint (DataAuthorityEndpoint):

    • Handles HTTP requests (GET, POST, PUT, DELETE) and exposes the REST API for the entity.
    • Responsible for request validation, serialization/deserialization, and calling the appropriate service methods.
    • Uses the DataAuthorityEndpoint.reify factory for standardized construction.

  2. Service (DataAuthorityService):

    • Contains the business logic for managing the entity.
    • Orchestrates operations by interacting with the Universe layer.
    • Handles transaction management and notifications.

  3. Universe (Universe):

    • Provides the core bitemporal persistence logic.
    • Interacts directly with the database (e.g., PostgreSQL with Exposed ORM).
    • Manages entity versions, effective times, and recorded times.

This separation of concerns allows for a clean and maintainable architecture.

Steps to Implement a New DataAuthorityEndpoint

Here’s a step-by-step guide to implementing a Data Authority for a new entity, let’s call it MyEntity.

1. Define Entity Payload and Metadata

First, define the data structures for your entity’s payload and its metadata. These will be Kotlin data classes.

  • Payload (MyEntityPayload.kt): Represents the actual data of your entity. It must implement EntityPayload.
  • Metadata (MyEntityMetadata.kt): Contains metadata associated with the entity, such as tenant ID, version, or other contextual information. It must implement PayloadMetadata.

Example:

// lib/src/main/kotlin/cards/arda/yourmodule/model/MyEntityPayload.kt

@Serializable
data class MyEntityPayload(
    override val id: LocalResourceId, // ID is typically set via ip2Payload
    val name: String,
    val description: String? = null,
    // other fields relevant to your entity
) : EntityPayload

// lib/src/main/kotlin/cards/arda/yourmodule/model/MyEntityMetadata.kt

@Serializable
data class MyEntityMetadata(
    val tenantId: UUID,
    val version: Int = 1
    // other metadata fields
) : PayloadMetadata

Ensure these classes are @Serializable if they need to be sent over the network or persisted in formats like JSON.

2. Create the DataAuthorityService Implementation

Next, create a service interface and its implementation for MyEntity.

  • Interface (MyEntityDataAuthorityService.kt): Extends DataAuthorityService<MyEntityPayload, MyEntityMetadata>.
  • Implementation (MyEntityDataAuthorityServiceImpl.kt): Implements MyEntityDataAuthorityService. This class will require a Universe instance.

Example:

// lib/src/main/kotlin/cards/arda/yourmodule/service/MyEntityDataAuthorityService.kt

interface MyEntityDataAuthorityService : DataAuthorityService<MyEntityPayload, MyEntityMetadata> {
    // You can add entity-specific service methods here if needed
}


// lib/src/main/kotlin/cards/arda/yourmodule/service/MyEntityDataAuthorityServiceImpl.kt

class MyEntityDataAuthorityServiceImpl(
    override val db: Database, // Injected database instance
    override val universe: Universe<MyEntityPayload, MyEntityMetadata>
) : MyEntityDataAuthorityService,
    ObserverManager<DataAuthorityNotification<MyEntityPayload, MyEntityMetadata>> by DelegatedObserverManager() {
    // The base DataAuthorityService methods (getAsOf, add, update, delete, listEntities)
    // are mostly implemented by the interface default implementations or through the Universe.
    // You may override them if specific pre/post processing or custom logic is needed.

}

2.1. Universe Implementation Details

The Universe layer is the heart of the Data Authority pattern, directly managing the persistence and bitemporal aspects of your entities.

Role of the Universe Interface:

The cards.arda.common.lib.persistence.universe.Universe<EP : EntityPayload, M : PayloadMetadata> interface defines the contract for data storage and retrieval. Its key responsibilities include:

  • Bitemporal Data Management: Handling effectiveAsOf and recordedAsOf timestamps for all entity versions, typically encapsulated in a TimeCoordinates object.
  • CRUD Operations: Providing methods to create, read (specific versions or latest), update, and delete (logically) entities.
  • Listing and Querying: Offering methods like list to fetch collections of entities based on filter criteria (defined by a Query object, which includes filter, sort, and pagination instructions) and specific TimeCoordinates.
  • History Tracking: Allowing retrieval of an entity’s version history.
  • Idempotency: Ensuring that repeated operations with the same parameters can be handled safely, often relevant for create and update.

Key Methods to Implement (from Universe interface)

When creating a Universe for your specific entity (MyEntityPayload, MyEntityMetadata), you’ll typically implement methods such as:

  • create(payload: EP, metadata: M, asOf: TimeCoordinates, author: String): DBIO<BitemporalEntity<EP, M>>
  • read(eId: LocalResourceId, asOf: TimeCoordinates, includeDeleted: Boolean = false): DBIO<BitemporalEntity<EP, M>?>
  • update(update: Update<EP, M>): DBIO<BitemporalEntity<EP, M>> (where Update is a data class from cards.arda.common.lib.module.dataauthority.Update holding payload, metadata, time, author, etc.)
  • delete(originEId: LocalResourceId, metadata: M, asOf: TimeCoordinates, author: String): DBIO<BitemporalEntity<EP, M>>
  • list(query: Query, asOf: TimeCoordinates, includeDeleted: Boolean = false, withTotal: Boolean = false): DBIO<Page<EP, M>> (The Page type here is cards.arda.common.lib.module.dataauthority.Page, which is then converted to PageResult by the service layer).
  • readRecord(rId: LocalResourceId): DBIO<BitemporalEntity<EP, M>?>
  • history(eId: LocalResourceId, asOf: TimeCoordinates, asc: Boolean): DBIO<List<BitemporalEntity<EP, M>>>
  • findOne(filter: Filter, asOf: TimeCoordinates, includeDeleted: Boolean = false): DBIO<BitemporalEntity<EP, M>?>
  • count(filter: Filter, asOf: TimeCoordinates, includeRetired: Boolean = false): DBIO<Long>

Database Interaction (Example with Exposed ORM)

The Universe implementation usually interacts with a database. If using Kotlin’s Exposed DSL/ORM, you would define an EntityTable that maps your MyEntityPayload and MyEntityMetadata to database columns. This table would also include standard bitemporal columns like entity_id, record_id, effective_from, effective_to, recorded_from, recorded_to, author, payload_json, metadata_json, is_deleted.

  • EntityPayload (EP): Your specific entity data (e.g.,MyEntityPayload).
  • PayloadMetadata (M): Your specific metadata (e.g., MyEntityMetadata).

Sample Universe implementation using AbstractScopedUniverse

Although a Universe can be implemented from scratch, A real implementation should use AbstractUniverse or ScopedAbstractUniverse to provide most of the required functionality and
provide specialized access methods if required.

@Serializable
data class Sample(
  @Serializable(with = UUIDSerializer::class)
  override val eId: EntityId,
  val name: String
) : EntityPayload {
  override fun validate(ctx: ApplicationContext, mutation: Mutation): Result<Unit> = Result.success(Unit)
}

@Serializable
data class SampleMetadata(
  @Serializable(with = UUIDSerializer::class)
  override val tenantId: UUID
) : ScopedMetadata

object SampleTableConfiguration : TableConfiguration {
  override val name = "SAMPLE"
}

object SampleTable : ScopedTable(SampleTableConfiguration) {
  val name = varchar("ITEM_NAME", 255)
}

class SampleRecord(rId: EntityID<UUID>) : ScopedRecord<Sample, SampleMetadata, SampleTable>(rId, SampleTable) {
  companion object : Persistence<Sample, SampleMetadata, SampleTable, SampleRecord>(
    SampleTable,
    SampleRecord::class.java,
    { SampleRecord(it) },
    {
      name = payload.name
      tenantId = metadata.tenantId
    }) {
  }

  var name by SampleTable.name

  override fun fillPayload() {
    payload = Sample(this.eId,this.name)
  }
  override fun fillMetadata() {
    metadata = SampleMetadata(this.tenantId)
  }
}

object SampleValidator: ScopingValidator<Sample, SampleMetadata>() {
  private fun validateName(name: String): DBIO<Unit> = suspend {
    if (name.isBlank()) Result.failure(AppError.ArgumentValidation("Sample name cannot be blank"))
    else Result.success(Unit)
  }

  override suspend fun validateForCreate(
    ctx: ApplicationContext,
    payload: Sample,
    metadata: SampleMetadata,
    asOf: TimeCoordinates,
    author: String
  ): DBIO<Unit> = super.validateForCreate(ctx, payload, metadata, asOf, author)
    .liftMap { validateName(payload.name) }

  override suspend fun validateForUpdate(
    ctx: ApplicationContext,
    payload: Sample,
    metadata: SampleMetadata,
    asOf: TimeCoordinates,
    author: String,
    idempotency: Idempotency,
    previous: BitemporalEntity<Sample, SampleMetadata>?
  ) : DBIO<Unit> =
    super.validateForUpdate(ctx, payload, metadata, asOf, author, Idempotency.CONFIRM, null)
      .liftMap { validateName(payload.name) }
}

object SampleUniversalCondition: ScopedUniversalCondition()

class SampleUniverse : AbstractScopedUniverse<Sample, SampleMetadata, SampleTable, SampleRecord>(),
  LogEnabled by LogProvider(SampleUniverse::class) {
  override val persistence = SampleRecord.Companion
  override val validator = SampleValidator
  override val universalCondition = SampleUniversalCondition
}

Providing the Universe to the DataAuthorityService:

The MyEntityUniverse instance (or a more generic one configured for MyEntity) is typically created and managed by a dependency injection framework (e.g., Koin, Kodein). It’s then injected into the constructor of your MyEntityDataAuthorityServiceImpl.

This structured approach ensures that the DataAuthorityService can operate on entities without being tightly coupled to the specific database access logic, which is encapsulated within the Universe.

3. Create the DataAuthorityEndpoint Implementation

Now, create the endpoint class. This class will extend DataAuthorityEndpoint and use the DataAuthorityEndpoint.reify factory.

Example:

@Serializable
data class SampleInput(val name: String)

typealias SampleIP = String
typealias SampleIM = UUID

val ip2Payload: SampleIP.(UUID) -> Sample = { eId ->
  Sample(eId, this)
}
val im2Metadata: SampleIM.() -> SampleMetadata = { SampleMetadata(this) }
val retrieveMetadata:  RoutingCall.() -> Result<SampleIM> = {
  requireHeader(StandardHeaders.tenantId.name).flatMap { tIdStr ->
    when(val tenantId = tIdStr.validUUIDOrNull()) {
      null -> Result.failure(AppError.ArgumentValidation(StandardHeaders.tenantId.name, "TenantId must be a valid UUID"))
      else -> Result.success(tenantId)
    }
  }
}

val recordRespond: suspend RoutingCall.(EntityRecord<Sample, SampleMetadata>) -> Result<Unit> = {
  respond(it)
  Result.success(Unit)
}

class SampleDataAuthorityEndpoint(override val reference: EndpointLocator.Rest, service: SampleDataAuthorityService) :
  DataAuthorityEndpoint
  by DataAuthorityEndpoint.Companion.reify<SampleIP, SampleIM, Sample, SampleMetadata> (
    reference,
    service,
    ip2Payload,
    im2Metadata,
    retrieveMetadata,
    recordRespond
  )

Key parts of DataAuthorityEndpoint.reify:

  • reference: An EndpointLocator.Rest instance defining the base path for this endpoint (e.g., /v1/myentity).
  • service: Your MyEntityDataAuthorityService instance.
  • ip2Payload: A lambda function that converts the input payload (received in POST/PUT requests) to the EntityPayload type that will be stored.
  • im2Metadata: A lambda function that converts the input metadata (extracted from the request) to the PayloadMetadata type that will be stored.
  • retrieveMetadata: A function that extracts input metadata (like tenant ID, user info) from the current request context (e.g., Ktor’s RoutingCall). This often involves reading headers or security context.
  • recordRespond: A lambda that handles sending the EntityRecord back to the client. This typically involves using Ktor’s call.respond() or similar.

4. Configure Serializers and Metadata Handlers

Serialization

Ensure Ktor (or your chosen HTTP framework) is configured to serialize/deserialize your MyEntityPayload, MyEntityMetadata, Query, and PageResult classes. For Kotlinx Serialization, this usually means installing the ContentNegotiation feature with the json format.

// In your Ktor application setup
install(ContentNegotiation) {
    json(JsonConfig.standardJson) // JsonConfig.standardJson is a common library convention
}

Metadata Extraction (retrieveMetadataLogic)

This function is crucial for security and multi-tenancy. It needs to safely extract information like tenant IDs, user identifiers, etc., from request headers, JWT tokens, or other request properties.

Update or create a corresponding DataAuthorityEndpointSpec.kt file for your new entity. This class defines the OpenAPI (Swagger) documentation for your endpoint’s routes.

// lib/src/main/kotlin/cards/arda/yourmodule/api/MyEntityDataAuthorityEndpointSpec.kt

class MyEntityDataAuthorityEndpointSpec(
    resourceName: String, // e.g., "MyEntity"
    openApiSpecId: String // e.g., "/v1/myentity"
) : DataAuthorityEndpointSpec<MyEntityPayload, MyEntityMetadata>(resourceName, openApiSpecId) {
    // The base class provides builders for get, post, put, delete.
    // You might need to customize them if your input types (IP, IM) for create/update
    // are different from your entity types (EP, M), or if you have custom query params.

    // For the new query endpoints (POST /query, GET /query/{page}),
    // you'll use postQueryBuilder and getQueryByPageBuilder, for example:
    // val myEntityPostQueryBuilder = postQueryBuilder
    // val myEntityGetQueryByPageBuilder = getQueryByPageBuilder
}

You would then reference these builders in your MyEntityDataAuthorityEndpoint when setting up the routes in configureSecureRoutes. The epSpec variable used in DataAuthorityEndpoint.Impl would be an instance of this spec class.

6. Register the Endpoint in Your Ktor Module

Finally, instantiate your MyEntityDataAuthorityEndpoint and MyEntityDataAuthorityService (likely via dependency injection) and configure the routes within your Ktor application module.

// In your Ktor module configuration (e.g., MyModule.kt)

val myEntityService = MyEntityDataAuthorityServiceImpl.create(db, rdbmsUniverseFactory) // Or inject
val myEntityEndpointLocator = EndpointLocator.Rest("http", null, "v1", "MyApplication", "myentity")

val retrieveMetadataLogic: RoutingCall.() -> Result<MyEntityInputMetadata> = { /- ... see example above ... */ }

val myEntityApi = MyEntityDataAuthorityEndpoint(
  reference = myEntityEndpointLocator,
  service = myEntityService,
  retrieveMetadataLogic = retrieveMetadataLogic
)

  // This calls myEntityApi.endpoint.configureSecureRoutes(this)
  // which sets up all the GET, POST, PUT, DELETE, and query routes.
  // Ensure proper authentication and authorization are applied.
authenticate(API_KEY_AUTH_PROVIDER) { // Example auth provider
    myEntityApi.endpoint.configure(this)
}

This involves setting up the EndpointConfigurator which then calls configureSecureRoutes to define all the standard CRUD+Q routes.

Comments