Module Concept

Important

This document in very much work in progress and may contradict other parts of the documentation. Once it reaches a more
mature state and is reviewed by the team, it will be used to update other parts of the documentation and guide design
and refactoring of the system.

Role of Modules in the system¶

Role in the Functional Design of the System¶

A Module is the core unit for Functional Decomposition for Arda’s system. A module
contributes a set of cohesive capabilities to the overall system and interacts with other modules in a controlled and well-defined
manner.

Modules are grouped into Functional Domains that represent major areas of related product capabilities.

The Functional decomposition of the system should define Modules that have high cohesion and low harmful coupling between them. The design of modules
should also aim for high coherence of the artifacts it defines and implements.

A Module encapsulates a portion of the state of the system that can only be accessed through the Module’s Services. The minimal requirement of encapsulation is at the
Module level, it is strongly desired that Modules that support different services further partition their state and encapsulates it within each Service. Exceptions can be
made for performance reasons (e.g. two Services in a Module may have database foreign keys between them) but they should NEVER break the encapsulation at the Module level.

Role in other viewpoints¶

A Module is one of the basic units of change and version control all the artifacts that define it or are generated from it are identified with
the version of the Module.

A Module is typically implemented within a technology ecosystem (e.g. Kotlin/JVM, TypeScript/NodeJS, etc.).

Modules are deployed as part of Runtime Components that package them into deployable artifacts and interact with the Runtime Environments to execute the artifacts (e.g. Docker images).

Interaction between Modules¶

Modules interact with each other through well-defined mechanisms:

API Endpoints or Services: A module invokes Services exposed by other modules. Services are collections of typed, named operations that perform a well-defined behavior.
References: A module may keep references to entities managed by other modules. A Reference at the system level needs to be able to uniquely identify the entity and the instance of
the module that manages it. References in the system need to be expressed as Data Types to be able to be exchanged between Modules.
Data Types: For complex interactions, the information that Modules exchange by invoking each other’s Services and operations needs to have more structure than simple programming language primitives.
A Module exposes the structure of the information they exchange through Data Types. At a minimum, Data Types define the structure of the information exchanged. When the technology allows it, a Module
may also define basic behaviors and constraints of the Data Types it exposes, similar to how Abstract Data Types are defined in programming languages. Note that Data Types need to have STRICT value
semantics, meaning that:
1. Two instances of a Data Type are equal if and only if the information they contain is equal.
2. Data Type instances are immutable, meaning that once created, their information cannot be changed.
Bindings: A Binding is the expression of a Module’s Data Types and Services in a particular technology, protocol and address space. A Binding defines how a Module can invoke an operation on another Module by
using a particular protocol (e.g. REST over HTTP), a particular data representation and encoding (e.g. JSON on UTF-8) and deployed at a particular base URL.

DRY¶

Two modules to interact with each other, must, at a minimum agree on the Bindings that allow them to invoke Services. This enables Modules implemented in different technology ecosystems to interact with each other
regardless of their internal implementation. In this case, each interacting Module needs to implement the Services or Service Accessors and the Data Types in their own technology.

This approach is always possible, but when Modules share a technology ecosystem, it leads to duplication of code, artifacts and effort, both in the initial development, and more importantly in subsequent
maintenance and evolution of the system. This duplication is very harmful at scale, violating the DRY principle, and leading to inconsistencies and errors.

To minimize duplication within a technology ecosystem, Modules can share Data Types and Service Accessors by depending on shared libraries that define them, or they can rely on code generation from
a shared IDL (Interface Definition Language) that defines the Data Types and Services. Using IDLs for sharing Data Types is limited to the specific information structure, and they usually impose
certain conventions and limitations on the expressiveness of Data Types in a particular technology ecosystem. Programming languages have much richer type systems (e.g. polymorphism, genericity, etc.)
and can share not only the structure of the information but also ADT like behaviors, validations, etc… as well as incorporate system-wide cross-cutting concerns and designs like logging, error handling etc.

Implementation¶

Arda’s products are primarily implemented in two technology ecosystems:

Kotlin/JVM for backend modules.
TypeScript/NodeJS/React for frontend modules.

Interaction between modules will be done through:

RESTful APIs over HTTP1.1 using JSON as the data representation for interaction from Front-End Modules to Backend Modules as well as between External System and Backend Modules.
gRPC over HTTP/2 using binary Protobuf representation for interaction between Backend Modules.

In this context, modules may publish (focused on backend Modules, but analogous approach for Frontend Modules):

OpenAPI specifications for RESTful APIs to access Services from outside the System or from Front-End Modules.
Protobuf IDL files for gRPC APIs to access Services from other Backend Modules.
A Library in Kotlin that defined ADT like Kotlin Interfaces and Data Classes that represent the structure of the information the Module supports.
A Library of Bindings of the ADT types for the OpenAPI and the Protobuf specification respectively. These “Binding” libraries bridge the differences between
the pure ADT like Kotlin types (POJOs) and either the wire representation of the type in the protocol (e.g. JSON serialization/deserialization) or the Types generated by
tools from the corresponding IDL’s (e.g. protobuf generated types or OpenAPI generated types).
For Data Types that may be persisted as part of larger structures in different modules (e.g. the concept of an Address may be part of a UserAccount in the system.useraccount module and part
of a BusinessAffiliate in the reference.businessaffiliate module). A module may also choose to publish a library that includes persistence mappings for the ecosystems persistence mechanisms
(e.g. Kotlin exposed)

Important

A module may not publish all of the above artifacts. The minimum requirement is to publish the IDL specifications for the protocols it supports. The rest of the libraries can be published
to reduce duplication within the system.
A module may choose to use another module’s published libraries or not depending on its own design priorities and constraints. The only requirement is compliance with the published IDL specifications
of the services it needs to use. If a module chooses not to use another module’s published libraries it will need to implement its own adaptors to the published IDL, but it will also likely
suffer less “churn” of versions when the other module changes, as API changes are usually less frequent and more controlled than changes to ADT implementations or Bindings.

Choosing how much to publish and how much to reuse from other modules is a design decision with trade-offs of coupling, maintainability and development effort that each module needs to make. That said,
in the context of Arda’s system, where the complexity of the system is derived from the complexity of the concepts in the business domain, developing a set of ADT like types that capture
and model the business domain with high fidelity will be a key factor in the success of the system.

Common or Shared Types¶

While Services tend to be very specific to the module that offers them, Data Types live in a range of specificity from very general to very specific:

Primitive types: (e.g. String) that are the foundation of programming languages.
Common technology types (e.g. URI, UUID, DateTime, Email) which are frequently even standardized in software systems.
Commonly used business concepts like Address, Money, Length, Volume, GeoLocation etc. that correspond to ubiquitous concepts in the business domain.
Some of them may be even standardized by external bodies (e.g. Money with ISO 4217 currency codes).
Industry specific concepts that sometimes are standardized (e.g ShippingContainer) or at least widely used (e.g. WorkOrder, BillOfMaterials,…).
Concepts specific to a particular Module closely related to its capabilities (e.g. UserAccount in the system.useraccount module).

It is a design decision to determine which Module will define and publish these Data Types. For the most specific types, it is clear that the Module that owns the concept should
also own its Data Type. For the most general types, (Primitive types and Common technology types), it is also fairly clear that they should be defined in a shared library like common-module.

For the intermediate types, it is a design decision with the following options:

In a specific Module that first introduces the concept or is its main user. This is easiest to implement and manage, but may lead to creating too many dependencies on that Module
from other Modules that need to use some of the Data Types but not its Services.
In a dedicated Sister module to the one that defines the Services, this is a clean separation but may lead to a proliferation of small modules with a significant management overhead.
In a shared module at the Domain or Sub-Domain level that collects Data Types that, although may not be specific to a particualr Module in the Domain, they may clearly belong to the
wider Domain.
In a shared module at the system level, either common-module, a dedicated domain-types module or even several of them if the number of shared types grows too large.

The decision on where to define and publish these shared types may evolve over time and specific Data Types may start at a very specific Module and later be promoted
to a more general shared module as their use becomes more widespread across Modules or Domains.

Extension to IDL Schemas¶

API IDL’s define Services (gRPC) or Routes (OpenApi) and also Messages or Schemas to represent the structure of the information exchanged. The definition and usage of
Message or Schema definitions is very similar to the usage and definition of Data Types. The same considerations of where to define and publish them apply, just adding
a scoping level below the Module level for a particular API Endpoint. Definition of Schemas and Messages may be done within:

A Single API Endpoint definition (usually corresponding to a single Service)
The Module that publishes the set of API Endpoints that share the message definition.
A Domain or Sub-Domain
Globally at the system level.

Warning

Some IDL’s have severe limitations on how to compose API Endpoint specifications from partial definitions. E.g. OpenAPI does not specify a standard way to
resolve references outside a single file, much less across files that may be versioned independently in different repositories.

The selection of technologis for how to define API Endpoints needs careful consideration of these limitations and trade-off the ability to stay DRY vs.
the complexity of managing a tool chain to support composition of API Endpoint definitions from multiple sources.

Underlying Concepts¶

Cohesion: A measure of how closely related are a set of responsibilities and capabilities within a system. Indicated by how many unrelated primitives or workflows exist in
the element under design. Cohesion is related to the definition of a Module, Domain or Product regardless of their internal design. It can be an indication of the
quality of the design or decomposition of the containing element.
Coherence: A measure of how consistent all the artifacts that contribute to a capability or set of capabilities are. Indicated by the
number of exceptions, special cases and different ways to do things that needs to be handled to understand, maintain and use the capability. Coherence is related to
the design of the artifacts that support the capability.
Coupling: A measure of how interdependent two elements of the system are. Indicated by the number of items in one element that the other references, either explicitly
or implicitly (e.g. assumptions on behavior) and whether the references are in one direction or bidirectional. There are different kinds of coupling that affect the
design and maintainability of the system in different ways, some more harmful than others. There is always coupling between elements of a system,
as a system is made of interacting elements and not simply a collection of unrelated parts. The goal of a good design is to minimize harmful coupling.
Encapsulation: A part of the state of the system that can only be accessed and modified through a Service which is responsible for maintaining the integrity of the state it encapsulates and
providing appropriate querying and inspection capabilities.
API Endpoint/Service: A named collection of operations that together provide a self-contained business capability. Note that we will use for now the terms API Endpoint and Service interchangeably.
Later we’ll specialize the term API Endpoint to refer to the expression of a Service in a particular technology, protocol and at a particular address (typically a URI).
Operation: A typed, named signature that can be invoked in a Service that triggers the execution of a well-defined behavior. An operation will have a name unique within the Service it is defined,
a typed input parameter (which may be empty, or a Tuple of parameters) and a typed output parameter (which may also be empty or a Tuple).
Behavior: The observable effects and results of invoking an operation on a Service. Effects are actions that the Behavior performs in the system, including changes to the internal state of the Service that executes the
behavior, invocations of other Services, interactions with external systems, etc. Results are the output values returned at the end of the execution of the Behavior by the operation that triggered it.