Arda's Platform Design.

The description of the design of Arda’s platform requires multiple viewpoints to capture its different aspects and provide a productive environment for engineers to understand and contribute to its design.

Architectural Viewpoints¶

Arda develops products to support manufacturing and supply chain operations at the intersection of the physical and information worlds. They share many characteristics with SaaS and Enterprise applications with additional requirements to interact with users and systems in the shop floor. Development of these systems requires a firm foundation of concepts to describe them and to allow engineers to communicate with minimal ambiguity.

There are many alternative frameworks to describe systems and software architectures as referenced in the notes below. At Arda, we have adapted their concepts to early architectural decisions and the practices and tools we use and the system that Arda needs to build.

Arda’s system and the development process rely on some early decisions that are worth making explicit:

The bulk of the system will be deployed in the AWS cloud and will also use other ancillary cloud services where appropriate (e.g. Coda, Documint, …)
User access to the system will be through browsers and mobile devices over the public internet. Some devices used to access the system may not be traditional screen-mouse-keyboard based devices (e.g., barcode scanners)
The system will be built as a structured set of loosely-coupled elements with well-defined capabilities and strict state encapsulation. Interaction of elements among themselves and with the external world will be through well-defined interfaces and without implicit sharing of their internal state.
The system needs to be easily duplicated when provided with a set of resources (compute, network, storage, ingress/egress, oam) to support independent development, testing, and verification of new capabilities.
All items that go into building and deploying the system will be versioned and their content and configuration tracked. All changes to the system or any of its parts will require at least one review and approval by a team member other than the author of the change.
Development practices will avoid manual (and error-prone) intervention of engineers in the building and deployment of the system or parts of it from a set of verified elements. Building and deploying the system or its parts will be completely reproducible given a set of configuration parameters.

Development of a system can be viewed as a sequence of transformations that take a description of the desired capabilities of the system and produce a runtime system that exhibits behaviors that support these capabilities. As a high-level view we adopt:

Design: Transform desired capabilities into a set of functional elements, each of them with defined behaviors and internal state that, interacting together, support the desired capabilities.
Implementation: Transform the elements resulting from the design into source code, configuration parameters, etc… that express the behaviors and internal state of the elements in a way that can be accepted by computing tools (compilers, etc…).
Building: Transform the results of implementation into a set of artifacts like images, packages, scripts, etc. that can be directly deployed to create the running system.
Deployment: The process of installing the results of building onto a set of available resources (e.g., AWS accounts) that expose the access points, processing, and stores that are used by Arda’s customers.
Operation: Although not strictly a transformation in the way the previous ones are, it is the set of activities that continuously configure, monitor, diagnose, secure and pay for the system while it runs.

The viewpoints we adopt correspond to the Inputs & Outputs of these transformations.

This document describes the intent and structure of each viewpoint without covering the Requirements viewpoint, for which we will only describe the concepts associated with the systems’ Context.

Notation¶

Important

Any architectural description necessarily uses a lot of terms that already have assigned meanings in other contexts or in common usage. Terms like System, Component, Module, Service, etc. are hopelessly overloaded and can mean different things in different contexts. This framework also uses many of these terms.

Within this framework, those terms are intended to have very specific meanings and the reader is encouraged to put aside their use in other contexts as much as possible. To avoid confusion, we will strive to use terms consistently and highlight them with code-style in the text.

Diagrams are essential for describing system architecture patterns. To avoid confusion, we will use consistent notation based on the C4 Model and UML. While UML has strict formal semantics, we will prioritize clarity and brevity over strict adherence, adjusting the level of detail as needed.

Design Viewpoints¶

Context: Describes how the system interacts with its environment and its users. It should describe specific interactions as well as how the system uses resources or other systems to perform its functions.

Functional: Describes the system’s functions and its interface with the environment, as outlined in the context viewpoint. It focuses on the system’s functions and managed information, aiming for clarity and simplicity. Functional decomposition identifies interaction points (Endpoints), components (Domains and Modules), and managed information. This viewpoint is central, defining requirements and specifications while guiding design decisions in other viewpoints. Unlike with other viewpoints, there is no clear set of technologies or concrete elements that support it. The practical realization of functional elements is in technical documentation and diagrams.

Sources and Development: Represents the tangible elements created by engineers, subject to change control and review. This includes source code, configuration files, and build scripts. The organization should simplify locating and modifying elements, ensuring minimal changes are needed to address specific functionality updates.

Artifacts: Describes the elements that will be directly used to create the runtime system. Artifacts are also subject to change and version control. They are NOT directly changed by engineers. They are the result of transforming and grouping Sources. Artifacts include docker images, jar libraries, node packages, helm charts, node applications (aws-cdk), etc.

Runtime Viewpoint: Describes the structure of the resources that, working together realize the functionality of the systems that users access. Runtime viewpoint elements include resources to support computing workloads, communication among elements and with the external world, information and data storage, and the ancillary elements needed to keep the system itself running (the elements that run the OAM viewpoint). The runtime viewpoint includes resources like K8s clusters and resources, databases, queues and streams, network addresses and routes, etc. It also describes how these basic resources are grouped in to higher level constructs that work together and act as pre-defined units. The goal of this viewpoint is to enable engineers to understand the physical structure of the system and how it can be created, operated, changed and maintained, again with a goal of minimizing information overload for any given individual task.

Operation, Administration, and Maintenance (OAM) Viewpoint: Describes how the system is operated, administered and maintained. It includes the mechanisms used to provision and configure it, monitor it while it runs, detect and diagnose problems, secure it against unauthorized access and attacks and report on the cost of operation to Arda.

Context¶

The context viewpoint describes the systems interactions with external systems and users. In this viewpoint, the complete Arda system is an opaque concept that represents all the offerings of Arda to its customers. The description is centered on the other elements in the diagram:

Arda System: The complete set of products and services offered by Arda Systems to its customers.
Enterprise: An Enterprise is a legal entity that does business with Arda Systems and has a contractual agreement to use Arda’s services or to provide services to Arda. An Enterprise acts through the Personas that act on their behalf (are agents of the enterprise) or through information systems that interface with Arda’s system. Arda Systems itself is also an Enterprise in the sense that it has personas that act on its behalf to interact with the system.
Persona: A Persona is a human that interacts with the system on behalf of an Enterprise. Personas can be characterized by the way they access the system (desktop, mobile, etc.), the functionality they access (Administrators, Tenant Administrators, Shop Operators, Facility Managers, …).
External System: An External System is an information system that interacts with Arda’s system on behalf of an Enterprise through a machine to machine API. It can be a Server system that provides functionality to Arda System (e.g., Documint ), or a Client system that uses the functionality of Arda System (e.g., The ERP of an Arda Customer).
Application: An Application is a human accessible interface to interact with Arda System. They may be web applications, mobile applications, or specialized software to support other HMI devices like barcode scanners.
API: An API is a machine accessible interface to interact with Arda System. These APIs will preferably be implemented as REST over HTTP but other protocols may be used when appropriate (e.g. Ftp for file transfer, …).

Functional Viewpoint¶

The functional viewpoint decomposes the system into smaller units that contribute to the overall functionality. The decomposition aims to minimize coupling between units and maximize cohesion within them. The decomposition is first informed by partitioning the state of the system (the information it keeps) into internally consistent and cohesive information models, then by the functions and algorithms that operate on these information models. Finally, the functional decomposition will extract common cross-cutting concerns like security, observability, data access and querying patterns, etc. into generic units to be used across the system.

Note

The Functional Viewpoint, unlike other viewpoints does not have commonly adopted tools and artifacts to represent its concepts so we need to rely on documentation and diagrams to represent it. Nevertheless, these concepts need to be understood as concrete as Classes or Functions in source code or docker containers or K8s deployments in the runtime viewpoint that have concrete, well-understood technology artifacts to represent them. There are experimental frameworks and tools to represent functional-like elements but they are too complex and immature to be used with an advantage in the Arda system.

System: The complete set of capabilities of a product (Product Launches) offered by Arda offers. It is decomposed into Domains that are loosely coupled among themselves so that they can be defined, developed and deployed as independently of each other as possible.
Domain: A cohesive set of information, workflows, functions of the system. It is composed of Applications, Modules and Capabilities that together, provide a cohesive set of functionality in the system.

Application: A Functional unit that provides human accessible UI to interact with the system and through which human users may exercise the functionality of the system or access the information it contains. Regardless of the technology used, an application will rely on a delivery mechanism (e.g., web browser, mobile app, etc.) and will follow a UX design specification.

Module: A collection of Algorithms and Information models that implement the functionality represented by one or multiple Endpoints belonging to a Domain. Modules can be classified into several types (which may overlap in specific cases) attending to their main functionality like UI elements, Data Authorities, Transformers, etc.

Endpoint: The mechanism to access a well-defined function or information offered by a system either to external systems or internally between Modules or between Applications and Modules. They may be two-way endpoints expressed as Request/Response pairs or Message based Streaming endpoints that either produce or consume one-way messages.

Endpoints should have an associated Specification that describes them formally using OpenAPI or gRPC for Request/Response endpoints or using protobuf schemas or json schemas for Streaming endpoints.

Should this be called Service and free the term Endpoint for the Runtime Viewpoint?

Important

Endpoints are the only, exclusive way to access the functionality of a Module.

Capability: A closely related set of algorithms, information models and designs that express a common cross-cutting concern across Modules and Applications or express and adopted standard or best practice to support certain common functions. Capabilities are never mapped by themselves into runtime elements but always embedded in the Modules and Applications that use them. Examples of Capabilities range from simple libraries for Physical Quantities and Units of Measure to complex technology patterns like OAuth2 authentication, ABAC authorization, etc. On the User Interface side, examples of Capabilities include table and form interaction patterns, navigation, etc.

Sources Viewpoint¶

All Sources for Arda are managed by GitHub. The Sources Viewpoint follows the concepts and structure of GitHub and the build/packaging tools that Arda has adopted with standards practices:

Arda Sources are organized in a Multi-Repo structure.
Repositories may contain multiple projects in them and may be multi-technology.
Projects are defined by the build and packaging system they use. Arda’s adopted build systems are:
- Java/Kotlin: Gradle
- Node/Typescript: npm
- Python: poetry
- K8s Manifests: Helm
- Images Specifications: Docker
- Build and Deployment Scripts: GitHub Actions
- Cloud Resources: AWS CDK V2 (supported by Node typescript)
- Coda Packs: Coda Packs (supported by Node typescript)
- Documentation: MkDocs (supported by Python)
Note that other tools or systems may be adopted in the future (e.g., Makefile) to support specific needs.

Organization

Represents the complete set of Source Code assets that Arda owns. It is implemented as a GitHub Organization. Aside from containing the repositories, it also defines access control for users, common policies and special values (e.g. Security Tokens, Environment Variables, etc.). Although it might be desirable to have a way of nesting organizations, particularly as Arda grows, GitHub does not support it. We leave it for the future if needed (could be supported with a naming scheme for repositories tied to GH teams).

Repository

A set of source code with common change reasons, likely to have a common lifecycle and have tighter dependencies than cross repositories. Repositories may contain multiple technologies in them (e.g., Java/Kotlin, Node, Python, etc.) and may contain multiple projects in them.

Project

A project is a set of sources in a single technology from a build point of view. Projects tend to directly implement a Module, Capability, or Application from the Functional Viewpoint. Although a project may produce multiple Artifacts in the Artifact Viewpoint, the most common case is that a project produces a single Artifact.

Package

A package is a closely related set of files that together implement a cohesive functionality part of a Module, Capability, or Application. It corresponds to:

A package in Java/Kotlin
A module in Node/Typescript
A package in Python
A Chart in Helm

Packages, following the programming language practice can nest and may include sub-packages.

File: A single unit of source code in GitHub or the development operating system.
Export: An export is a name in the specific technology that can be used outside the package where it is defined. The roughly correspond to public types, functions, variables in Java/Kotlin, exported symbold in Node/Typescript, etc. Different programming languages have different visibilty designs and some of them (e.g., Java Modules) are not as widely used. In the context of Arda Architecture, Exports refer specifically to symbols that are intended to be accessed outside the Artifacts the sources contribute to.

Artifacts Viewpoint¶

Note

TODO: This Viewpoint should also describe the concepts involved in transforming and qualifying the artifacts for deployment through Stages to serve the different “Purposes” of the Organization.

Organization: Similar to the Sources Viewpoint, the Organization is the complete set of artifacts that Arda owns. Arda Uses GitHub package registries to store its artifacts, so the organization is the same as it is for the Source Viewpoint.
Registry: A registry is a repository of artifacts that can be accessed by build tools and scripts to store and retreive artifacts. It is implemented as a GitHub Package Registry. The registry contains the artifacts produced by the sources in the Sources Viewpoint that are then used to deploy the runtime system. Arda uses registries for:

- Docker images
- Java/Kotlin jar files in a Maven structure.
- Helm Charts
- Node Packages in a npm structure. (planned, not implemented yet)

Currently, Arda does not use registries for:

- Python packages
- CDK Applications (packaged as Node packages, Docker images or AWS Lambda functions)

Category: A category is a set of artifacts that are used to build the runtime system. It is implemented as a GitHub Organization. The registry contains the artifacts produced by the sources in the Sources Viewpoint and is used to build the runtime system.
Artifact; An Artifact is a versioned, packaged unit representing reusable capabilities. It bundles code (source or compiled), necessary resources, and descriptive metadata (like name, version, dependencies) according to the conventions of a specific technology ecosystem, facilitating its distribution, deployment, and management as a dependency by other software or directly for deployment. Artifacts can be:

- Library: An artifact intended to be used as a dependency by other artifacts:
- Java/Kotlin: A jar file in a Maven structure.
- Node/Typescript: A module in a npm structure.
- Python: A package in a poetry structure.
- Image: An Artifact intended to be deployed to a container runtime.
- Chart: An Artifact that describes a complete set of K8s resources to be deployed together in a K8s cluster.
- CDK Application: An Artifact that can be deployed to an AWS account using the AWS CDK and creates a set of AWS resources. It is implemented as a node typescript application.

Runtime Viewpoint¶

Cloud Provider: An entity that supplies resources to Arda’s platform, typically through a contract or terms-of-service
agreement. The main providers are Coda, AWS, GitHub, etc… with other specialized suppliers like Documint
and Short.io.
Component: The smallest unit of the Arda platform that is deployed and managed independently. It may consist of a
collection of resources that are deployed together and managed as a single unit. Components are the direct
support of the functionality of functional modules. A Component may include resources from multiple technologies (e.g. AWS Resources like API Gateway Routes, Micro-Services implemented as Helm Charts to deploy in a K8s cluster or Database configurations deployed as SQL scripts and initialization containers to execute them). In the future, as the concept matures, Arda may develop tools to coordinate resource deployment across technologies withing a component (e.g. Shell scripts, Makefiles, GH Actions) and formalize this concept further.

Component is a particularly overloaded term. It is used in the context of Arda to refer to a unit of deployment and management that is not necessarily a single unit of functionality. This concept may be renamed to make it clearer.

Partition: A set of runtime resources deployed on top of an Infrastructure that provide all the computing, network, ingress/egress and storage resources needed to deploy components. Different Partitions may have different resource contents and configurations depending on their intended use. A Partition intended to host production Components will have higher security, reliability and performance requirements than one intended to host development components. Also, depending on the Components it needs to host, it may have different combinations of resources, providing a RDBMS or not, messaging queues or not, etc.
Infrastructure: An Infrastructure provides a collection of shared resources that different Partitions deployed to that Infrastructure use in a shared fashion. There will be a very small number of Infrastructures in use at any given time. The main reason to have Partitions and Infrastructures as distinct concepts is to be able to optimize cost and usage of resources that may be scarce or expensive. For example, K8s clusters are complex and expensive to deploy but they can scale to host multiple Partitions. Infrastructures are the main unit of cost and governance in the platform. There will be two main types of Infrastructures: Production dedicated to support commercial operations and NonProduction dedicated to support development, testing, and qualification operations.
Root: The collection of Resources that are global to the complete platform and shared by all other elements of the platform like root DNS entries, e-mail domains, etc.
OAM: OAM is the collection of resources that are used to provision, operate and monitor a number of Infrastructures
with their contained Purposes and Components. There may be more than one OAM at a time, but only for special
projects like building new types of Infrastructures or developing OAM capabilities themselves.
Resource: A Resource is a capability needed to support the execution and operation of the system. They are obtained from Cloud Providers. They can be classified into:

- Compute: Resources that provide processing capabilities for workloads. They can be virtual machines, containers, serverless functions, etc.
- Network: Resources that provide connectivity and routing capabilities for workloads. They can be virtual networks, load balancers, gateways, etc.
- Storage: Resources that provide persistent storage capabilities for workloads. They can be databases, object storage, file systems, etc.
- Ingress/Egress: Resources that provide access to the system from the external world. They can be API Gateways, Load Balancers, etc.
- OAM: Resources that provide the capabilities needed to operate, administer and maintain the system. They can be monitoring tools, logging tools, security tools, etc.
- External Services: Cloud or SaaS Resources consumed as a service by the system.

OAM Viewpoint¶

Warning

TBD

Notes¶

Architectural viewpoints have long been used in the fields of software and systems engineering, with standards from IEEE, IETC, ISO and others. The approach taken at Arda is strongly influenced by the 4+1 architectural view model while trying to keep it as simple as possible for practical use. Other architecture description models used in defining this design include Abstractions of the C4 Model and the System Model from Backstage Software Catalog and Developer Platform