Skip to content

Design: Sentry Configuration as Code

Audience: platform engineers and on-call operators Reading time: ~20 min full; ~4 min for §1–§5

The problem. Arda’s Sentry alerting is configured by hand in the UI: seven rules across two projects, with overlapping scope, one outright duplicate, and two defects nobody could see. One rule was scoped to an environment its backend project does not have, so half of it had never matched anything. Nothing recorded what any of it was for.

The approach. Alerting becomes declared configuration, reconciled by snapshot → plan → apply → verify. This establishes OperationsManagement, a fourth platform element for resources that observe and administer the platform rather than run it; Sentry is its first inhabitant, and the engine that reconciles it names no vendor.

Three decisions shape everything else:

The Application produces a desired-state tree, applied by our own engineSentry workflows have no CloudFormation representation, so there is nothing to synthesize (DQ-002)
Merge applies with no approval stepWhich makes the engine’s refusals — never delete the unmanaged, never apply without a valid rollback point — the actual safety mechanism (DQ-023)
Identity lives in the workflow nameSentry assigns ids and offers no metadata field, so the name is the only field we control that survives a round trip (DQ-029)

One premise was wrong. The brief described the legacy issue-alert-rule API and cited rule ids verified in the UI. The org had been migrated to Sentry’s workflow engine without a version bump; those ids do not exist in the legacy namespace, which still answers 200 with a lossy translation (DQ-004). Every type here was derived from payloads captured live, not from documentation — and the freshness block says when.

TermMeaning
WorkflowSentry’s unit of alerting configuration in the new engine: a trigger condition group plus one or more action filter groups. Replaces the legacy “issue alert rule”.
DetectorA project-scoped monitor (error, issue_stream, uptime_domain_failure). Workflows attach to projects through detectors, via detectorIds.
Data condition group{logicType, conditions[], actions[]} — the shape of both triggers and each entry in actionFilters.
Logical environmentAn Arda-level environment name (prod, stage, dev, demo) that resolves to a different physical Sentry environment name per project.
Managed setThe workflows this tooling owns: those carrying the required arda/ name prefix. Everything else is untracked and never touched.
ClaimedA managed workflow whose embedded tier key a declared tier carries. Only an unclaimed managed workflow is ever proposed for deletion (DQ-028, DQ-029).
Tier keyThe stable identifier a tier declares and never changes, embedded in the workflow name — arda/High — platform-be [high-be]. The only thing reconcile matches on.
TierA severity band with its own trigger and its own action. High creates a Linear ticket at priority 2; Urgent posts a Slack notification. They differ in action, not just in priority — see DQ-017.

ViewpointThis design’s position
ProductPlatform Operator — the on-call engineer who receives alerts and maintains alerting config. Persona authored by this project (DQ-014); no operator persona previously existed.
Functionaloamfault. Consumes no Arda services; produces no Arda endpoints. It configures a third-party control plane that observes every component.
Artifactsinfrastructure repo only. New modules under platform/, tools/lib/, platform/constructs/, stacks/, apps/, instances/, plus two drivers and a scheduled workflow. No new deployable image or chart.
RuntimeNothing runs in an Arda partition. Execution is operator-initiated or CI-scheduled, against Sentry’s API. The effect is on alerting for arda-frontend and platform-be.
OAMThis design is OAM-plane. Its own operational surface is covered in §11.
TechnologyInfrastructure layer; TypeScript on the repo’s existing ts-node + Jest stack. No new runtime dependency — SentryClient uses native fetch.
MechanismAdded by this designConsumed by this design
API Endpoints / ServicesNone. This design exposes no Arda endpoint.Sentry Workflows API: GET/POST /organizations/{org}/workflows/, GET/PUT/DELETE /organizations/{org}/workflows/{id}/, GET /organizations/{org}/detectors/, GET /projects/{org}/{project}/environments/.
ReferencesNone. Sentry entities are referenced by opaque ID within OperationsManagement config and never cross into an Arda Universe.1Password item reference for the alerts-scoped token.
Data TypesWorkflowSpec, DataConditionGroupSpec, ConditionSpec, ActionSpec, LinearActionSettings, LogicalEnvironment — all derived from live payloads, all in-process (no wire contract of ours).Sentry’s workflow JSON, treated as an external schema pinned by a dated observations note.
BindingsHTTPS + bearer token to https://us.sentry.io.1Password (OpResolver), reused unchanged.

Capability framing: DesiredStateEngine is a Capability, not a Sentry feature. It is the reconcile mechanism the OperationsManagement plane needs for any provider whose resources are declared rather than deployed. Grafana or a future uptime-monitor provider implements the same seam.


Arda’s Sentry alerting is configured by hand in the Sentry UI. Seven workflows exist across two projects, and the hand-maintenance shows: one is an outright duplicate, the best-configured Linear routing sits on a rule that is disabled, and one rule is attached to both projects with an environment filter that can only ever match one of them — so backend Slack alerting from it has never fired. There is no record of intended state, no review path for changes, and no way to detect drift.

This design introduces OperationsManagement as a platform element and makes Sentry its first provider. Desired state is declared through the repo’s existing IaC vocabulary — an AlertTier Construct parametric on trigger, logical environment, and action, composed into a SentryFaultAlerting Stack per Sentry project, assembled by the OperationsManagementApp Application, and pinned by an ArdaSystems Instance. Because Sentry workflows have no CloudFormation representation, the Application emits a desired-state tree rather than synthesizing a template, and a vendor-neutral DesiredStateEngine reconciles it against live state (DQ-002). Two drivers consume it: OperationsManagementConfig for operator-initiated snapshot/plan/apply/verify, and OperationsManagementDrift for read-only scheduled conformance — the fourth member of the repo’s existing drift family. A third consumer, the OperationsManagementDeploy workflow, plans on every PR and reconciles automatically on merge to main, so main and live cannot diverge (DQ-023). The PR plan is deliberate: with no gate at merge, review is the only human checkpoint, so the changeset has to be visible while it is still reviewable.

In scope: the engine, the Sentry provider, the Fault-area alerting Stack, and a migration that consolidates high-priority routing into Linear, adds an Urgent tier, and retires the frontend Slack rules. Deliberately deferred: the Performance-area resources (metric alerts, dashboards, uptime monitors). The seam accommodates them; this project does not build them (DQ-007).

#DecisionChosen Option
DQ-001Where the code livesOperationsManagement element, layered per repo vocabulary
DQ-002Execution modelDeclarative App + our own reconcile engine
DQ-003Identity and managed setPinned IDs + required name prefix — superseded in part by DQ-028
DQ-004API surfaceOrg-scoped /workflows/ only
DQ-005DetectorsRead-only, resolved symbolically
DQ-006Fixtures and snapshotsCommitted
DQ-007Performance lookaheadSeam designed, Fault implemented
DQ-008Urgent window3 in 15m, a tunable starting value (30m unavailable)
DQ-009Urgent scopingTwo rules, one per project
DQ-010Linear routingKTLO / BE and KTLO / FE
DQ-011Rule 3190653 defectFix in this migration
DQ-012Environment divergenceLogical-environment abstraction
DQ-013Uptime attachmentPreserve, flag in plan
DQ-014Product Viewpoint personaAuthor the Platform Operator persona
DQ-015assigneeIdStrip explicitly
DQ-016stateIdReuse Triage
DQ-017High → Urgent escalationUrgent notifies via Slack; only High tickets
DQ-018Removal modeDelete outright
DQ-019Split 3153115Delete, create two fresh
DQ-020Trigger scopeDrop existing_high_priority_issue
DQ-021Urgent Slack channel#1-dev-teamsuperseded by DQ-030
DQ-022Configuration shapeNested Configuration/Props/Built tree as one SentryAlerts value
DQ-023Deploy on mergeFully automatic; plan posted on the PR
DQ-024CI snapshotWorkflow artifact, 90 days, uploaded before apply
DQ-025Enablement timingAfter the migration lands and plan verifies empty
DQ-026Disposition of 3190653Delete and recreate, superseding adopt-by-rename
DQ-027Absent-environment assumptionMeasured and confirmed; 3190653 was inert
DQ-028What the managed set isPrefix ∩ declared; supersedes part of DQ-003
DQ-029Where identity livesA key embedded in the workflow name; no pinned ids
DQ-030Urgent destination#sre-production (C0BNTF2AJ6N)

Full rationale and rejected alternatives in decision-log.md. All decisions are settled; §13 records one stated assumption and one scheduled review.

  1. The Sentry schema is external and under-documented. Every type is derived from live payloads or round-trip discovery. Guessing a field name is a stop-and-ask condition. The observations note carries the date each shape was observed.
  2. event_frequency_count intervals are a closed enum: 1m, 5m, 15m, 1h, 1d, 1w, 30d. A 30-minute window cannot be expressed.
  3. A workflow carries exactly one environment string and one action config per action. Per-project environment names and per-discipline Linear routing therefore force one workflow per (tier × project).
  4. The workflow schema has no free-form metadata field, and POST does not accept a caller-supplied id. The name is therefore the only field this tooling controls that survives a round trip, and identity is a key embedded in it (DQ-003, DQ-028, DQ-029).
  5. Deletes are not reversible in place. Recreated workflows receive new IDs.
  6. SENTRY_AUTH_TOKEN is already taken in this repo — amm.sh resolves a partition-scoped, source-map-upload token under that name. The alerts-scoped token must use a distinct variable to avoid collision.
  7. apply is dry-run by default and refuses unless a snapshot still describes live state. For operator-initiated runs it additionally requires --confirm. In CI there is no confirmation step — merge to main applies automatically (DQ-023). This is a deliberate reduction from the original constraint (“never applies unattended”), taken with the trade-off understood: the PR review becomes the only human checkpoint, which is why the plan is posted on the PR rather than only after merge.
  8. Configuration follows the repo’s Configuration / Props / Built discipline. Each layer owns its own Configuration interface; the Instance file holds values only. Thresholds, channels, routing targets, and frequencies are nodes in one SentryAlerts tree, so retuning alerting is a config PR rather than a code change, and adding a tier or a provider is adding a node rather than a parallel family of constants (DQ-022).
AttributeTargetSatisfied by
IdempotenceSecond apply produces an empty planIssueAlertResource.normalize stripping all server-managed fields recursively
Blast radiusNever mutates an unmanaged workflowManaged set = the arda/ prefix; deletion needs managed and unclaimed; untracked reported, never touched
AuditabilityEvery mutation reconstructible after the factCommitted snapshots + JSONL audit log under --audit
RecoverabilityAny applied change reversible from a snapshotrollback restores from a named snapshot; new-ID limitation documented
ExtensibilityNew resource type = one file + registry entryResource<TSpec, TRemote> seam; engine is vendor-neutral
StakeholderPrimary concern
Platform OperatorAlerts fire when they should and route to the right place; changes are reviewable
Backend / frontend engineersTickets land in their discipline’s KTLO project, already triaged
Whoever runs the migrationNothing is deleted without a recorded snapshot and an explicit confirmation

The component diagram below shows the four layers. platform holds vendor constants with no I/O; tools/lib holds the transport and the vendor-neutral engine; the cdk package declares desired state through the repo’s IaC vocabulary; and the drivers tie them together. The dependency direction is strictly downward — instances → apps → stacks → constructs, with drivers consuming both the Application and the engine.

PlantUML diagram

Signatures live in the code. What follows is why each element exists and what it is not allowed to do.

ElementExists to
SentryServiceHold Sentry constants and the API-surface freshness block. Every shape was derived from live payloads, not documentation, because Sentry migrated this org between alerting engines without a version bump (DQ-004).
LogicalEnvironmentMap an Arda environment to each project’s physical Sentry name, absorbing a divergence that is an accident of project creation order (DQ-011 / DQ-012).
Workflow namingCarry identity in the workflow name, since Sentry assigns ids and has no metadata field (DQ-029).

The environment mapping returns null — never a blank filter — for a pair that does not exist, and callers must treat that as this tier does not apply here. Collapsing the two is precisely the defect that left rule 3190653 inert on the backend for months, so the type makes the collapse impossible rather than discouraged.

ElementExists toConstraint it must respect
SentryClientBe the single authenticated transportHonours Retry-After, backs off on 5xx, never retries other 4xx. Mirrors postmark-client.ts so the two age together
DesiredStateEngineReconcile desired against live — diff, plan, apply, snapshotNames no vendor. A second provider implements Resource and nothing else changes (DQ-002, DQ-007)
IssueAlertResourceTranslate between declared spec and Sentry’s workflow JSONnormalize strips server-managed fields recursively; the nested ids matter as much as the envelope, or the diff never converges and every apply rewrites every rule
DetectorResolverResolve symbolic detector names to the ids a workflow needsRead-only. A detector is somebody’s monitoring, not ours to rewrite (DQ-005). An unresolvable name is a plan-time error, never a silent empty attachment

Rendering ids back to symbolic names in diff output was designed and dropped: it requires teaching the vendor-neutral formatter about Sentry detectors, which is the coupling the engine exists to prevent. Plan output shows raw ids. A value-renderer hook on formatPlan would fix it without the import.

Produces one workflow spec for one (tier × project) pair. Three properties carry the design:

  • Parametric on its action, not Linear-specific. The TierAction variant selects the emitted shape. This is why the Construct is not named for Linear: only the High tier opens tickets, because a second rule firing for an already-ticketed issue creates nothing and upgrades nothing, so escalation routed through Linear is silently lost (DQ-017).
  • Discriminated unions, not optional fields. A new trigger or action kind is a new variant the compiler forces every consumer to handle, rather than a field that is silently absent.
  • Rejects at plan time what would otherwise become a workflow that looks configured and matches nothing: an interval outside Sentry’s closed enum, an environment unmapped for the target project, an empty detector list, a Linear action with no project.

It is deliberately not a constructs.Construct — it emits nothing into a template — so it exports validateProps as a free function, the documented convention for non-Construct value objects. assigneeId is emitted explicitly empty: the template rule this derives from carries a real assignee that must not propagate (DQ-015).

SentryFaultAlerting, OperationsManagementApp, ArdaSystems

Section titled “SentryFaultAlerting, OperationsManagementApp, ArdaSystems”

The Stack is the deployable unit for one project’s Fault-area alerting, and is where the Fault/Performance split lands — a Performance stack would be a sibling, deployed independently. It injects project and detectors into each tier, so a tier definition never names its own project and can be reused across them.

The App composes Stacks into the plane’s complete desired state. It constructs no cdk.App(): there is nothing to synthesize (DQ-002). Its Configuration aggregates and projects rather than unioning its stacks’ shapes — linearWorkspace and managedNamePrefix are hoisted above the per-project level, so two tiers cannot disagree about which Linear team owns Arda’s tickets.

The Instance holds const values only. Defining a Configuration in an instance file is a named anti-pattern in the IaC rules, and a missing value should be a compile error rather than a runtime guard.

The shape is the extensibility argument (DQ-022). The whole configuration is one JSON-like value, and every axis of growth is an edit to one node of it:

To addEdit
A Performance tierAnother entry in a tiers array
A third Sentry projectAnother entry in projects
A Grafana providerA sibling key beside sentryAlerts

None of them touch the Construct, the engine, or the drivers. This is also why every operationally-tunable value — the Urgent threshold, its Slack destination, the notification frequency — lives here and nowhere else: retuning after real traffic should be a one-line pull request, not a code change.

OperationsManagementConfig is the operator entry point: snapshot, plan, apply, verify. OperationsManagementDrift is the same plan with no intent to mutate, which is what stops drift detection and apply from ever disagreeing about what the world should look like. OperationsManagementMigrate owns the one-time retirement and nothing else.

OperationsManagementDeploy — the GitHub Actions workflow

Section titled “OperationsManagementDeploy — the GitHub Actions workflow”

Reconciles on merge to main, with no approval step (DQ-023). That is a deliberate reduction from the original “never applies unattended” constraint, and the consequences are load-bearing:

  • The PR check is the human checkpoint, because merge no longer has one. It posts the plan as a comment rather than blocking, since a pending changeset is the normal state of a PR that edits alerting.
  • The snapshot uploads before apply, 90-day retention, so a partial apply still has a rollback source (DQ-024).
  • Path filters are directory globs, so adding or removing a file never requires editing the workflow. Deliberately broad: a spurious run yields an empty plan and applies nothing.
  • It ships disabled, until the migration has landed and plan verifies empty (DQ-025). The first automated apply must never be the destructive one.

Behaviors are grouped below. Each cross-links to the Key Element that owns it and maps to entries in Behavior Verification.

Owner: DesiredStateEngine, IssueAlertResource.

  • Normalization makes specs and remotes comparable. Server-managed fields are stripped recursively before comparison, so an unchanged workflow yields an empty diff. Verified by round-trip rather than assumed.

  • Membership is the prefix; claiming is the key. A live workflow is managed when its name carries the arda/ prefix. A managed workflow is claimed when a declared tier carries the key embedded in that name. Deletion requires managed and unclaimed (DQ-028).

    Identity is a key rather than a server-assigned id because no id exists before the resource does. An earlier revision pinned ids in configuration, recorded after each create, and the window before that recording was destructive rather than merely untidy: the workflow carried the prefix, so it was managed, and no declaration held its id, so nothing claimed it — which is the delete rule. verify, run immediately after the first live apply, proposed deleting all six workflows that apply had just created. The key removes the window by existing before the workflow does (DQ-029).

    Two consequences worth stating plainly. arda/ is a reserved namespace: a hand-made rule using it, and declaring no matching key, will be proposed for deletion. And keys must be unique within the managed set — a collision is refused rather than guessed, and the App asserts uniqueness at build time so it never reaches the live org.

  • Untracked is not drift. Workflows outside the managed set are listed in plan output as untracked and never proposed for deletion.

  • Ordering is not semantic where Sentry does not make it so. detectorIds and data.settings are order-insensitive and sorted before comparison. Condition order within a group is preserved, because logicType: any-short short-circuits and order is therefore observable.

Owner: AlertTier, LogicalEnvironment, SentryFaultAlerting.

  • A tier resolves its own environment. The Construct takes a logical environment and resolves the physical name for its project. An unmapped pair means the tier is not emitted for that project at all — never emitted with a name that cannot match.
  • Linear routing is per discipline. platform-beKTLO / BE, arda-frontendKTLO / FE, both at Triage with the Bug label and no assignee.
  • The High tier fires on escalation only. new_high_priority_issue alone; existing_high_priority_issue is dropped (DQ-020).
  • The Urgent tier fires on occurrence rate and notifies rather than tickets. event_frequency_count at 3 occurrences in 15 minutes — deliberately narrower than the specified 3-in-30m, since 30m is not an available interval and starting strict produces too few Urgent notifications rather than too many (DQ-008). Its action is Slack, not Linear — a second rule firing for an already-ticketed Sentry issue is a silent no-op that neither creates a ticket nor upgrades priority, proven by probe on 2026-08-06 (DQ-017). Routing escalation to a notification channel makes it visible without depending on behavior the integration does not provide.
  • Uptime attachment is preserved and surfaced. The backend High tier keeps the two uptime_domain_failure detectors, and plan output states this explicitly so the carry-over is a visible choice (DQ-013).

Owner: OperationsManagementConfig, DesiredStateEngine.

  • Dry-run is the default. apply without --confirm prints the plan and exits without mutating.

  • In CI the checkpoint moves earlier, to the PR. The pull_request run posts the plan while the change is still reviewable; the push run applies without asking. A reviewer who reads the posted plan is the only thing standing between a merged config error and production alerting — the design does not pretend otherwise.

  • No apply without a snapshot that still describes live state. The gate compares the snapshot against live and refuses on any difference, naming each one. Age is not the test: a snapshot taken a second ago is worthless if something changed in that second, and an hour-old one is a perfectly good rollback source if nothing did. The comparison runs on the normalized form, so a rule merely firing between snapshot and apply — which moves lastTriggered — is not a refusal. In CI the snapshot is taken and uploaded before apply, so a partial apply always has a rollback source.

    An earlier revision of this design specified a session-scoped refusal — “a snapshot from the current run”. That could not work: snapshot and apply are separate processes, so a session identifier established at module load could never agree between them, and the gate would have refused every real invocation. Discovered by running the sequence end-to-end against the live org rather than by unit test, which calls the gate inside one process and cannot see it.

  • Destructive changes are marked and ordered last. Deletes and detaches carry a DESTRUCTIVE marker; a detach that would strip the last detector from a project requires its own confirmation flag.

  • plan exits non-zero when changes are pending, so CI can gate on it.

Owner: OperationsManagementConfig.

The migration runs as ordered, individually confirmable phases. Creates precede deletes so the window in which high-priority routing is uncovered — an unavoidable consequence of recreating 3153115 as two rules (DQ-019) — is as short as possible.

PlantUML diagram

The two halves run through different tools, and that is deliberate. The creates are ordinary reconcile work: OperationsManagementConfig apply performs them, is snapshot-gated, and runs again on every subsequent deploy. The deletes cannot go through reconcile at all — the retired rules carry legacy names, so identify() returns null and the engine classifies them untracked, the one category it is built never to touch. Making reconcile able to delete them would mean weakening exactly the guarantee that protects every unmanaged rule in the org.

So the destructive half lives in OperationsManagementMigrate, which holds the one-time managed-set exemption, and it refuses to act unless three conditions hold:

PreconditionFailure it prevents
Every retired id still names what it named at design timeDeleting a rule somebody renamed or recreated in the UI. Ids are opaque; a mismatch is a hard stop, not a warning.
All six replacements are liveThe ordering rule, enforced rather than trusted. Nothing otherwise stops the delete phase running first and leaving alerting uncovered.
A snapshot still describes live stateDeleting with no usable rollback path.

An id that is already absent reads as already retired rather than an error, so an interrupted migration resumes rather than restarting.

The call sequence from wrapper to API is unremarkable and reads plainly in the drivers. Two boundaries it enforces are not obvious from any single file:

  • Credentials are resolved in exactly one place and never persisted. The shell wrapper resolves the token for an operator; CI passes a 1Password service-account token and the driver resolves through the SDK. Neither writes it to disk, and snapshots record a token fingerprint rather than the token, so a snapshot can be committed without leaking the credential that produced it.
  • Every decision happens in tools/. The wrapper parses arguments and maps exit codes; it makes no API calls and evaluates no state. This is the repo’s tools-versus-scripts split, and it is what allows the drift check, the operator, and CI to share one implementation of what the world should look like rather than three that can disagree.
  • Halt on first error, print the rollback command. Partial application is reported precisely — which changes landed, which did not.
  • Rollback recreates from a snapshot. Recreated workflows get new IDs; this is an API property, stated in code and in operator output rather than worked around.
  • Round-trip instability is a stop condition. If a workflow does not round-trip cleanly after normalization, the tool reports it rather than papering over it — an unstable field means the schema understanding is wrong.

Test identifiers follow BV-<group>-<seq>, mirroring §9.

Test IDBehavior TestedRequired SetupTest Fixtures
BV-1-01Normalization strips every server-managed field, including nested idsLive fixture workflow-3433641.jsonCaptured workflow fixtures
BV-1-02Normalized fixture compared against itself yields an empty diffSame fixture twiceCaptured workflow fixtures
BV-1-03Pinned ID with a non-prefixed name raises a hard errorFixture renamed without prefixCaptured workflow fixtures
BV-1-04Unmanaged workflow is reported untracked, never in the delete setFixture set including a non-prefixed workflowCaptured workflow fixtures
BV-1-05A declared tier claims the live workflow carrying its key, rather than deleting itPrefixed live workflow with a matching keyCaptured workflow fixtures
BV-1-07A tier key round-trips through compose and parse, and survives a change of display name
BV-1-08A malformed key suffix parses as no key rather than as a guess
BV-1-06Two managed workflows sharing a name raise AmbiguousAdoptionError rather than guessingTwo prefixed fixtures with one nameCaptured workflow fixtures
BV-1-05detectorIds reordering produces no diff; condition reordering doesTwo fixtures differing only in orderCaptured workflow fixtures
BV-1-06Diff paths are addressable (actionFilters[0].actions[1].data.settings.priority)Fixture pair with one changed settingCaptured workflow fixtures
Test IDBehavior TestedRequired SetupTest Fixtures
BV-1-07Live GETnormalizeplan against unchanged desired state yields empty planRead-only tokenLive read-only harness
Test IDBehavior TestedRequired SetupTest Fixtures
BV-2-01Tier resolves physical environment per projectLogicalEnvironment table
BV-2-02Unmapped (project, logical) pair emits no workflow rather than an unmatched filterdemo × arda-frontend
BV-2-03Linear action carries empty assigneeId even when derived from the templateTemplate fixture with assignee setCaptured workflow fixtures
BV-2-04Per-discipline projectId routingBoth projects
BV-2-05High tier omits existing_high_priority_issue
BV-2-06Urgent tier interval is a member of the closed enumInterval enum guard
BV-2-07Backend High tier retains uptime detectorsDetector fixtureDetector fixture
BV-2-08validateProps rejects an out-of-enum interval, an unmapped logical environment, an empty detector list, and a blank Linear projectIdMalformed Props per caseInterval enum guard
BV-2-09linearWorkspace projected from the App reaches every tier identicallyTwo-project SentryAlerts value
Test IDBehavior TestedRequired SetupTest Fixtures
BV-3-01apply without --confirm mutates nothingRecording client stubRecording SentryClient stub
BV-3-02apply refuses when no snapshot exists, and when the newest no longer matches liveEmpty snapshot dir; snapshot diverging from liveRecording SentryClient stub
BV-3-03plan exits non-zero with pending changes, zero when cleanTwo desired statesRecording SentryClient stub
BV-3-04Last-detector detach requires its own flagFixture with one detectorDetector fixture
BV-3-05Destructive entries ordered after non-destructiveMixed plan
BV-3-06Path filter matches a new file added under a globbed directory, and does not match platform/one-password.tsWorkflow YAML + candidate pathsPath-filter matcher
BV-3-07A pull_request-triggered run plans but never applies, and still leaves a snapshotStubbed client, PR eventRecording SentryClient stub
BV-3-08The snapshot artifact is written before any mutating call is issuedClient stub failing on the first writeRecording SentryClient stub
Test IDBehavior TestedRequired SetupTest Fixtures
BV-4-01Phases run in order; --phase and --from honoredStubbed engineRecording SentryClient stub
BV-4-02Failure mid-apply halts and reports the rollback commandClient stub failing on Nth callRecording SentryClient stub
BV-4-03Second run of the full migration produces an empty plan (idempotence)Post-migration fixture setCaptured workflow fixtures
Test IDBehavior TestedRequired SetupTest Fixtures
BV-4-04Throwaway workflow create → GET → delete round-trips and leaves inventory unchangedWrite-scoped token; disabled, detector-less workflowThrowaway workflow probe

The seven live workflow payloads captured 2026-08-05, committed under the repo (DQ-006). They are simultaneously the type-derivation evidence, the unit-test corpus, and the migration’s rollback baseline.

The GET /detectors/ payload — eight detectors across the two projects, including the two uptime_domain_failure monitors whose attachment BV-2-07 asserts.

A SentryClient implementation that records calls and returns scripted responses without network access. Every safety-gating test asserts on what was not called, which is the property that matters for a destructive tool.

A compile-time union plus a runtime assertion pinning 1m | 5m | 15m | 1h | 1d | 1w | 30d, dated to the observation. Its purpose is to fail loudly if someone writes 30m again.

Evaluates the deploy workflow’s on.push.paths globs against candidate file paths using the same matching semantics GitHub applies. Its purpose is to catch the failure the requirement names directly: a filter so specific that adding a file silently stops triggering deploys. Asserts both directions — a newly-added file under a globbed directory does match, and platform/one-password.ts does not.

Runs plan against the real org with a read-only token. Asserts an empty plan when config matches live — the conformance check, also the drift driver’s core.

Creates one workflow that is enabled: false with detectorIds: [], so it cannot fire; captures its payload; deletes it; asserts a 404 and an unchanged inventory. This is the mechanism by which any future unknown shape gets discovered rather than guessed.


  • JSONL audit log under --audit: one record per request/response, with the token fingerprint (a hash) and never the token.
  • DriftReport from OperationsManagementDrift, in the same shape the three existing drift drivers emit, so it feeds the same failure-issue mechanism.
  • Snapshot manifests: org, projects, workflow IDs, token fingerprint, timestamp.
ControlMechanismEffect when set
Dry-runDefault; --confirm opts outNo mutation without explicit intent
PR planpull_request triggerPosts the changeset while the change is still reviewable; the only human checkpoint
Deploy kill switchDisable the workflow in the Actions UIStops automatic apply without reverting code — the emergency stop, since there is no approval gate
Deploy triggeron.push path filter, directory globsWhich changes cause a deploy; broad by design
Workflow enablementWorkflow disabled until post-migrationThe first automated apply is never the destructive one
Snapshot gateAutomaticapply refuses unless a snapshot still matches live
Detach guard--allow-last-detector-detachPermits a detach that would leave a project unattached
Phase selection--phase=<name>, --from=<name>Run or resume a single migration phase
Audit log--auditFull request/response paper trail
TokenAlerts-scoped variable, distinct from SENTRY_AUTH_TOKENAbsent ⇒ fail fast with a named error

Negligible. A full plan is a handful of API calls (one workflow list, one detector list, one environment list per project) against endpoints with no per-call cost. The scheduled drift check runs monthly, matching the cadence of the existing drift workflows.

The real cost is Linear ticket volume, and it should fall. Dropping existing_high_priority_issue removes the churn source, and the Urgent tier adds no tickets at all — it notifies Slack (DQ-017). One ticket per Sentry issue per project is the steady state. Volume should still be measured after the first week, since per-discipline routing splits the stream in a way nobody has observed yet.

  • Drift check fails. Read the DriftReport. Divergence means someone edited a managed workflow in the Sentry UI. Run plan to see the changeset; either fold the change into config or re-apply.
  • Migration halts mid-apply. The driver prints the exact rollback command with the snapshot path. Recreated workflows get new IDs; re-pin them in the Instance file afterward.
  • Alerts stop arriving. Check the workflow is enabled, its detectorIds are non-empty, and its environment exists on the target project — the last is the failure mode that made rule 3190653 silently inert.
  • Token expired or descoped. SentryClient fails fast with a named error. Rotate in Arda-SystemsOAM; no code change.
  • Deploy workflow ran but nothing changed. Expected. The path filter is deliberately broad, so an engine or tooling edit triggers a run whose plan is empty; nothing is applied and the job ends green.
  • A bad change was merged and has already applied. There is no approval to withhold, so recovery is the path: download the snapshot artifact from that run and roll back, then revert the config commit so the next merge does not re-apply it. Do both — rolling back without reverting means main still describes the bad state and the next unrelated merge restores it.
  • Deploy failed part-way. Same halt-and-report contract as a manual apply: the run prints the rollback command and the snapshot artifact is attached to that run. Rollback needs the artifact within its 90-day retention; beyond that, reconstruct from the committed fixtures and the Instance file’s git history (DQ-024).
  • Need to stop automatic deploys right now. Disable the workflow in the Actions UI. That is the emergency stop; it does not require a PR.

Where the work lands, by layer. Contents are not enumerated — read the code.

LayerPathHolds
Vendor factssrc/main/cdk/platform/sentry-*.tsConstants, the API-surface freshness block, workflow wire types, the validating parser, and the naming functions that carry identity
Constructsrc/main/cdk/platform/constructs/sentry/AlertTier — one workflow spec per (tier × project)
Stack / App / Instancesrc/main/cdk/{stacks/operations-management,apps/OperationsManagement,instances/OperationsManagement}/The deployable unit, the composition, and the declared values
Enginetools/lib/desired-state/Resource seam, diff, plan, apply, snapshot. Names no vendor
Providertools/lib/sentry/The Sentry Resource, transport, detector resolution, token resolution
Driverstools/operations-management-*.tsconfig (snapshot/plan/apply/verify), drift (read-only), rollback, migrate (the one-time retirement)
Operator surfacescripts/sentry/Shell wrapper and README. Ergonomy only — no logic
CI.github/workflows/operations-management-{deploy,drift}.ymlDeploy on merge (ships disabled) and monthly drift
Fixturestools/lib/sentry/__fixtures__/Seven workflow payloads captured live on 2026-08-05, plus detectors and projects

Two placements are deliberate and would otherwise look arbitrary. The workflow types live under platform/ rather than tools/lib/, because the Construct layer needs them and src/ may not import tools/. The Sentry client lives under tools/lib/sentry/ rather than flat, so the deploy trigger can name one directory glob instead of tracking files (DQ-023).

Recorded because a reader comparing design to code will otherwise wonder.

PlannedWhy it is absent
tools/lib/sentry/registry.ts — kind → resourceBuilt, then removed. Nothing consumed it, and a registry with one entry and no reader is speculation rather than extensibility. The seam that matters is Resource itself
tools/lib/drift/sentry-probe.ts — read helpersThe Resource seam already gives drift and apply one shared read path. A second one is the divergence that rule exists to prevent
An import command on the config driverUnnecessary once identity moved into the name: adoption happens on any ordinary plan
A Jest suite parsing the deploy workflow YAML (BV-3-06)Not built. The path filter is unverified by test
Symbolic rendering of detector ids in plan outputRequires teaching the vendor-neutral formatter about Sentry. Plan output shows raw ids
  • Performance-area resources (metric alerts, dashboards, uptime monitors). Revisit when: Performance Management work starts. See DQ-007.
  • Managing detectors. Revisit when: uptime monitors need declaring, which is the same trigger as above. See DQ-005.
  • Unifying Sentry environment names across projects. The design absorbs the divergence rather than fixing it. Revisit when: someone changes SDK configuration in either codebase. See DQ-012.
  • The #sentry-fe-prod naming problem. That channel carries backend alerts (rule 3462179) despite its name, and the retained Slack rules keep pointing at it. The Urgent tier uses #sre-production instead, so this design neither fixes nor worsens the existing mismatch. Revisit when: Slack routing is restructured, or the retained Slack rules are themselves brought under management.
  • Reconciling Sentry project topology with the Sentry Integration design (arda-operations / arda-accounts vs. the live arda-frontend / platform-be). Revisit when: a third component is onboarded to Sentry.
  • Wiring plan into a per-PR CI gate. The command is CI-shaped; only the monthly drift schedule is built. Revisit when: config changes become frequent enough that monthly detection is too slow.
  • Linear-side automation (triage rules, duplicate merging). Revisit when: simultaneous rule firings produce enough duplicate tickets to be a nuisance — the probe showed the dedup race is real but narrow (DQ-017).
  • Two-tier Linear ticketing. Ruled out because a later rule firing neither creates a ticket nor upgrades priority. Revisit when: Linear’s Sentry integration gains documented priority-update semantics, or the H2 hypothesis (dedup keyed on Sentry issue × Linear project) is confirmed — under H2, routing Urgent to its own Linear project would restore ticketed escalation.

All decisions are settled, and what was the design’s one open assumption has since been measured. What remains is a scheduled review, which does not block implementation.

  • Settled by measurement: a Sentry environment filter naming an environment absent from a project matches nothing, not everything. This is what makes rule 3190653 inert and therefore a defect rather than a design choice. It was carried through the design as an untested assumption, and was tested on 2026-08-06 with a controlled probe — a workflow scoped to a valid environment fired, an otherwise identical one scoped to production on platform-be did not (DQ-027). Retiring 3190653 is a repair, and the PR describes it as one.
  • Review: the Urgent threshold is a starting value, not a conclusion. {3, "15m"} is deliberately strict, and its destination #sre-production (DQ-030) has not yet received a real alert. Both are nodes in the Instance file’s SentryAlerts tree specifically so the first weeks of real traffic can retune them in a one-line PR (DQ-022). Worth revisiting after roughly a week of live operation.

A required gate between “implementation complete” and “PR opened for external review”. It exists because this project twice discovered, mid-build, that governing guidance had not been read — once for skills, once for the repository’s knowledge-base/. Both times the code was already written and already passing, and both times the guidance had something to say about it.

The audit is cheap and mechanical. Run it against the finished branch, record the result in the PR, and treat any gap as work rather than a footnote.

#CheckWhy it is here
1Every skill the routing table names for this work was loaded and its doc page read — not just the stubtypescript-coding was a stub over a page that never existed; that was only discovered by trying to read it
2knowledge-base/ was listed, every file assessed for relevance, and the relevant ones read in fullTwo of nine were read; testing-and-ci.md documented the exact lint rules discovered by trial, and tooling-and-scripts.md held the entry-point convention the drivers need
3Conventions encoded in tooling were consulted directly: eslint.config.mjs, tsconfig*.json, jest.config.jsThe enforced standard is the config, not the prose; where they disagree the config wins and the prose is stale
4Layer rules hold — import/no-restricted-paths, tools/ vs scripts/ splitSilent violations are possible where a zone is not configured
5Both typecheck configurations pass, not just the test suiteisolatedModules means a green suite proves nothing about compilation
6Guidance found stale or wrong during the work has been corrected or reportedLeaving it costs the next person the same discovery
7Documentation contradicted by the code has been updatedA design that no longer describes the code is worse than no design
8CHANGELOG entry present and describing intent, not file listsRepo gate; also the only artifact most readers see

State plainly in the PR which checks passed and what each turned up. “Audit clean” is a claim; “read these four knowledge-base files, two were stale, both corrected” is evidence. A reviewer should not have to take the audit on trust any more than the tests.


  • Decision Log — settled decisions and rejected alternatives.
  • Goal — the project goal this design serves.
  • Sentry Observability — how Arda uses Sentry today.
  • Sentry Integration — the project that introduced Sentry and deferred alert rules.
  • Platform Operator persona — the Product Viewpoint actor.
  • IaC Functional Design — layer responsibilities, the script → instances → apps → stacks → constructs direction, and the anti-patterns this design’s configuration shape avoids.
  • AWS CDK Infrastructure — the Configuration / Props / Built pattern and repo-local CDK conventions.
  • infrastructure/knowledge-base/cdk-construct-patterns.md — the mechanical triad rules, including the non-Construct value-object exception that AlertTier follows.
  • infrastructure/knowledge-base/platform-architecture.md — platform elements and the generalized IaC vocabulary.
  • infrastructure/knowledge-base/tools-vs-scripts-split.md — logic in tools/, ergonomy in scripts/.


Copyright: (c) Arda Systems 2025-2026, All rights reserved