Checks That Cannot Fail

A guard existed to stop a mock-mode server being tested and reported as a real-backend pass. It waited five seconds for the mock worker’s marker and, on timeout, concluded the server was real.

// The bad way: absence of a signal, inferred from a clock.
await page.waitForFunction(() => window.__msw_worker, { timeout: 5000 })
  .catch(() => { /* timed out — must be the real server */ });

The mock installs that worker through dynamic imports. A slow mock startup times out and passes the guard — producing exactly the false green the guard was added to prevent. Nothing ever went red.

This page is about that shape, and the rule that closes it.

A check must derive its answer from a fact that the failure it guards against cannot produce.

A check that breaks this rule still runs, still passes, and still appears in the record as evidence. It answers “fine” using something the broken case produces as readily as the working case, so the failure is indistinguishable from a legitimate negative result. Nothing goes red. The gate reports a pass, and the pass is worth nothing.

This page states the rule, names the two shapes it usually takes, and works through seven instances found on two pull requests in one evening.

Two shapes

A timeout treated as evidence. The check waits for a signal, does not get it, and concludes the signal is absent. Slowness and absence produce the same observation, so a slow success is read as a failure — or, more dangerously, a slow failure is read as a success.

An empty result for an input the reader did not recognize. A reader receives a shape it has no branch for and returns [] or undefined. Downstream, that is indistinguishable from “the server had nothing to return.”

Both are silent. Both look exactly like the good outcome. Neither fails a test.

The good way, on the same example

The comment justifying those five seconds argued about cost — five seconds is cheap, so five seconds it is. The defect was never in the duration. It was in the inference.

The fix is not a longer timeout. It is a different fact.

MSWInit returns null until worker.start() resolves; it blocks the React tree deliberately, so that data fetches cannot outrun the worker. On a mock build there is therefore nothing on the page until MSW is installed. The check now waits for rendered content, then reads the marker once. Absence of the marker after the app has rendered is conclusive, because the fact it rests on — the app rendered — is one a mock build cannot produce without first installing MSW.

That is the shape to look for when applying the rule. Ask what fact the answer rests on, then ask whether the failure could produce that fact too.

Seven witnesses

Seven instances, all from #1290 and #1305 in arda-frontend-app. Five are readers, one is a guard, one is a configuration default.

#	Site	What it answered with	What the failure looked like
1	`cardsForItem` (list read)	`[]` for an envelope shape it did not know	an item with no cards
2	`statusOf` (single-record read)	`undefined` for `data.records[0]`	a card whose event never landed
3	`recordIn` (the fix for #2)	`undefined` for an unrecognized envelope	a record missing a field
4	`cardsIn` / `recordIn` (flat records)	`[]` after filtering every flat record away	an item with no cards
5	`recordIn` (array branches)	a raw record where callers expected a normalized one	a payload that is legitimately absent
6	`assertServerNotMocked`	”the server is real” on a five-second timeout	a real-backend run
7	the Playwright project’s inherited `retries`	a green run on the third attempt	a run that passed

Weigh these before citing them. They are not seven independent confirmations. Witnesses 1 to 5 are one wrong assumption found five times, and the root is the next section. Witness 6 is independent and is the sharpest case. Witness 7 is independent and reaches a register the others do not — configuration inheritance rather than code.

Witness 7 is worth stating on its own terms. The dev-backend project inherited retries: CI ? 2 from the shared configuration. A retry on a suite that mints real records into a shared tenant it cannot clean up turns one flake into three runs of orphans, and the inheritance was silent: it predated the project, and nobody had asked what CI was already doing to it. Scope a project’s settings to that project rather than riding a global switch, and read what the switch already does before inheriting it.

Why five of them were in one place

A Next.js backend-for-frontend that forwards an upstream body untouched — forwardAsNextResponse(upstream, data) — does not own the response envelope. The backend does.

The production code has always known this and carries defensive shape-lists to prove it. ManageCardsPanel.fetchCards accepts four shapes: data.records, data.data.records, a bare data array, and data.results. It falls back from result.payload to result for a flat record. getKanbanCardBare checks data.records[0] before data.

The end-to-end helper file was written as though the envelope were settled. Five of the seven witnesses are that single assumption, found five times.

The durable lesson is not “handle four shapes.”

When a boundary forwards a payload it does not own, the shapes it can deliver are part of its contract, and exactly one place in a codebase should know them. Spread that knowledge across readers and each new shape costs another round of the same review. Concentrate it and the next shape is a one-line change.

The fix that covers half the branches

Witness 5 is the one to keep, because of what its fix had to be.

recordIn normalized its flat branch and returned its two array branches raw, so a valid data.records entry carrying its fields flat reached statusOf, createItem and mintCard as an undefined payload. The obvious repair is to add the missing branch. That repair would have been wrong, because cardsIn held a second copy of the same payload ?? record unwrap, written one round earlier while fixing witness 2.

The answer was to make one function the only thing that knows a record’s shape. asRecord is now that function, and the property was confirmed by counting the remaining readers afterwards rather than asserting it: every other .payload in the file reads an already-normalized record.

The fix that covers half the branches is not a fix. It is the next instance. Witness 3 is the same story one round earlier — written while fixing witness 2, it reintroduced witness 1’s failure mode one function away. Both were caught in review rather than by a test, which is the point of the page: no test could catch them, because each returned a value the good case returns too.

Applying the rule

Ask three questions of any check that guards against a failure.

What fact does the answer rest on? Name it. If the answer is “it did not throw” or “nothing came back,” keep going.
Can the failure produce that fact? If yes, the check cannot fail. Find a different fact rather than a longer wait or a stricter matcher.
Who else knows this shape? If two places decode the same payload, a fix to one is half a fix. Count the readers.

Three rules follow, and they hold outside this codebase.

No timeouts as evidence. A timeout distinguishes slow from absent only when you have independently established that the thing cannot be slow.

Fail loudly on an unrecognized input. A reader that meets a shape it has no branch for throws. Returning empty converts a decoding defect into a plausible business answer, and plausible business answers do not get investigated.

Count the readers after a shape fix, do not assert the count. The claim “now only one place knows this” is cheap to make and cheap to check. Check it.

Sources

The practice was adopted on arda-frontend-app during the Orders v2 Phase-III work. The submission it was written from is /workbooks/notebooks/domain-ontology/streams/storefront/parmandil-submission-c4.md.

Witness	Commit
1 — list envelope	`111c8888` (#1305)
2 — record envelope	`712a4fa4` (#1305)
3 — unwrapper fails loudly	`ddf2eacc` (#1305)
4 — flat records	`310167fd` (#1305)
5 — array branches, `asRecord`	`10fd7376` (#1305)
6 — timeout as proof	`310167fd` (#1305)
7 — inherited retries	`10fd7376` (#1305)

The expect.poll note in End-to-End Tests Against a Real Deployment is a member of the same family, recorded there because it belongs with the suite mechanics.