Skip to main content

0022: Require Consumer-Complete Verification

Status: Accepted

Date: 2026-07-21

Context​

ADR 0021 established consumer-complete visual review after an in-engine Almanac omitted the production HUD it was meant to evaluate. The same failure shape appears outside visual work.

The Circussy One repeatedly produced locally convincing but systemically incomplete evidence:

  • mouse interaction worked while controller focus and submit paths did not;
  • audio files and Wwise events previewed while generated media, runtime event posting, emitter lifetime, or menu-to-gameplay handoff still failed;
  • a loading overlay existed and async stages completed while main-thread work still froze the visible frame loop;
  • imported prefabs and materials were valid while runtime factories continued to compose placeholders; and
  • isolated lifecycle behavior passed while the VContainer-composed path lacked a registration or reset participant.

These are not arguments for replacing focused tests with one enormous end-to-end suite. They show that evidence answers only claims whose material consumers and boundaries it actually exercises. A pure rule may be completely verified at a unit boundary. A claim about controller navigation, middleware playback, frame continuity, runtime composition, or player-visible feedback is not.

Decision​

  • For every material claim, identify its production consumer and intended environment before selecting evidence. A production consumer may be a code caller, composed subsystem, engine loop, input route, middleware backend, authoring tool, platform, or player.
  • Require consumer-complete verification: use the smallest truthful route that includes the production consumer and every integration or environmental dependency material to the claim.
  • Treat consumer completeness as relative to the claim, not as a synonym for a full-game or end-to-end test. Narrow claims may have narrow complete routes.
  • Keep unit tests, pure rules tests, static checks, fakes, isolated previews, diagnostics, and component tests. They provide fast guardrails and precise fault localization. Do not inflate them into claims about omitted consumers.
  • When a material production consumer cannot be exercised, narrow the reported claim and mark the broader result inconclusive, unavailable, or not attempted. Do not manufacture a substitute harness merely to obtain a passing result.
  • Separate mechanical verification from product validation. Real users or the Director still judge intended use, visual quality, control feel, audio mix, pacing, accessibility, and fun when those properties are material.
  • Apply Earned Infrastructure. Reuse an existing production route first; retain a fixture, driver, scenario, or operation only when recurring setup or risk gives it a demonstrated job.

Consequences​

  • Evidence selection starts with the claim and its consumer, not with the easiest available command.
  • A direct method call cannot prove an input mapping and focus path; a Wwise authoring preview cannot prove Unity runtime playback; task completion cannot prove loading-frame continuity; and asset import cannot prove runtime prefab composition.
  • Full composed tests remain selective because they are slower and harder to diagnose. Focused evidence remains the default support layer around the smallest consumer-complete route.
  • Visual Review remains a specialized application of this rule. ADR 0021 continues to define its visual surface-selection consequences.
  • STAGE gains no required artifact, schema, test pyramid, or custom harness. This is a claim boundary used by planning, rehearsal, and reporting.

Research Boundary​

This decision is consistent with established verification and validation principles, but STAGE's exact formulation remains a practitioner synthesis. Saltzer, Reed, and Clark's end-to-end argument explains why some functions can be established completely only with application-end knowledge, while NASA defines validation around intended use in the intended environment. Google SRE guidance retains layered unit, integration, and system evidence rather than promoting one test scale universally. STAGE applies those ideas to evidence routing; it does not claim that the end-to-end argument is itself a software testing prescription.

Last updated on