Capstone: System Design Methodology & QA

Phase 10 added security and privacy to the growing pile of nonfunctional concerns a real embedded system has to satisfy alongside timing, power, and cost. This closing phase steps back from any single technical concern and returns, deliberately, to where the whole series began: Phase 0 introduced the requirements → specification → architecture → components → integration design flow in the abstract, before any of the modeling, hardware, software, concurrency, or verification machinery that filled the rest of this series existed. This capstone chapter revisits that same flow with real methodologies, real specification techniques, and two genuinely devastating real-world failure case studies showing exactly what goes wrong when a stage gets shortchanged.

Why Have an Explicit Design Process at All?

Everyone already has a design process, whether they can articulate it or not; the case for making it explicit only starts to bite once more than one person is involved, since a team needs shared, communicable agreement on who does what and when. Wolf names time-to-market, design cost, and quality as the three goals a methodology serves on top of raw function, performance, and power, which is worth remembering: “the system works” is necessary but not the whole scorecard a real product gets judged against.

Four design-flow shapes recur, each suited to a different situation. The waterfall model (requirements flow one-way into architecture, then coding, then testing, then maintenance) is clean to describe, but Wolf is explicit that nobody treats it as a realistic literal process anymore, since real projects need bottom-up feedback that a strictly one-way model can’t account for. It’s worth knowing because it’s the reference point everything else reacts against, not because it’s good practice to follow.

The spiral model fixes waterfall’s rigidity by building several increasingly complete versions, cycling through requirements, construction, and test at each turn, with the spiral widening as more is learned; it’s more realistic, at the cost of risking excessive iteration when time is tight. Successive refinement has a similar spirit but a different purpose: it’s specifically for when you’re unfamiliar with the application domain, so you build a rough prototype first and refine repeatedly, and some iterations may deliberately stop partway (through detailed design only, say) purely to learn something before committing to a full build. And a hierarchical, concurrent hardware-plus-software flow treats the front end (requirements, spec, architecture) and back end (integration, system test) as joint hardware/software concerns, while letting the middle stages proceed independently; for genuinely complex systems this whole flow recurses, with each “implementation” box itself a complete flow, different teams handling different abstraction levels and relying on each other’s specs and results.

The explicit countermeasure to “over-the-wall” design, where one person finishes a task and tosses the result to the next person with minimal interaction, is concurrent engineering: cross-functional teams doing subsystems in parallel rather than sequentially, sharing information incrementally rather than only at handoffs, with one person owning the whole project’s success rather than just their own stage. The real AT&T PBX case study is worth remembering as concrete proof this isn’t just theory:

$$\text{AT\&T PBX design time: } 18\text{–}30 \text{ months} \;\longrightarrow\; 11 \text{ months (via concurrent engineering restructuring)}$$

roughly a 40 to 60 percent reduction, matching the team’s own 40 percent target set from competitive benchmarking. Product development time dropped from 18 to 30 months down to 11 months by restructuring explicitly around concurrent-engineering principles, and tellingly, the team found their biggest bottleneck (too much sequential handoff, unclear task prioritization at the handoff points) was a managerial problem, not a technical one.

The Mars Climate Observer is the chapter’s proof that getting the requirements stage wrong can be catastrophic even when nothing downstream goes wrong at all. Lockheed Martin’s engineers returned trajectory values in pound-force; JPL’s flight software expected Newtons; the resulting 4.45x error was caught by neither configuration management nor manual review, and the spacecraft was lost approaching Mars. The lesson generalizes precisely: this wasn’t a coding bug or an architecture flaw, it was a units mismatch in a requirements-level communication gap between two organizations, the cheapest possible class of bug to prevent (an explicit units field in the spec) causing the most expensive possible outcome (total mission loss). It’s a direct, real-world instance of the “informal specifications are dangerously ambiguous” argument this series has made before, now playing out at organizational scale rather than within a single program.

Requirements Analysis

Phase 0’s model-train chapter already drew the line between requirements and specification; this chapter restates it with a formal quality checklist attached. Requirements are the informal description of what the customer wants; the specification is the detailed, precise, consistent description you can actually build an architecture from. Both describe outward behavior, not internal structure, and that behavior-versus-structure distinction is exactly what separates requirements and spec from architecture proper.

Functional and nonfunctional requirements are also familiar territory from Phase 0, but Wolf attaches a genuinely useful seven-point quality checklist worth applying to any requirements document you write. Correctness means matching what the customer actually wants, including not over-requiring unnecessary conditions. Unambiguousness means exactly one plain-language interpretation is possible. Completeness speaks for itself. Verifiability means a cost-effective way exists to check the requirement at all: “attractive” fails this test without an agreed definition of attractive. Consistency means no internal contradictions. Modifiability means a requirement can change without breaking the rest of the document. And traceability means the link runs both ways, back to why a requirement exists and forward to how it’s satisfied in the implementation; that bidirectional link is what makes audits and later impact analysis possible at all.

Direct customer contact is called out as valuable specifically because it’s unfiltered: hearing what customers want firsthand beats reading marketing’s summary of it, a channel that loses information at every step, and direct contact also tends to build the kind of empathy that produces cleaner interfaces.

Specification Techniques

Wolf’s specification toolkit revisits several notations this series has already seen, with one genuinely new idea buried inside it. SDL, an event-oriented state-machine language built for telecom protocols, is presented mainly as one more concrete syntax in the same family as the UML state machines from Phase 0, worth knowing as existing rather than deeply new.

Statecharts’ OR and AND states are the real addition, and they’re worth understanding structurally. An OR state groups several states that share a common outgoing transition (from any of s1, s2, or s3, input i2 goes to s4, say) into one drawn transition instead of several duplicated ones; it’s pure notational compression, with the same underlying semantics as drawing every transition separately. An AND state is the more powerful idea: it represents the machine being in two states simultaneously, component sa in state s1 and component sb in state s3 at the same time. That is literally the synchronous side-by-side composition from Phase 1, just drawn as nested boxes instead of separate composed machines; recognizing that Statecharts’ AND-states and Phase 1’s concurrent composition are the same underlying idea in two different notations is the single most valuable connection in this section. AND/OR tables, a third notation from Leveson and colleagues, express the same underlying boolean and state logic with don’t-cares made explicit as table dashes, reinforcing that no single notation here is uniquely correct, only more or less convenient for a given audience or analysis need.

TCAS II, the real aircraft collision-avoidance system specified via Leveson and colleagues’ RSML notation, is the chapter’s proof that this machinery scales to genuinely safety-critical, real-world complexity: explicit input and output lists per state, AND-decomposition into parallel concurrent subsystems (tracking your own aircraft plus up to 30 other aircraft plus up to 15 ground stations, all simultaneously active), and deliberately named inhibit-states, such as Climb-Inhibit and Descend-Inhibit, that encode hard-won operational safety knowledge directly into the state machine’s structure rather than leaving it as a comment or an external rule.

Architecture Design: CRC Cards

CRC, for Classes, Responsibilities, and Collaborators, is a deliberately low-tech, index-card-based method for turning a specification into an initial object-oriented architecture. Its real value, Wolf states explicitly, isn’t technical sophistication but social accessibility: domain experts who aren’t computer specialists (mechanics, human-factors people, automobile designers) can meaningfully participate in a CRC card session in a way they can’t in a UML modeling session, a genuinely underrated point about matching a technique’s formality to who actually needs to contribute to it.

The walkthrough is the actual mechanism that finds problems, not the initial card-writing. Team members physically act out usage scenarios, each person “being” one or more classes, calling out what their class does and which collaborators it invokes; this is where holes in the specification and missing or wrong responsibilities actually surface. The elevator-system worked example, reusing the elevator design from Phase 8, is a good template for the method: separate real-world classes (the elevator car, the passenger, floor control) from architectural classes invented purely to support the design (a scheduler, car state, control readers), then define concrete usage scenarios, including edge cases like simultaneous requests from two floors, to walk through together.

Quality Assurance

If one incident deserves to be the single most important cautionary example in this entire series, it’s the Therac-25. This radiation-therapy machine delivered fatal overdoses because of a race condition between two concurrently running tasks, Datent and Hand, sharing an unprotected variable (the mode and energy offset) with no synchronization mechanism at all: non-atomic test-and-set, no mutual exclusion. The failure was timing-dependent on the operator’s own typing speed, triggered only if the operator edited mode or energy data within roughly an eight-second window while a subroutine was mid-execution, meaning the bug got more likely as operators became faster and more experienced, the exact opposite of what you’d naively expect. This is Phase 6‘s race-condition and mutual-exclusion material, Phase 9‘s “informal specs hide real bugs” argument, and Phase 0’s “specification teaches you about the system” lesson, all converging in one real, fatal incident. Leveson and Turner’s root-cause list is worth keeping close to verbatim: limited safety analysis, with probabilities assigned to failure modes with no real justification; removed mechanical safety backups that earlier versions of the machine had included; and reliance on system-level testing with insufficient module-level testing or formal analysis, which is exactly the clear-box and formal-verification discipline from Phase 5 and Phase 9 that this incident shows are not optional extras for safety-critical embedded systems.

The cost-of-bugs-over-time curve is the chapter’s central quantitative argument for why any of this discipline is worth the effort in the first place: a bug’s fix cost grows the longer it survives undetected through the design flow, so a requirements-level bug caught only at maintenance can force a full redesign rather than a simple component fix.

That curve is the same underlying economic argument, generalized, behind why the formal-verification machinery from Phase 9 exists at all: prove correctness before building rather than relying on testing alone. Wolf names several concrete requirements and spec validation techniques that catch problems while they’re still cheap. Prototypes let the customer see, touch, or hear something concrete instead of parsing abstract technical language, especially valuable for UI work and less so for nonfunctional properties like true execution speed. Preexisting systems make critique easier, since it’s simpler for a customer to say what they like or dislike about the old machine than to describe a new one from scratch in the abstract. Simulation is called out as especially important for cyber-physical systems, since requirements partly depend on the physical plant’s own dynamics, a direct callback to Phase 1‘s continuous-dynamics and actor-model machinery being a prerequisite for writing correct CPS requirements, not just an implementation detail. And formal techniques, mathematical proof carried out manually or with tool support, are named as most valuable for systems whose spec is compact but whose behavior over time is complex, exactly the model-checking sweet spot from Phase 9.

Design reviews round out the toolkit as a structured meeting: designers present, a review leader runs it, a scribe records issues, and an audience, ideally including people from other projects for an outside perspective, hunts for problems. They catch bugs two ways at once, through the audience’s scrutiny and simply through the discipline of the presenter having to explain the design clearly enough to prepare for the meeting. They’re cheap, low-tech, and, per Wolf, very effective in practice, worth remembering alongside CRC cards as evidence that some of the highest-leverage QA techniques in this whole book are organizational and social practices, not tools or formalisms. At the organizational level, ISO 9000 and the CMM’s five maturity levels (Initial, Repeatable, Defined, Managed, Optimizing) measure an organization’s process quality rather than any single product’s, worth knowing primarily as vocabulary for describing where an organization sits on the spectrum from haphazard to continuously self-improving, since Wolf notes that truly Optimizing-level organizations are rare in practice.

Where the Whole Series Leaves Off

This chapter is deliberately a full-circle chapter rather than a new topic, and so is a good vantage point for looking back at the whole series from Phase 0 to here. Phase 0 opened with the requirements → specification → architecture → components → integration design flow described only in the abstract, alongside Lee and Seshia’s modeling, design, and analysis framing of the same loop. Everything in between was, in a real sense, filling in what each of those abstract stages actually requires in practice: Phase 1 gave the flow a rigorous behavioral vocabulary (continuous dynamics, state machines, concurrent composition) to specify against; Phase 2 and Phase 3 grounded architecture and components in real processors, memory, and I/O; Phase 4 extended that down to the buses and physical-world interfaces a platform actually needs; Phase 5 and Phase 6 worked through how software gets built and scheduled on top of that hardware; Phase 7 and Phase 8 scaled the same questions up to multiple cores and multiple networked devices; and Phase 9 and Phase 10 asked how you know the resulting system is correct and how you keep it that way once it’s exposed to an adversary.

This closing chapter revisits that entire arc through real named methodologies (waterfall, spiral, successive refinement, concurrent engineering) and a real organizational case study proving the payoff, and it revisits the specification techniques from Phase 0 by explicitly connecting Statecharts’ AND-states to Phase 1’s synchronous composition, the same underlying idea recognized twice, years apart in the reading, in two different notations. Quality assurance then revisits testing, race conditions and mutual exclusion, and formal verification all at once through the Therac-25 case study, arguably the single most important real-world story in the entire series, since it shows a known bug class covered many phases earlier (unsynchronized shared-variable access) causing real deaths, not just a hypothetical exercise. The Mars Climate Observer and Therac-25 stories make the same point from two different angles: Mars Climate Observer is a requirements-level failure, ambiguous units, with no code bug at all; Therac-25 is a code-level failure, a race condition, that better specification, module testing, or formal analysis could plausibly have caught. Between them, they make the best possible case for why this whole study plan, from Phase 0’s emphasis on precise specification through Phase 9’s formal-verification machinery, was never academic overhead. It is the actual, demonstrated difference between a working system and a fatal one, and it is a genuinely open question, worth carrying forward past these notes, whether the same root causes (ambiguous requirements, unsynchronized shared state, insufficient verification) still dominate today’s embedded-system failures, or whether the field has actually moved the needle since 1999 and 1987.