04 · Computer Architecture

This chapter is the floor where physics becomes logic — the lowest floor this book visits, and the one every floor above it stands on.

How do you get arithmetic out of electricity?

A single Nvidia Blackwell B200 processor holds 208 billion transistors. A transistor does exactly one thing: it conducts, or it does not. It cannot add. It has no notion of a number, or of quantity. Yet somewhere between that switch and the arithmetic running underneath this sentence, meaning appears — and it appears in no component. This chapter is about where it actually is.

The answer is composition, and it is older than the computer. In the master's thesis he submitted to MIT in 1937, Claude Shannon showed that the two-valued algebra George Boole had built to formalize logic described, exactly, the behavior of circuits made of switches (MIT's copy). Two switches in series are AND; in parallel, OR. That correspondence is the hinge of the entire tower: a circuit can be designed as an argument rather than tinkered into working. After it, the climb is short and mechanical. Five gates — two XOR, two AND, one OR — make a full adder, a circuit that adds two bits plus a carry. Chain sixty-four of them and the machine adds sixty-four-bit integers. Nothing was introduced along the way except arrangement. Arithmetic is not in the electricity. It is in the shape.

The second idea arrived in a document nobody finished. In June 1945, Herman Goldstine circulated twenty-four copies of a 101-page draft by John von Neumann, the First Draft of a Report on the EDVAC (scan). Its central move — the stored program — was to keep a machine's instructions in the same memory as its data. This sounds like a filing decision. It is the reason the upper tower exists: if a program is just numbers in memory, then a program can read, write, and generate other programs — which is what every tool in chapter 06 does. It carried von Neumann's name alone, and caused sixty years of argument: J. Presper Eckert and John Mauchly held that the idea came out of group meetings at the Moore School, and the report's wide circulation later counted as prior art that helped invalidate the ENIAC patent in 1973. The lesson survives the quarrel: the machine's most consequential feature was a decision about what to not keep separate.

The third idea is the one that gives this floor its name. On 7 April 1964 IBM announced System/360, a family of machines of differing size, speed, and price built to run the same programs. Gene Amdahl, Gerrit Blaauw, and Frederick Brooks defined architecture as "the attributes of a system as seen by the programmer, i.e., the conceptual structure and functional behavior, as distinct from the organization of the data flow and controls, the logical design, and the physical implementation" (Amdahl, Blaauw & Brooks, 1964). That distinction is the floor's surface. An instruction set is a contract: the hardware promises what each instruction means, and promises nothing about how it will keep the promise. Behind an unchanged contract, engineers have replaced nearly everything — pipelining instructions so several are in flight at once, executing them out of order, guessing at branches and rolling back wrong guesses. The programmer's side of the contract never mentions any of it. This is why software outlives the hardware it was written for.

But the contract has fine print, and the fine print is where this floor leaks. In 1995 William Wulf and Sally McKee published a two-page note called "Hitting the Memory Wall: Implications of the Obvious". Their arithmetic used numbers everyone already had: memory was getting about 7 percent faster a year, processors about 80. Compounded, that gap projected an average memory access costing roughly 99 processor cycles by 2010 — a machine spending most of its life waiting. What has held the wall back is not a theorem but an empirical regularity: locality. Programs touch the same data again soon, and data near what they just touched. So architects built a hierarchy of small fast memories in front of large slow ones and bet on that habit — the most exploited empirical regularity in computing, and it usually wins. When it loses, two loops identical in logic — same operations, same results — can differ enormously in time, for reasons the instruction set does not mention and the source code does not show. Knowing that such a gap is a memory problem, three floors down, rather than an algorithm problem, is a large fraction of what practical expertise in computing means.

The last idea is that this floor stopped being free. Gordon Moore's 1965 magazine article observed that the component count on a chip was doubling every year, a rate he revised to every two years in 1975; the famous "eighteen months" appears in neither. What made shrinking transistors feel free was a second rule from Robert Dennard and colleagues in 1974: scale dimensions and voltage together and power density stays constant, so more transistors cost no more heat (Dennard et al., 1974). Around 2005 that stopped working — voltages could not keep falling without leakage — and the whole tower felt it, because for four decades every floor above had been getting faster while doing nothing. Architects responded by spending transistors differently: first on more cores, then on chips that do less but do it better. Google's first Tensor Processing Unit, described at ISCA in 2017, dropped nearly everything a general processor offers and reported, on its intended workload, 25 to 29 times the operations per watt of the contemporary Nvidia K80 GPU and 41 to 83 times a contemporary Intel CPU — the spread depending on how power is accounted (Jouppi et al., 2017). John Hennessy and David Patterson, who received the 2017 Turing Award for the quantitative approach that made such comparisons possible, called the result a new golden age for computer architecture. Note what changed: speed used to arrive from below as a gift; now it is negotiated.

The people are unusually entangled with the ideas. Amdahl architected System/360 and then left IBM in 1970 to found a company building machines plug-compatible with the very contract he had drafted — the architecture outlived its author's employment, which is the strongest proof it was a real interface. Dennard invented the one-transistor memory cell in 1966 (patent granted June 1968) and then wrote the scaling rule that let processors outrun it: one man is responsible both for the memory that is too slow and for the reason it matters.

Connections

Below this chapter, in this book, there is nothing — 01 through 03 are about what can be computed and what information is, questions that hold for any machine or none. Above it, 05 takes this single machine and makes it behave like many, using the protection and timer hardware this floor provides; 06 turns the instruction-set contract into something a person can write in. Chapter 02's cost model quietly assumes memory that answers in constant time, which this floor does not supply; read the two against each other and the disagreement is instructive. Chapter 12 is where specialization now points, and chapter 14 asks what the energy wall does to all of it.

Open questions

Is a fixed instruction set still worth its cost, when the workloads that matter change faster than chips can be designed? If memory is the bottleneck, should computation move into memory instead of data moving to the processor? Speculative execution bought decades of speed and then turned out to leak secrets across the security boundary chapter 11 depends on; can a machine be fast and honest about what it is doing? And after specialization, what is left for a general-purpose computer to be?

Go deeper

• John L. Hennessy & David A. Patterson, Computer Architecture: A Quantitative Approach, 7th ed. (with Christos Kozyrakis), Morgan Kaufmann, 2025 — the book that made architecture an experimental science. Publisher.
• Gene Amdahl, Gerrit Blaauw & Frederick Brooks, "Architecture of the IBM System/360," IBM Journal of Research and Development 8, no. 2 (1964): 87–101 — the paper that defined the word.
• William A. Wulf & Sally A. McKee, "Hitting the Memory Wall: Implications of the Obvious," ACM SIGARCH Computer Architecture News 23, no. 1 (1995): 20–24 — two pages, thirty years of consequences.