From the ISA to the energy bill

A computing system is a stack of contracts. Each layer hides the one below it, and architecture lives at the boundary where software stops and hardware begins. The instruction set architecture (ISA) is that boundary: the registers, instructions, data types, addressing modes and memory model a program can rely on. The microarchitecture is one particular way of implementing it. x86-64 has been implemented by dozens of very different microarchitectures; the same binary runs on all of them.

The classic organization is the von Neumann model: a single memory holds both instructions and data, and a processor fetches, decodes and executes one instruction at a time. Its weakness, the shared path between processor and memory, is still called the von Neumann bottleneck. A Harvard organization separates instruction and data paths; modern cores are effectively a hybrid, with split L1 instruction and data caches backed by a unified memory.

ISAs differ in philosophy. CISC designs (x86) offer variable-length instructions that can operate on memory directly. RISC designs (Arm, RISC-V, MIPS) use fixed-length, load/store instructions that are simple to decode and pipeline. Modern x86 cores decode CISC instructions into RISC-like micro-operations internally, so the distinction now matters most for decoder complexity, code density and licensing, rather than raw speed.

The datapath is the hardware that moves and transforms data (register file, ALU, memory ports); the control unit tells it what to do each cycle. Pipelining splits instruction execution into stages so several instructions are in flight at once, like an assembly line. The textbook five-stage RISC pipeline is Fetch, Decode, Execute, Memory access and Write-back.

Pipelines are limited by hazards. Structural hazards occur when two instructions need the same hardware. Data hazards occur when an instruction needs a result that is not ready yet; forwarding (bypassing) solves most of them. Control hazards occur at branches, because the next instruction to fetch is unknown until the branch resolves. Deeper pipelines allow faster clocks but make every misprediction and stall more expensive, which is one reason clock frequencies stopped climbing in the mid-2000s.

Processors got faster much more quickly than DRAM, a gap known as the memory wall. Caches close the gap by exploiting temporal locality (recently used data is reused) and spatial locality (nearby data is used soon). Each level is larger, slower and cheaper per byte than the one above it.

Cache design comes down to a few parameters: capacity, block (line) size, associativity, replacement policy, and write policy (write-through or write-back). Misses fall into the "three Cs": compulsory, capacity and conflict, with a fourth, coherence, on multicore chips. Virtual memory adds a translation layer: page tables map virtual to physical addresses, and the TLB caches those translations so most accesses skip the page-table walk.

A single core finds parallelism inside one instruction stream. Superscalar cores issue several instructions per cycle. Out-of-order execution lets an instruction run as soon as its operands are ready, using register renaming to remove false dependences and a reorder buffer to retire results in program order. The core idea traces back to Tomasulo's algorithm for the IBM System/360 Model 91.

Branch prediction and speculative execution keep the pipeline full by guessing the outcome of branches and running ahead. Modern predictors are right well over 95% of the time on typical code. Speculation has a cost beyond wasted work: Spectre and Meltdown showed that squashed instructions can still leave measurable traces in caches, turning a performance feature into a security vulnerability.

Multicore chips share memory, so they need cache coherence (snooping protocols such as MESI, or directories at larger scale) and a defined memory consistency model that says which orderings of loads and stores other cores may observe. x86 provides fairly strong ordering (TSO); Arm and RISC-V are weaker and rely on explicit fences.

GPUs trade single-thread speed for massive throughput: thousands of lightweight threads hide memory latency by switching among themselves. Domain-specific accelerators such as Google's TPU go further, building systolic arrays of multiply-accumulate units with low-precision arithmetic and software-managed memory. Heterogeneous systems now combine big and little CPU cores, GPUs, neural engines and media blocks on one die or across chiplets connected in one package.

Compatibility is sticky. The System/360 decision to fix an ISA and vary the implementation is why x86 and Arm binaries outlive the chips they were compiled for, and why new ISAs succeed mainly in new markets.

RISC won inside CISC. The RISC argument for simple, pipelinable operations was absorbed by x86 through micro-op translation. Today's debate is less RISC versus CISC and more open versus licensed ISAs.

The free lunch ended. With Dennard scaling gone, performance has to come from parallelism and specialization rather than clock speed. That shift explains multicore chips, GPUs in data centers, and dark silicon: parts of a chip that cannot all be powered at once.

Specialization is the new scaling. Hennessy and Patterson call this a new golden age: domain-specific architectures, domain-specific languages, open hardware and agile chip design, all driven by the end of general-purpose scaling.

Data movement dominates. Moving data now costs far more time and energy than computing on it, pushing designs toward larger caches, stacked high-bandwidth memory, chiplets and processing near or in memory.

Each term belongs to different layers of the stack. The algorithm, compiler and ISA set the instruction count. The microarchitecture and memory system set cycles per instruction (CPI). The circuit technology and pipeline depth set the clock. Improving one term often worsens another, which is why a single metric like GHz or MIPS misleads.

Worked example. A program executes 2.0×10⁹ instructions at CPI 1.25 on a 3 GHz core: 2.0×10⁹ × 1.25 / 3×10⁹ = 0.833 s. A compiler change cuts instructions by 10% but raises CPI to 1.40: 1.8×10⁹ × 1.40 / 3×10⁹ = 0.840 s. Fewer instructions, slower program.

Amdahl's law bounds the speedup from improving a fraction p of the work by a factor n. The serial part sets a hard ceiling of 1/(1 − p) no matter how many cores you add. Gustafson's law gives the complementary view: as machines grow, people solve bigger problems, so the parallel fraction grows with them. Hill and Marty applied Amdahl's law to multicore chip design and showed why asymmetric big-plus-little cores can beat both extremes.

Arithmetic intensity is floating-point operations per byte moved from memory. Kernels left of the ridge point are memory-bound and gain nothing from more compute; kernels to the right are compute-bound. The model tells you which optimization to try: improve data reuse (blocking, fusion) to move right, or vectorize and parallelize to climb toward the roof.

Dynamic power scales with activity factor, switched capacitance, the square of supply voltage, and frequency. Lowering voltage and frequency together (DVFS) cuts power roughly with the cube of frequency, which is why many slower cores can beat one fast core on throughput per watt. Static (leakage) power flows even when transistors are idle and is what ended Dennard scaling.

Use execution time of real workloads as the ground truth. Standard suites such as SPEC CPU, MLPerf and HPL make results comparable; summarize normalized ratios with the geometric mean. Report efficiency alongside speed (performance per watt, per dollar, per mm²), and for services, report tail latency (p99) as well as averages. Be wary of peak numbers, MIPS, clock rate alone, and benchmarks tuned for one machine.