Beyond the basic pipeline

An ALU result exists at the end of EX, but a simple pipeline waits until WB to write it to the register file, two cycles later. Forwarding (also called bypassing) adds multiplexers in front of the ALU that can take an operand straight from the EX/MEM or MEM/WB pipeline register instead of the register file. With it, back-to-back dependent ALU instructions run with no stalls at all.

A forwarding unit compares register numbers carried in the pipeline registers. For the first ALU operand, in the textbook five-stage RISC-V design:

Forwarding cannot send a value backward in time. A load's data arrives at the end of MEM, but an instruction right behind it needs that value at the start of EX in the same cycle. This load-use hazard needs one stall. A hazard detection unit in ID spots it:

Holding PC and IF/ID makes the two younger instructions repeat their stages; clearing the control signals turns the slot behind the load into a bubble that writes nothing. One cycle later the forwarding unit supplies the loaded value from MEM/WB.

Hazard detection predates RISC. The CDC 6600 (1964) used a scoreboard to track which functional unit would write each register and to hold instructions until their operands were ready. Tomasulo's algorithm for the IBM 360/91 went further, letting later independent instructions proceed past a stalled one, the root of today's out-of-order processors.

The "Unscheduled" example computes a = b + e; c = b + f; in source order and hits two load-use stalls: 13 cycles even with forwarding. In "Scheduled", the compiler moves the third load up so that an independent instruction sits between each load and its use: 11 cycles, no stalls, same result. The hardware still checks for hazards; the compiler simply arranges the code so the checks rarely fire.

A branch history table, indexed by low bits of the branch address, holds a 2-bit saturating counter for each entry. Two wrong guesses in a row are needed to change the prediction, so a loop branch that is taken many times and falls through once costs one misprediction per loop rather than two. A branch target buffer supplies the target address during fetch, so a predicted-taken branch costs nothing when the prediction is right.

A superpipelined processor splits the slowest stages, usually instruction fetch and data memory access, into several shorter ones so the clock can run faster. The MIPS R4000 (1991) is the classic example: its eight stages split both cache accesses in two.

The price is longer hazard penalties measured in cycles. On the R4000 a load's value arrives two cycles later than in the five-stage pipeline, so a load-use hazard costs 2 stalls instead of 1, and a branch costs 3 cycles instead of 2. Intel's Pentium 4 pushed further, to 20 stages and later 31, before its successors cut the pipeline back as mispredictions and clock power grew.

Studies of optimal depth (Kunkel and Smith; Hartstein and Puzak; Sprangle and Carmean; Hrishikesh and colleagues) agree on the shape of the trade-off. Performance rises as stages are added, peaks, then falls as register overhead and hazard penalties dominate. Where the peak lies depends on the workload, the predictor and the circuit overhead.

A very long instruction word processor issues one wide instruction per cycle, often called a bundle. Each bundle holds several independent operations, one slot per functional unit. The compiler, not the hardware, decides which operations can run together and fills unused slots with no-ops. With no dependence checking between slots and no dynamic scheduling, the issue logic stays simple even at high width.

VLIW came out of Fisher's trace-scheduling research and machines such as the Multiflow TRACE in the 1980s. Intel and HP's Itanium (EPIC) added predication, speculation support and bundle templates. Itanium was discontinued, but VLIW thrives where code is regular and power is tight: digital signal processors such as TI's C6000 family and Qualcomm's Hexagon, and many AI accelerators.

VLIW's weaknesses are the flip side of its strength. Schedules are fixed at compile time, so an unexpected cache miss stalls the whole bundle. Binaries encode the slot layout, so a wider implementation needs recompiled code. Empty slots waste instruction memory unless the encoding compresses no-ops, which most modern VLIWs do.

Every technique on this page attacks one term of this product, and many make another term worse. Forwarding shrinks data stalls but adds multiplexers to the critical path. Resolving branches early cuts the control penalty but may lengthen the cycle. Superpipelining shortens the cycle but multiplies stall penalties. The only fair comparison is the product, on a realistic workload.

Things to try. With the defaults, the 8-stage design is fastest: its shorter cycle more than pays for its longer penalties, but at 10 stages the penalties win. Turn forwarding off or switch to "Stall" and the 5-stage design wins instead, because deeper pipelines magnify every unhandled hazard. Raise the register overhead and the advantage of depth shrinks.

Jouppi and Wall compared superpipelined and superscalar machines on the same code in 1989 and found that, for the same degree of parallelism, the two delivered roughly the same performance, because both were limited by the parallelism in the programs. Modern processors combine all of these techniques: a moderately deep pipeline, forwarding everywhere, accurate prediction, and several instructions issued per cycle.

Speed is only one axis. Power, area, design and verification effort, code size, and the ability to run existing binaries all weigh in, and they usually decide which strategy a product uses.