How Microprocessors Execute Pipelined Instructions

Imagine a commercial laundry service with four simple tasks: washing a load of clothes (30 minutes), drying it (40 minutes), folding it (20 minutes), and ironing it (30 minutes). Total time per load: 120 minutes (2 hours).

Non-Pipelined Sequential Laundry (Single-Cycle):
Load 1: [ Wash (30m) ] -> [ Dry (40m) ] -> [ Fold (20m) ] -> [ Iron (30m) ] === Done at 2:00
Load 2:                                                                      [ Wash (30m) ] -> ...
(While Load 1 is drying, the washer sits empty and idle for 90 minutes!)

If you do laundry sequentially, you wait until Load 1 is completely washed, dried, folded, and ironed before placing Load 2 into the washer. Processing four loads of laundry will take 8 hours.

While Load 1 is in the dryer, the washing machine sits empty and useless. While Load 1 is being ironed, the dryer and washer sit completely dark.

Now consider how any sensible human actually does laundry: As soon as Load 1 finishes washing and moves into the dryer, you immediately toss Load 2 into the washer. When Load 1 moves to the folding table, Load 2 moves into the dryer, and Load 3 enters the washer.

Pipelined Overlapped Laundry:
Time:     0m       30m      70m      90m      120m     150m     190m
Washer:   [Load 1] [Load 2] [Load 3] [Load 4]
Dryer:             [Load 1] [Load 2] [Load 3] [Load 4]
Folder:                     [Load 1] [Load 2] [Load 3] [Load 4]
Ironer:                              [Load 1] [Load 2] [Load 3] [Load 4]
                                              ▲        ▲        ▲        ▲
                                            Done!    Done!    Done!    Done!
(After the initial pipeline fill, a completed load pops out every 40 minutes!)

By overlapping the stages, all four machines run simultaneously. The time to wash a single load has not changed (it still takes 120 minutes from dirty hamper to clean shelf), but the throughput of your laundry operation has skyrocketed. Instead of finishing a load every 2 hours, a freshly cleaned, folded load pops off the line every 40 minutes!

This simple industrial assembly line insight is the foundation of modern computing: Instruction Pipelining.

---

1. The Single-Cycle Bottleneck

In early microprocessors, instructions were executed like sequential laundry.

To execute an instruction, digital hardware must perform several distinct electronic operations:

1. Instruction Fetch (IF): Read the 32-bit binary instruction word from memory.
2. Instruction Decode (ID): Read the opcode, inspect the register addresses, and fetch the operand values from the register file.
3. Execute (EX): Pass the operands through the Arithmetic Logic Unit (ALU) to perform an addition, subtraction, logical bitwise operation, or address calculation.
4. Memory Access (MEM): If the instruction is a LOAD or STORE, read from or write to the data cache.
5. Writeback (WB): Write the final computational result back into the register file.

Hardware Delay Profile of a MIPS Instruction (Picoseconds):
+----------------+----------------+----------------+----------------+----------------+
|   Fetch (IF)   |   Decode (ID)  |   ALU (EX)     |   Memory (MEM) | Writeback (WB) |
|     200 ps     |     100 ps     |     200 ps     |     200 ps     |     100 ps     |
+----------------+----------------+----------------+----------------+----------------+
Total Critical Path Delay for a LOAD Instruction: 200 + 100 + 200 + 200 + 100 = 800 ps

In a non-pipelined single-cycle processor, every single instruction must execute from start to finish within a single tick of the CPU clock.

Because the clock period must accommodate the slowest possible instruction (the critical path), the clock cycle time is dictated by the memory LOAD instruction:

Even if an instruction is a fast integer addition (ADD R1, R2, R3) that requires zero memory access and could physically complete in 500 picoseconds, it must sit idle and wait for the entire 800 picosecond clock period to expire!

Worse, hardware utilization is abysmal: while the ALU is calculating an addition, the memory buses, instruction caches, and register writeback circuits sit 100% idle.

---

2. The Pipelined Solution: Interstage Registers and Ideal Speedup

Pipelining breaks the monolithic execution datapath into independent, synchronized stages separated by clocked storage elements called pipeline registers (or interstage latches).

Instead of waiting for an instruction to travel through all 800 picoseconds of silicon before fetching the next one, the CPU fetches a brand new instruction on every single clock tick!

The Classic 5-Stage Pipelined Datapath:
Clock Cycle:      1       2       3       4       5       6       7
Instruction 1:  [ IF ]  [ ID ]  [ EX ]  [ MEM]  [ WB ]
Instruction 2:          [ IF ]  [ ID ]  [ EX ]  [ MEM]  [ WB ]
Instruction 3:                  [ IF ]  [ ID ]  [ EX ]  [ MEM]  [ WB ]
Instruction 4:                          [ IF ]  [ ID ]  [ EX ]  [ MEM]  [ WB ]
Instruction 5:                                  [ IF ]  [ ID ]  [ EX ]  [ MEM]  [ WB ]
                                                  ▲       ▲       ▲       ▲
                                              1 Inst  1 Inst  1 Inst  1 Inst
                                              Retires Retires Retires Retires

Notice what happens once the pipeline is full (at Clock Cycle 5):

• Instruction 1 is writing its result back to registers (WB).
• Instruction 2 is reading data from memory (MEM).
• Instruction 3 is crunching numbers in the ALU (EX).
• Instruction 4 is reading operands from the register file (ID).
• Instruction 5 is being fetched from the instruction cache (IF).

Five different instructions are in flight simultaneously!

The Mathematical Speedup

What is the clock cycle time of this pipelined processor?

The clock period is no longer the sum of all stages; it is the delay of the single slowest stage, plus the small overhead of the interstage latch flip-flops (t_latch ≈ 20 ps):

Compare the two processors:

• In the single-cycle processor, one instruction finished every 800 ps.
• In the pipelined processor, after an initial 5-cycle startup delay (the pipeline latency), one instruction completes every 220 ps!

In an ideal pipeline with k balanced stages, the theoretical speedup approaches k, and the metric Cycles Per Instruction (CPI) approaches 1.0:

---

---

3. The Classic 5-Stage RISC Datapath

To see how a CPU executes an instruction in a pipeline, examine the classic 5-stage architecture developed by John L. Hennessy and David A. Patterson (the MIPS and RISC-V foundational model).

Between each stage sits a synchronous pipeline register named after the stages it separates: IF/ID, ID/EX, EX/MEM, and MEM/WB. On every rising edge of the CPU clock, these registers capture and latch the state of the datapath, isolating the stages from one another.

The Physical Datapath and Pipeline Latches:
[ PC ] ---> [ Inst Cache ] ===( IF/ID )===> [ Register File ] ===( ID/EX )===> [    ALU    ]
                                                   |                                 |
                                                   v                                 v
                                              Immediate Sign Ext                ( EX/MEM )
                                                                                     |
                                                                                     v
                                              [ Data Cache ] <=======================+
                                                     |
                                                     v
                                                 ( MEM/WB ) ===> [ Mux to Reg File Write ]

Stage 1: Instruction Fetch (IF)

• The Program Counter (PC) holds the 64-bit or 32-bit memory address of the next instruction.
• The address is sent to the Level 1 Instruction Cache (L1 I-cache).
• While the cache reads the 32-bit instruction word, an adder computes PC + 4 (the address of the sequential next instruction).
• At the clock edge, the fetched instruction word and the PC + 4 value are latched into the IF/ID register.

Stage 2: Instruction Decode & Register Fetch (ID)

• The instruction word is cracked open into its constituent bitfields:
  • Opcode and function bits (determining what operation to perform).
  • Source register specifiers (rs1 and rs2).
  • Destination register specifier (rd).
• The register numbers are routed to the Dual-Port Register File, which reads out the 64-bit data values stored in rs1 and rs2.
• If the instruction contains an immediate constant (like an offset in LW R1, 100(R2)), the Sign-Extension Unit expands the 12-bit or 16-bit constant to a full 64-bit word.
• At the clock edge, the operand values, sign-extended immediate, register numbers, and control signals are latched into the ID/EX register.

Stage 3: Execution / Effective Address Calculation (EX)

• The Arithmetic Logic Unit (ALU) performs the core computational work:
  • For arithmetic instructions (ADD, SUB, AND, OR), the ALU computes the mathematical result of the two register operands.
  • For memory instructions (LOAD, STORE), the ALU adds the base register value to the sign-extended immediate offset to calculate the effective memory address.
  • For conditional branch instructions (BEQ, BNE), the ALU subtracts the two operands to check for equality while a branch adder calculates the target address: Target = PC + (Offset × 4).
• At the clock edge, the ALU result, write-data, and control bits are latched into the EX/MEM register.

Stage 4: Memory Access (MEM)

• If the instruction is a LOAD, the effective address generated by the ALU is sent to the Level 1 Data Cache (L1 D-cache) to read out a word of data.
• If the instruction is a STORE, the data read from rs2 during the ID stage is written into the L1 D-cache at the calculated address.
• If the instruction is an ordinary arithmetic operation, the MEM stage is completely passive: the ALU result simply passes straight through to the next register.
• At the clock edge, the memory read data and ALU result are latched into the MEM/WB register.

Stage 5: Writeback (WB)

• A multiplexer selects between the memory read data (for loads) and the ALU result (for arithmetic).
• The selected value is routed back across the chip to the Register File and written into the destination register rd.
• The instruction is officially retired—its architectural state change is permanent.

---

---

4. The Three Great Pipeline Hazards

If every instruction were completely independent of every other instruction, pipelining would be simple. A CPU could execute instructions endlessly with CPI = 1.0.

In real software, instructions are deeply interdependent.

An instruction frequently needs the mathematical result of the instruction immediately preceding it. Other instructions alter control flow based on conditions that have not yet been evaluated.

When the hardware cannot execute the next instruction in the designated clock cycle, the pipeline encounters a Hazard.

Pipeline hazards fall into three fundamental classes:

1. Structural Hazards (hardware conflicts),
2. Data Hazards (data dependencies),
3. Control Hazards (branch instruction decisions).

Understanding how microprocessors solve these hazards without crashing or corrupting data is the pinnacle of computer architecture.

---

5. Structural Hazards: Resource Contention

A structural hazard occurs when two different instructions in different pipeline stages attempt to use the exact same physical hardware resource simultaneously.

Consider what would happen if a microprocessor had a single, unified memory cache for both instructions and data (a unified Princeton/von Neumann cache):

The Structural Collision:
Clock Cycle 4:
Instruction 1 (LOAD):   In MEM stage  ===> Trying to read DATA from Cache!
Instruction 4 (FETCH):  In IF stage   ===> Trying to read INSTRUCTION from Cache!
                                           |
                                  COLLISION: Structural Hazard!
                   Single memory bus cannot service both requests simultaneously!

Instruction 1 needs to read memory in the MEM stage at the exact same clock tick that Instruction 4 needs to fetch an instruction from memory in the IF stage!

The Hardware Solutions

1. Harvard Architecture Caches: Modern processors physically separate the Level 1 cache into two independent, parallel SRAM structures:

   • A dedicated L1 Instruction Cache (L1 I-cache) connected exclusively to the IF stage.
   • A dedicated L1 Data Cache (L1 D-cache) connected exclusively to the MEM stage. Because both caches have independent address and data buses, IF and MEM can access memory simultaneously on every single clock cycle with zero conflict.

2. Multi-Ported Register Files: A structural conflict also threatens the Register File: Instruction 1 wants to write to register rd in WB, while Instruction 3 wants to read from registers rs1 and rs2 in ID. Hardware engineers solve this with split-phase clocking:

   • Register writes occur on the first half of the clock cycle (falling edge to midpoint).
   • Register reads occur on the second half of the clock cycle (midpoint to rising edge). If Instruction 3 reads the same register that Instruction 1 is writing, it reads the freshly updated value in the very same clock tick!

---

6. Data Hazards: Read-After-Write (RAW) Dependencies

A data hazard occurs when an instruction depends on the result of an earlier instruction that is still in flight in the pipeline.

Consider this simple assembly code sequence:

ADD R1, R2, R3    # R1 = R2 + R3
SUB R4, R1, R5    # R4 = R1 - R5 (Depends on R1!)

Trace this sequence through the pipeline clock cycles:

Data Dependency Timeline:
Clock Cycle:       1      2      3      4      5      6
ADD R1, R2, R3:  [IF]   [ID]   [EX]   [MEM]  [WB]   
                                       (R1 written HERE at Cycle 5!)
                                               ▲
SUB R4, R1, R5:         [IF]   [ID]   [EX]   [MEM]  [WB]
                                 ▲
                     (SUB reads R1 HERE at Cycle 3!)

Look at the fatal collision:

• The SUB instruction reads its input operands from the register file during its ID stage (Clock Cycle 3).
• But the ADD instruction does not write its calculated sum into R1 until its WB stage (Clock Cycle 5)!

If the hardware executes blindly, the SUB instruction will read the old, stale value of R1 that existed before the ADD ran. The calculation will be completely wrong! This is a Read-After-Write (RAW) Hazard.

Naive Solution: Pipeline Stalls (Bubbles)

A primitive processor detects this dependency using hazard detection logic: it compares the destination register number in ID/EX with the source register numbers in IF/ID.

If they match, the hazard detector stalls the pipeline:

1. It disables the write-enable on the PC and IF/ID register, freezing the SUB instruction in the ID stage for two full clock cycles.
2. It injects a bubble (No-Operation / NOP) into the ID/EX register.
3. Two empty cycles bubble through the pipeline like air in a hydraulic brake line, delaying execution and degrading CPI from 1.0 to 3.0.

Stalling with Pipeline Bubbles:
Clock Cycle:     1     2     3     4     5     6     7
ADD R1, R2, R3: [IF]  [ID]  [EX]  [MEM] [WB]
SUB R4, R1, R5:       [IF]  [ID]  [ID]  [ID]  [EX]  [MEM] [WB]
                              ▲     ▲     ▲
                           Stall Stall Valid
                           Bubble Bubble Read

The Elegant Solution: Register Forwarding (Bypassing)

In the late 1970s, computer architects realized something profound:

Where is the new value of R1 physically located inside the processor at Clock Cycle 3?

It is not in the register file yet. But it has already been calculated!

At the end of Clock Cycle 3, the output of the ALU is sitting right there, latched into the EX/MEM pipeline register! It is a set of stable electrical voltages waiting to be passed to memory.

Why wait two whole cycles for that electrical charge to travel into EX/MEM, then into MEM/WB, and finally into the register file, only to read it right back out?

Register Forwarding (Bypassing) Hardware Path:
[ ID Stage ] ===> [ ALU Input Multiplexer ] <====================+
                         ▲                                       | (Forwarding Path)
                         |                                       |
                     [  ALU  ] ===> ( EX/MEM Latch ) ===> [ MEM Stage ] ===> ( MEM/WB Latch )
                         |                |                                          |
                         +----------------+------------------------------------------+
                                          (Direct Hardware Bypasses!)

Architects built dedicated electrical wires called Forwarding Paths (Bypass Multiplexers):

• Dedicated comparator circuits check if the destination register in EX/MEM matches the source register in ID/EX:

• If a match is detected, the forwarding multiplexer snatches the freshly calculated ALU output directly from the EX/MEM register and routes it straight into the ALU input for the very next cycle!
• The SUB instruction gets the exact, correct value of R1 with zero stalls, zero bubbles, and zero lost clock cycles!

The One Unavoidable Hazard: The Load-Use Delay

Forwarding solves almost all data hazards, with one critical exception: the Load-Use Data Hazard.

LW  R1, 0(R2)     # Load R1 from Data Cache
ADD R3, R1, R4    # ADD needs R1 immediately!

Trace the timing:

• The ADD instruction needs the loaded data at the start of its EX stage (Clock Cycle 4).
• But for a LOAD instruction, the data is not read from the L1 Data Cache until the end of its MEM stage (Clock Cycle 4)!

Even with forwarding, the data physically does not exist inside the processor until Clock Cycle 4 ends. Time travel is impossible: the hardware cannot forward a value backwards in time from Cycle 4 to Cycle 3.

The Load-Use Stall:
Clock Cycle:       1     2     3     4     5     6
LW R1, 0(R2):    [IF]  [ID]  [EX]  [MEM] [WB]
                                    | (Data available at END of Cycle 4)
                                    v (Forwarded to EX stage!)
ADD R3, R1, R4:        [IF]  [ID]  [ID]  [EX]  [MEM] [WB]
                               ▲     ▲
                            Fetch  STALL!
                            (1-cycle unavoidable bubble)

The processor must insert a mandatory 1-cycle pipeline bubble.

Compiler Optimization: Instruction Scheduling

Can software eliminate this 1-cycle penalty? Yes!

Modern optimizing compilers perform Instruction Scheduling: the compiler inspects the dependency graph of the basic block and finds an unrelated, independent instruction from elsewhere in the code, moving it into the gap between the LOAD and the dependent instruction:

# UNOPTIMIZED (1-cycle hardware stall):
LW   R1, 0(R2)
ADD  R3, R1, R4   # Stalls for 1 cycle!
SUB  R6, R7, R8   # Independent instruction

# COMPILER-SCHEDULED (Zero stalls!):
LW   R1, 0(R2)
SUB  R6, R7, R8   # Fills the delay slot while LW accesses cache!
ADD  R3, R1, R4   # Data is now ready; zero stall!

---

7. Control Hazards: The Cost of Branches

The third and most dangerous class of hazard is the Control Hazard (or Branch Hazard).

In computer programs, execution is rarely a pure, straight line. Programs are filled with if-else statements, loops (for, while), switch blocks, and function calls. In typical compiled code, one out of every five instructions (15% to 20%) is a branch or jump!

Consider a conditional branch:

BEQ R1, R2, Target   # If R1 == R2, jump to Target
ADD R3, R4, R5       # Next sequential instruction (PC + 4)

When the processor fetches the BEQ instruction in the IF stage (Cycle 1), it does not know whether the branch will be taken or not taken!

• The condition (R1 == R2) is not evaluated until the ALU subtracts them in the EX stage (Cycle 3).
• The branch target address is not computed until the EX stage.

What instruction should the processor fetch on Clock Cycle 2?

• Should it fetch the next sequential instruction (PC + 4)?
• Or should it fetch the instruction at Target?

If the processor simply freezes and waits until the branch resolves at Cycle 3, it must discard two whole clock cycles on every single branch:

The Branch Penalty:
Clock Cycle:      1     2     3     4     5
BEQ R1, R2, Lbl: [IF]  [ID]  [EX]  [MEM] [WB]
                             (Branch resolves HERE!)
Next Inst:             Wait  Wait  [IF]  [ID]  [EX]
                        ▲     ▲
                     2-Cycle Penalty on EVERY branch!

If 20% of all instructions are branches, and every branch costs 2 stall cycles:

The processor loses nearly 30% of its total theoretical performance to branch stalls! On deeply pipelined modern processors with 15 to 20 stages, the branch penalty can be 15 to 20 lost clock cycles!

---

---

8. Defeating the Branch Hazard: Speculative Execution and Branch Prediction

Microprocessors cannot afford to wait. Instead, they guess.

This is Speculative Execution: the processor predicts whether the branch will be taken or not taken, predicts the target address, and immediately begins fetching instructions down that predicted path.

Speculative Execution Timeline:
1. Branch fetched in Cycle 1.
2. Hardware predictor guesses: "Branch is TAKEN to Address 0x8040!"
3. Cycle 2 fetches from 0x8040. Cycle 3 fetches from 0x8044. Cycle 4 fetches from 0x8048.
4. Cycle 4: Branch resolves in EX stage.
   - If Guess was CORRECT: Execution proceeds at full speed! ZERO LOST CYCLES!
   - If Guess was WRONG:   PIPELINE FLUSH! Purge speculative instructions and restart!

Static vs. Dynamic Branch Prediction

1. Static Prediction: Fixed rules baked into hardware without looking at runtime history:

   • Predict Not Taken: Always assume the branch will fail; fetch PC + 4.
   • Backward Taken, Forward Not Taken (BTFN): If the branch offset is negative (backward in memory), predict Taken (because backward branches are almost always loop boundaries: for, while). If the branch offset is positive (forward in memory), predict Not Taken (often error-handling if checks).

2. Dynamic Branch Prediction: Dedicated hardware tables track the historical behavior of branches at runtime.

The 2-Bit Saturating Counter

A simple 1-bit predictor (remembering whether the branch was taken last time) fails miserably on loops. At the end of an inner loop that runs 1,000 times:

• It mispredicts on the final exit (when the loop finishes).
• Then, when the loop starts again, it mispredicts on the very first iteration because it remembered the exit! Two mispredictions for a single loop.

In 1981, Jim Smith introduced the 2-Bit Saturating Counter Finite State Machine:

The 2-Bit Saturating Counter State Machine:
               Taken                        Taken
         +---------------->           +---------------->
       [ 00 ]                       [ 01 ]                       [ 10 ]                       [ 11 ]
    Strongly Not                 Weakly Not                  Weakly Taken                 Strongly
       Taken                        Taken                                                  Taken
         <----------------+           <----------------+           <----------------+
             Not Taken                    Not Taken                    Not Taken

PREDICTION: If state is 00 or 01 ===> Predict NOT TAKEN
            If state is 10 or 11 ===> Predict TAKEN

The counter requires two consecutive opposing outcomes to change its prediction!

• In a loop that runs 1,000 times, it sits in state 11 (Strongly Taken).
• On the final exit iteration, it transitions to 10 (Weakly Taken). It mispredicts once.
• On the next entry into the loop, it is still in state 10—so it correctly predicts Taken on the first iteration!
• Misprediction rate on loops drops by 50%!

Modern Branch Predictors: TAGE and Perceptrons

Modern processors (like Intel Raptor Lake, AMD Zen 5, and Apple M-series) do not use simple 2-bit counters. They use TAGE (TAgged GEometric history length) predictors and neural perceptron predictors:

• They maintain global branch history registers tracking the pattern of the last 100 to 500 branches across the entire program.
• They recognize complex correlated patterns (e.g., "if branch A was taken and branch B was not taken, branch C is 99.4% likely to be taken").
• Modern predictors achieve accuracy exceeding 97% to 99%!

The Pipeline Flush: Paying the Misprediction Penalty

What happens when that 1% error occurs?

When the branch resolves in the EX stage and hardware discovers the prediction was wrong:

1. The Trap Door Opens: The speculative instructions currently sitting in the IF, ID, and EX stages are wrong. They must never be allowed to write their results to memory or registers!
2. Synchronous Clear: The control unit asserts the synchronous Flush (clear) lines on the IF/ID and ID/EX registers, resetting their control bits to zero. This converts the speculative instructions into harmless No-Operations (NOPs).
3. Redirect the Fetch: The correct branch target address is forced into the Program Counter, and the instruction cache restarts fetching down the correct path.
4. The Penalty Paid: All cycles spent fetching the wrong path are discarded. On deep modern pipelines, this costs 15 to 20 clock cycles. This is why branch prediction accuracy is the single most critical performance bottleneck in modern computer architecture.

---

9. Beyond 5 Stages: Superscalar and Out-of-Order Execution

The classic 5-stage pipeline achieves an ideal CPI = 1.0: one instruction completes per clock cycle.

Can a processor do better than CPI = 1.0? Can it retire four, six, or eight instructions on every single clock tick (IPC > 1)?

Yes. This is the domain of modern Superscalar Out-of-Order (OoO) Processors.

Modern Superscalar Out-of-Order Execution Pipeline:
[ In-Order Fetch: 6 Insts/Cycle ] ===> [ Branch Predictor (TAGE) ]
                 |
[ In-Order Decode & Register Renaming: Map Arch Regs to Physical Regs ]
                 |
[ DISPATCH TO RESERVATION STATIONS / ISSUE QUEUES ]
                 |
   +-------------+-------------+-------------+-------------+
   | (ALU 1)     | (ALU 2)     | (Vector FPU)| (Load/Store)|  <--- Out-of-Order Execution!
   +-------------+-------------+-------------+-------------+       (Instructions execute as
                 |                                                 soon as operands are ready!)
                 v
[ REORDER BUFFER (ROB) ] ===> IN-ORDER RETIREMENT & COMMIT!
                              (Preserves precise architectural exceptions!)

1. Superscalar Execution

Instead of a single execution pipeline, the chip contains multiple parallel execution pipes: three ALUs, two floating-point vector units, and two memory load/store pipelines. It fetches and decodes a bundle of 4 to 8 instructions simultaneously.

2. Register Renaming (Eliminating False Dependencies)

In addition to True Data Dependencies (RAW), code contains "false dependencies" caused by compilers reusing a limited number of architectural register names:

• Write-After-Read (WAR): Anti-dependency.
• Write-After-Write (WAW): Output dependency.

Modern processors contain 128 to 256 physical registers, while the assembly architecture only exposes 32 (e.g., R0--R31). A hardware Register Alias Table (RAT) dynamically renames architectural registers to physical registers on the fly, breaking all WAR and WAW false dependencies!

3. Out-of-Order Dispatch (Tomasulo's Algorithm)

If Instruction 1 is waiting on a slow cache miss, why should Instruction 2 and Instruction 3 sit behind it waiting? Instructions are placed into Reservation Stations (Issue Queues). They wait there until their specific operands are ready. As soon as its inputs are available, an instruction fires into an available execution unit, completely out of program order!

4. The Reorder Buffer (ROB) and In-Order Commit

If instructions execute out of order, how does the processor maintain correct program semantics and handle exceptions (like a page fault or divide-by-zero)? All in-flight instructions are tracked in a circular FIFO queue called the Reorder Buffer (ROB):

• Instructions execute out of order.
• But they commit their results to architectural state strictly in program order!
• If an instruction triggers a fault, all subsequent speculative instructions in the ROB are discarded, leaving the architectural register state perfectly pristine—maintaining the illusion of sequential, deterministic execution.

---

10. The Silicon Symphony

When you type a command, stream a video, or run a simulation, your computer's microprocessor is not plodding through machine code one step at a time.

It is conducting a high-speed, multi-stage silicon symphony:

• Tens of instructions march through the pipeline registers on every fraction of a nanosecond.
• Bypass multiplexers snatch freshly computed numbers from the outputs of arithmetic units and slingshot them across the chip before they even touch memory.
• Dynamic branch predictors look into the future, guessing loop exits with 99% accuracy.
• Out-of-order issue queues shuffle instructions around slow cache stalls, keeping execution units blazing at billions of cycles per second.

Pipelining took the raw, unyielding physics of silicon propagation delays and transformed it into the fluid, effortless speed that powers modern digital civilization.