A1.1.6 Describe the process of pipelining in multi-core architectures. (HL only).

📚 You can find additional information in the course companion pages 21 to 25

The Big Idea

Modern CPUs are designed to execute billions of instructions per second, and to achieve this level of performance, they rely on advanced techniques such as instruction pipelining and multi-core parallelism. These techniques allow for overlapping operations and distributed computation across multiple cores, maximizing throughput while minimizing latency.

For an advanced look at the relationship between scheduling, pipelining and multi-core architecture, please click this link.

Definition: Pipelining

Pipelining is a CPU design technique where multiple instructions are overlapped in execution — like an assembly line.
While one instruction is being fetched, another can be decoded, and another can be executed.
This increases the overall instruction throughput (how many instructions are completed per second), even though each instruction still goes through all stages.

Example:
Imagine washing clothes:

• Load clothes (Fetch)
• Wash (Decode)
• Rinse (Execute)
• Spin (Write back)
  With pipelining, while one batch is washing, another can be rinsing — so multiple batches are processed at once.

 

Definition: Multi-Core Architectures 

A multi-core processor has two or more independent cores (processing units) on a single chip.
Each core can execute its own instructions at the same time as the others, allowing true parallel processing.

Example:
Think of having multiple workers in a kitchen — each can cook a different dish at the same time, so the overall meal is ready faster.

 

1. The Pipeline Model in a Single Core

A pipeline in CPU architecture is analogous to an assembly line in a factory. Instead of executing one instruction at a time through all stages (which would be inefficient), pipelining allows the CPU to work on multiple instructions simultaneously, with each instruction at a different stage.

Basic Pipeline Stages:

1. Instruction Fetch (IF): Retrieve the next instruction from memory.
2. Instruction Decode (ID): Interpret the opcode and operands.
3. Execute (EX): Perform the required operation (e.g., ALU computation).
4. Memory Access (MEM): Read from or write to memory (optional, depending on the instruction).
5. Write-Back (WB): Store the result in a register.

Each stage is handled by a different part of the CPU hardware. Once the pipeline is full, a new instruction can enter the pipeline every clock cycle, dramatically improving instruction throughput.

Example:

Cycle	Stage 1	Stage 2	Stage 3	Stage 4	Stage 5
1	IF A
2	IF B	ID A
3	IF C	ID B	EX A
4	IF D	ID C	EX B	MEM A
5	IF E	ID D	EX C	MEM B	WB A

---

2. The Role of Write-Back and Performance Optimization

The write-back (WB) stage is critical in ensuring that computed results are stored properly—typically in a register or memory location. Without this stage, intermediate results would not persist, and the next instructions might use stale or undefined data.

Write-back supports:

• Instruction chaining: Later instructions depend on results of earlier ones.
• Out-of-order execution: Ensures correct values are committed to registers in program order.
• Hazard resolution: Coordination with register renaming and forwarding logic to avoid data hazards.

In high-performance CPUs, write-back logic is tightly coupled with forwarding/bypassing paths, allowing results to be used by subsequent instructions before they are formally written back, reducing pipeline stalls.

---

3. Multi-Core Architectures: Independent and Parallel Pipelines

While pipelining improves instruction-level parallelism (ILP) within a single core, multi-core processors exploit task-level parallelism (TLP) by duplicating the entire core (and pipeline) across the chip.

Key Features of Multi-Core Systems:

• Each core has its own pipeline, registers, and instruction decoder.
• Cores operate independently and in parallel, executing separate threads or processes concurrently.
• The operating system scheduler assigns threads to available cores.
• In symmetric multi-core systems, all cores are identical and share a common memory hierarchy (L1 private, L2/L3 shared).
• Some architectures use heterogeneous cores (e.g., ARM big.LITTLE), where different cores are optimized for different workloads.

Coordination and Shared Resources

• Cores communicate via shared memory, which requires cache coherence protocols (e.g., MESI).
• Synchronization primitives (like mutexes or semaphores) are used to manage access to shared data structures.

---

4. Pipelining + Multi-Core = Parallelism at Two Levels

By combining deep pipelining in each core with multiple independent cores, modern CPUs achieve performance through parallelism at multiple levels:

• Instruction-level parallelism (ILP): Multiple instructions in flight within one core (pipelining, superscalar).
• Thread-level parallelism (TLP): Multiple threads across multiple cores.
• Data-level parallelism (DLP): SIMD execution within vector units inside each core.

These levels complement one another. For example:

• While one core is executing a heavy arithmetic task using its pipeline, another core may be processing user input.
• Within one core, one instruction is being fetched while another is being written back, with no delays.

---

Summary

Concept	Description
Pipelining	Overlapping stages of instruction execution (fetch, decode, execute, write-back) to increase throughput.
Write-back	Final pipeline stage that stores results; critical for correctness and forwarding.
Multi-core	Multiple independent CPU cores, each with its own pipeline, running in parallel to execute multiple threads simultaneously.
System Benefit	Increased throughput, better multitasking, reduced latency for parallel workloads.

---

Conclusion

Pipelining transforms the CPU into a highly efficient execution engine by enabling parallelism across instruction stages, while multi-core architectures scale this model horizontally by duplicating cores. Together, these approaches allow modern processors to handle complex, high-throughput workloads by exploiting concurrent execution both within and across processing units.