Minimizing Memory Stalls

As Chapter 2 detailed, hardware vendors heavily innovated on implicit parallelism until 2005 through aggressive microarchitectural techniques (e.g., pipelining, superscalar execution, out-of-order execution, SIMD, etc.). Despite the differences across these techniques, all these innovations aim at one thing: minimizing the stall cycles where a core cannot retire an instruction. An instruction that completes all the five pipeline stages (described in Section 2.1) is retired. When an instruction gets stuck at one of the five pipeline stages, its execution is stalled. All the implicit parallelism techniques in modern hardware, become ineffective when an application exhibits excessive memory stalls. In other words, processor cannot retire instructions as efficiently as possible due to waiting for the instructions or data to be fetched from the memory hierarchy to be able to complete the five pipeline stages for those instructions.