Power Scaling: the Ultimate Obstacle to 1K-Core Chips

As Moore' s Law continues for at least another decade, the number of cores on chip and the on-chip cache size will continue to grow at an exponential rate. While workloads with limited parallelism pose performance challenges with multicore processors, server workloads with abundant parallelism are believed to be immune, capable of scaling to the parallelism available in the hardware. However, despite the inherent scalability in threaded server workloads, increasing core counts cannot directly translate into performance improvements because chips are physically constrained in power and off-chip bandwidth. In this work, we explore the design space of physically-constrained multicore chips across technologies and show that, even with conservative estimates, chips will not scale beyond a few tens of cores due to physical power and off-chip bandwidth constraints, potentially leaving the die real-estate underutilized in future technology generations. We observe that customized heterogenous multicores can leverage die area to overcome the initial power barrier, resulting in bandwidth constrained designs. Overcoming the bandwidth wall, e.g. through the use of large multi-gigabyte 3D-stacked caches, fully exposes multicore designs to the power wall, requiring innovation in low-power interconnects and on-chip hierarchies to further improve the performance of future servers.