System-aware Architectural Co-design to Tackle the Memory Wall

The ever-growing gap between the performance of processing and memory elements, known as the memory wall, remains a major contributor to the runtime and energy overheads in modern computing systems. This concern is particularly acute in emerging machine learning (ML) and scientific computing applications, where heavy memory traffic leads to frequent computation stalls and high latency. To address these challenges, this thesis introduces novel architectural techniques designed and assessed from a cross-layer perspective. These target the alleviation of four main constraints in the communication between processing logic and main memory: (1) limited bandwidth, (2) high energy costs, (3) low computational intensity, and (4) inefficient interconnects. This thesis first focus on Compute-near-Memory (CnM) architectures, interfacing processing units (PUs) directly to the DRAM memory banks to increase compute bandwidth at low energy per bit. To enable the fast DSE of the paradigm, a CnM simulation framework is proposed. This methodology allows to co-optimize PU configuration and DRAM standard choice according to the targeted workloads. Following this analysis, the thesis proposes SideDRAM, a novel bank-level CnM architecture to address the limitations of previous designs. It leverages heterogeneous quantization, row-wide access to banks, and a scalable control plane to enable massively parallel and area-/energy-efficient processing near DRAM. This solution achieves up to 83% EDAP reduction compared to a state-of-the-art industrial design. Next, this thesis targets the co-design of hardware and software to optimize data transfers in ML workloads on resource-constrained heterogeneous edge devices. By aligning the pruning weight tiles with the dimensions of accelerators, entire computations can be skipped to increase locality and computational intensity. The assessment of this technique with a purpose-built co-design framework showcases speedups of up to 73x. Finally, this thesis targets the inefficient, resource-limited chiplet interconnects in heterogeneous systems. In-package wireless communication is explored as a flexible alternative that supports data movement while freeing up physical interconnect resources. Results from system-level simulations highlight the potential of this technology, achieving up to 2.64x speedup for convolutional neural networks. In addition, thanks to co-designed mapping heuristics, reductions in silicon temperature by 24% are also accomplished. In summary, this thesis demonstrates the effectiveness of system-aware architectural design to alleviate memory wall constraints in heterogeneous systems. By adopting a cross-layer perspective, this approach allows to mitigate the bottlenecks in the communication between computing logic and memory, enabling more efficient execution of data-intensive applications. As a result, system-wide improvements in runtime, energy consumption and thermal behavior are accomplished, underscoring the