Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

GPU Memory Hierarchy: Optimization

In course

CS-307: Introduction to multiprocessor architecture

Multiprocessors are a core component in all types of computing infrastructure, from phones to datacenters. This course will build on the prerequisites of processor design and concurrency to introduce the essential technologies required to combine multiple ...

Description

This lecture covers GPU memory hierarchy, including global, local, shared memory, and caches. It explains the CUDA processing flow, GPU optimizations, and control-flow divergence. The instructor discusses strategies to optimize algorithms for GPUs, exploit shared memory, and coalesce memory accesses. Various techniques to efficiently use parallelism and resources on GPUs are explored, such as reduction operations and addressing bank conflicts. The lecture concludes with a focus on scalability with array size and a summary of optimizing code for GPUs.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructor

Babak Falsafi

Babak is a Professor in the School of Computer and Communication Sciences and the founding director of the EcoCloud, an industrial/academic consortium at EPFL investigating scalable data-centric technologies. He has made numerous contributions to computer system design and evaluation including a scalable multiprocessor architecture which was prototyped by Sun Microsystems (now Oracle), snoop filters and memory streaming technologies that are incorporated into IBM BlueGene/P and Q and ARM cores, and computer system performance evaluation methodologies that have been in use by AMD, HP and Google PerKit . He has shown that hardware memory consistency models are neither necessary (in the 90's) nor sufficient (a decade later) to achieve high performance in multiprocessor systems. These results eventually led to fence speculation in modern microprocessors. His latest work on workload-optimized server processors laid the foundation for the first generation of Cavium ARM server CPUs, ThunderX. He is a recipient of an NSF CAREER award, IBM Faculty Partnership Awards, and an Alfred P. Sloan Research Fellowship. He is a fellow of IEEE and ACM.

Official source

Ontological neighbourhood

Computer engineering

High-performance computing: Parallel computing

Related concepts (32)

Memory access pattern

In computing, a memory access pattern or IO access pattern is the pattern with which a system or program reads and writes memory on secondary storage. These patterns differ in the level of locality of reference and drastically affect cache performance, and also have implications for the approach to parallelism and distribution of workload in shared memory systems. Further, cache coherency issues can affect multiprocessor performance, which means that certain memory access patterns place a ceiling on parallelism (which manycore approaches seek to break).

Shared memory

In computer science, shared memory is memory that may be simultaneously accessed by multiple programs with an intent to provide communication among them or avoid redundant copies. Shared memory is an efficient means of passing data between programs. Depending on context, programs may run on a single processor or on multiple separate processors. Using memory for communication inside a single program, e.g. among its multiple threads, is also referred to as shared memory.

General-purpose computing on graphics processing units

General-purpose computing on graphics processing units (GPGPU, or less often GPGP) is the use of a graphics processing unit (GPU), which typically handles computation only for computer graphics, to perform computation in applications traditionally handled by the central processing unit (CPU). The use of multiple video cards in one computer, or large numbers of graphics chips, further parallelizes the already parallel nature of graphics processing.

Distributed shared memory

In computer science, distributed shared memory (DSM) is a form of memory architecture where physically separated memories can be addressed as a single shared address space. The term "shared" does not mean that there is a single centralized memory, but that the address space is shared—i.e., the same physical address on two processors refers to the same location in memory. Distributed global address space (DGAS), is a similar term for a wide class of software and hardware implementations, in which each node of a cluster has access to shared memory in addition to each node's private (i.

Memory management

Memory management is a form of resource management applied to computer memory. The essential requirement of memory management is to provide ways to dynamically allocate portions of memory to programs at their request, and free it for reuse when no longer needed. This is critical to any advanced computer system where more than a single process might be underway at any time. Several methods have been devised that increase the effectiveness of memory management.

Related lectures (37)

GPU Advanced: Memory Management and Optimization

Explores advanced GPU memory management, CUDA programming, and data-parallel computing for optimizing performance.

Multi Masters Systems

Explores Multi Masters Systems, discussing architectures with multiple processors, shared memory, mutual exclusion, and hardware accelerators.

Cache Memory

Explores cache memory design, hits, misses, and eviction policies in computer systems, emphasizing spatial and temporal locality.

Virtual Memory

Covers virtual memory, program relocation, memory protection, TLB, and memory hierarchy interactions.

Concurrency and Parallelism: Visibility