Lecture

Task Parallelism and Explicit Communication

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture by the instructor covers the topics of task parallelism, explicit communication, remote direct memory access, and architectural support for task parallelism and explicit communication. The discussion includes the need for human intelligence in managing parallel programming, the differences between implicit and explicit communication, and the use of runtime systems for prefetching data. The lecture also delves into the implementation of synchronization primitives in hardware, such as counters and queues, for efficient multi-party synchronization. The instructor emphasizes the importance of a balanced approach between human ingenuity and compiler/runtime support in achieving efficient parallel programming.

Official source

Ontological neighbourhood

Computer engineering

High-performance computing: Parallel computing

Related concepts (20)

Parallel computing

Parallel computing is a type of computation in which many calculations or processes are carried out simultaneously. Large problems can often be divided into smaller ones, which can then be solved at the same time. There are several different forms of parallel computing: bit-level, instruction-level, data, and task parallelism. Parallelism has long been employed in high-performance computing, but has gained broader interest due to the physical constraints preventing frequency scaling.

Parallel programming model

In computing, a parallel programming model is an abstraction of parallel computer architecture, with which it is convenient to express algorithms and their composition in programs. The value of a programming model can be judged on its generality: how well a range of different problems can be expressed for a variety of different architectures, and its performance: how efficiently the compiled programs can execute. The implementation of a parallel programming model can take the form of a library invoked from a sequential language, as an extension to an existing language, or as an entirely new language.

Task parallelism

Task parallelism (also known as function parallelism and control parallelism) is a form of parallelization of computer code across multiple processors in parallel computing environments. Task parallelism focuses on distributing tasks—concurrently performed by processes or threads—across different processors. In contrast to data parallelism which involves running the same task on different components of data, task parallelism is distinguished by running many different tasks at the same time on the same data.

Data parallelism

Data parallelism is parallelization across multiple processors in parallel computing environments. It focuses on distributing the data across different nodes, which operate on the data in parallel. It can be applied on regular data structures like arrays and matrices by working on each element in parallel. It contrasts to task parallelism as another form of parallelism. A data parallel job on an array of n elements can be divided equally among all the processors.

Synchronization (computer science)

In computer science, synchronization refers to one of two distinct but related concepts: synchronization of processes, and synchronization of data. Process synchronization refers to the idea that multiple processes are to join up or handshake at a certain point, in order to reach an agreement or commit to a certain sequence of action. Data synchronization refers to the idea of keeping multiple copies of a dataset in coherence with one another, or to maintain data integrity.

Related lectures (64)

Parallel Programming I

Covers the basics of parallel programming, including concurrency, forms of parallelism, synchronization, and programming models like PThreads and OpenMP.

Parallel Computing: Principles and OpenMP

Covers the principles of parallel computing and introduces OpenMP for creating concurrent code from serial code.

Introduction to Multiprocessor Architecture

Covers multiprocessor architecture, sustainable computing, AI model training impact, and parallel programming fundamentals.

Principles of Parallelism

Covers the basics of parallelism, including physical examples, historical context, multicore era, and parallel collections in Scala.

Parallelism: Programming and Performance