Lecture

MPI Basics: Point-to-Point and Collective Communications

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MATH-454: Parallel and high-performance computing

This course provides insight into a broad variety of High Performance Computing (HPC) concepts and the majority of modern HPC architectures. Moreover, the student will learn to have a feeling about what architectures are suited for several types of algorit ...

Description

This lecture covers the basics of MPI, including the distributed memory programming paradigm, point-to-point communications, collective communications, and synchronizations. It explains how to run multiple instances of a program, different types of communications in MPI, and the importance of matching sends with receives to avoid race conditions or deadlocks. The lecture also delves into blocking and non-blocking point-to-point communications, MPI barriers, wildcard receives, and collective communications like MPI_Bcast, MPI_Scatter, MPI_Gather, and MPI_Reduce. Additionally, it discusses receiving image parts in order, out-of-order, and with a collective, as well as the importance of avoiding collective calls within conditional clauses and customizing communicators and datatypes.

Instructors (2)

Nicolas Richart

I graduated as an engineer in computer science in 2005 at the Ecole National Superieur en Electronique Informatique et Radiocomunication de Bordeaux (ENSEIRB). My degree is colored by an option od High Performance Computing (HPC). Then I made my thesis at LaBRI/INRIA Bordeaux in the ScalApplix/HiePACS team, on the in-situ steering of coupled parallel numerical simulations. I graduated as a Ph D. of computer science of the Universite de Bordeaux 1 in 2010. Since then I am working at EPFL in the Laboratoire de Simulations en Mechanique des Solides (LSMS) as a scientific collaborator.

Pablo Antolin Sanchez

Teaching & PhD PhD Students Philipp Christoph Weder, Davor Dobrota Past EPFL PhD Students as codirector Ondine Chanon Courses Analysis III MATH-202(c) The course studies the fundamental concepts of vector analysis and Fourier-Laplace analysis with a view to their use in solving multidisciplinary problems in scientific engineering. Analysis III (for IC) MATH-203(d) The course studies the fundamental concepts of vector analysis and Fourier-Laplace analysis with a view to thei

Official source

Related concepts (17)

Message Passing Interface

Message Passing Interface (MPI) is a standardized and portable message-passing standard designed to function on parallel computing architectures. The MPI standard defines the syntax and semantics of library routines that are useful to a wide range of users writing portable message-passing programs in C, C++, and Fortran. There are several open-source MPI implementations, which fostered the development of a parallel software industry, and encouraged development of portable and scalable large-scale parallel applications.

Race condition

A race condition or race hazard is the condition of an electronics, software, or other system where the system's substantive behavior is dependent on the sequence or timing of other uncontrollable events. It becomes a bug when one or more of the possible behaviors is undesirable. The term race condition was already in use by 1954, for example in David A. Huffman's doctoral thesis "The synthesis of sequential switching circuits". Race conditions can occur especially in logic circuits, multithreaded, or distributed software programs.

Deadlock

In concurrent computing, deadlock is any situation in which no member of some group of entities can proceed because each waits for another member, including itself, to take action, such as sending a message or, more commonly, releasing a lock. Deadlocks are a common problem in multiprocessing systems, parallel computing, and distributed systems, because in these contexts systems often use software or hardware locks to arbitrate shared resources and implement process synchronization.

Race and health

Race and health refers to how being identified with a specific race influences health. Race is a complex concept that has changed across chronological eras and depends on both self-identification and social recognition. In the study of race and health, scientists organize people in racial categories depending on different factors such as: phenotype, ancestry, social identity, genetic makeup and lived experience. "Race" and ethnicity often remain undifferentiated in health research.

Race and genetics

Researchers have investigated the relationship between race and genetics as part of efforts to understand how biology may or may not contribute to human racial categorization. Many constructions of race are associated with phenotypical traits and geographic ancestry, and scholars like Carl Linnaeus have proposed scientific models for the organization of race since at least the 18th century. Following the discovery of Mendelian genetics and the mapping of the human genome, questions about the biology of race have often been framed in terms of genetics.

Related lectures (19)

Locking Primitives: Avoiding Race Conditions in Threads

Covers locking primitives necessary to prevent race conditions in multithreaded programming, focusing on mutual exclusion and atomic operations.

Concurrency Overview

Explores the significance of concurrency in enhancing system performance and responsiveness, emphasizing the need for synchronization and atomicity to prevent race conditions and non-determinism.

Principles of Parallel Computing: OpenMP

Explores the principles of parallel computing, focusing on OpenMP as a tool for creating concurrent code from serial code.

Atomic execution: Synchronization in threads

Covers atomic execution in threads, race conditions, shared variables, and synchronized blocks.

LabVIEW Design Patterns