Lecture

MapReduce: Execution Models for Distributed Computing

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Related lectures (64)

General Introduction to Big Data

Covers data science tools, Hadoop, Spark, data lake ecosystems, CAP theorem, batch vs. stream processing, HDFS, Hive, Parquet, ORC, and MapReduce architecture.

Big Data Ecosystems: Technologies and Challenges

Covers the fundamentals of big data ecosystems, focusing on technologies, challenges, and practical exercises with Hadoop's HDFS.

Data-Parallel Programming: Vector & SIMD Processors

Explores data-parallel programming with vector processors and SIMD, and introduces MapReduce, Pregel, and TensorFlow.

GPU Introduction: Performance and Programming

Explores the motivation and advantages of using GPUs for computation, focusing on their performance and programming through CUDA.

Advanced Spark Optimization Techniques: Managing Big Data

Discusses advanced Spark optimization techniques for managing big data efficiently, focusing on parallelization, shuffle operations, and memory management.

Parallel Computing: Principles and OpenMP

Covers the principles of parallel computing and introduces OpenMP for creating concurrent code from serial code.

Data Wrangling with Hive: Managing Big Data Efficiently

Covers data wrangling techniques using Apache Hive for efficient big data management.

Hadoop Ecosystem: Architectural Choices & MapReduce Programming

Explores the Hadoop ecosystem's architecture and MapReduce programming model, emphasizing strengths and limitations.

Big Data Best Practices and Guidelines

Covers best practices and guidelines for big data, including data lakes, architecture, challenges, and technologies like Hadoop and Hive.

Scaling up: Spark and Big Data

Explores the challenges of big data processing and introduces Spark as a solution.

Big Data: Best Practices and Guidelines

Covers best practices and guidelines for big data, including data lakes, typical architecture, challenges, and technologies used to address them.

Parallel Scan Left

Introduces parallel scan left in Scala, covering its properties, sequential solutions, and efficient parallel computation techniques.

Parallel Programming I

Covers the basics of parallel programming, including concurrency, forms of parallelism, synchronization, and programming models like PThreads and OpenMP.

Principles of Parallel Computing: OpenMP

Explores the principles of parallel computing, focusing on OpenMP as a tool for creating concurrent code from serial code.

Big Data Challenges: Scaling to Massive Data

Explores challenges of handling massive data in the era of big data, discussing solutions like MapReduce and Spark.

Synchronization

Covers the principles of synchronization in parallel computing, focusing on shared memory synchronization and different methods like locks and barriers.

Big Data Challenges: Distributed Computing with Spark

Explores big data challenges, distributed computing with Spark, RDDs, hardware requirements, MapReduce, transformations, and Spark DataFrames.

Synchronization: Locks and Barriers

Explores synchronization principles using locks and barriers, emphasizing efficient hardware-supported implementations and coordination mechanisms like OpenMP.

Data Wrangling with Hadoop

Covers data wrangling techniques using Hadoop, focusing on row versus column-oriented databases, popular storage formats, and HBase-Hive integration.

Asynchronous Programming with Futures

Explores asynchronous programming with futures, showcasing benefits, challenges, and efficient implementations.