Lecture

Programming Models: Overview and Examples

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

CS-422: Database systems

This course is intended for students who want to understand modern large-scale data analysis systems and database systems. It covers a wide range of topics and technologies, and will prepare students to be able to build such systems as well as read and und ...

Description

This lecture covers different programming models for big data processing, comparing MapReduce with more powerful models like Spark and Dryad. It explains the concepts of DAGs, declarative languages, functional programming, and domain-specific languages. The presentation also delves into Spark's RDDs, actions, transformations, lazy evaluation, and optimizations. Additionally, it discusses the design choices of big data systems regarding storage, programming models, execution engines, resource management, and fault tolerance.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructor

Anastasia Ailamaki

Expertise Database management systems, scientific applications, computer architecture Teaching & PhD PhD Students Hamish Nicholson, Anna Herlihy, Yi Jiang, Eleni Zapridou, Georgiy Konstantinovich Lebedev, Ioanna Tsakalidou Past EPFL PhD Students Farhan Tauheed, Radu Ioan Stoica, Manos Athanassoulis, Pinar Tözün, Adrian Daniel Popescu, Ioannis Alagiannis, Renata Borovica-Gajic, Danica Porobic, Iraklis Psaroudakis, Manos Karpathiotakis, Matthaios Alexandros Olma, Mirjana Pavlovic, Eleni Tzirita Zacharatou,

Official source

Related concepts (31)

Declarative programming

In computer science, declarative programming is a programming paradigm—a style of building the structure and elements of computer programs—that expresses the logic of a computation without describing its control flow. Many languages that apply this style attempt to minimize or eliminate side effects by describing what the program must accomplish in terms of the problem domain, rather than describing how to accomplish it as a sequence of the programming language primitives (the how being left up to the language's implementation).

Optimizing compiler

In computing, an optimizing compiler is a compiler that tries to minimize or maximize some attributes of an executable computer program. Common requirements are to minimize a program's execution time, memory footprint, storage size, and power consumption (the last three being popular for portable computers). Compiler optimization is generally implemented using a sequence of optimizing transformations, algorithms which take a program and transform it to produce a semantically equivalent output program that uses fewer resources or executes faster.

Program optimization

In computer science, program optimization, code optimization, or software optimization, is the process of modifying a software system to make some aspect of it work more efficiently or use fewer resources. In general, a computer program may be optimized so that it executes more rapidly, or to make it capable of operating with less memory storage or other resources, or draw less power. Although the word "optimization" shares the same root as "optimal", it is rare for the process of optimization to produce a truly optimal system.

Peephole optimization

Peephole optimization is an optimization technique performed on a small set of compiler-generated instructions; the small set is known as the peephole or window. Peephole optimization involves changing the small set of instructions to an equivalent set that has better performance.

Evaluation strategy

In a programming language, an evaluation strategy is a set of rules for evaluating expressions. The term is often used to refer to the more specific notion of a parameter-passing strategy that defines the kind of value that is passed to the function for each parameter (the binding strategy) and whether to evaluate the parameters of a function call, and if so in what order (the evaluation order). The notion of reduction strategy is distinct, although some authors conflate the two terms and the definition of each term is not widely agreed upon.

Related lectures (64)

Spark Ecosystem: Architectural Choices

Explores the Spark ecosystem's architectural choices, including RDDs and fault tolerance.

Hadoop Ecosystem: Architectural Choices & MapReduce Programming

Explores the Hadoop ecosystem's architecture and MapReduce programming model, emphasizing strengths and limitations.

Compiler Extension Lab

Covers the Compiler Extension Lab, focusing on adding new functionality to a simple functional language compiler.

Optimizing Recursive Queries

Explores optimizing recursive queries in database systems using Datalog and semirings, discussing the challenges and solutions in data analytics.

Programming for Engineers: Advanced MATLAB Techniques