Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Graph
Search

Course

COM-490: Large-scale data science for real-world data

Summary

This hands-on course covers tools and methods used by data scientists, from researching solutions to scaling prototypes on Spark clusters. Students engage with the full data engineering and data science pipeline, from data acquisition to extracting insights, applied to real-world problems.

Lectures

Instructors (3)

Sofiane Sarni

Olivier Verscheure

Eric Pierre Bouillet

Eric Bouillet received his PhD degree in Electrical Engineering from Columbia University, New York, NY in June 1999. Eric Bouillet has been working at IBM T.J. Watson Research Center, Hawthorne, NY since June 2004, and at the IBM Smarter City Technical Centre, Dublin from October 2010 to August 2016. While at IBM he has been working on scalable data stream analytics applied to a number of fields, including finances, law-enforcement, telecommunications, environmental monitoring, intelligent transport systems, and aircraft reliability control systems. Before joining IBM Research, Eric Bouillet was at Tellium, Oceanport, NJ where he was part of the research team who invented and designed the first commercial optical mesh restoration network (deployed nationwide and documented in their book Path Routing in Mesh Optical Networks), and at Lucent Technologies’ Mathematical Science Center in the department of Mathematics of Networks and System Research department where he worked on the design optimization and sizing of circuit and packet switched networks. Since September 2016 Eric is the Chief Architect and Head of Engineering of the newly created Swiss Data Science Center, a national R&D center organizationally hosted by both EPFL in Lausanne and ETH Zurich.

Official source

Moodle Page

http://go.epfl.ch/COM-490

Summary

Lectures

Instructors (3)

Sofiane Sarni

Olivier Verscheure

Eric Pierre Bouillet

Official source

Moodle Page

http://go.epfl.ch/COM-490

Lectures in this course (41)

Big Data Best Practices and Guidelines

Covers best practices and guidelines for big data, including data lakes, architecture, challenges, and technologies like Hadoop and Hive.

Introduction to Spark Runtime Architecture

Introduces Apache Spark, covering its architecture, RDDs, transformations, actions, fault tolerance, deployment options, and practical exercises in Jupyter notebooks.

Spark DataFrames: Basics and Optimization

Covers the basics of Spark DataFrames, their advantages, performance comparison with RDDs, and practical demos.

Advanced Spark Optimizations and Partitioning

Covers advanced Spark optimizations, memory management, shuffle operations, and data partitioning strategies to improve big data processing efficiency.

Advanced Spark Optimizations and Partitioning

Delves into advanced Spark optimizations, partitioning, data skew, persistency, MLlib, and best practices.

Related courses (30)

CS-401: Applied data analysis

This course teaches the basic techniques, methodologies, and practical skills required to draw meaningful insights from a variety of data, with the help of the most acclaimed software tools in the data science world (pandas, scikit-learn, Spark, etc.) ...

ENG-209: Data science for engineers with Python

Ce cours propose une immersion progressive et complète dans le domaine de la Data Science à travers le langage Python. Il guide les étudiants depuis la manipulation de données brutes jusqu'à la modélisation et l'extraction de connaissances utiles à partir ...

CS-423: Distributed information systems

This course introduces the foundations of information retrieval, data mining and knowledge bases, which constitute the foundations of today's Web-based distributed information systems. ...

ME-213: Programmation pour ingénieur

Mettre en pratique les bases de la programmation vues au semestre précédent. Développer un logiciel structuré. Méthode de debug d'un logiciel. Introduction à la programmation scientifique. Introduction à l'instrumentation virtuelle. ...

PHYS-467: Machine learning for physicists

Machine learning and data analysis are becoming increasingly central in sciences including physics. In this course, fundamental principles and methods of machine learning will be introduced and practised. ...