Lecture

Big Data Best Practices and Guidelines

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Related lectures (64)

Real-time Intelligence: Data Challenges and Hardware Evolution

Explores data challenges and hardware evolution for real-time intelligence in the era of big data.

Principal Component Analysis: Dimension Reduction

Covers Principal Component Analysis for dimension reduction in biological data, focusing on visualization and pattern identification.

Handling Data: Data Models and Wrangling

Explores data handling fundamentals, including models, sources, and wrangling, emphasizing the importance of understanding and addressing data problems.

Advanced Spark Optimization

Delves into advanced Spark optimization techniques, emphasizing data partitioning, shuffle operations, and memory management.

Handling Data: Intro to Pandas

Introduces the fundamentals of handling data, emphasizing the importance of Pandas and data modeling for effective analysis.

Spark DataFrames: Basics and Optimization

Covers the basics of Spark DataFrames, their advantages, performance comparison with RDDs, and practical demos.

Introduction to Database Systems

Covers the fundamentals of database systems, including data modeling, information processing, and the challenges of managing large volumes of data.

Hadoop Ecosystem: Architectural Choices & MapReduce Programming

Explores the Hadoop ecosystem's architecture and MapReduce programming model, emphasizing strengths and limitations.

Data Warehousing: Overview and Challenges

Introduces data warehousing fundamentals, challenges, and the innovative concept of a 'lakehouse'.

ServiceNow: Creating Dashboards and Reports

Explores creating dashboards in ServiceNow, focusing on advantages, transition from homepages, and important concepts like tasks and incidents.

Introduction to R Programming for Genetics & Genomics

Introduces a course on Genetics & Genomics, focusing on R programming with interactive exercises.

Python Lists: Manipulation and Comprehension

Covers Python list manipulation and comprehension, emphasizing memory representation and mutability.

Total scattering and PDF analysis

Explores total scattering and PDF analysis in materials science, covering in-situ synthesis, data analysis techniques, and applications in host-guest systems.

Data Warehouses and Data Lakes

Covers data warehouses, data lakes, OLTP vs. OLAP, data quality, and the Data Lakehouse concept.

Introduction to Data Stream Processing: Concepts and Applications

Covers data stream processing concepts, focusing on Apache Kafka and Spark Streaming integration, event time management, and project implementation guidelines.

Scaling up: Spark and Big Data

Explores the challenges of big data processing and introduces Spark as a solution.

Collaborative Data Science

Covers collaborative data science tools, big data concepts, Spark, and data stream processing, with tips for the final project.

Singular Value Decomposition

Explores Singular Value Decomposition and its role in unsupervised learning and dimensionality reduction, emphasizing its properties and applications.

Big Data Challenges: Distributed Computing with Spark

Explores big data challenges, distributed computing with Spark, RDDs, hardware requirements, MapReduce, transformations, and Spark DataFrames.

Data Wrangling and Analysis

Covers a homework assignment on data wrangling and analysis using Python's pandas library for real-world datasets.