Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Course

COM-490: Large-scale data science for real-world data

Lectures in this course (41)

Integrating Scalable Data Storage and Map Reduce Processing with Hadoop

Covers the integration of scalable data storage and map reduce processing using Hadoop, including HDFS, Hive, Parquet, ORC, Spark, and HBase.

Introduction to Spark Runtime Architecture

Covers the Spark runtime architecture, including RDDs, transformations, actions, and caching for performance optimization.

Spark Data Frames: Overview and Performance Analysis

Covers Spark Data Frames, their performance benefits, Catalyst Optimizer, PySpark integration, and Gutenberg corpus analysis.

Advanced Spark: Partitioning and Optimization

Explores advanced Spark topics like partitioning strategies, memory optimization, and shuffle operations for efficient job execution.

Introduction to Data Stream Processing

Covers the fundamentals of data stream processing, including real-time insights, industry applications, and practical exercises on Kafka and Spark Streaming.

Advanced Data Stream Processing Concepts

Explores event time vs. processing time, stream processing operations, stream-stream joins, and handling late/out-of-order data in data stream processing.

Advanced Data Stream Processing Concepts

Explores advanced data stream processing concepts using real-time train data from Dutch railways and historical data from Swiss Federal Railways.

Collaborative Data Science: Tools and Techniques

Introduces collaborative data science tools like Git and Docker, emphasizing teamwork and practical exercises for effective learning.

Big Data Ecosystems: Technologies and Challenges

Covers the fundamentals of big data ecosystems, focusing on technologies, challenges, and practical exercises with Hadoop's HDFS.

Data Wrangling Techniques: HBase and Hive Integration

Covers data wrangling techniques using HBase and Hive, focusing on integration and practical applications.

Data Wrangling with Hive: Managing Big Data Efficiently

Covers data wrangling techniques using Apache Hive for efficient big data management.

Data Wrangling with Hadoop: Advanced Techniques

Covers advanced data wrangling techniques using Hadoop, focusing on Hive and HBase integration.

Untitled

Advanced Spark Optimization Techniques: Managing Big Data

Discusses advanced Spark optimization techniques for managing big data efficiently, focusing on parallelization, shuffle operations, and memory management.

Introduction to Data Stream Processing: Concepts and Applications

Covers the principles of data stream processing and its applications in real-time data analysis.

Introduction to Data Stream Processing: Concepts and Applications

Covers data stream processing concepts, focusing on Apache Kafka and Spark Streaming integration, event time management, and project implementation guidelines.

Introduction: What do we mean by Data Science?

Introduces the team, provides a crash course on Python, and explores the journey into Data Science and the importance of refining data.

General Introduction to Data Science

Offers a comprehensive introduction to Data Science, covering Python, Numpy, Pandas, Matplotlib, and Scikit-learn, with a focus on practical exercises and collaborative work.

Collaborative Data Science: Tools and Git Workflow

Explores tools like Git and Docker for collaborative data science projects.