Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Home

Lecture

Advanced Spark Optimization Techniques: Managing Big Data

About
Privacy
Disclaimer

Graph Chatbot

Related lectures (49)

Handling Data: Intro to Pandas

Introduces the fundamentals of handling data, emphasizing the importance of Pandas and data modeling for effective analysis.

Collaborative Data Science: Tools and Techniques

Introduces collaborative data science tools like Git and Docker, emphasizing teamwork and practical exercises for effective learning.

Big Data Challenges: Scaling to Massive Data

Explores challenges of handling massive data in the era of big data, discussing solutions like MapReduce and Spark.

Data Modeling: Concepts and Applications

Explores data modeling concepts, SQL implementations, and practical applications in handling missing data.

Data Stream Processing: Apache Kafka and Spark

Covers data stream processing with Apache Kafka and Spark, including event time vs processing time, stream processing operations, and stream-stream joins.

Spark DataFrames: Basics and Optimization

Covers the basics of Spark DataFrames, their advantages, performance comparison with RDDs, and practical demos.

Analytics on Data at Rest and Data in Motion

Explores combining data at rest with data in motion, emphasizing the Lambda architecture complexities and quality assessment of streams and batches.

Accelerating Data Analytics: Innovations in Post-Moore Era Log in to Mediaspace to watch this video

Covers advancements in data analytics systems and the role of hardware-software co-design in enhancing performance in the Post-Moore era.

Data Cleaning Challenges: Optimizing Error Detection Log in to Mediaspace to watch this video

Addresses challenges in data cleaning for analysis, proposing optimizations to reduce processing time.

Introduction to Data Science Log in to Mediaspace to watch this video

Introduces the basics of data science, covering decision trees, machine learning advancements, and deep reinforcement learning.

Data Science: Python for Engineers - Part II Log in to Mediaspace to watch this video

Explores data wrangling, numerical data handling, and scientific visualization using Python for engineers.

Water Consumption in Geneva Log in to Mediaspace to watch this video

Explores water consumption data in Geneva, including charts on consumption and losses, available datasets, and data processing phases.

Information Systems: Overview Log in to Mediaspace to watch this video

Covers the overview of information systems, data modeling, managing data, and the distinction between data and information.

Image Processing Basics Log in to Mediaspace to watch this video

Covers the basics of image processing, focusing on writing a program to process images.

Numpy Tutorial Log in to Mediaspace to watch this video

Covers the basics of Numpy, including importing the library, creating arrays, and performing linear algebra operations.

Causal & Total Order Broadcast Log in to Mediaspace to watch this video

Covers exercises on reliable broadcast, uniform reliable broadcast, and total order broadcast in distributed algorithms.

Cost Modelling for Sustainable Innovation Log in to Mediaspace to watch this video

Introduces cost modelling as a tool for sustainable innovation and covers topics such as transport CO2 emissions and material properties.

Deep and Convolutional Networks: Generalization and Optimization Log in to Mediaspace to watch this video

Explores deep and convolutional networks, covering generalization, optimization, and practical applications in machine learning.

Unsupervised Learning: Movie Recommendation Log in to Mediaspace to watch this video

Covers unsupervised learning for movie recommendation using singular value decomposition.

Data Quality in Life Cycle Assessment Log in to Mediaspace to watch this video

Explores data quality in Life Cycle Assessment, covering inventory format, control, measurement procedures, uncertainty factors, and the Data Quality Rating system.

Page 2 of 3