Big Data Challenges: Distributed Computing with Spark

About
Privacy
Disclaimer

Graph Chatbot

Related lectures (53)

Scaling up: Spark and Big Data

Explores the challenges of big data processing and introduces Spark as a solution.

Big Data Best Practices and Guidelines

Covers best practices and guidelines for big data, including data lakes, architecture, challenges, and technologies like Hadoop and Hive.

Big Data Challenges: Scaling to Massive Data

Explores challenges of handling massive data in the era of big data, discussing solutions like MapReduce and Spark.

Advanced Spark Optimization Techniques: Managing Big Data

Discusses advanced Spark optimization techniques for managing big data efficiently, focusing on parallelization, shuffle operations, and memory management.

Big Data: Best Practices and Guidelines

Covers best practices and guidelines for big data, including data lakes, typical architecture, challenges, and technologies used to address them.

Introduction to Spark Runtime Architecture

Covers the Spark runtime architecture, including RDDs, transformations, actions, and caching for performance optimization.

Integrating Scalable Data Storage and Map Reduce Processing with Hadoop

Covers the integration of scalable data storage and map reduce processing using Hadoop, including HDFS, Hive, Parquet, ORC, Spark, and HBase.

Big Data Ecosystems: Technologies and Challenges

Covers the fundamentals of big data ecosystems, focusing on technologies, challenges, and practical exercises with Hadoop's HDFS.

Introduction to Spark Runtime Architecture

Introduces Apache Spark, covering its architecture, RDDs, transformations, actions, fault tolerance, deployment options, and practical exercises in Jupyter notebooks.

Introduction to Data Stream Processing: Concepts and Applications

Covers the principles of data stream processing and its applications in real-time data analysis.

Introduction to Spark runtime architecture

Introduces Apache Spark, covering its key features, history, RDDs, architecture, and distributed computing framework.

Data Issues in Research

Explores challenges in data assumptions, biases, and more in research, including incomplete write-ups and frustrations of newcomers.

Data Wrangling Techniques: HBase and Hive Integration

Covers data wrangling techniques using HBase and Hive, focusing on integration and practical applications.

Data Analysis to AI and ML, Social Media

Explores the evolution from data analysis to AI and ML, emphasizing big data, machine learning, and social media interaction.

Data Wrangling with Hadoop: Storage Formats and Hive

Explores data wrangling with Hadoop, emphasizing storage formats and Hive for big data processing.

Spark Data Frames

Covers Spark Data Frames, distributed collections of data organized into named columns, and the benefits of using them over RDDs.

Regulations: Figures of Regulations

Covers the analysis of ECG heart rate data and respiratory flow measurements using Excel.

Digital Transformation: Solutions and Data

Explores digital transformation opportunities, big data, analytics, and technology innovations in business and research.

Big Data: Processing and Dimensions

Explores Big Data generation, storage, processing, and dimensions, along with challenges in data analytics, cloud computing elasticity, and security.

Hadoop Ecosystem: Architectural Choices & MapReduce Programming Log in to Mediaspace to watch this video

Explores the Hadoop ecosystem's architecture and MapReduce programming model, emphasizing strengths and limitations.

Page 1 of 3