Lecture

Operational Analytics: Fresh Data Management Techniques

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture discusses the concept of operational analytics and the importance of fresh data in analytical processing. The instructor introduces the STEP (Hybrid Transactional Analytical Processing) approach, which focuses on maintaining data freshness while ensuring performance in transactional analytics. The lecture highlights the challenges faced by existing systems that either sacrifice data freshness or performance due to static assumptions about data access. The instructor presents the adaptive STEP system, which optimizes the trade-off between freshness and performance dynamically at runtime. This system is built into an open-source database called Produce, which does not rely on fixed assumptions about data freshness. Instead, it adapts to workload requirements through elastic resource scheduling and workload-driven access methods. The lecture concludes by emphasizing how adaptive STEP enhances operational analytics by allowing for efficient access to fresh data, thus improving overall system performance.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Official source

Ontological neighbourhood

Computer engineering

High-performance computing: Distributed computing

Related concepts (32)

Concurrency control

In information technology and computer science, especially in the fields of computer programming, operating systems, multiprocessors, and databases, concurrency control ensures that correct results for concurrent operations are generated, while getting those results as quickly as possible. Computer systems, both software and hardware, consist of modules, or components. Each component is designed to operate correctly, i.e., to obey or to meet certain consistency rules.

Multiversion concurrency control

Multiversion concurrency control (MCC or MVCC), is a concurrency control method commonly used by database management systems to provide concurrent access to the database and in programming languages to implement transactional memory. Without concurrency control, if someone is reading from a database at the same time as someone else is writing to it, it is possible that the reader will see a half-written or inconsistent piece of data.

Object code optimizer

An object code optimizer, sometimes also known as a post pass optimizer or, for small sections of code, peephole optimizer, forms part of a software compiler. It takes the output from the source language compile step - the object code or - and tries to replace identifiable sections of the code with replacement code that is more algorithmically efficient (usually improved speed). The earliest "COBOL Optimizer" was developed by Capex Corporation in the mid 1970s for COBOL.

Optimizing compiler

In computing, an optimizing compiler is a compiler that tries to minimize or maximize some attributes of an executable computer program. Common requirements are to minimize a program's execution time, memory footprint, storage size, and power consumption (the last three being popular for portable computers). Compiler optimization is generally implemented using a sequence of optimizing transformations, algorithms which take a program and transform it to produce a semantically equivalent output program that uses fewer resources or executes faster.

Optimistic concurrency control

Optimistic concurrency control (OCC), also known as optimistic locking, is a concurrency control method applied to transactional systems such as relational database management systems and software transactional memory. OCC assumes that multiple transactions can frequently complete without interfering with each other. While running, transactions use data resources without acquiring locks on those resources. Before committing, each transaction verifies that no other transaction has modified the data it has read.

Related lectures (64)

Transaction Management & Concurrency Control

Explores transaction management, concurrency control, ACID properties, anomalies, and conflict serializability in database systems.

Distributed Transactions: Architectures and Protocols

Explores parallel database architectures and distributed transaction protocols, focusing on coordination, concurrency control, and recovery challenges.

Introduction to Database Systems

Covers the basics of database systems, including data modeling, DBMS, data independence, and the course overview.

Transactional Memory: Hardware Concurrency Control

Explores transactional memory for hardware concurrency control, discussing locking mechanisms, performance trade-offs, and hardware changes.

Architecture of CockroachDB