Lecture

High-order Correlations in Massive Datasets

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture by the instructor explores the discovery of high-order correlations in massive datasets, discussing the natural occurrence of correlations in various fields such as finance, health, and genomics. The presentation delves into the extension of pairwise correlations to account for multiple variables concurrently, highlighting the importance of high-order correlations and the challenges in computing them efficiently.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Official source

Related concepts (32)

Correlation

In statistics, correlation or dependence is any statistical relationship, whether causal or not, between two random variables or bivariate data. Although in the broadest sense, "correlation" may indicate any type of association, in statistics it usually refers to the degree to which a pair of variables are linearly related. Familiar examples of dependent phenomena include the correlation between the height of parents and their offspring, and the correlation between the price of a good and the quantity the consumers are willing to purchase, as it is depicted in the so-called demand curve.

Pearson correlation coefficient

In statistics, the Pearson correlation coefficient (PCC) is a correlation coefficient that measures linear correlation between two sets of data. It is the ratio between the covariance of two variables and the product of their standard deviations; thus, it is essentially a normalized measurement of the covariance, such that the result always has a value between −1 and 1. As with covariance itself, the measure can only reflect a linear correlation of variables, and ignores many other types of relationships or correlations.

Big data

Big data primarily refers to data sets that are too large or complex to be dealt with by traditional data-processing application software. Data with many entries (rows) offer greater statistical power, while data with higher complexity (more attributes or columns) may lead to a higher false discovery rate. Though used sometimes loosely partly because of a lack of formal definition, the interpretation that seems to best describe big data is the one associated with a large body of information that we could not comprehend when used only in smaller amounts.

Health

In common usage and medicine, health, according to the World Health Organization, is "a state of complete physical, mental and social well-being and not merely the absence of disease and infirmity". A variety of definitions have been used for different purposes over time. Health can be promoted by encouraging healthful activities, such as regular physical exercise and adequate sleep, and by reducing or avoiding unhealthful activities or situations, such as smoking or excessive stress.

Data

In common usage and statistics, data (USˈdætə; UKˈdeɪtə) is a collection of discrete or continuous values that convey information, describing the quantity, quality, fact, statistics, other basic units of meaning, or simply sequences of symbols that may be further interpreted formally. A datum is an individual value in a collection of data. Data is usually organized into structures such as tables that provide additional context and meaning, and which may themselves be used as data in larger structures.

Related lectures (64)

Describing Data: Statistics and Hypothesis Testing

Covers descriptive statistics, hypothesis testing, and correlation analysis with various probability distributions and robust statistics.

Red bus/Blue bus paradox

Explores the Red bus/Blue bus paradox, nested logit models, and multivariate extreme value models in transportation.

Dependence Concepts and Copulas

Explores dependence concepts, copulas, correlation fallacies, and rank correlations in statistics.

Genomics and Society

Explores the impact of genomics on health, society, and personalized medicine, addressing ethical, legal, and social implications.

Engineering Multi-Scale Systems: Nano-Tera 2016