Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Maximum Entropy Modeling: Applications & Inference

In course

BIO-369: Randomness and information in biological data

Biology is becoming more and more a data science, as illustrated by the explosion of available genome sequences. This course aims to show how we can make sense of such data and harness it in order to understand biological processes in a quantitative way. ...

Description

This lecture delves into the concept of maximum entropy modeling, focusing on its applications in neuroscience data and protein sequence data. It covers topics such as quantifying randomness, statistical dependence, inferring probability distributions, and finding relevant dimensions in data through dimension reduction techniques like principal component analysis. The lecture also explores the link between maximum entropy modeling and physics, discussing the inference of probability distributions from data and the prediction of protein structure from sequence data.

Instructor

Anne-Florence Raphaëlle Bitbol

Official source

Related concepts (32)

Dimensionality reduction

Dimensionality reduction, or dimension reduction, is the transformation of data from a high-dimensional space into a low-dimensional space so that the low-dimensional representation retains some meaningful properties of the original data, ideally close to its intrinsic dimension. Working in high-dimensional spaces can be undesirable for many reasons; raw data are often sparse as a consequence of the curse of dimensionality, and analyzing the data is usually computationally intractable (hard to control or deal with).

Principal component analysis

Principal component analysis (PCA) is a popular technique for analyzing large datasets containing a high number of dimensions/features per observation, increasing the interpretability of data while preserving the maximum amount of information, and enabling the visualization of multidimensional data. Formally, PCA is a statistical technique for reducing the dimensionality of a dataset. This is accomplished by linearly transforming the data into a new coordinate system where (most of) the variation in the data can be described with fewer dimensions than the initial data.

Nonlinear dimensionality reduction

Nonlinear dimensionality reduction, also known as manifold learning, refers to various related techniques that aim to project high-dimensional data onto lower-dimensional latent manifolds, with the goal of either visualizing the data in the low-dimensional space, or learning the mapping (either from the high-dimensional space to the low-dimensional embedding or vice versa) itself. The techniques described below can be understood as generalizations of linear decomposition methods used for dimensionality reduction, such as singular value decomposition and principal component analysis.

Joint probability distribution

Given two random variables that are defined on the same probability space, the joint probability distribution is the corresponding probability distribution on all possible pairs of outputs. The joint distribution can just as well be considered for any given number of random variables. The joint distribution encodes the marginal distributions, i.e. the distributions of each of the individual random variables. It also encodes the conditional probability distributions, which deal with how the outputs of one random variable are distributed when given information on the outputs of the other random variable(s).

Probability distribution

In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon in terms of its sample space and the probabilities of events (subsets of the sample space). For instance, if X is used to denote the outcome of a coin toss ("the experiment"), then the probability distribution of X would take the value 0.5 (1 in 2 or 1/2) for X = heads, and 0.

Related lectures (30)

Entropy in Neuroscience and Ecology

Delves into entropy in neuroscience data and ecology, exploring the representation of sensory information and the diversity of biological populations.

Quantifying Randomness and Information in Biological Data

Explores entropy, randomness, and information quantification in biological data analysis, including neuroscience and protein structure prediction.

Conditional Entropy and Information Theory Concepts

Discusses conditional entropy and its role in information theory and data compression.

Mutual Information: Continued

Explores mutual information for quantifying statistical dependence between variables and inferring probability distributions from data.

Quantifying Entropy in Neuroscience Data