Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Mutual Information: Continued

In course

BIO-369: Randomness and information in biological data

Biology is becoming more and more a data science, as illustrated by the explosion of available genome sequences. This course aims to show how we can make sense of such data and harness it in order to understand biological processes in a quantitative way. ...

Description

This lecture delves into the concept of mutual information, focusing on quantifying statistical dependence between variables. It covers the definition of mutual information, its properties, and its role in identifying coevolving sites in interacting proteins. The lecture also explores inferring probability distributions from data, including model selection, parameter estimation, and predicting protein structure from sequence data.

Instructor

Anne-Florence Raphaëlle Bitbol

Official source

Ontological neighbourhood

Statistics

Statistical inference: Mathematical statistics

Information engineering

Information theory: Channel capacity

Related concepts (32)

Mutual information

In probability theory and information theory, the mutual information (MI) of two random variables is a measure of the mutual dependence between the two variables. More specifically, it quantifies the "amount of information" (in units such as shannons (bits), nats or hartleys) obtained about one random variable by observing the other random variable. The concept of mutual information is intimately linked to that of entropy of a random variable, a fundamental notion in information theory that quantifies the expected "amount of information" held in a random variable.

Statistical inference

Statistical inference is the process of using data analysis to infer properties of an underlying distribution of probability. Inferential statistical analysis infers properties of a population, for example by testing hypotheses and deriving estimates. It is assumed that the observed data set is sampled from a larger population. Inferential statistics can be contrasted with descriptive statistics. Descriptive statistics is solely concerned with properties of the observed data, and it does not rest on the assumption that the data come from a larger population.

Maximum likelihood estimation

In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed statistical model, the observed data is most probable. The point in the parameter space that maximizes the likelihood function is called the maximum likelihood estimate. The logic of maximum likelihood is both intuitive and flexible, and as such the method has become a dominant means of statistical inference.

Rényi entropy

In information theory, the Rényi entropy is a quantity that generalizes various notions of entropy, including Hartley entropy, Shannon entropy, collision entropy, and min-entropy. The Rényi entropy is named after Alfréd Rényi, who looked for the most general way to quantify information while preserving additivity for independent events. In the context of fractal dimension estimation, the Rényi entropy forms the basis of the concept of generalized dimensions. The Rényi entropy is important in ecology and statistics as index of diversity.

Information theory

Information theory is the mathematical study of the quantification, storage, and communication of information. The field was originally established by the works of Harry Nyquist and Ralph Hartley, in the 1920s, and Claude Shannon in the 1940s. The field, in applied mathematics, is at the intersection of probability theory, statistics, computer science, statistical mechanics, information engineering, and electrical engineering. A key measure in information theory is entropy.

Related lectures (56)

Quantifying Statistical Dependence

Delves into quantifying statistical dependence through covariance, correlation, and mutual information.

Mutual Information in Biological Data

Explores mutual information in biological data, emphasizing its role in quantifying statistical dependence and analyzing protein sequences.

Quantifying Statistical Dependence: Covariance and Correlation

Explores covariance, correlation, and mutual information in quantifying statistical dependence between random variables.

Variational Formulation: Information Measures

Explores variational formulation for measuring information content and divergence between probability distributions.

Quantifying Randomness and Information in Biological Data