Lecture

Quantifying Entropy in Neuroscience Data

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

BIO-369: Randomness and information in biological data

Biology is becoming more and more a data science, as illustrated by the explosion of available genome sequences. This course aims to show how we can make sense of such data and harness it in order to understand biological processes in a quantitative way. ...

Description

This lecture explores the concept of entropy in neuroscience data, focusing on quantifying randomness and information in biological data. The instructor discusses the response of a neuron to sensory input, statistical dependence, and inferring probability distributions from data. Through examples of binary representations of neuron activity and experimental setups, the lecture delves into how sequences of spikes convey information from stimuli. The process of discretizing signals to obtain binary counts of spikes is explained, along with the selection of time intervals for accurate representation. The lecture concludes with a discussion on the maximum entropy of sequences of binary digits and their implications in understanding sensory perception.

Instructor

Anne-Florence Raphaëlle Bitbol

I was trained in physics, and always had a strong interest in biology. I am broadly interested in understanding biological phenomena in a quantitative way, through physical concepts and mathematical and computational tools. I investigate the relative impacts of optimization and historical contingency in biological evolution, both at the scale of protein sequences and at the scale of microbial populations.Here is my research group website.Curriculum vitae Here is my complete CV. Professional experience:Since

Official source

Ontological neighbourhood

Physics

Thermodynamics: Statistical mechanics

Related concepts (32)

Entropy (information theory)

In information theory, the entropy of a random variable is the average level of "information", "surprise", or "uncertainty" inherent to the variable's possible outcomes. Given a discrete random variable , which takes values in the alphabet and is distributed according to : where denotes the sum over the variable's possible values. The choice of base for , the logarithm, varies for different applications. Base 2 gives the unit of bits (or "shannons"), while base e gives "natural units" nat, and base 10 gives units of "dits", "bans", or "hartleys".

Information theory

Information theory is the mathematical study of the quantification, storage, and communication of information. The field was originally established by the works of Harry Nyquist and Ralph Hartley, in the 1920s, and Claude Shannon in the 1940s. The field, in applied mathematics, is at the intersection of probability theory, statistics, computer science, statistical mechanics, information engineering, and electrical engineering. A key measure in information theory is entropy.

Algorithmic information theory

Algorithmic information theory (AIT) is a branch of theoretical computer science that concerns itself with the relationship between computation and information of computably generated objects (as opposed to stochastically generated), such as strings or any other data structure. In other words, it is shown within algorithmic information theory that computational incompressibility "mimics" (except for a constant that only depends on the chosen universal programming language) the relations or inequalities found in information theory.

Joint probability distribution

Given two random variables that are defined on the same probability space, the joint probability distribution is the corresponding probability distribution on all possible pairs of outputs. The joint distribution can just as well be considered for any given number of random variables. The joint distribution encodes the marginal distributions, i.e. the distributions of each of the individual random variables. It also encodes the conditional probability distributions, which deal with how the outputs of one random variable are distributed when given information on the outputs of the other random variable(s).

Probability distribution

In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon in terms of its sample space and the probabilities of events (subsets of the sample space). For instance, if X is used to denote the outcome of a coin toss ("the experiment"), then the probability distribution of X would take the value 0.5 (1 in 2 or 1/2) for X = heads, and 0.

Related lectures (64)

Lecture: Shannon

Covers the basics of information theory, focusing on Shannon's setting and channel transmission.

Entropy in Neuroscience and Ecology

Delves into entropy in neuroscience data and ecology, exploring the representation of sensory information and the diversity of biological populations.

Source Coding Theorems: Entropy and Source Models

Covers source coding theorems, entropy, and various source models in information theory.

Information Measures: Entropy and Information Theory

Explains how entropy measures uncertainty in a system based on possible outcomes.

Entropy and Mutual Information