Lecture

Maximal Correlation: Information Measures

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

COM-621: Advanced Topics in Information Theory

The class will focus on information-theoretic progress of the last decade. Topics include: Network Information Theory ; Information Measures: definitions, properties, and applications to probabilistic models. ...

Description

This lecture covers the concept of maximal correlation in information theory, focusing on mutual information and its properties. It discusses the calculation of mutual information for discrete variables, the chain rule, and the concept of maximal correlation as a measure of dependence between random variables. The lecture also delves into Renyi's generalized information measures and their applications in characterizing the relationship between variables. Furthermore, it explores the continuity and concavity of mutual information, providing insights into the mathematical foundations of information theory.

Instructor

Michael Christoph Gastpar

Michael Gastpar is a (full) Professor at EPFL. From 2003 to 2011, he was a professor at the University of California at Berkeley, earning his tenure in 2008. He received his Dipl. El.-Ing. degree from ETH Zürich, Switzerland, in 1997 and his MS degree from the University of Illinois at Urbana-Champaign, IL, USA, in 1999. He defended his doctoral thesis at EPFL on Santa Claus day, 2002. He was also a (full) Professor at Delft University of Technology, The Netherlands. His research interests are in network information theory and related coding and signal processing techniques, with applications to sensor networks and neuroscience. He is a Fellow of the IEEE. He is the co-recipient of the 2013 Communications Society & Information Theory Society Joint Paper Award. He was an Information Theory Society Distinguished Lecturer (2009-2011). He won an ERC Starting Grant in 2010, an Okawa Foundation Research Grant in 2008, an NSF CAREER award in 2004, and the 2002 EPFL Best Thesis Award. He has served as an Associate Editor for Shannon Theory for the IEEE Transactions on Information Theory (2008-11), and as Technical Program Committee Co-Chair for the 2010 International Symposium on Information Theory, Austin, TX.

Official source

Ontological neighbourhood

Information engineering

Information theory: Channel capacity

Related concepts (32)

Information theory

Information theory is the mathematical study of the quantification, storage, and communication of information. The field was originally established by the works of Harry Nyquist and Ralph Hartley, in the 1920s, and Claude Shannon in the 1940s. The field, in applied mathematics, is at the intersection of probability theory, statistics, computer science, statistical mechanics, information engineering, and electrical engineering. A key measure in information theory is entropy.

Interaction information

The interaction information is a generalization of the mutual information for more than two variables. There are many names for interaction information, including amount of information, information correlation, co-information, and simply mutual information. Interaction information expresses the amount of information (redundancy or synergy) bound up in a set of variables, beyond that which is present in any subset of those variables. Unlike the mutual information, the interaction information can be either positive or negative.

Back pain

Back pain is pain felt in the back. It may be classified as neck pain (cervical), middle back pain (thoracic), lower back pain (lumbar) or coccydynia (tailbone or sacral pain) based on the segment affected. The lumbar area is the most common area affected. An episode of back pain may be acute, subacute or chronic depending on the duration. The pain may be characterized as a dull ache, shooting or piercing pain or a burning sensation. Discomfort can radiate to the arms and hands as well as the legs or feet, and may include numbness or weakness in the legs and arms.

Low back pain

Low back pain (LBP) or lumbago is a common disorder involving the muscles, nerves, and bones of the back, in between the lower edge of the ribs and the lower fold of the buttocks. Pain can vary from a dull constant ache to a sudden sharp feeling. Low back pain may be classified by duration as acute (pain lasting less than 6 weeks), sub-chronic (6 to 12 weeks), or chronic (more than 12 weeks). The condition may be further classified by the underlying cause as either mechanical, non-mechanical, or referred pain.

Algorithmic information theory

Algorithmic information theory (AIT) is a branch of theoretical computer science that concerns itself with the relationship between computation and information of computably generated objects (as opposed to stochastically generated), such as strings or any other data structure. In other words, it is shown within algorithmic information theory that computational incompressibility "mimics" (except for a constant that only depends on the chosen universal programming language) the relations or inequalities found in information theory.

Related lectures (40)

Conditional Entropy and Information Theory Concepts

Discusses conditional entropy and its role in information theory and data compression.

Information Measures: Entropy and Information Theory

Explains how entropy measures uncertainty in a system based on possible outcomes.

Gibbs Entropy and Information Theory

Explores Gibbs's entropy, information theory, and the information content of events in non-equiprobable scenarios.

Variational Formulation: Information Measures

Explores variational formulation for measuring information content and divergence between probability distributions.

Huffman Coding