Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Concept

Information theory

Related lectures (31)

Variational Formulation: Information Measures

Explores variational formulation for measuring information content and divergence between probability distributions.

Lecture: Shannon

Covers the basics of information theory, focusing on Shannon's setting and channel transmission.

Information Theory: Review and Mutual Information

Reviews information measures like entropy and introduces mutual information as a measure of information between random variables.

Information Measures: Entropy and Information Theory

Explains how entropy measures uncertainty in a system based on possible outcomes.

Building a Decision Tree

Covers building decision trees to classify mushrooms as poisonous or not.

Quantum Entropy

Covers quantum entropy, density matrices, and entanglement in quantum information theory.

Probability Theory: Joint Marginals and Granger Causality

Covers joint marginals and Granger causality in probability theory, explaining their implications in predicting outcomes.

Interpretation of Entropy

Explores the concept of entropy expressed in bits and its relation to probability distributions, focusing on information gain and loss in various scenarios.

Quantifying Randomness and Information in Biological Data

Explores entropy, randomness, and information quantification in biological data analysis, including neuroscience and protein structure prediction.

Information Theory: Quantifying Messages and Source Entropy

Covers quantifying information in messages, source entropy, common information, and communication channel capacity.

Introduction to Signals and Filters

Covers the conversion of analog signals to digital, data compression, and signal reconstruction, highlighting the significance of signal processing in communication systems.

Signal Representation

Discusses signal representation, focusing on mathematical expressions and inequalities in signal processing.

Random Walks and Moran Model in Population Genetics

Explores random walks, Moran model, bacterial chemotaxis, entropy, information theory, and coevolving sites in proteins.

Entropy: Examples and Properties MOOC: Information, Computation, Communication: Introduction to computational thinking

Explores examples of guessing letters, origins of entropy, and properties in information theory.

Mutual Information and Entropy

Explores mutual information and entropy calculation between random variables.

Introduction: Course Structure and Fundamentals of Computing

Explores the role of Computing in society and the basics of computing, algorithms, communication systems, and computer security.

Information in Networked Systems: Functional Representation and Data Compression

Explores traditional information theory, data compression, data transmission, and functional representation lemmas in networked systems.

Understanding Computer Information

Explores how information is coded, stored, and processed within computers, aiming to introduce computer thinking to a broad audience.

Entropy and Mutual Information

On entropy and mutual information explores quantifying information in data science through probability distributions.

Information Measures

Covers information measures like entropy, Kullback-Leibler divergence, and data processing inequality, along with probability kernels and mutual information.