Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Untitled

In course

CS-119(g): Information, Computation, Communication

L'objectif de ce cours est d'initier les étudiants à la pensée algorithmique, de les familiariser avec les fondamentaux de l'informatique et des communications et de développer une première compétence en programmation (langage C++). ...

Instructors (2)

Jamila Sam

Jamila Sam is currently senior researcher and lecturer at the Ecole Polytechnique Fédérale de Lausanne (Switzerland).

Barbara Jobstmann

Official source

https://mediaspace.epfl.ch/media/0_8yykgvrs

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related concepts (32)

Entropy

Entropy is a scientific concept, as well as a measurable physical property, that is most commonly associated with a state of disorder, randomness, or uncertainty. The term and the concept are used in diverse fields, from classical thermodynamics, where it was first recognized, to the microscopic description of nature in statistical physics, and to the principles of information theory.

Originalism

In the context of United States law, originalism is a theory of constitutional interpretation that asserts that all statements in the Constitution must be interpreted based on the original understanding. This concept views the Constitution as stable from the time of enactment and that the meaning of its contents can be changed only by the steps set out in Article Five. This notion stands in contrast to the concept of the Living Constitution, which asserts that the Constitution should be interpreted based on the context of current times, even if such interpretation is different from the original interpretations of the document.

Entropy (information theory)

In information theory, the entropy of a random variable is the average level of "information", "surprise", or "uncertainty" inherent to the variable's possible outcomes. Given a discrete random variable , which takes values in the alphabet and is distributed according to : where denotes the sum over the variable's possible values. The choice of base for , the logarithm, varies for different applications. Base 2 gives the unit of bits (or "shannons"), while base e gives "natural units" nat, and base 10 gives units of "dits", "bans", or "hartleys".

Rényi entropy

In information theory, the Rényi entropy is a quantity that generalizes various notions of entropy, including Hartley entropy, Shannon entropy, collision entropy, and min-entropy. The Rényi entropy is named after Alfréd Rényi, who looked for the most general way to quantify information while preserving additivity for independent events. In the context of fractal dimension estimation, the Rényi entropy forms the basis of the concept of generalized dimensions. The Rényi entropy is important in ecology and statistics as index of diversity.

Cross-entropy

In information theory, the cross-entropy between two probability distributions and over the same underlying set of events measures the average number of bits needed to identify an event drawn from the set if a coding scheme used for the set is optimized for an estimated probability distribution , rather than the true distribution . The cross-entropy of the distribution relative to a distribution over a given set is defined as follows: where is the expected value operator with respect to the distribution .

Related lectures (51)

Data Compression: Entropy Definition

Explores data compression through entropy definition, types, and practical examples, illustrating its role in efficient information storage and transmission.

Data Compression and Shannon-Fano Algorithm

Explores the Shannon-Fano algorithm for data compression and its efficiency in creating unique binary codes for letters.

Entropy and Compression I

Explores entropy theory, compression without loss, and the efficiency of the Shannon-Fano algorithm in data compression.

Generative Models: Boltzmann Machine

Covers generative models, focusing on Boltzmann machines and constrained maximization using Lagrange multipliers.

Entropy and Algorithms