Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Data Compression and Shannon's Theorem: Huffman Codes

In course

CS-119(d): Information, Computation, Communication

L'objectif de ce cours est d'introduire les étudiants à la pensée algorithmique, de les familiariser avec les fondamentaux de l'Informatique et de développer une première compétence en programmation (langage C++). ...

Description

This lecture discusses the performance analysis of Shannon-Fano algorithm, showing that it is close to the optimal performance. It introduces Huffman codes, which offer excellent compression rates and are optimal for lossless compression. The lecture explains the simple idea behind Huffman codes, where longer codes are assigned to less frequent letters. It also presents an example of Huffman coding, highlighting the construction process and the optimal nature of Huffman codes. The lecture concludes by emphasizing the importance of starting with the least frequent letters when constructing Huffman codes.

Instructor

Jean-Cédric Chappelier

Jean-Cédric Chappelier is currently senior researcher and lecturer in Computational Linguistics at the Ecole Polytechnique Fédérale de Lausanne (Switzerland). He holds an Ms.Sci. and a Ph.D. in Computer Science from the Ecole Nationale Supérieure des Télécommunications de Paris (France).

Official source

Related concepts (20)

Huffman coding

In computer science and information theory, a Huffman code is a particular type of optimal prefix code that is commonly used for lossless data compression. The process of finding or using such a code is Huffman coding, an algorithm developed by David A. Huffman while he was a Sc.D. student at MIT, and published in the 1952 paper "A Method for the Construction of Minimum-Redundancy Codes". The output from Huffman's algorithm can be viewed as a variable-length code table for encoding a source symbol (such as a character in a file).

Noisy-channel coding theorem

In information theory, the noisy-channel coding theorem (sometimes Shannon's theorem or Shannon's limit), establishes that for any given degree of noise contamination of a communication channel, it is possible to communicate discrete data (digital information) nearly error-free up to a computable maximum rate through the channel. This result was presented by Claude Shannon in 1948 and was based in part on earlier work and ideas of Harry Nyquist and Ralph Hartley.

Lossless compression

Lossless compression is a class of data compression that allows the original data to be perfectly reconstructed from the compressed data with no loss of information. Lossless compression is possible because most real-world data exhibits statistical redundancy. By contrast, lossy compression permits reconstruction only of an approximation of the original data, though usually with greatly improved compression rates (and therefore reduced media sizes).

Data compression

In information theory, data compression, source coding, or bit-rate reduction is the process of encoding information using fewer bits than the original representation. Any particular compression is either lossy or lossless. Lossless compression reduces bits by identifying and eliminating statistical redundancy. No information is lost in lossless compression. Lossy compression reduces bits by removing unnecessary or less important information.

Lossy compression

In information technology, lossy compression or irreversible compression is the class of data compression methods that uses inexact approximations and partial data discarding to represent the content. These techniques are used to reduce data size for storing, handling, and transmitting content. The different versions of the photo of the cat on this page show how higher degrees of approximation create coarser images as more details are removed. This is opposed to lossless data compression (reversible data compression) which does not degrade the data.

Related lectures (31)

Compression: Prefix-Free Codes

Explains prefix-free codes for efficient data compression and the significance of uniquely decodable codes.

Data Compression and Shannon's Theorem Summary

Summarizes Shannon's theorem, emphasizing the importance of entropy in data compression.

Entropy and Compression I

Explores entropy theory, compression without loss, and the efficiency of the Shannon-Fano algorithm in data compression.

Data Compression and Entropy: Conclusion

Covers the definition of entropy, Shannon–Fano algorithm, and upcoming topics.

Data Compression and Shannon's Theorem: Lossy Compression