Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Lossless Compression: Shannon-Fano and Huffman

In course

CS-119(a): Information, Computation, Communication

D'une part, le cours aborde: (1) la notion d'algorithme et de représentation de l'information, (2) l'échantillonnage d'un signal et la compression de données et (3) des aspects liés aux systèmes: ordinateur, mémoire, etc. D'autre part, le cours donne une i ...

Description

This lecture covers the principles of lossless compression, focusing on exploiting data redundancy to shorten sequences efficiently. It introduces the Shannon-Fano algorithm, which divides letters based on their frequency, and the Huffman algorithm, which assigns variable-length codes to optimize compression. By comparing the two methods, the instructor demonstrates how Huffman coding outperforms Shannon-Fano in terms of efficiency and speed. The lecture also delves into the concept of entropy, calculating the entropy of a given sequence to highlight the effectiveness of Huffman coding in minimizing the average number of bits per letter.

In MOOC

Information, Computation, Communication: Introduction to computational thinking

Instructors (3)

Dan-Cristian Tomozei

Mirjana Stojilovic

Mirjana Stojilović received the Dipl. Ing. and Ph.D. degrees from the School of Electrical Engineering, University of Belgrade, in 2006 and 2013, respectively. From 2010 to 2013, she was collaborating with the Processor Architecture Laboratory at EPFL, visiting periodically as a Guest Researcher. From 2013 to 2016, she was working at the University of Applied Sciences Western Switzerland as a senior researcher, and at EPFL as a lecturer. She joined Parallel Systems Architecture Lab at EPFL in October 2016. Mirjana's main research interests include electronic design automation, reconfigurable computing, electromagnetic-compatibility and signal-integrity issues, and hardware security. Mirjana Stojilović serves on the program committee of the FPGA, FPL, and FCCM conferences and as a reviewer for IEEE TCAD, TVLSI, TC, TEMC, IEEE Access and ACM TRETS. She received the Best Paper Award at 2016 International Symposium on Electromagnetic Compatibility (EMC Europe 2016), Young Scientist Award at 33rd International Conference on Lightning Protection (ICLP2016), and the Young Author Best Paper Award at the 20th Telecommunication Forum in Belgrade (TELFOR 2012). In 2015, the EPFL School of Computer and Communication Sciences (IC) presented her with the Teaching Award.

Olivier Lévêque

Olivier Lévêque was born in Switzerland in 1971. He received the physics diploma from EPFL in 1995 and completed his PhD in mathematics at EPFL in 2001. Since then, he has been with the Laboratory of Information Theory at EPFL. He spent the academical year 2005-2006 at the Electrical Engineering Department of Stanford University, where he was appointed as lecturer. His research interests include stochastic analysis, random matrices, wireless communications and information theory.

Official source

Quiz

Related concepts (32)

Data compression

In information theory, data compression, source coding, or bit-rate reduction is the process of encoding information using fewer bits than the original representation. Any particular compression is either lossy or lossless. Lossless compression reduces bits by identifying and eliminating statistical redundancy. No information is lost in lossless compression. Lossy compression reduces bits by removing unnecessary or less important information.

Huffman coding

In computer science and information theory, a Huffman code is a particular type of optimal prefix code that is commonly used for lossless data compression. The process of finding or using such a code is Huffman coding, an algorithm developed by David A. Huffman while he was a Sc.D. student at MIT, and published in the 1952 paper "A Method for the Construction of Minimum-Redundancy Codes". The output from Huffman's algorithm can be viewed as a variable-length code table for encoding a source symbol (such as a character in a file).

Lossless compression

Lossless compression is a class of data compression that allows the original data to be perfectly reconstructed from the compressed data with no loss of information. Lossless compression is possible because most real-world data exhibits statistical redundancy. By contrast, lossy compression permits reconstruction only of an approximation of the original data, though usually with greatly improved compression rates (and therefore reduced media sizes).

Information theory

Information theory is the mathematical study of the quantification, storage, and communication of information. The field was originally established by the works of Harry Nyquist and Ralph Hartley, in the 1920s, and Claude Shannon in the 1940s. The field, in applied mathematics, is at the intersection of probability theory, statistics, computer science, statistical mechanics, information engineering, and electrical engineering. A key measure in information theory is entropy.

Image compression

Image compression is a type of data compression applied to s, to reduce their cost for storage or transmission. Algorithms may take advantage of visual perception and the statistical properties of image data to provide superior results compared with generic data compression methods which are used for other digital data. Image compression may be lossy or lossless. Lossless compression is preferred for archival purposes and often for medical imaging, technical drawings, clip art, or comics.

Related lectures (28)

Compression: Prefix-Free Codes

Explains prefix-free codes for efficient data compression and the significance of uniquely decodable codes.

Data Compression and Shannon-Fano Algorithm

Explores the Shannon-Fano algorithm for data compression and its efficiency in creating unique binary codes for letters.

Entropy and Compression I

Explores entropy theory, compression without loss, and the efficiency of the Shannon-Fano algorithm in data compression.

Data Compression: Entropy Definition

Explores data compression through entropy definition, types, and practical examples, illustrating its role in efficient information storage and transmission.

Entropy and Data Compression: Huffman Coding Techniques