Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Data Compression and Shannon's Theorem

In course

CS-119(d): Information, Computation, Communication

L'objectif de ce cours est d'introduire les étudiants à la pensée algorithmique, de les familiariser avec les fondamentaux de l'Informatique et de développer une première compétence en programmation (langage C++). ...

Description

This lecture discusses lossless data compression, where all letters are coded without exceptions, and the average code length is always greater than or equal to entropy. It explores the relationship between compression, entropy, and data loss, highlighting the threshold effect and the importance of not compressing below the entropy level.

Instructor

Jean-Cédric Chappelier

Jean-Cédric Chappelier is currently senior researcher and lecturer in Computational Linguistics at the Ecole Polytechnique Fédérale de Lausanne (Switzerland). He holds an Ms.Sci. and a Ph.D. in Computer Science from the Ecole Nationale Supérieure des Télécommunications de Paris (France).

Official source

Related concepts (18)

Lossless compression

Lossless compression is a class of data compression that allows the original data to be perfectly reconstructed from the compressed data with no loss of information. Lossless compression is possible because most real-world data exhibits statistical redundancy. By contrast, lossy compression permits reconstruction only of an approximation of the original data, though usually with greatly improved compression rates (and therefore reduced media sizes).

Data compression

In information theory, data compression, source coding, or bit-rate reduction is the process of encoding information using fewer bits than the original representation. Any particular compression is either lossy or lossless. Lossless compression reduces bits by identifying and eliminating statistical redundancy. No information is lost in lossless compression. Lossy compression reduces bits by removing unnecessary or less important information.

Lossy compression

In information technology, lossy compression or irreversible compression is the class of data compression methods that uses inexact approximations and partial data discarding to represent the content. These techniques are used to reduce data size for storing, handling, and transmitting content. The different versions of the photo of the cat on this page show how higher degrees of approximation create coarser images as more details are removed. This is opposed to lossless data compression (reversible data compression) which does not degrade the data.

Lossless JPEG

Lossless JPEG is a 1993 addition to JPEG standard by the Joint Photographic Experts Group to enable lossless compression. However, the term may also be used to refer to all lossless compression schemes developed by the group, including JPEG 2000 and JPEG-LS. Lossless JPEG was developed as a late addition to JPEG in 1993, using a completely different technique from the lossy JPEG standard. It uses a predictive scheme based on the three nearest (causal) neighbors (upper, left, and upper-left), and entropy coding is used on the prediction error.

Audio coding format

An audio coding format (or sometimes audio compression format) is a content representation format for storage or transmission of digital audio (such as in digital television, digital radio and in audio and video files). Examples of audio coding formats include MP3, AAC, Vorbis, FLAC, and Opus. A specific software or hardware implementation capable of audio compression and decompression to/from a specific audio coding format is called an audio codec; an example of an audio codec is LAME, which is one of several different codecs which implements encoding and decoding audio in the MP3 audio coding format in software.

Related lectures (33)

Data Compression: Entropy Definition

Explores data compression through entropy definition, types, and practical examples, illustrating its role in efficient information storage and transmission.

Data Compression and Shannon-Fano Algorithm

Explores the Shannon-Fano algorithm for data compression and its efficiency in creating unique binary codes for letters.

Entropy and Compression I

Explores entropy theory, compression without loss, and the efficiency of the Shannon-Fano algorithm in data compression.

Data Compression and Entropy: Conclusion

Covers the definition of entropy, Shannon–Fano algorithm, and upcoming topics.

Compression: Prefix-Free Codes