Channel capacity

Channel capacity, in electrical engineering, computer science, and information theory, is the tight upper bound on the rate at which information can be reliably transmitted over a communication channel. Following the terms of the noisy-channel coding theorem, the channel capacity of a given channel is the highest information rate (in units of information per unit time) that can be achieved with arbitrarily small error probability. Information theory, developed by Claude E. Shannon in 1948, defines the notion of channel capacity and provides a mathematical model by which it may be computed. The key result states that the capacity of the channel, as defined above, is given by the maximum of the mutual information between the input and output of the channel, where the maximization is with respect to the input distribution. The notion of channel capacity has been central to the development of modern wireline and wireless communication systems, with the advent of novel error correction coding mechanisms that have resulted in achieving performance very close to the limits promised by channel capacity. The basic mathematical model for a communication system is the following: where: is the message to be transmitted; is the channel input symbol ( is a sequence of symbols) taken in an alphabet ; is the channel output symbol ( is a sequence of symbols) taken in an alphabet ; is the estimate of the transmitted message; is the encoding function for a block of length ; is the noisy channel, which is modeled by a conditional probability distribution; and, is the decoding function for a block of length . Let and be modeled as random variables. Furthermore, let be the conditional probability distribution function of given , which is an inherent fixed property of the communication channel. Then the choice of the marginal distribution completely determines the joint distribution due to the identity which, in turn, induces a mutual information . The channel capacity is defined as where the supremum is taken over all possible choices of .

Ontology

Official source

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Related exercises (4)

Irregular tube

An incompressible liquid circulates in a tube. At a certain location, the section of the tube strongly decreases (cf. figure). The continuity equation indicates us that :

\frac{\partial\rho}{\partial t} + \nabla\cdot \rho \vec{v} = 0~.

Show that this is equivalent to the simpler equation:

\quad \langle v_n\rangle=\frac{1}{S_n}\iint_{S_n}\vec{v}\cdot d\vec{\sigma} ~.$$ If we add to the figure the velocity vectors as well as the vect

Excitation of a vertical harmonic oscillator

A mass m is suspended on a spring of natural length l₀ and of elastic constant k. The spring is attached to a piston. The vertical motion of the piston is described by the equation s(t)u2004=u2004s₀cosu2006(ωt)xa0.

Give explicit expressions for the external forces and determine the equation of motion.
Show that the equation of motion can be recast as, $\ddot{x}\left(t\right) + c_1\,x\left(t\right) = c_1\,s\left(t\right)\ ,$ where x(t)u2004=u2004z(t)u2005+u2005c₂ with z(t) the vertica

The crow and the fox

A few days after the events narrated in the famous fable[1], the crow and the fox meet again. Having learned from its mistakes, the crow is determined not to be fooled again by the fox’s flattery. The fox, however, is set on getting a second piece of cheese. This time , it chooses a more direct method and throws a stone at the crow to scare it. It works beyond expectations: as soon as the stone is thrown, the crow drops the cheese. The crow is perched on a tree of height H; the fox is a distance L from the

Model of elastic collision

A chariot considered as a material point of mass m moves without friction on an horizontal plane; a spring of constant k is mounted at one end of the chariot. Through this spring, the chariot that has a speed v₀, collides with a fixed wall (considered as having an infinite mass); the spring first contracts, then expands, and the chariot moves in the opposite direction. We assume that the collision is perfectly elastic, i.e. the mechanical energy is conserved.

Determine the time interval Δt during which

Related notebooks (3)

Understanding the Haar Basis for Grayscale Image Compression

A Basis for Grayscale Images[1]¶

                                      [2]Paolo Prandoni, [3]LCAV, EPFL

The concept of "basis" for a vector space is one of the most profound and fruitful ideas in linear algebra. Basis vectors are the "building blocks" that all other vectors are built from; by "disassembling" a vector into a linear combination of basis elements we are able to "look inside" the vector and highlight many of its relevant features. A vector space has an infinite number o

Advantages of Digital Transmission over Analog Transmission

[1]Paolo Prandoni, [2]EPFL Analog vs Digital Transmission In this notebook we will explore the advantages of digital transmission over analog transmission. We will model the case of transmission over a long (e.g. transoceanic) cable in which several repeaters are used to compensate for the attenuation incurred by the signal. In [1]: import matplotlib import matplotlib.pyplot as plt import numpy as np import IPython from scipy.io import wavfile plt.rcParams["figure.figsize"] = (14,4)

Fairness Week 2: COMPAS and Fairness in Machine Learning – Solutions

Fairness Week 2: Predicting recidivism¶{.anchor-link} {#Fairness-Week-2:-Predicting-recidivism}

How to use this notebook Simply read the text and follow the instructions.
This notebook contains code cells, which can be modified and must be executed to see the result of their content.
To execute a cell, select it and click on the play button (▶) in the tool bar, or type Shift + Enter or Ctr + Enter. As the variables contained in a cell are stor

Related exercises (4)

Irregular tube

An incompressible liquid circulates in a tube. At a certain location, the section of the tube strongly decreases (cf. figure). The continuity equation indicates us that :

\frac{\partial\rho}{\partial t} + \nabla\cdot \rho \vec{v} = 0~.

Show that this is equivalent to the simpler equation:

\quad \langle v_n\rangle=\frac{1}{S_n}\iint_{S_n}\vec{v}\cdot d\vec{\sigma} ~.$$ If we add to the figure the velocity vectors as well as the vect

Excitation of a vertical harmonic oscillator

Give explicit expressions for the external forces and determine the equation of motion.
Show that the equation of motion can be recast as, $\ddot{x}\left(t\right) + c_1\,x\left(t\right) = c_1\,s\left(t\right)\ ,$ where x(t)u2004=u2004z(t)u2005+u2005c₂ with z(t) the vertica

The crow and the fox

Model of elastic collision

Determine the time interval Δt during which

Channel capacity

Chat with Graph Search

Fairness Week 2: Predicting recidivism¶{.anchor-link} {#Fairness-Week-2:-Predicting-recidivism}

Fairness Week 2: Predicting recidivism¶{.anchor-link} {#Fairness-Week-2:-Predicting-recidivism}

Informative yet compact monitoring for building retrofits: Joint information entropy–based measurement system design applied to a real office building

A Finite-Sample Strong Converse for Binary Hypothesis Testing via (Reverse) Rényi Divergence

Contraction of Rényi Divergences for Discrete Channels: Properties and Applications