Publications associées à Typical set

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

We derive a conditional version of the classical regret-capacity theorem. This result can be used in universal prediction to find lower bounds on the minimal batch regret, which is a recently introduced generalization of the average regret, when batches of training data are available to the predictor. As an example, we apply this result to the class of binary memoryless sources. Finally, we generalize the theorem to Rényi information measures, revealing a deep connection between the conditional Rényi diverg

With the rapid discovery of emergent phenomena in deep learning and large language models, understanding their cause has become an urgent need. Here, we propose a rigorous entropic-force theory for understanding the learning dynamics of neural networks trained with stochastic gradient descent (SGD) and its variants. Building on the theory of parameter symmetries and an entropic loss landscape, we show that representation learning is crucially governed by emergent entropic forces arising from stochasticity a

This work tests several classification techniques and acoustic features and further combines them using late fusion to classify paralinguistic information for the ComParE 2018 challenge. We use Multiple Linear Regression (MLR) with Ordinary Least Squares ( ...

Spatially coupled codes have been of interest recently owing to their superior performance over memoryless binary-input channels. The performance is good both asymptotically, since the belief propagation thresholds approach the Shannon limit, as well as fo ...

Arikan's 'polar coding' is a technique to achieve the symmetric capacity of binary input memoryless channels. In this talk I will attempt to describe this technique, and briefly discuss its extensions to q-ary input channels, multiple access channels and rate-distortion coding. The underlying principle of polar coding allows one to view randomness from a different vantage. I will try to illustrate this with a recent result of Sasoglu: when a binary ergodic process is transformed by Arikan's 'polar transform

Polar coding is a recent channel coding technique invented by Arikan to achieve the 'symmetric capacity' of binary-input memoryless channels. Subsequently it was observed by Korada and Urbanke that such codes are also good for lossy channel coding, achieving the 'symmetric rate distortion' bound, when the representation alphabet is binary. In this note we extend this result to the case when the representation alphabet is q-ary, for q a prime number.

In this paper lossless compression with polar codes is considered. A polar encoding algorithm is developed and a method to design the code and compute the average compression rate for finite lengths is given. It is shown that the scheme achieves the optimal compression rate asymptotically. Furthermore, the proposed scheme has a very good performance at finite lengths. Both the encoding and decoding operations can be accomplished with complexity O(N logN) where N denotes the length of the code.

Shannon in his seminal work \cite{paper:shannon} formalized the framework on the problem of digital communication of information and storage. He quantified the fundamental limits of compression and transmission rates. The quantity \textit{channel capacity} was coined to give an operational meaning on the ultimate limit of information transfer between two entities, transmitter and receiver. By using a random coding argument, he showed the achievability of reliable communication on any noisy channel as long a

Fountain codes have been successfully employed for reliable and efficient transmission of information via erasure channels with unknown erasure rates. This paper introduces the notion of fountain capacity for arbitrary channels, and shows that it is equal to the conventional Shannon capacity for stationary memoryless channels. In contrast, when the channel is not stationary or has memory, Shannon capacity and fountain capacity need not be equal.

The goal of a class of sensor networks is to monitor an underlying physical reality at the highest possible fidelity. Sensors acquire noisy measurements and have to communicate them over a power- and possibly bandwidth-constrained interference channel to a set of base stations. The goal of this paper is to analyze, as a function of the number of sensors, the trade-offs between the degrees of freedom of the underlying physical reality, the communication resources (power, temporal and spatial bandwidth), and

Sensors acquire data, and communicate this to an interested party. The arising coding problem is often split into two parts: First, the sensors compress their respective acquired signals, potentially applying the concepts of distributed source coding. Then, they communicate the compressed version to the interested party, the goal being not to make any errors. This coding paradigm is inspired by Shannon’s separation theorem for point-to-point communication, but it leads to suboptimal performance in general n

For classification problems, it is important that the classifier is trained with data which is likely to appear in the future. Discriminative models, because of their nature to focus on the boundary between classes rather than data itself, usually do not have the capability to deal with noisy training data. We propose the use of generative models as filters to make discriminative models more robust against noise. Firstly the distribution of the training data is estimated, then examples which do not satisfy

What is the meaning' of a single spike? Spike-triggered averaging (reverse correlations') yields the typical input just before a spike. Similarly, cross-correlations describe the probability of firing an output spike given (one additional) presynaptic input spike. In this paper we analytically calculate reverse and cross-correlations for a spiking neuron model with escape noise. The influence of neuronal parameters (such as the membrane time constant, the noise level, and the mean firing rate) on the form

The use of multiresolution (MR) joint source-channel coding in the context of digital terrestrial broadcasting of high-definition television (HDTV) is shown to be an efficient alternative to single-resolution techniques, which suffer from a sharp threshold effect in the fringes of the broadcast area. It is shown how matched multiresolution source and channel coding can provide a stepwise graceful degradation and improve the behavior, in terms of coverage and robustness of the transmission scheme, over syste

We investigate the interactionof congestion control with the partitioning of source information into components of varying importance for variable bit-rate packet voice and packet video. High-priority transport for the more important signal components results in substantially increased objective service quality. Using a Markov chain voice source model with simple PCM speech encoding and a priority queue, simulation results show a signal-to=noise ratio improvement of 45 dB with two priorities over unprioriti

Typical set

Chattez avec Graph Search

The Conditional Regret-Capacity Theorem for Batch Universal Prediction

Neural Thermodynamics: Entropic Forces in Deep and Universal Representation Learning

Implementing Fusion Techniques for the Classification of Paralinguistic Information

Windowed Decoding of Spatially Coupled Codes

Polar coding: a brief tour

Polar Codes for Q-ary Source Coding

Lossless Source Coding with Polar Codes

Linear Universal Decoder for Compound Discrete Memoryless Channels

Fountain Capacity

Power-bandwidth-distortion scaling laws for sensor networks

Source-Channel Communication in Sensor Networks

Noise Robust Discriminative Models

Coding properties of spiking neurons: reverse- and cross-correlations

Multiresolution broadcast for digital HDTV using joint source-channel coding

Joint source/channel coding of statistically multiplexed real-time services on packet networks