Michael Christoph Gastpar

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

The aim of this work is to provide bounds connecting two probability measures of the same event using Rényi

\alpha

-Divergences and Sibson’s

\alpha

-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence and Shannon’s Mutual ...

Learning discrete distributions from i.i.d. samples is a well-understood problem. However, advances in generative machine learning prompt an interesting new, non-i.i.d. setting: after receiving a certain number of samples, an estimated distribution is fixed, and samples from this estimate are drawn and introduced into the sample corpus, undifferentiated from real samples. Subsequent generations of estimators now face contaminated environments, a scenario referred to in the machine learning literature as sel

Given two probability measures P and Q and an event E, we provide bounds on P(E) in terms of Q(E) and f-divergences. In particular, the bounds are instantiated when the measures considered are a joint distribution and the corresponding product of marginals ...

The Karhunen–Loève transform (KLT) is a key ele- ment of many signal processing and communication tasks. Many recent applications involve distributed signal processing, where it is not generally possible to apply the KLT to the entire signal; rather, the K ...

Determining the capacity of multi-receiver networks with arbitrary message demands is an open problem in the network coding literature. In this paper, we consider a multi-source, multi-receiver symmetric deterministic network model parameterized by channel coefficients (inspired by wireless network flow) in which the receivers compute a sum of the symbols generated at the sources. Scalar and vector linear coding strategies are analyzed. It is shown that computation alignment over finite field vector spaces

The conditional mean is a fundamental and important quantity whose applications include the theories of estimation and rate-distortion. It is also notoriously difficult to work with. This paper establishes novel bounds on the differential entropy of the co ...

In this work, we analyse how to define a conditional version of Sibson's alpha-Mutual Information. Several such definitions can be advanced and they all lead to different information measures with different (but similar) operational meanings. We will analy ...

We consider a setup in which confidential i.i.d. samples X1, ..., Xn from an unknown discrete distribution PX are passed through a discrete memoryless privatization channel (a.k.a. mechanism) which guarantees an epsilon-level of local differential privacy. ...

Function computation over Gaussian networks with orthogonal components is studied for arbitrarily correlated discrete memoryless sources. Two classes of functions are considered: 1) the arithmetic sum function and 2) the type function. The arithmetic sum f ...

Systems of interest for theoretical or experimental work often exhibit high-order interactions, corresponding to statistical interdependencies in groups of variables that cannot be reduced to dependencies in subsets of them. While still under active develo ...

In this paper, we discuss a framework for the distributed compression of vector sources, based on our previous work on distributed transform coding. In particular, our goal is to develop a strategy of first applying a suitable distributed Karhunen-Loeve transform, whereafter each component can be andled by standard distributed compression techniques. In the present paper, we first study the scenario where all but one terminal furnish a noisy approximation of their observation. For the case where the underly

Learning and compression are driven by the common aim of identifying and exploiting statistical regularities in data, which opens the door for fertile collaboration between these areas. A promising group of compression techniques for learning scenarios is ...

In this paper, we study the data exchange problem, where a set of users is interested in gaining access to a common file, but where each has only partial knowledge about it as side-information. Assuming that the file is broken into packets, the side-inform ...

A particular instance of the shortest vector problem (SVP) appears in the context of compute-and-forward. Despite the NP-hardness of the SVP, we will show that this certain instance can be solved in complexity order O(nψlog(nψ)) , where ψ=sqrt(P ||h||^2+1) ...

We formalize the problem of prompt compression for large language models (LLMs) and present a framework to unify token-level prompt compression methods which create hard prompts for black-box models. We derive the distortion-rate function for this setup as ...

Linear receivers are often used to reduce the implementation complexity of multiple-antenna systems. In a traditional linear receiver architecture, the receive antennas are used to separate out the codewords sent by each transmit antenna, which can then be ...

For a class of sensor networks, the task is to monitor an underlying physical phenomenon over space and time through an imperfect observation process. The sensors can communicate back to a central data collector over a noisy channel. The key pa- rameters i ...

In recent years, attention-based transformers have achieved tremendous success across a variety of disciplines including natural languages. A key ingredient behind their success is the generative pretraining procedure, during which these models are trained on a large text corpus in an auto-regressive manner. To shed light on this phenomenon, we propose a new framework that allows both theory and systematic experiments to study the sequential modeling capabilities of transformers through the lens of Markov c

Building on polar code constructions proposed by the authors for deterministic broadcast channels, two theorems are introduced in the present paper for noisy two-user broadcast channels. The theorems establish polar code constructions for two important ...

Michael Christoph Gastpar

Chat with Graph Search

Robust Generalization via $\alpha$-Mutual Information

Model Non-Collapse: Minimax Bounds for Recursive Discrete Distribution Estimation

Model Non-Collapse: Minimax Bounds for Recursive Discrete Distribution Estimation

Robust Generalization via f−Mutual Information

The Distributed Karhunen-Loève Transform

Network Coding with Computation Alignment

Differential Entropy of the Conditional Expectation Under Additive Gaussian Noise

On conditional Sibson's alpha-Mutual Information

Locally differentially-private distribution estimation

Computation Over Gaussian Networks With Orthogonal Components

Characterising directed and undirected metrics of high-order interdependence

On Compression Using the Distributed Karhunen-Loeve Transform

Learning, compression, and leakage: Minimising classification error via meta-universal compression principles

Efficient Algorithms for the Data Exchange Problem

Polynomially Solvable Instances of the Shortest and Closest Vector Problems With Applications to Compute-and-Forward

Fundamental Limits of Prompt Compression: A Rate- Distortion Framework for Black-Box Language Models

Integer-forcing Linear Receivers

Power, Spatio-Temporal Bandwidth, and Distortion in Large Sensor Networks

Attention with Markov: A Framework for Principled Analysis of Transformers via Markov Chains

Polar Codes For Broadcast Channels