Publication

The Gray-Wyner Network and Wyner's Common Information for Gaussian Sources

Publications associées (31)

Mutual Information Disentangles Interactions from Changing Environments

Daniel Maria Busiello, Giorgio Nicoletti

Real-world systems are characterized by complex interactions of their internal degrees of freedom, while living in ever-changing environments whose net effect is to act as additional couplings. Here, we introduce a paradigmatic interacting model in a switc ...

AMER PHYSICAL SOC2021

Differential Entropy of the Conditional Expectation under Gaussian Noise

Michael Christoph Gastpar, Alper Köse, Ahmet Arda Atalik

This paper considers an additive Gaussian noise channel with arbitrarily distributed finite variance input signals. It studies the differential entropy of the minimum mean-square error (MMSE) estimator and provides a new lower bound which connects the diff ...

IEEE2021

Generalization Error Bounds Via Rényi-, f-Divergences and Maximal Leakage

Michael Christoph Gastpar, Ibrahim Issa, Amedeo Roberto Esposito

In this work, the probability of an event under some joint distribution is bounded by measuring it with the product of the marginals instead (which is typically easier to analyze) together with a measure of the dependence between the two random variables. ...

2021

Low-Cost Side-Channel Secure Standard 6T-SRAM-Based Memory With a 1% Area and Less Than 5% Latency and Power Overheads

Robert Giterman

Side-channel attacks constitute a concrete threat to IoT systems-on-a-chip (SoCs). Embedded memories implemented with 6T SRAM macrocells often dominate the area and power consumption of these SoCs. Regardless of the computational platform, the side-channel ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

Statistical limits of high-dimensional inference problems

Clément Dominique Luneau

This thesis focuses on two kinds of statistical inference problems in signal processing and data science. The first problem is the estimation of a structured informative tensor from the observation of a noisy tensor in which it is buried. The structure com ...

EPFL2021

Deep Boltzmann Machines: Rigorous Results at Arbitrary Depth

Emanuele Mingione, Diego Alberici

A class of deep Boltzmann machines is considered in the simplified framework of a quenched system with Gaussian noise and independent entries. The quenched pressure of a K-layers spin glass model is studied allowing interactions only among consecutive laye ...

2021

Learning Constrained Distributions of Robot Configurations with Generative Adversarial Network

Sylvain Calinon, Emmanuel Pignat, Teguh Santoso Lembono, Julius Maximilian Jankowski

In high dimensional robotic system, the manifold of the valid configuration space often has a complex shape, especially under constraints such as end-effector orientation or static stability. We propose a generative adversarial network approach to learn th ...

IEEE2021

Validating model-based data interpretation methods for quantification of reserve capacity

Ian Smith, Sai Ganesh Sarvotham Pai

Optimal performance of civil infrastructure is an important aspect of liveable cities. A judicious combination of physics-based models with monitoring data in a validated methodology that accounts for uncertainties is explored in this paper. This methodolo ...

2021

Robust Generalization via $\alpha$-Mutual Information

Michael Christoph Gastpar, Ibrahim Issa, Amedeo Roberto Esposito

The aim of this work is to provide bounds connecting two probability measures of the same event using Rényi

\alpha

-Divergences and Sibson’s

\alpha

-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence and Shannon’s Mutual ...

ETHZ2020

Mutual Information and Optimality of Approximate Message-Passing in Random Linear Estimation

Nicolas Macris, Mohamad Baker Dia, Jean François Emmanuel Barbier, Florent Gérard Krzakala

We consider the estimation of a signal from the knowledge of its noisy linear random Gaussian projections. A few examples where this problem is relevant are compressed sensing, sparse superposition codes, and code division multiple access. There has been a ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

On quantifying the quality of acoustic models in hybrid DNN-HMM ASR

Hervé Bourlard, Afsaneh Asaei, Pranay Dighe

We propose an information theoretic framework for quantitative assessment of acoustic models used in hidden Markov model (HMM) based automatic speech recognition (ASR). The HMM backend expects that (i) the acoustic model yields accurate state conditional e ...

ELSEVIER2020

Publications associées (31)

\alpha

-Divergences and Sibson’s

\alpha

-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence and Shannon’s Mutual ...

ETHZ2020

Mutual Information and Optimality of Approximate Message-Passing in Random Linear Estimation

Nicolas Macris, Mohamad Baker Dia, Jean François Emmanuel Barbier, Florent Gérard Krzakala

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

On quantifying the quality of acoustic models in hybrid DNN-HMM ASR

Hervé Bourlard, Afsaneh Asaei, Pranay Dighe

ELSEVIER2020