Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Concept

Principle of maximum entropy

Related publications (30)

Information Spectrum Converse for Minimum Entropy Couplings and Functional Representations

Yanina Yurina Shkel, Anuj Kumar Yadav

Given two jointly distributed random variables (X,Y), a functional representation of X is a random variable Z independent of Y, and a deterministic function g(⋅,⋅) such that X=g(Y,Z). The problem of finding a minimum entropy functional representation is kn ...

2023

Learning curves for the multi-class teacher-student perceptron

Lenka Zdeborová, Bruno Loureiro, Elisabetta Cornacchia, Rodrigo Soares Veiga

One of the most classical results in high-dimensional learning theory provides a closed-form expression for the generalisation error of binary classification with a single-layer teacher-student perceptron on i.i.d. Gaussian inputs. Both Bayes-optimal (BO) ...

IOP Publishing Ltd2023

T-5 configurations and hyperbolic systems

Riccardo Tione, Carl Johan Peter Johansson

In this paper, we study the rank-one convex hull of a differential inclusion associated to entropy solutions of a hyperbolic system of conservation laws. This was introduced in [B. Kirchheim, S. Muller and V. S(sic)ver & aacute;k, Studying Nonlinear PDE by ...

WORLD SCIENTIFIC PUBL CO PTE LTD2023

Differential Entropy of the Conditional Expectation Under Additive Gaussian Noise

Michael Christoph Gastpar, Alper Köse, Ahmet Arda Atalik

The conditional mean is a fundamental and important quantity whose applications include the theories of estimation and rate-distortion. It is also notoriously difficult to work with. This paper establishes novel bounds on the differential entropy of the co ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Exploring quantum perceptron and quantum neural network structures with a teacher-student scheme

Patrick Hümbeli

Near-term quantum devices can be used to build quantum machine learning models, such as quantum kernel methods and quantum neural networks (QNN), to perform classification tasks. There have been many proposals on how to use variational quantum circuits as ...

2022

A Maximum-Likelihood-Based Two-User Receiver for LoRa Chirp Spread-Spectrum Modulation

Andreas Peter Burg, Alexios Konstantinos Balatsoukas Stimming, Orion Afisiadis, Joachim Tobias Tapparel

Long Range (LoRa) is an emerging low-power wide-area network technology offering long-range wireless connectivity to Internet of Things (IoT) devices. For energy efficiency reasons, LoRa end nodes implement a nonslotted ALOHA multiple access scheme to tran ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Alpha-NML Universal Predictors

Michael Christoph Gastpar, Marco Bondaschi

Inspired by Sibson’s alpha-mutual information, we introduce a new parametric class of universal predictors. This class interpolates two well-known predictors, the mixture estimator, that includes the Laplace and the Krichevsky-Trofimov predictors, and the ...

2022

Atomic subgraphs and the statistical mechanics of networks

Sofia Charlotta Olhede

We develop random graph models where graphs are generated by connecting not only pairs of vertices by edges, but also larger subsets of vertices by copies of small atomic subgraphs of arbitrary topology. This allows for the generation of graphs with extens ...

2021

Differential Entropy of the Conditional Expectation under Gaussian Noise

Michael Christoph Gastpar, Alper Köse, Ahmet Arda Atalik

This paper considers an additive Gaussian noise channel with arbitrarily distributed finite variance input signals. It studies the differential entropy of the minimum mean-square error (MMSE) estimator and provides a new lower bound which connects the diff ...

IEEE2021

Gaussian Process Regression for Maximum Entropy Distribution

Mohammadhossein Gorji, Mohsen Sadr

Maximum-Entropy Distributions offer an attractive family of probability densities suitable for moment closure problems. Yet finding the Lagrange multipliers which parametrize these distributions, turns out to be a computational bottleneck for practical clo ...

2020

Entropy in Spin Relaxation, Spintronics, and Magnetic Resonance

Ferenc Simon

The entropy change during spin relaxation for a realistic model system is studied, whose spin dynamics can be handled with the Boltzmann equation. The time evolution of the von Neumann entropy is monitored during the process and is compared with the recent ...

WILEY-V C H VERLAG GMBH2020

Coarse-grained entropy production with multiple reservoirs: Unraveling the role of time scales and detailed balance in biology-inspired systems

Amos Maritan, Daniel Maria Busiello

A general framework to describe a vast majority of biology-inspired systems is to model them as stochastic processes in which multiple couplings are in play at the same time. Molecular motors, chemical reaction networks, catalytic enzymes, and particles ex ...

2020

Artificial Neural Network Approach to the Analytic Continuation Problem

Oleg Yazyev, Quansheng Wu, Lei Wang

Inverse problems are encountered in many domains of physics, with analytic continuation of the imaginary Green's function into the real frequency domain being a particularly important example. However, the analytic continuation problem is ill defined and c ...

AMER PHYSICAL SOC2020

Information Theoretic Characterization of Uncertainty Distinguishes Surprise From Accuracy Signals in the Brain

Kerstin Preuschoff, Leyla Loued-Khenissi

Uncertainty presents a problem for both human and machine decision-making. While utility maximization has traditionally been viewed as the motive force behind choice behavior, it has been theorized that uncertainty minimization may supersede reward motivat ...

2020

Gaussian Process Regression for Maximum Entropy Distribution

Mohsen Sadr

2019

Entropy as a tool for crystal discovery

Pablo Miguel Piaggi

The computational prediction of crystal structures has emerged as an useful alternative to expensive and often cumbersome experiments. We propose an approach to the prediction of crystal structures and polymorphism based on reproducing the crystallization ...

EPFL2019

Generalized maximum entropy estimation

John Lygeros, Peyman Mohajerin Esfahani, Tobias Sutter

We consider the problem of estimating a probability distribution that maximizes the entropy while satisfying a finite number of moment constraints, possibly corrupted by noise. Based on duality of convex programming, we present a novel approximation scheme ...

2019

Entropy production for coarse-grained dynamics

Daniel Maria Busiello, Amos Maritan

Systems out of equilibrium exhibit a net production of entropy. We study the dynamics of a stochastic system represented by a Master equation (ME) that can be modeled by a Fokker-Planck equation in a coarse-grained, mesoscopic description. We show that the ...

2019

A sequence-dependent coarse-grain model of B-DNA with explicit description of bases and phosphate groups parametrised from large scale Molecular Dynamics simulations

Alessandro Samuele Patelli

We introduce a sequence-dependent coarse-grain model of double-stranded DNA with an explicit description of both the bases and the phosphate groups as interacting rigid-bodies. The model parameters are trained on extensive, state-of-the-art large scale mol ...

EPFL2019

A power-balance model of the density limit in fusion plasmas: application to the L-mode tokamak

Olivier Sauter, Ambrogio Fasoli, Basil Duval, Martinus Adela Maria Gijs, Richard Pitts, Stefano Coda, Henri Weisen, Yves Martin, Yann Camenen, Jan Horacek, Patrick Blanchard, Javier García Hernández, Duccio Testa, Marco Wischmeier, Nicola Vianello, Mikhail Maslov, Alessandro Pau, Davide Galassi, Antonio José Pereira de Figueiredo, Liang Yao, Dalziel Joseph Wilson, Cristian Sommariva, Jonathan Graves, Federico Nespoli, Hamish William Patten, Samuel Lanthaler, David Pfefferlé, Jonathan Marc Philippe Faustin

A power-balance model, with radiation losses from impurities and neutrals, gives a unified description of the density limit (DL) of the stellarator, the L-mode tokamak, and the reversed field pinch (RFP). The model predicts a Sudo-like scaling for the stel ...

IOP Publishing Ltd2019