Strengthened Information-theoretic Bounds on the Generalization Error

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

As large, data-driven artificial intelligence models become ubiquitous, guaranteeing high data quality is imperative for constructing models. Crowdsourcing, community sensing, and data filtering have long been the standard approaches to guaranteeing or imp ...

Local and global inference methods have been developed to infer structural contacts from multiple sequence alignments of homologous proteins. They rely on correlations in amino acid usage at contacting sites. Because homologous proteins share a common ance ...

Given two jointly distributed random variables (X,Y), a functional representation of X is a random variable Z independent of Y, and a deterministic function g(⋅,⋅) such that X=g(Y,Z). The problem of finding a minimum entropy functional representation is kn ...

We propose a structured prediction approach for robot imitation learning from demonstrations. Among various tools for robot imitation learning, supervised learning has been observed to have a prominent role. Structured prediction is a form of supervised le ...

Since the birth of Information Theory, researchers have defined and exploited various information measures, as well as endowed them with operational meanings. Some were born as a "solution to a problem", like Shannon's Entropy and Mutual Information. Other ...

Biochemistry, ecology, and neuroscience are examples of prominent fields aiming at describing interacting systems that exhibit nontrivial couplings to complex, ever-changing environments. We have recently shown that linear interactions and a switching envi ...

This paper presents explicit solutions for two related non-convex information extremization problems due to Gray and Wyner in the Gaussian case. The first problem is the Gray-Wyner network subject to a sum-rate constraint on the two private links. Here, ou ...

We consider the problem of parameter estimation in a Bayesian setting and propose a general lower-bound that includes part of the family of f-Divergences. The results are then applied to specific settings of interest and compared to other notable results i ...

We consider increasingly complex models of matrix denoising and dictionary learning in the Bayes-optimal setting, in the challenging regime where the matrices to infer have a rank growing linearly with the system size. This is in contrast with most existin ...

With the increase in massive digitized datasets of cultural artefacts, social and cultural scientists have an unprecedented opportunity for the discovery and expansion of cultural theory. The WikiArt dataset is one such example, with over 250,000 high qual ...

Measuring conditional dependencies among the variables of a network is of great interest to many disciplines. This paper studies some shortcomings of the existing dependency measures in detecting direct causal influences or their lack of ability for group ...

Unstructured neural network pruning algorithms have achieved impressive compression ratios. However, the resulting-typically irregular-sparse matrices hamper efficient hardware implementations, leading to additional memory usage and complex control logic t ...

In this thesis, timing is everything. In the first part, we mean this literally, as we tackle systems that encode information using timing alone. In the second part, we adopt the standard, metaphoric interpretation of this saying and show the importance of ...

Inferring protein-protein interactions from sequences is an important task in computational biology. Recent methods based on Direct Coupling Analysis (DCA) or Mutual Information (MI) allow to find interaction partners among paralogs of two protein families ...

It was recently shown that almost all solutions in the symmetric binary perceptron are isolated, even at low constraint densities, suggesting that finding typical solutions is hard. In contrast, some algorithms have been shown empirically to succeed in fin ...

This thesis consists of three applications of machine learning techniques to risk management. The first chapter proposes a deep learning approach to estimate physical forward default intensities of companies. Default probabilities are computed using artifi ...

Inexpensive and accessible sensors are accelerating data acquisition in animal ecology. These technologies hold great potential for large-scale ecological understanding, but are limited by current processing approaches which inefficiently distill data into ...

In this work, the probability of an event under some joint distribution is bounded by measuring it with the product of the marginals instead (which is typically easier to analyze) together with a measure of the dependence between the two random variables. ...

Due to conservative approaches in construction design and practice, infrastructure often has hidden reserve capacity. When quantified, this reserve has potential to improve decisions related to asset management. Field measurements, collected through load t ...

Optimal performance of civil infrastructure is an important aspect of liveable cities. A judicious combination of physics-based models with monitoring data in a validated methodology that accounts for uncertainties is explored in this paper. This methodolo ...

Strengthened Information-theoretic Bounds on the Generalization Error

Chat with Graph Search

Extensions of Peer Prediction Incentive Mechanisms

Impact of phylogeny on structural contact inference from protein sequence data

Information Spectrum Converse for Minimum Entropy Couplings and Functional Representations

A structured prediction approach for robot imitation learning

A Functional Perspective on Information Measures

Mutual information in changing environments: Nonlinear interactions, out-of-equilibrium systems, and continuously varying diffusivities

The Gray-Wyner Network and Wyner's Common Information for Gaussian Sources

Lower-bounds on the Bayesian Risk in Estimation Procedures via f–Divergences

Statistical limits of dictionary learning: Random matrix theory and the spectral replica method

WikiArtVectors: Style and Color Representations of Artworks for Cultural Analysis via Information Theoretic Measures

A Wasserstein-based measure of conditional dependence

Dynamic Probabilistic Pruning: A General Framework for Hardware-Constrained Pruning at Different Granularities

Timing is Everything

Correlations from structure and phylogeny combine constructively in the inference of protein partners from sequences

Binary Perceptron: Efficient Algorithms Can Find Solutions in a RareWell-Connected Cluster

Financial Risk Management with Machine Learning

Perspectives in machine learning for wildlife conservation

Generalization Error Bounds Via Rényi-, f-Divergences and Maximal Leakage

A Comparison of optimal measurement-system design with engineering judgement for bridge load testing

Validating model-based data interpretation methods for quantification of reserve capacity