Publications by Bruno Loureiro

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

While classical in many theoretical settings-and in particular in statistical physics-inspired works-the assumption of Gaussian i.i.d. input data is often perceived as a strong limitation in the context of statistics and machine learning. In this study, we ...

A recent line of work in high-dimensional statistics working under the Gaussian mixture hypothesis has led to a number of results in the context of empirical risk minimization, Bayesian uncertainty quantification, separation of kernel methods and neural ne ...

Uncertainty quantification is a central challenge in reliable and trustworthy machine learning. Naive measures such as last-layer scores are well-known to yield overconfident estimates in the context of overparametrized neural networks. Several methods, ra ...

For high-dimensional Gaussian data, we investigate theoretically how the features of a two-layer neural network adapt to the structure of the target function through a few large batch gradient descent steps, leading to an improvement in the approximation c ...

Inverse probability problems whose generative models are given by strictly nonlinear Gaussian random fields show the all-or-nothing behavior: There exists a critical rate at which Bayesian inference exhibits a phase transition. Below this rate, the optimal ...

Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achieve global convergence under gradient descent. The picture can be radically different for narrow net-works, which tend to get stuck in badly-generalizing loca ...

This work investigates adversarial training in the context of margin-based linear classifiers in the high-dimensional regime where the dimension d and the number of data points n diverge with a fixed ratio α = n/d. We introduce a tractable mathematical mod ...

Despite their incredible performance, it is well reported that deep neural networks tend to be overoptimistic about their prediction confidence. Finding effective and efficient calibration methods for neural networks is therefore an important endeavour tow ...

We consider the phase retrieval problem of reconstructing a n -dimensional real or complex signal X ⋆ from m (possibly noisy) observations Y μ = | ∑ n i = 1 Φ μ i X ⋆ i / √ n | , for a large class of correlated real and complex random sensing matrices Φ , ...

We study generalised linear regression and classification for a synthetically generated dataset encompassing different problems of interest, such as learning with random features, neural networks in the lazy training regime, and the hidden manifold model. ...

Generalised linear models for multi-class classification problems are one of the fundamental building blocks of modern machine learning tasks. In this manuscript, we characterise the learning of a mixture of K Gaussians with generic means and covariances v ...

In this manuscript we consider Kernel Ridge Regression (KRR) under the Gaussian design. Exponents for the decay of the excess generalization error of KRR have been reported in various works under the assumption of power-law decay of eigenvalues of the feat ...

Teacher-student models provide a framework in which the typical-case performance of high-dimensional supervised learning can be described in closed form. The assumptions of Gaussian i.i.d. input data underlying the canonical teacher-student model may, howe ...

In this manuscript we consider kernel ridge regression (KRR) under the Gaussian design. Exponents for the decay of the excess generalization error of KRR have been reported in various works under the assumption of power-law decay of eigenvalues of the feat ...

We investigate popular resampling methods for estimating the uncertainty of statistical models, such as subsampling, bootstrap and the jackknife, and their performance in high-dimensional supervised regression tasks. We provide a tight asymptotic descripti ...

In this manuscript, we investigate the problem of how two-layer neural networks learn features from data, and improve over the kernel regime, after being trained with a single gradient descent step. Leveraging the insight from (Ba et al., 2022), we model t ...

A simple model to study subspace clustering is the high-dimensional k -Gaussian mixture model where the cluster means are sparse vectors. Here we provide an exact asymptotic characterization of the statistically optimal reconstruction error in this model i ...

From the sampling of data to the initialisation of parameters, randomness is ubiquitous in modern Machine Learning practice. Understanding the statistical fluctuations engendered by the different sources of randomness in prediction is therefore key to unde ...

Understanding the impact of data structure on the computational tractability of learning is a key challenge for the theory of neural networks. Many theoretical works do not explicitly model training data, or assume that inputs are drawn component-wise inde ...

We study the impact of the batch

n_b

on the iteration time

T

of training two-layer neural networks with one-pass stochastic gradient descent (SGD) on multi-index target functions of isotropic covariates. We characterize the optimal batch size minimizin ...

Bruno Loureiro

Chat with Graph Search

Gaussian universality of perceptrons with random labels

Universality laws for Gaussian mixtures in generalized linear models

On double-descent in uncertainty quantification in overparametrized models

How Two-Layer Neural Networks Learn, One (Giant) Step at a Time

Secure Coding via Gaussian Random Fields

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offs

Expectation consistency for calibration of neural networks

Phase retrieval in high dimensions: Statistical and computational phase transitions

Generalisation error in learning with random features and the hidden manifold model*

Learning Gaussian Mixtures with Generalized Linear Models: Precise Asymptotics in High-dimensions

Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regime

Learning curves of generic features maps for realistic datasets with a teacher-student model

Generalization error rates in kernel regression: the crossover from the noiseless to noisy regime*

Analysis of Bootstrap and Subsampling in High-dimensional Regularized Regression

Asymptotics of feature learning in two-layer networks after one gradient-step

Subspace clustering in high-dimensions: Phase transitions & Statistical-to-Computational gap

Fluctuations, bias, variance and ensemble of learners: exact asymptotics for convex losses in high-dimension

The Gaussian equivalence of generative models for learning with shallow neural networks

Online Learning and Information Exponents: On The Importance of Batch size, and Time/Complexity Tradeoffs