Publications de Florent Gérard Krzakala

Publications associées (66)

On the atypical solutions of the symmetric binary perceptron

Lenka Zdeborová, Florent Gérard Krzakala, Damien Yves Valery Barbier

We study the random binary symmetric perceptron problem, focusing on the behavior of rare high-margin solutions. While most solutions are isolated, we demonstrate that these rare solutions are part of clusters of extensive entropy, heuristically correspond ...

Iop Publishing Ltd2024

Gaussian universality of perceptrons with random labels

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Ludovic Théo Stephan, Federica Gerace

While classical in many theoretical settings-and in particular in statistical physics-inspired works-the assumption of Gaussian i.i.d. input data is often perceived as a strong limitation in the context of statistics and machine learning. In this study, we ...

Amer Physical Soc2024

Phase Diagram of Compressed Sensing with l0-norm Regularization

Lenka Zdeborová, Florent Gérard Krzakala, Damien Yves Valery Barbier

Noiseless compressive sensing is a two-step setting that allows for undersampling a sparse signal and then reconstructing it without loss of information. The LASSO algorithm, based on l(1) regularization, provides an efficient and robust way to address thi ...

IOP Publishing Ltd2025

Universality laws for Gaussian mixtures in generalized linear models

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Ludovic Théo Stephan, Yatin Dandi

A recent line of work in high-dimensional statistics working under the Gaussian mixture hypothesis has led to a number of results in the context of empirical risk minimization, Bayesian uncertainty quantification, separation of kernel methods and neural ne ...

2024

Asymptotic Errors for Teacher-Student Convex Generalized Linear Models (Or: How to Prove Kabashima’s Replica Formula)

Florent Gérard Krzakala, Alia Abbara

There has been a recent surge of interest in the study of asymptotic reconstruction performance in various cases of generalized linear estimation problems in the teacher-student setting, especially for the case of i.i.d standard normal matrices. Here, we g ...

2022

Bayes-optimal Learning of Deep Random Networks of Extensive-width

Lenka Zdeborová, Florent Gérard Krzakala, Hugo Chao Cui

We consider the problem of learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights. We consider the asymptotic limit where the number of samples, the input dimension and the network width ...

2023

On double-descent in uncertainty quantification in overparametrized models

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Lucas Andry Clarte

Uncertainty quantification is a central challenge in reliable and trustworthy machine learning. Naive measures such as last-layer scores are well-known to yield overconfident estimates in the context of overparametrized neural networks. Several methods, ra ...

PMLR Proceedings of Machine Learning Research2023

Multi-layer State Evolution Under Random Convolutional Design

Lenka Zdeborová, Florent Gérard Krzakala

Signal recovery under generative neural network priors has emerged as a promising direction in statistical inference and computational imaging. Theoretical analysis of reconstruction algorithms under generative priors is, however, challenging. For generati ...

2022

Optimal errors and phase transitions in high-dimensional generalized linear models

Nicolas Macris, Florent Gérard Krzakala, Lenka Zdeborová, Jean François Emmanuel Barbier

Generalized linear models (GLMs) are used in high-dimensional machine learning, statistics, communications, and signal processing. In this paper we analyze GLMs when the data matrix is random, as relevant in problems such as compressed sensing, error-corre ...

National Academy of Sciences2019

Bayes-optimal learning of deep random networks of extensive-width*

Lenka Zdeborová, Florent Gérard Krzakala, Hugo Chao Cui

We consider the problem of learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights. We consider the asymptotic limit where the number of samples, the input dimension and the network width ...

IOP Publishing Ltd2025

Statistical mechanics of the maximum-average submatrix problem

Rodrigo Emilio Pérez Ortiz, Lenka Zdeborová, Florent Gérard Krzakala, Vittorio Erba

We study the maximum-average submatrix problem, wherein given an N x N matrix J, one needs to find the k x k submatrix with the largest average number of entries. We investigate the problem for random matrices J, whose entries are i.i.d. random variables, ...

Iop Publishing Ltd2024

How Two-Layer Neural Networks Learn, One (Giant) Step at a Time

Florent Gérard Krzakala, Bruno Loureiro, Ludovic Théo Stephan, Yatin Dandi, Luca Pesce

For high-dimensional Gaussian data, we investigate theoretically how the features of a two-layer neural network adapt to the structure of the target function through a few large batch gradient descent steps, leading to an improvement in the approximation c ...

2024

Secure Coding via Gaussian Random Fields

Florent Gérard Krzakala, Bruno Loureiro

Inverse probability problems whose generative models are given by strictly nonlinear Gaussian random fields show the all-or-nothing behavior: There exists a critical rate at which Bayesian inference exhibits a phase transition. Below this rate, the optimal ...

IEEE2022

Compressed sensing with ℓ 0-norm: statistical physics analysis & algorithms for signal recovery

Lenka Zdeborová, Florent Gérard Krzakala, Damien Yves Valery Barbier

Noiseless compressive sensing is a protocol that enables undersampling and later recovery of a signal without loss of information. This compression is possible because the signal is usually sufficiently sparse in a given basis. Currently, the algorithm off ...

IEEE2023

Rigorous Dynamical Mean-Field Theory for Stochastic Gradient Descent Methods

Lenka Zdeborová, Florent Gérard Krzakala, Emanuele Troiani

We prove closed-form equations for the exact high-dimensional asymptotics of a family of first-order gradient-based methods, learning an estimator (e.g., M-estimator, shallow neural network) from observations on Gaussian data with empirical risk minimizati ...

Society for Industrial & Applied Mathematics (SIAM)2024

Bayesian reconstruction of memories stored in neural networks from their connectivity

Lenka Zdeborová, Florent Gérard Krzakala

The advent of comprehensive synaptic wiring diagrams of large neural circuits has created the field of connectomics and given rise to a number of open research questions. One such question is whether it is possible to reconstruct the information stored in ...

2023

Quenches in the Sherrington-Kirkpatrick model

Lenka Zdeborová, Florent Gérard Krzakala, Freya Behrens, Vittorio Erba

The Sherrington-Kirkpatrick model is a prototype of a complex non-convex energy landscape. Dynamical processes evolving on such landscapes and locally aiming to reach minima are generally poorly understood. Here, we study quenches, i.e. dynamics that local ...

2024

Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimization

Lenka Zdeborová, Florent Gérard Krzakala

We consider a commonly studied supervised classification of a synthetic dataset whose labels are generated by feeding a one-layer non-linear neural network with random iid inputs. We study the generalization performances of standard classifiers in the high ...

Curran Associates, Inc.2020

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Ludovic Théo Stephan, Rodrigo Soares Veiga

Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achieve global convergence under gradient descent. The picture can be radically different for narrow net-works, which tend to get stuck in badly-generalizing loca ...

2022

The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents

Lenka Zdeborová, Florent Gérard Krzakala, Yatin Dandi, Emanuele Troiani, Luca Pesce, Luca Arnaboldi

We investigate the training dynamics of two-layer neural networks when learning multi-index target functions. We focus on multi-pass gradient descent (GD) that reuses the batches multiple times and show that it significantly changes the conclusion about wh ...

JMLR2024