Home

Person

Florent Gérard Krzakala

About
Privacy
Disclaimer

Graph Chatbot

Related publications (66)

Page 3 of 4

Generalization error rates in kernel regression: the crossover from the noiseless to noisy regime*

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Hugo Chao Cui

In this manuscript we consider kernel ridge regression (KRR) under the Gaussian design. Exponents for the decay of the excess generalization error of KRR have been reported in various works under the assumption of power-law decay of eigenvalues of the feat ...

2022

Analysis of Bootstrap and Subsampling in High-dimensional Regularized Regression

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Adrien Vandenbroucque, Lucas Andry Clarte, Guillaume Cyprien Dalle

We investigate popular resampling methods for estimating the uncertainty of statistical models, such as subsampling, bootstrap and the jackknife, and their performance in high-dimensional supervised regression tasks. We provide a tight asymptotic descripti ...

ACM2024

The committee machine: Computational to statistical gaps in learning a two-layers neural network

Nicolas Macris, Florent Gérard Krzakala, Lenka Zdeborová, Jean François Emmanuel Barbier

Heuristic tools from statistical physics have been used in the past to locate the phase transitions and compute the optimal learning and generalization errors in the teacher-student scenario in multi-layer neural networks. In this contribution, we provide ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2018

Theoretical characterization of uncertainty in high-dimensional linear classification

Lenka Zdeborová, Florent Gérard Krzakala, Lucas Andry Clarte

Being able to reliably assess not only the accuracy but also the uncertainty of models' predictions is an important endeavor in modern machine learning. Even if the model generating the data and labels is known, computing the intrinsic uncertainty after le ...

IOP Publishing Ltd2023

Asymptotics of feature learning in two-layer networks after one gradient-step

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Hugo Chao Cui, Yatin Dandi, Luca Pesce, Yue Lu

In this manuscript, we investigate the problem of how two-layer neural networks learn features from data, and improve over the kernel regime, after being trained with a single gradient descent step. Leveraging the insight from (Ba et al., 2022), we model t ...

2024

The Mutual Information in Random Linear Estimation Beyond i.i.d. Matrices

Nicolas Macris, Florent Gérard Krzakala, Jean François Emmanuel Barbier

There has been definite progress recently in proving the variational single-letter formula given by the heuristic replica method for various estimation problems. In particular, the replica formula for the mutual information in the case of noisy linear esti ...

IEEE2018

The committee machine: computational to statistical gaps in learning a two-layers neural network

Nicolas Macris, Florent Gérard Krzakala, Lenka Zdeborová, Jean François Emmanuel Barbier

IOP PUBLISHING LTD2019

Asymptotic Characterisation of the Performance of Robust Linear Regression in the Presence of Outliers

Florent Gérard Krzakala, Emanuele Troiani, Vittorio Erba, Matteo Vilucchio

We study robust linear regression in highdimension, when both the dimension d and the number of data points n diverge with a fixed ratio α = n/d, and study a data model that includes outliers. We provide exact asymptotics for the performances of the empiri ...

2024

Subspace clustering in high-dimensions: Phase transitions & Statistical-to-Computational gap

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Luca Pesce

A simple model to study subspace clustering is the high-dimensional k -Gaussian mixture model where the cluster means are sparse vectors. Here we provide an exact asymptotic characterization of the statistically optimal reconstruction error in this model i ...

2022

Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification*

Lenka Zdeborová, Florent Gérard Krzakala

We analyze in a closed form the learning dynamics of the stochastic gradient descent (SGD) for a single-layer neural network classifying a high-dimensional Gaussian mixture where each cluster is assigned one of two labels. This problem provides a prototype ...

2021

Sampling with flows, diffusion, and autoregressive neural networks from a spin-glass perspective

Lenka Zdeborová, Florent Gérard Krzakala, Yatin Dandi, Davide Ghio

Recent years witnessed the development of powerful generative models based on flows, diffusion, or autoregressive neural networks, achieving remarkable success in generating data from examples with applications in a broad range of areas. A theoretical anal ...

Proceedings of the National Academy of Sciences2024

Fluctuations, bias, variance and ensemble of learners: exact asymptotics for convex losses in high-dimension

Florent Gérard Krzakala, Bruno Loureiro

From the sampling of data to the initialisation of parameters, randomness is ubiquitous in modern Machine Learning practice. Understanding the statistical fluctuations engendered by the different sources of randomness in prediction is therefore key to unde ...

Iop Publishing Ltd2023

Phase diagram of stochastic gradient descent in high-dimensional two-layer neural networks

Lenka Zdeborová, Florent Gérard Krzakala, Ludovic Théo Stephan, Rodrigo Soares Veiga

Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achieve global convergence under gradient descent. The picture can be radically different for narrow networks, which tend to get stuck in badly-generalizing local ...

Iop Publishing Ltd2023

The Gaussian equivalence of generative models for learning with shallow neural networks

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro

Understanding the impact of data structure on the computational tractability of learning is a key challenge for the theory of neural networks. Many theoretical works do not explicitly model training data, or assume that inputs are drawn component-wise inde ...

2021

Entropy and mutual information in models of deep neural networks

Nicolas Macris, Florent Gérard Krzakala, Lenka Zdeborová, Jean François Emmanuel Barbier, Clément Dominique Luneau

We examine a class of stochastic deep learning models with a tractable method to compute information-theoretic quantities. Our contributions are three-fold: (i) we show how entropies and mutual informations can be derived from heuristic statistical physics ...

2019

Entropy and mutual information in models of deep neural networks

Nicolas Macris, Florent Gérard Krzakala, Lenka Zdeborová, Jean François Emmanuel Barbier, Clément Dominique Luneau

We examine a class of stochastic deep learning models with a tractable method to compute information-theoretic quantities. Our contributions are three-fold: (i) We show how entropies and mutual informations can be derived from heuristic statistical physics ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2018

Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures

Florent Gérard Krzakala

Despite being the workhorse of deep learning, the backpropagation algorithm is no panacea. It enforces sequential layer updates, thus preventing efficient parallelization of the training process. Furthermore, its biological plausibility is being challenged ...

Curran Associates, Inc.2020

Adversarial Robustness by Design Through Analog Computing And Synthetic Gradients

Florent Gérard Krzakala

We propose a new defense mechanism against adversarial at-tacks inspired by an optical co-processor, providing robustness without compromising natural accuracy in both white-box and black-box settings. This hardware co-processor performs a nonlinear fixed ...

IEEE2022

Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification

Lenka Zdeborová, Florent Gérard Krzakala

We analyze in a closed form the learning dynamics of stochastic gradient descent (SGD) for a single layer neural network classifying a high-dimensional Gaussian mixture where each cluster is assigned one of two labels. This problem provides a prototype of ...

Curran Associates, Inc.2020

Online Learning and Information Exponents: On The Importance of Batch size, and Time/Complexity Tradeoffs

Florent Gérard Krzakala, Bruno Loureiro, Ludovic Théo Stephan, Yatin Dandi, Luca Pesce, Luca Arnaboldi

We study the impact of the batch

n_b

on the iteration time

T

of training two-layer neural networks with one-pass stochastic gradient descent (SGD) on multi-index target functions of isotropic covariates. We characterize the optimal batch size minimizin ...

JMLR2024