Home

Person

Lenka Zdeborová

About
Privacy
Disclaimer

Graph Chatbot

Related publications (72)

Page 3 of 4

Learning curves of generic features maps for realistic datasets with a teacher-student model

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Hugo Chao Cui

Teacher-student models provide a framework in which the typical-case performance of high-dimensional supervised learning can be described in closed form. The assumptions of Gaussian i.i.d. input data underlying the canonical teacher-student model may, howe ...

2021

Generalization error rates in kernel regression: the crossover from the noiseless to noisy regime*

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Hugo Chao Cui

In this manuscript we consider kernel ridge regression (KRR) under the Gaussian design. Exponents for the decay of the excess generalization error of KRR have been reported in various works under the assumption of power-law decay of eigenvalues of the feat ...

2022

High-dimensional Asymptotics of Denoising Autoencoders

Lenka Zdeborová, Hugo Chao Cui

We address the problem of denoising data from a Gaussian mixture using a twolayer non-linear autoencoder with tied weights and a skip connection. We consider the high-dimensional limit where the number of training samples and the input dimension jointly te ...

2023

Integer traffic assignment problem: Algorithms and insights on random graphs

Rayan Harfouche, Lenka Zdeborová, Giovanni Piccioli

Path optimization is a fundamental concern across various real-world scenarios, ranging from traffic congestion issues to efficient data routing over the Internet. The traffic assignment problem (TAP) is a classic continuous optimization problem in this fi ...

2025

Analysis of Bootstrap and Subsampling in High-dimensional Regularized Regression

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Adrien Vandenbroucque, Lucas Andry Clarte, Guillaume Cyprien Dalle

We investigate popular resampling methods for estimating the uncertainty of statistical models, such as subsampling, bootstrap and the jackknife, and their performance in high-dimensional supervised regression tasks. We provide a tight asymptotic descripti ...

ACM2024

The committee machine: Computational to statistical gaps in learning a two-layers neural network

Nicolas Macris, Florent Gérard Krzakala, Lenka Zdeborová, Jean François Emmanuel Barbier

Heuristic tools from statistical physics have been used in the past to locate the phase transitions and compute the optimal learning and generalization errors in the teacher-student scenario in multi-layer neural networks. In this contribution, we provide ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2018

Theoretical characterization of uncertainty in high-dimensional linear classification

Lenka Zdeborová, Florent Gérard Krzakala, Lucas Andry Clarte

Being able to reliably assess not only the accuracy but also the uncertainty of models' predictions is an important endeavor in modern machine learning. Even if the model generating the data and labels is known, computing the intrinsic uncertainty after le ...

IOP Publishing Ltd2023

Large deviations of semisupervised learning in the stochastic block model

Lenka Zdeborová, Hugo Chao Cui, Luca Saglietti

In semisupervised community detection, the membership of a set of revealed nodes is known in addition to the graph structure and can be leveraged to achieve better inference accuracies. While previous works investigated the case where the revealed nodes ar ...

AMER PHYSICAL SOC2022

Asymptotics of feature learning in two-layer networks after one gradient-step

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Hugo Chao Cui, Yatin Dandi, Luca Pesce, Yue Lu

In this manuscript, we investigate the problem of how two-layer neural networks learn features from data, and improve over the kernel regime, after being trained with a single gradient descent step. Leveraging the insight from (Ba et al., 2022), we model t ...

2024

Gibbs sampling the posterior of neural networks

Lenka Zdeborová, Emanuele Troiani, Giovanni Piccioli

In this paper, we study sampling from a posterior derived from a neural network. We propose a new probabilistic model consisting of adding noise at every pre- and post-activation in the network, arguing that the resulting posterior can be sampled using an ...

Iop Publishing Ltd2024

The committee machine: computational to statistical gaps in learning a two-layers neural network

Nicolas Macris, Florent Gérard Krzakala, Lenka Zdeborová, Jean François Emmanuel Barbier

IOP PUBLISHING LTD2019

The planted k-factor problem

Lenka Zdeborová, Gabriele Sicuro

We consider the problem of recovering an unknown k-factor, hidden in a weighted random graph. For k = 1 this is the planted matching problem, while the k = 2 case is closely related to the planted traveling salesman problem. The inference problem is solved ...

IOP PUBLISHING LTD2021

Subspace clustering in high-dimensions: Phase transitions & Statistical-to-Computational gap

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Luca Pesce

A simple model to study subspace clustering is the high-dimensional k -Gaussian mixture model where the cluster means are sparse vectors. Here we provide an exact asymptotic characterization of the statistically optimal reconstruction error in this model i ...

2022

Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification*

Lenka Zdeborová, Florent Gérard Krzakala

We analyze in a closed form the learning dynamics of the stochastic gradient descent (SGD) for a single-layer neural network classifying a high-dimensional Gaussian mixture where each cluster is assigned one of two labels. This problem provides a prototype ...

2021

Aligning random graphs with a sub-tree similarity message-passing algorithm

Lenka Zdeborová, Giovanni Piccioli

The problem of aligning Erdos-Renyi random graphs is a noisy, average-case version of the graph isomorphism problem, in which a pair of correlated random graphs is observed through a random permutation of their vertices. We study a polynomial time message- ...

IOP Publishing Ltd2022

Sampling with flows, diffusion, and autoregressive neural networks from a spin-glass perspective

Lenka Zdeborová, Florent Gérard Krzakala, Yatin Dandi, Davide Ghio

Recent years witnessed the development of powerful generative models based on flows, diffusion, or autoregressive neural networks, achieving remarkable success in generating data from examples with applications in a broad range of areas. A theoretical anal ...

Proceedings of the National Academy of Sciences2024

Bilinear Sequence Regression: A Model for Learning from Long Sequences of High-Dimensional Tokens

Lenka Zdeborová, Emanuele Troiani, Vittorio Erba, Luca Biggio

Current progress in artificial intelligence is centered around so-called large language models that consist of neural networks processing long sequences of high-dimensional vectors called tokens. Statistical physics provides powerful tools to study the fun ...

American Physical Society2025

Phase diagram of stochastic gradient descent in high-dimensional two-layer neural networks

Lenka Zdeborová, Florent Gérard Krzakala, Ludovic Théo Stephan, Rodrigo Soares Veiga

Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achieve global convergence under gradient descent. The picture can be radically different for narrow networks, which tend to get stuck in badly-generalizing local ...

Iop Publishing Ltd2023

Machine learning and statistical physics: preface

Lenka Zdeborová

IOP PUBLISHING LTD2020

The Gaussian equivalence of generative models for learning with shallow neural networks

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro

Understanding the impact of data structure on the computational tractability of learning is a key challenge for the theory of neural networks. Many theoretical works do not explicitly model training data, or assume that inputs are drawn component-wise inde ...

2021