Recherche

Afficher tous les résultats pour

Accueil

Personne

Lenka Zdeborová

À propos
Confidentialité
Mentions légales

Graph Chatbot

Publications associées (72)

Page 2 sur 4

Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem

Lenka Zdeborová

In this paper we investigate how gradient-based algorithms such as gradient descent (GD), (multi-pass) stochastic GD, its persistent variant, and the Langevin algorithm navigate non-convex loss-landscapes and which of them is able to reach the best general ...

2021

Perturbative construction of mean-field equations in extensive-rank matrix factorization and denoising

Lenka Zdeborová, Florent Gérard Krzakala

Factorization of matrices where the rank of the two factors diverges linearly with their sizes has many applications in diverse areas such as unsupervised representation learning, dictionary learning or sparse coding. We consider a setting where the two fa ...

2022

Probing transfer learning with a model of synthetic correlated datasets

Lenka Zdeborová, Luca Saglietti, Federica Gerace

Transfer learning can signiﬁcantly improve the sample eﬃciency of neural networks, by exploiting the relatedness between a data-scarce target task and a data-abundant source task. Despite years of successful applications, transfer learning practice often r ...

2022

Low-rank matrix estimation with inhomogeneous noise

Lenka Zdeborová, Florent Gérard Krzakala

We study low-rank matrix estimation for a generic inhomogeneous output channel through which the matrix is observed. This generalizes the commonly considered spiked matrix model with homogeneous noise to include for instance the dense degree-corrected stoc ...

Oxford University Press2025

A phase transition between positional and semantic learning in a solvable model of dot-product attention

Lenka Zdeborová, Florent Gérard Krzakala, Hugo Chao Cui, Freya Behrens

Many empirical studies have provided evidence for the emergence of algorithmic mechanisms (abilities) in the learning of language models, that lead to qualitative improvements in the model capabilities. Yet, a theoretical characterization of how such mecha ...

2025

Multi-layer state evolution under random convolutional design

Lenka Zdeborová, Florent Gérard Krzakala

Signal recovery under generative neural network priors has emerged as a promising direction in statistical inference and computational imaging. Theoretical analysis of reconstruction algorithms under generative priors is, however, challenging. For generati ...

2023

Expectation consistency for calibration of neural networks

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Lucas Andry Clarte

Despite their incredible performance, it is well reported that deep neural networks tend to be overoptimistic about their prediction confidence. Finding effective and efficient calibration methods for neural networks is therefore an important endeavour tow ...

2023

Phase retrieval in high dimensions: Statistical and computational phase transitions

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro

We consider the phase retrieval problem of reconstructing a n -dimensional real or complex signal X ⋆ from m (possibly noisy) observations Y μ = | ∑ n i = 1 Φ μ i X ⋆ i / √ n | , for a large class of correlated real and complex random sensing matrices Φ , ...

Curran Associates, Inc.2020

Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed

Lenka Zdeborová, Florent Gérard Krzakala, Maria Refinetti

A recent series of theoretical works showed that the dynamics of neural networks with a certain initialisation are well-captured by kernel methods. Concurrent empirical work demonstrated that kernel methods can come close to the performance of neural netwo ...

PMLR2021

(Dis)assortative partitions on random regular graphs

Lenka Zdeborová, Freya Behrens, Yaroslav Kivva

We study the problem of assortative and disassortative partitions on random dregular graphs. Nodes in the graph are partitioned into two non-empty groups. In the assortative partition every node requires at least H of their neighbors to be in their own gro ...

IOP Publishing Ltd2022

Counting and hardness-of-finding fixed points in cellular automata on random graphs

Cédric Xavier Koller, Lenka Zdeborová, Freya Behrens

We study the fixed points of outer-totalistic cellular automata on sparse random regular graphs. These can be seen as constraint satisfaction problems, where each variable must adhere to the same local constraint, which depends solely on its state and the ...

IOP Publishing2024

Large deviations in the perceptron model and consequences for active learning

Lenka Zdeborová, Luca Saglietti

Active learning (AL) is a branch of machine learning that deals with problems where unlabeled data is abundant yet obtaining labels is expensive. The learning algorithm has the possibility of querying a limited number of samples to obtain the corresponding ...

2021

Construction of optimal spectral methods in phase retrieval

Lenka Zdeborová, Florent Gérard Krzakala

We consider the phase retrieval problem, in which the observer wishes to recover a n-dimensional real or complex signal X⋆ from the (possibly noisy) observation of |ΦX⋆|, in which Φ is a matrix of size m×n. We consider a \emph{high-dimensional} setting whe ...

2021

Neural-prior stochastic block model

Lenka Zdeborová, Odilon Marie Cyprien Duranthon

The stochastic block model (SBM) is widely studied as a benchmark for graph clustering aka community detection. In practice, graph data often come with node attributes that bear additional information about the communities. Previous works modeled such data ...

2023

Planted XY model: Thermodynamics and inference

Lenka Zdeborová, Guanhao Huang, Siyu Chen, Giovanni Piccioli

In this paper we study a fully connected planted spin glass named the planted XY model. Motivation for studying this system comes both from the spin glass field and the one of statistical inference where it models the angular synchronization problem. We de ...

2022

Generalisation error in learning with random features and the hidden manifold model*

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Federica Gerace

We study generalised linear regression and classification for a synthetically generated dataset encompassing different problems of interest, such as learning with random features, neural networks in the lazy training regime, and the hidden manifold model. ...

IOP Publishing Ltd2021

Optimal Inference in Contextual Stochastic Block Models

Lenka Zdeborová, Odilon Marie Cyprien Duranthon

The contextual stochastic block model (CSBM) was proposed for unsupervised community detection on attributed graphs where both the graph and the high-dimensional node information correlate with node labels. In the context of machine learning on graphs, the ...

Journal of Machine Learning Research Inc.2024

Learning Gaussian Mixtures with Generalized Linear Models: Precise Asymptotics in High-dimensions

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Gabriele Sicuro

Generalised linear models for multi-class classification problems are one of the fundamental building blocks of modern machine learning tasks. In this manuscript, we characterise the learning of a mixture of K Gaussians with generic means and covariances v ...

2021

Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval

Lenka Zdeborová, Florent Gérard Krzakala

Despite the widespread use of gradient-based algorithms for optimizing high-dimensional non-convex functions, understanding their ability of finding good minima instead of being trapped in spurious ones remains to a large extent an open problem. Here we fo ...

Curran Associates, Inc.2020

Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regime

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Hugo Chao Cui

In this manuscript we consider Kernel Ridge Regression (KRR) under the Gaussian design. Exponents for the decay of the excess generalization error of KRR have been reported in various works under the assumption of power-law decay of eigenvalues of the feat ...

2021