Publications by Yatin Dandi | EPFL Graph Search

Related publications (9)

Universality laws for Gaussian mixtures in generalized linear models

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Ludovic Théo Stephan, Yatin Dandi

A recent line of work in high-dimensional statistics working under the Gaussian mixture hypothesis has led to a number of results in the context of empirical risk minimization, Bayesian uncertainty quantification, separation of kernel methods and neural ne ...

2024

How Two-Layer Neural Networks Learn, One (Giant) Step at a Time

Florent Gérard Krzakala, Bruno Loureiro, Ludovic Théo Stephan, Yatin Dandi, Luca Pesce

For high-dimensional Gaussian data, we investigate theoretically how the features of a two-layer neural network adapt to the structure of the target function through a few large batch gradient descent steps, leading to an improvement in the approximation c ...

2024

A gentle introduction to gradient-based optimization and variational inequalities for machine learning

Yatin Dandi

The rapid progress in machine learning in recent years has been based on a highly productive connection to gradient-based optimization. Further progress hinges in part on a shift in focus from pattern recognition to decision-making and multi-agent problems ...

2024

The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents

Lenka Zdeborová, Florent Gérard Krzakala, Yatin Dandi, Emanuele Troiani, Luca Pesce, Luca Arnaboldi

We investigate the training dynamics of two-layer neural networks when learning multi-index target functions. We focus on multi-pass gradient descent (GD) that reuses the batches multiple times and show that it significantly changes the conclusion about wh ...

JMLR2024

Implicit Gradient Alignment in Distributed and Federated Learning

Martin Jaggi, Yatin Dandi, Luis Felipe Barba Flores

A major obstacle to achieving global convergence in distributed and federated learning is the misalignment of gradients across clients, or mini-batches due to heterogeneity and stochasticity of the distributed data. In this work, we show that data heteroge ...

ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE2022

Model-Agnostic Learning to Meta-Learn

Yatin Dandi, Arnout Jan J Devos

In this paper, we propose a learning algorithm that enables a model to quickly exploit commonalities among related tasks from an unseen task distribution, before quickly adapting to specific tasks from that same distribution. We investigate how learning wi ...

PMLR2021

Asymptotics of feature learning in two-layer networks after one gradient-step

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Hugo Chao Cui, Yatin Dandi, Luca Pesce, Yue Lu

In this manuscript, we investigate the problem of how two-layer neural networks learn features from data, and improve over the kernel regime, after being trained with a single gradient descent step. Leveraging the insight from (Ba et al., 2022), we model t ...

2024

Sampling with flows, diffusion, and autoregressive neural networks from a spin-glass perspective

Lenka Zdeborová, Florent Gérard Krzakala, Yatin Dandi, Davide Ghio

Recent years witnessed the development of powerful generative models based on flows, diffusion, or autoregressive neural networks, achieving remarkable success in generating data from examples with applications in a broad range of areas. A theoretical anal ...

Proceedings of the National Academy of Sciences2024

Online Learning and Information Exponents: On The Importance of Batch size, and Time/Complexity Tradeoffs

Florent Gérard Krzakala, Bruno Loureiro, Ludovic Théo Stephan, Yatin Dandi, Luca Pesce, Luca Arnaboldi

We study the impact of the batch

n_b

on the iteration time

T

of training two-layer neural networks with one-pass stochastic gradient descent (SGD) on multi-index target functions of isotropic covariates. We characterize the optimal batch size minimizin ...

JMLR2024