Publication

Convergence Of Variance-Reduced Learning Under Random Reshuffling

Publications associées (30)

Data-Driven Convergence Prediction of Astrobots Swarms

Denis Gillet, Jean-Paul Richard Kneib, Matin Macktoobian, Francesco Basciani

Astrobots are robotic artifacts whose swarms are used in astrophysical studies to generate the map of the observable universe. These swarms have to be coordinated with respect to various desired observations. Such coordination\footnote{\z{A coordination sa ...

2022

On the nonlinear Dirichlet-Neumann method and preconditioner for Newton's method

Tommaso Vanzan

The Dirichlet-Neumann (DN) method has been extensively studied for linear partial differential equations, while little attention has been devoted to the nonlinear case. In this paper, we analyze the DN method both as a nonlinear iterative method and as a p ...

Springer-Verlag2023

Last iterate convergence of SGD for Least-Squares in the Interpolation regime

Aditya Vardhan Varre, Nicolas Henri Bernard Flammarion, Loucas Pillaud-Vivien

Motivated by the recent successes of neural networks that have the ability to fit the data perfectly \emph{and} generalize well, we study the noiseless model in the fundamental least-squares setup. We assume that an optimum predictor fits perfectly inputs ...

Advances in Neural Information Processing Systems 20212021

Coupling-dependent convergence behavior of phase oscillators with tegotae-control

Auke Ijspeert, Jonathan Patrick Arreguit O'Neill, Simon Lukas Hauser, Matthieu Guillaume Marie Dujany

A bio-inspired way to model locomotion is using a network of coupled phase oscillators to create a Central Pattern Generator (CPG). The recently developed feedback control method tegotae includes exteroceptive force feedback into the governing phase update ...

IEEE2021

Is there an analog of Nesterov acceleration for gradient-based MCMC?

Xiang Cheng, Nicolas Henri Bernard Flammarion

We formulate gradient-based Markov chain Monte Carlo (MCMC) sampling as optimization on the space of probability measures, with Kullback-Leibler (KL) divergence as the objective functional. We show that an under-damped form of the Langevin algorithm perfor ...

2021

Learning convergence prediction of astrobots in multi-object spectrographs

Denis Gillet, Jean-Paul Richard Kneib, Matin Macktoobian, Francesco Basciani

Astrobot swarms are used to capture astronomical signals to generate the map of the observable universe for the purpose of dark energy studies. The convergence of each swarm in the course of its coordination has to surpass a particular threshold to yield a ...

2021

Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity

Scott William Pesme, Nicolas Henri Bernard Flammarion, Loucas Pillaud-Vivien

Understanding the implicit bias of training algorithms is of crucial importance in order to explain the success of overparametrised neural networks. In this paper, we study the dynamics of stochastic gradient descent over diagonal linear networks through i ...

2021

Efficient Parsing with Derivatives and Zippers

Romain Edelmann

Parsing is the process that enables a computer system to make sense of raw data. Parsing is common to almost all computer systems: It is involved every time sequential data is read and elaborated into structured data. The theory of parsing usually focuses ...

EPFL2021

A comment on performance guarantees of a greedy algorithm for minimizing a supermodular set function on comatroid

Orcun Karaca, Baiwei Guo, Maryam Kamgarpour

We provide a counterexample to the performance guarantee obtained in the paper "Il'ev, V., Linker, N., 2006. Performance guarantees of a greedy algorithm for minimizing a supermodular set function on comatroid", which was published in Volume 171 of the Eur ...

ELSEVIER2021

Variance-Reduced Stochastic Learning Under Random Reshuffling

Ali H. Sayed, Bicheng Ying, Kun Yuan

Several useful variance-reduced stochastic gradient algorithms, such as SVRG, SAGA, Finito, and SAG, have been proposed to minimize empirical risks with linear convergence properties to the exact minimizer. The existing convergence results assume uniform d ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020