Publication

Fast rates by transferring from auxiliary hypotheses

Publications associées (31)

STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization

In this work we investigate stochastic non-convex optimization problems wherethe objective is an expectation over smooth loss functions, and the goal is to find an approximate stationary point. The most popular approach to handling such problems is varianc ...

2021

Parametrized classifiers for optimal EFT sensitivity

Andrea Wulzer, Siyu Chen, Alfredo Glioti

We study unbinned multivariate analysis techniques, based on Statistical Learning, for indirect new physics searches at the LHC in the Effective Field Theory framework. We focus in particular on high-energy ZW production with fully leptonic decays, modeled ...

Springer Nature2021

Landscape and training regimes in deep learning

Mario Geiger, Matthieu Wyart, Leonardo Petrini

Deep learning algorithms are responsible for a technological revolution in a variety oftasks including image recognition or Go playing. Yet, why they work is not understood.Ultimately, they manage to classify data lying in high dimension – a feat generical ...

2021

Distributed Learning in Non-Convex Environments-Part I: Agreement at a Linear Rate

Ali H. Sayed, Stefan Vlaski

Driven by the need to solve increasingly complex optimization problems in signal processing and machine learning, there has been increasing interest in understanding the behavior of gradient-descent algorithms in non-convex environments. Most available wor ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

Continual Learning for Natural Language Generation in Task-oriented Dialog Systems

Boi Faltings, Fei Mi, Liangwei Chen, Mengjie Zhao

Natural language generation (NLG) is an essential component of task-oriented dialog systems. Despite the recent success of neural approaches for NLG, they are typically developed in an offline manner for particular domains. To better fit real-life applicat ...

2021

Regret Minimization in Stochastic Non-Convex Learning via a Proximal-Gradient Approach

Volkan Cevher, Shaul Nadav Hallak

This paper develops a methodology for regret minimization with stochastic first-order oracle feedback in online, constrained, non-smooth, non-convex problems. In this setting, the minimization of external regret is beyond reach for first-order methods, and ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Impact of quantum-chemical metrics on the machine learning prediction of electron density

Anne-Clémence Corminboeuf, Alberto Fabrizio, Ksenia Briling

Machine learning (ML) algorithms have undergone an explosive development impacting every aspect of computational chemistry. To obtain reliable predictions, one needs to maintain a proper balance between the black-box nature of ML frameworks and the physics ...

AMER INST PHYSICS2021

Disparity Between Batches as a Signal for Early Stopping

Patrick Thiran, Mahsa Forouzesh

We propose a metric for evaluating the generalization ability of deep neural networks trained with mini-batch gradient descent. Our metric, called gradient disparity, is the l2 norm distance between the gradient vectors of two mini-batches drawn from the t ...

Springer2021

Last iterate convergence of SGD for Least-Squares in the Interpolation regime

Aditya Vardhan Varre, Nicolas Henri Bernard Flammarion, Loucas Pillaud-Vivien

Motivated by the recent successes of neural networks that have the ability to fit the data perfectly \emph{and} generalize well, we study the noiseless model in the fundamental least-squares setup. We assume that an optimum predictor fits perfectly inputs ...

Advances in Neural Information Processing Systems 20212021

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Ahmet Alacaoglu

Stochastic gradient descent (SGD) and randomized coordinate descent (RCD) are two of the workhorses for training modern automated decision systems. Intriguingly, convergence properties of these methods are not well-established as we move away from the spec ...

EPFL2021

Fast screening of covariates in population models empowered by machine learning

Jan Sickmann Hesthaven, Nadia Terranova

One of the objectives of Pharmacometry (PMX) population modeling is the identification of significant and clinically relevant relationships between parameters and covariates. Here, we demonstrate how this complex selection task could benefit from supervise ...

2021