Barrier Smoothing for Nonsmooth Convex Minimization

À propos
Confidentialité
Mentions légales

Graph Chatbot

Publications associées (31)

Yapa: Accelerated Proximal Algorithm For Convex Composite Problems

Giovanni Chierchia, Mireille El Gheche

Proximal splitting methods are standard tools for nonsmooth optimization. While primal-dual methods have become very popular in the last decade for their flexibility, primal methods may still be preferred for two reasons: acceleration schemes are more effe ...

IEEE2021

A Newton Frank-Wolfe method for constrained self-concordant minimization

Volkan Cevher, Quoc Tran Dinh

We develop a new Newton Frank-Wolfe algorithm to solve a class of constrained self-concordant minimization problems using linear minimization oracles (LMO). Unlike L-smooth convex functions, where the Lipschitz continuity of the objective gradient holds gl ...

SPRINGER2022

Last iterate convergence of SGD for Least-Squares in the Interpolation regime

Aditya Vardhan Varre, Nicolas Henri Bernard Flammarion, Loucas Pillaud-Vivien

Motivated by the recent successes of neural networks that have the ability to fit the data perfectly \emph{and} generalize well, we study the noiseless model in the fundamental least-squares setup. We assume that an optimum predictor fits perfectly inputs ...

Advances in Neural Information Processing Systems 20212021

STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization

Volkan Cevher, Ali Kavis

In this work we investigate stochastic non-convex optimization problems wherethe objective is an expectation over smooth loss functions, and the goal is to find an approximate stationary point. The most popular approach to handling such problems is varianc ...

2021

Regret Minimization in Stochastic Non-Convex Learning via a Proximal-Gradient Approach

Volkan Cevher, Shaul Nadav Hallak

This paper develops a methodology for regret minimization with stochastic first-order oracle feedback in online, constrained, non-smooth, non-convex problems. In this setting, the minimization of external regret is beyond reach for first-order methods, and ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity

Scott William Pesme, Nicolas Henri Bernard Flammarion, Loucas Pillaud-Vivien

Understanding the implicit bias of training algorithms is of crucial importance in order to explain the success of overparametrised neural networks. In this paper, we study the dynamics of stochastic gradient descent over diagonal linear networks through i ...

2021

A unifying framework for form-finding and topology-finding of tensegrity structures

Yafeng Wang

This paper presents a unifying framework for the form-finding and topology-finding of tensegrity structures. The novel computational framework is based on rank-constrained linear matrix inequalities. For form-finding, given the topology (i.e., member conne ...

PERGAMON-ELSEVIER SCIENCE LTD2021

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

Martin Jaggi, Sebastian Urban Stich, Amirkeivan Mohtashami

It has been experimentally observed that the efficiency of distributed training with stochastic gradient (SGD) depends decisively on the batch size and—in asynchronous implementations—on the gradient staleness. Especially, it has been observed that the spe ...

MICROTOME PUBLISHING2021

Gradient-based Methods for Deep Model Interpretability

Suraj Srinivas

In this dissertation, we propose gradient-based methods for characterizing model behaviour for the purposes of knowledge transfer and post-hoc model interpretation. Broadly, gradients capture the variation of some output feature of the model upon unit vari ...

EPFL2021

Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem

Francesca Mignacco, Lenka Zdeborová

In this paper we investigate how gradient-based algorithms such as gradient descent (GD), (multi-pass) stochastic GD, its persistent variant, and the Langevin algorithm navigate non-convex loss-landscapes and which of them is able to reach the best general ...

2021

AKSEL: Fast Byzantine SGD

Rachid Guerraoui, El Mahdi El Mhamdi, Alexandre David Olivier Maurer, Sébastien Louis Alexandre Rouault

Modern machine learning architectures distinguish servers and workers. Typically, a d-dimensional model is hosted by a server and trained by n workers, using a distributed stochastic gradient descent (SGD) optimization scheme. At each SGD step, the goal is ...

Schloss Dagstuhl--Leibniz-Zentrum für Informatik2021

Page 2 sur 2