Accelerated And Inexact Forward-Backward Algorithms

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Universal methods for optimization are designed to achieve theoretically optimal convergence rates without any prior knowledge of the problem’s regularity parameters or the accurarcy of the gradient oracle employed by the optimizer. In this regard, existin ...

Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achieve global convergence under gradient descent. The picture can be radically different for narrow net-works, which tend to get stuck in badly-generalizing loca ...

In this thesis, we study the stochastic heat equation (SHE) on bounded domains and on the whole Euclidean space

\R^d.

We confirm the intuition that as the bounded domain increases to the whole space, both solutions become arbitrarily close to one another ...

The parallel Schwarz method (PSM) is an overlapping domain decomposition (DD) method to solve partial differential equations (PDEs). Similarly to classical nonoverlapping DD methods, the PSM admits a substructured formulation, that is, it can be formulated ...

This article studies a class of nonsmooth decentralized multiagent optimization problems where the agents aim at minimizing a sum of local strongly-convex smooth components plus a common nonsmooth term. We propose a general primal-dual algorithmic framewor ...

State-of-the-art training algorithms for deep learning models are based on stochastic gradient descent (SGD). Recently, many variations have been explored: perturbing parameters for better accuracy (such as in Extra-gradient), limiting SGD updates to a sub ...

In this work we investigate stochastic non-convex optimization problems wherethe objective is an expectation over smooth loss functions, and the goal is to find an approximate stationary point. The most popular approach to handling such problems is varianc ...

There has been a recent surge of interest in the study of asymptotic reconstruction performance in various cases of generalized linear estimation problems in the teacher-student setting, especially for the case of i.i.d standard normal matrices. Here, we g ...

Computational protein design has emerged as a powerful tool capable of identifying sequences compatible with pre-defined protein structures. The sequence design protocols, implemented in the Rosetta suite, have become widely used in the protein engineering ...

Understanding the implicit bias of training algorithms is of crucial importance in order to explain the success of overparametrised neural networks. In this paper, we study the dynamics of stochastic gradient descent over diagonal linear networks through i ...

Astrobots are robotic artifacts whose swarms are used in astrophysical studies to generate the map of the observable universe. These swarms have to be coordinated with respect to various desired observations. Such coordination\footnote{\z{A coordination sa ...

Stochastic gradient descent (SGD) and randomized coordinate descent (RCD) are two of the workhorses for training modern automated decision systems. Intriguingly, convergence properties of these methods are not well-established as we move away from the spec ...

Motivated by the recent successes of neural networks that have the ability to fit the data perfectly \emph{and} generalize well, we study the noiseless model in the fundamental least-squares setup. We assume that an optimum predictor fits perfectly inputs ...

Driven by the need to solve increasingly complex optimization problems in signal processing and machine learning, there has been increasing interest in understanding the behavior of gradient-descent algorithms in non-convex environments. Most available wor ...

The Dirichlet-Neumann (DN) method has been extensively studied for linear partial differential equations, while little attention has been devoted to the nonlinear case. In this paper, we analyze the DN method both as a nonlinear iterative method and as a p ...

We consider the numerical approximation of an optimal control problem for an elliptic Partial Differential Equation (PDE) with random coefficients. Specifically, the control function is a deterministic, distributed forcing term that minimizes the expected ...

Inverse reconstruction from images is a central problem in many scientific and engineering disciplines. Recent progress on differentiable rendering has led to methods that can efficiently differentiate the full process of image formation with respect to mi ...

Accelerated And Inexact Forward-Backward Algorithms

Chat with Graph Search

UNDERGRAD: A Universal Black-Box Optimization Method with Almost Dimension-Free Convergence Rate Guarantees

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Localization errors of the stochastic heat equation

Spectral Coarse Spaces for the Substructured Parallel Schwarz Method

Decentralized Proximal Gradient Algorithms With Linear Convergence Rates

Masked Training of Neural Networks with Partial Gradients

STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization

Asymptotic Errors for Teacher-Student Convex Generalized Linear Models (Or: How to Prove Kabashima’s Replica Formula)

A Rosetta-based protein design protocol converging to natural sequences

Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity

Data-Driven Convergence Prediction of Astrobots Swarms

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Last iterate convergence of SGD for Least-Squares in the Interpolation regime

Distributed Learning in Non-Convex Environments-Part I: Agreement at a Linear Rate

On the nonlinear Dirichlet-Neumann method and preconditioner for Newton's method

Complexity analysis of stochastic gradient methods for PDE-constrained optimal control problems with uncertain parameters

Large Steps in Inverse Rendering of Geometry