Accelerated And Inexact Forward-Backward Algorithms

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In this work, we develop first-order (Hessian-free) and zeroth-order (derivative-free) implementations of the Cubically Regularized Newton Method for solving general non-convex optimization problems. For that, we employ finite difference approximations of ...

Context. Imaging in radio interferometry requires solving an ill-posed noisy inverse problem, for which the most adopted algorithm is the original CLEAN algorithm and its variants. Alternative explicit optimization methods have gained increasing attention, ...

Distributed learning is the key for enabling training of modern large-scale machine learning models, through parallelising the learning process. Collaborative learning is essential for learning from privacy-sensitive data that is distributed across various ...

Driven by the need for more efficient and seamless integration of physical models and data, physics -informed neural networks (PINNs) have seen a surge of interest in recent years. However, ensuring the reliability of their convergence and accuracy remains ...

Neuroprostheses have been used clinically for decades, to help restore or preserve brain functions, when pharmaceutical treatments are inefficient. Although great progress in the field has been made over the years to interface with the nervous system, surf ...

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

We present a framework for performing regression when both covariate and response are probability distributions on a compact and convex subset of

\R^d

. Our regression model is based on the theory of optimal transport and links the conditional Fr'echet m ...

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

Kalman and H-infinity filters, the most popular paradigms for linear state estimation, are designed for very specific specific noise and disturbance patterns, which may not appear in practice. State observers based on the minimization of regret measures ar ...

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

Functional data are typically modeled as sample paths of smooth stochastic processes in order to mitigate the fact that they are often observed discretely and noisily, occasionally irregularly and sparsely. The smoothness assumption is imposed to allow for ...

Local and global inference methods have been developed to infer structural contacts from multiple sequence alignments of homologous proteins. They rely on correlations in amino acid usage at contacting sites. Because homologous proteins share a common ance ...

This paper presents a theoretical analysis of linear interpolation as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss lan ...

While adversarial training and its variants have shown to be the most effective algorithms to defend against adversarial attacks, their extremely slow training process makes it hard to scale to large datasets like ImageNet. The key idea of recent works to ...

Sharpness-Aware Minimization (SAM) is a recent training method that relies on worst-case weight perturbations which significantly improves generalization in various settings. We argue that the existing justifications for the success of SAM which are based ...

This paper considers the multi-agent linear least-squares problem in a server-agent network architecture. The system comprises multiple agents, each with a set of local data points. The agents are connected to a server, and there is no inter-agent communic ...

We develop a novel 2D functional learning framework that employs a sparsity-promoting regularization based on second-order derivatives. Motivated by the nature of the regularizer, we restrict the search space to the span of piecewise-linear box splines shi ...

We propose a stochastic conditional gradient method (CGM) for minimizing convex finitesum objectives formed as a sum of smooth and non-smooth terms. Existing CGM variants for this template either suffer from slow convergence rates, or require carefully inc ...

We study generalization properties of random features (RF) regression in high dimensions optimized by stochastic gradient descent (SGD) in under-/overparameterized regime. In this work, we derive precise non-asymptotic error bounds of RF regression under b ...

Accelerated And Inexact Forward-Backward Algorithms

Chat with Graph Search

First and Zeroth-Order Implementations of the Regularized Newton Method with Lazy Approximated Hessians

PolyCLEAN: Atomic optimization for super-resolution imaging and uncertainty estimation in radio interferometry

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Residual-based attention in physics-informed neural networks

Unraveling behavior and cortical signals to guide the development of soft neuroprostheses for auditory restoration and spreading depolarization

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Understanding generalization and robustness in modern deep learning

Distributional Regression and Autoregression via Optimal Transport

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Minimal regret state estimation of time-varying systems

Universal and adaptive methods for robust stochastic optimization

Functional data analysis with rough sample paths?

Impact of phylogeny on structural contact inference from protein sequence data

Stable Nonconvex-Nonconcave Training via Linear Interpolation

Fast Adversarial Training With Adaptive Step Size

Towards Understanding Sharpness-Aware Minimization

Iterative pre-conditioning for expediting the distributed gradient-descent method: The case of linear least-squares problem

Learning of Continuous and Piecewise-Linear Functions With Hessian Total-Variation Regularization

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

On the Double Descent of Random Features Models Trained with SGD