Publications associées à Linear Speedup In Saddle-Point Escape For Decentralized Non-Convex Optimization

Publications associées (49)

Framework to engineer existing bridges for sustainability, validated by applications

A framework to engineer existing structures, in particular bridges, and novel engineering methods and technologies are presented to extend the service duration of existing bridges while meeting the requirements of modern use. Accurate determination of brid ...

TAYLOR & FRANCIS LTD2025

Safe Zeroth-Order Convex Optimization Using Quadratic Local Approximations

Giancarlo Ferrari Trecate, Yuning Jiang, Maryam Kamgarpour, Baiwei Guo

We address black-box convex optimization problems, where the objective and constraint functions are not explicitly known but can be sampled within the feasible set. The challenge is thus to generate a sequence of feasible points converging towards an optim ...

IEEE2023

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Scott William Pesme

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

Universal and adaptive methods for robust stochastic optimization

Ali Kavis

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

EPFL2023

End-to-End Learning for Stochastic Optimization: A Bayesian Perspective

Daniel Kuhn, Tobias Sutter, Yves Rychener

We develop a principled approach to end-to-end learning in stochastic optimization. First, we show that the standard end-to-end learning algorithm admits a Bayesian interpretation and trains a posterior Bayes action map. Building on the insights of this an ...

2023

BENIGN LANDSCAPES OF LOW-DIMENSIONAL RELAXATIONS FOR ORTHOGONAL SYNCHRONIZATION ON GENERAL GRAPHS

Nicolas Boumal

Orthogonal group synchronization is the problem of estimating n elements Z(1),& mldr;,Z(n) from the rxr orthogonal group given some relative measurements R-ij approximate to Z(i)Z(j)(-1). The least-squares formulation is nonconvex. To avoid its local minim ...

Siam Publications2024

Diffusion Moving-Average Adaptation Over Networks

Ali H. Sayed

Recently, the diffusion moving-average (D-MA) scheme has been proposed as a way to combat noisy links over adaptive networks. However, the current theoretical results focus on networks with mean-square error costs where the optimal local solution agrees wi ...

2024

Data-Driven Control and Optimization under Noisy and Uncertain Conditions

Baiwei Guo

Control systems operating in real-world environments often face disturbances arising from measurement noise and model mismatch. These factors can significantly impact the perfor- mance and safety of the system. In this thesis, we aim to leverage data to de ...

EPFL2023

Learning to Optimize with Convergence Guarantees Using Nonlinear System Theory

Luca Furieri, Andrea Martin

The increasing reliance on numerical methods for controlling dynamical systems and training machine learning models underscores the need to devise algorithms that dependably and efficiently navigate complex optimization landscapes. Classical gradient desce ...

2024

A comprehensive review of digital twin-part 2: roles of uncertainty quantification and optimization, a battery digital twin, and perspectives

Olga Fink

As an emerging technology in the era of Industry 4.0, digital twin is gaining unprecedented attention because of its promise to further optimize process design, quality control, health monitoring, decision- and policy-making, and more, by comprehensively m ...

SPRINGER2023

The effect of smooth parametrizations on nonconvex optimization landscapes

Nicolas Boumal

We develop new tools to study landscapes in nonconvex optimization. Given one optimization problem, we pair it with another by smoothly parametrizing the domain. This is either for practical purposes (e.g., to use smooth optimization algorithms with good g ...

Springer Heidelberg2024

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Negar Kiyavash, Saber Salehkaleybar

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

Finding stationary points on bounded-rank matrices: a geometric hurdle and a smooth remedy

Nicolas Boumal

We consider the problem of provably finding a stationary point of a smooth function to be minimized on the variety of bounded-rank matrices. This turns out to be unexpectedly delicate. We trace the difficulty back to a geometric obstacle: On a nonsmooth se ...

SPRINGER HEIDELBERG2022

Learning of Continuous and Piecewise-Linear Functions With Hessian Total-Variation Regularization

Michaël Unser, Shayan Aziznejad, Joaquim Gonçalves Garcia Barreto Campos

We develop a novel 2D functional learning framework that employs a sparsity-promoting regularization based on second-order derivatives. Motivated by the nature of the regularizer, we restrict the search space to the span of piecewise-linear box splines shi ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Optimal Combination Policies For Adaptive Social Learning

Stefan Vlaski, Virginia Bordignon, Ping Hu

This paper investigates the effect of combination policies on the performance of adaptive social learning in non-stationary environments. By analyzing the relation between the error probability and the underlying graph topology, we prove that in the slow a ...

IEEE2022

Neural System Level Synthesis: Learning over All Stabilizing Policies for Nonlinear Systems

Giancarlo Ferrari Trecate, Luca Furieri, Clara Lucía Galimberti

We address the problem of designing stabilizing control policies for nonlinear systems in discrete-time, while minimizing an arbitrary cost function. When the system is linear and the cost is convex, the System Level Synthesis (SLS) approach offers an effe ...

IEEE2022

STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization

Volkan Cevher, Ali Kavis

In this work we investigate stochastic non-convex optimization problems wherethe objective is an expectation over smooth loss functions, and the goal is to find an approximate stationary point. The most popular approach to handling such problems is varianc ...

2021

Distributed Learning in Non-Convex Environments-Part II: Polynomial Escape From Saddle-Points

Ali H. Sayed, Stefan Vlaski

The diffusion strategy for distributed learning from streaming data employs local stochastic gradient updates along with exchange of iterates over neighborhoods. In Part I [3] of this work we established that agents cluster around a network centroid and pr ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

Logical Team Q-learning: An approach towards factored policies in cooperative MARL

Ali H. Sayed, Lucas Cesar Eduardo Cassano

We address the challenge of learning factored policies in cooperative MARL scenarios. In particular, we consider the situation in which a team of agents collaborates to optimize a common cost. The goal is to obtain factored policies that determine the indi ...

MICROTOME PUBLISHING2021

High Resolution Light-Based 3D Printing Of Cell-Laden Bio Constructs

Christophe Moser, Damien Claude-Marie Loterie, Paul Delrot, Jorge Andres Madrid Wolff, Antoine Vincent Boniface

MARY ANN LIEBERT, INC2023