Publication

Distributed Optimization with Arbitrary Local Solvers

Publications associées (26)

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Distributed learning is the key for enabling training of modern large-scale machine learning models, through parallelising the learning process. Collaborative learning is essential for learning from privacy-sensitive data that is distributed across various ...

EPFL2024

Byzantine Machine Learning: A Primer

Rachid Guerraoui, Nirupam Gupta, Rafaël Benjamin Pinot

The problem of Byzantine resilience in distributed machine learning, a.k.a., Byzantine machine learning, consists in designing distributed algorithms that can train an accurate model despite the presence of Byzantine nodes, i.e., nodes with corrupt data or ...

2023

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Mehmet Fatih Sahin

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

EPFL2023

Byzantine-Resilient Learning Beyond Gradients: Distributing Evolutionary Search

Rachid Guerraoui, Andrei Kucharavy, Matteo Monti

Modern machine learning (ML) models are capable of impressive performances. However, their prowess is not due only to the improvements in their architecture and training algorithms but also to a drastic increase in computational power used to train them.|S ...

Assoc Computing Machinery2023

Predicting in Uncertain Environments: Methods for Robust Machine Learning

Paul Thierry Yves Rolland

One of the main goal of Artificial Intelligence is to develop models capable of providing valuable predictions in real-world environments. In particular, Machine Learning (ML) seeks to design such models by learning from examples coming from this same envi ...

EPFL2022

Automated Design: A Journey Across Modelling, Optimization, and Education

Cyril Picard

Machine intelligence greatly impacts almost all domains of our societies. It is profoundly changing the field of mechanical engineering with new technical possibilities and processes. The education of future engineers also needs to adapt in terms of techni ...

EPFL2021

Gramian-Based Adaptive Combination Policies For Diffusion Learning Over Networks

Ali H. Sayed, Stefan Vlaski, Yigit Efe Erginbas

This paper presents an adaptive combination strategy for distributed learning over diffusion networks. Since learning relies on the collaborative processing of the stochastic information at the dispersed agents, the overall performance can be improved by d ...

IEEE2021

Linear convergence of primal-dual gradient methods and their performance in distributed optimization

Ali H. Sayed, Sulaiman A S A E Alghunaim

In this work, we revisit a classical incremental implementation of the primal-descent dual-ascent gradient method used for the solution of equality constrained optimization problems. We provide a short proof that establishes the linear (exponential) conver ...

PERGAMON-ELSEVIER SCIENCE LTD2020

A Multi-Agent Primal-Dual Strategy for Composite Optimization over Distributed Features

Ming Yan, Ali H. Sayed, Sulaiman A S A E Alghunaim

This work studies multi-agent sharing optimization problems with the objective function being the sum of smooth local functions plus a convex (possibly non-smooth) function coupling all agents. This scenario arises in many machine learning and engineering ...

IEEE2020

Forward-Backward Splitting for Optimal Transport Based Problems

Pascal Frossard, Effrosyni Simou, Hermina Petric Maretic, Mireille El Gheche, Guillermo Ortiz Jimenez

Optimal transport aims to estimate a transportation plan that minimizes a displacement cost. This is realized by optimizing the scalar product between the sought plan and the given cost, over the space of doubly stochastic matrices. When the entropy regula ...

IEEE2020

Sustainability as a Competitive Tool for the Brave? Or for the Best? A Behavioral Theory Perspective

Lorenzo Massa

Who invests in sustainability as a strategic choice? We adopt a behavioral theory perspective to explain the heterogeneity of sustainability-driven investments. Building on problemistic search and organizational inertia arguments, and contrary to resource- ...

SAGE PUBLICATIONS INC2020

Glutamine-to-glutamate ratio in the nucleus accumbens predicts effort-based motivated performance in humans

Maria del Carmen Sandi Perez, Rolf Gruetter, Gediminas Luksys, Lijing Xin, Alina Veronika Irene Strasser

Substantial evidence implicates the nucleus accumbens in motivated performance, but very little is known about the neurochemical underpinnings of individual differences in motivation. Here, we applied 1H magnetic resonance spectroscopy (1H-MRS) at ultra-hi ...

2020

Methods for strong gravitational lens detection and analysis using machine learning and high performance computing

Christoph Ernst René Schäfer

The study of strong gravitational lenses is a relatively new scientific field in astronomy with many applications in cosmology. Its unique observables allow astronomers to trace dark matter, determine the expansion of the universe and study galaxy evolutio ...

EPFL2020

Distributed Learning over Networks under Subspace Constraints

Ali H. Sayed, Roula Nassif, Stefan Vlaski

This work presents and studies a distributed algorithm for solving optimization problems over networks where agents have individual costs to minimize subject to subspace constraints that require the minimizers across the network to lie in a low-dimensional ...

IEEE2019

An accelerated communication-efficient primal-dual optimization framework for structured machine learning

Chenxin Ma, Martin Jaggi

Distributed optimization algorithms are essential for training machine learning models on very large-scale datasets. However, they often suffer from communication bottlenecks. Confronting this issue, a communication-efficient primal-dual coordinate ascent ...

2021

Sparsified SGD with Memory

Jean-Baptiste Francis Marie Juliette Cordonnier, Martin Jaggi, Sebastian Urban Stich

Huge scale machine learning problems are nowadays tackled by distributed optimization algorithms, i.e. algorithms that leverage the compute power of many devices for training. The communication overhead is a key bottleneck that hinders perfect scalability. ...

2018

Dual Coupled Diffusion for Distributed Optimization with Affine Constraints

Ali H. Sayed, Kun Yuan, Sulaiman A S A E Alghunaim

In this work, a distributed multi-agent optimization problem is studied where different subsets of agents are coupled with each other through affine constraints. Moreover, each agent is only aware of its own contribution to the constraints and only knows w ...

IEEE2018

Distributed Optimization and Control using Operator Splitting Methods

Georgios Stathopoulos

The significant progress that has been made in recent years both in hardware implementations and in numerical computing has rendered real-time optimization-based control a viable option when it comes to advanced industrial applications. At the same time, t ...

EPFL2018

CoCoA: A General Framework for Communication-Efficient Distributed Optimization

Chenxin Ma, Martin Jaggi

The scale of modern datasets necessitates the development of efficient distributed optimization methods for machine learning. We present a general-purpose framework for distributed computing environments, CoCoA, that has an efficient communication scheme a ...

2018

Distributed Coupled Learning Over Adaptive Networks

Ali H. Sayed, Sulaiman A S A E Alghunaim

This work develops an effective distributed algorithm for the solution of stochastic optimization problems that involve partial coupling among both local constraints and local cost functions. While the collection of networked agents is interested in discov ...

IEEE2018