Publications by Kun Yuan | EPFL Graph Search

Related publications (19)

Walkman: A Communication-Efficient Random-Walk Algorithm for Decentralized Optimization

This paper introduces a new algorithm for consensus optimization in a multi-agent network, where all agents collaboratively find a minimizer for the sum of their private functions. All decentralized algorithms rely on communications between adjacent nodes. ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Supervised Learning Under Distributed Features

Ali H. Sayed, Kun Yuan, Bicheng Ying

This paper studies the problem of learning under both large datasets and large-dimensional feature space scenarios. The feature information is assumed to be spread across agents in a network, where each agent observes some of the features. Through local co ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2019

Multiagent Fully Decentralized Value Function Learning With Linear Convergence Rates

Ali H. Sayed, Kun Yuan, Lucas Cesar Eduardo Cassano

This article develops a fully decentralized multiagent algorithm for policy evaluation. The proposed scheme can be applied to two distinct scenarios. In the first scenario, a collection of agents have distinct datasets gathered by following different behav ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

Convergence Of Variance-Reduced Learning Under Random Reshuffling

Ali H. Sayed, Kun Yuan, Bicheng Ying

Several useful variance-reduced stochastic gradient algorithms, such as SVRG, SAGA, Finito, and SAG, have been proposed to minimize empirical risks with linear convergence properties to the exact minimizers. The existing convergence results assume uniform ...

IEEE2018

Variance-Reduced Stochastic Learning Under Random Reshuffling

Ali H. Sayed, Kun Yuan, Bicheng Ying

Several useful variance-reduced stochastic gradient algorithms, such as SVRG, SAGA, Finito, and SAG, have been proposed to minimize empirical risks with linear convergence properties to the exact minimizer. The existing convergence results assume uniform d ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Dual Coupled Diffusion for Distributed Optimization with Affine Constraints

Ali H. Sayed, Kun Yuan, Sulaiman A S A E Alghunaim

In this work, a distributed multi-agent optimization problem is studied where different subsets of agents are coupled with each other through affine constraints. Moreover, each agent is only aware of its own contribution to the constraints and only knows w ...

IEEE2018

Cover: A Cluster-Based Variance Reduced Method For Online Learning

Ali H. Sayed, Kun Yuan, Bicheng Ying

In this paper, we develop a stochastic-gradient learning algorithm for situations involving streaming data that arise from an underlying clustered structure. In such settings, the variance of gradient noise can be decomposed into the in-cluster variance si ...

IEEE2019

Efficient Variance-Reduced Learning Over Multi-Agent Networks

Ali H. Sayed, Kun Yuan, Bicheng Ying

This work develops a fully decentralized variance-reduced learning algorithm for multi-agent networks where nodes store and process the data locally and are only allowed to communicate with their immediate neighbors. In the proposed algorithm, there is no ...

IEEE COMPUTER SOC2018

A Proximal Diffusion Strategy for Multiagent Optimization With Sparse Affine Constraints

Ali H. Sayed, Kun Yuan, Sulaiman A S A E Alghunaim

This article develops a proximal primal-dual decentralized strategy for multiagent optimization problems that involve multiple coupled affine constraints, where each constraint may involve only a subset of the agents. The constraints are generally sparse, ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Exact Diffusion for Distributed Optimization and Learning-Part II: Convergence Analysis

Ali H. Sayed, Kun Yuan, Bicheng Ying

Part I of this paper developed the exact diffusion algorithm to remove the bias that is characteristic of distributed solutions for deterministic optimization problems. The algorithm was shown to be applicable to the larger set of locally balanced left-sto ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2019

Exact Diffusion for Distributed Optimization and Learning-Part I: Algorithm Development

Ali H. Sayed, Kun Yuan, Bicheng Ying

This paper develops a distributed optimization strategy with guaranteed exact convergence for a broad class of left-stochastic combination policies. The resulting exact diffusion strategy is shown in Part II of this paper to have a wider stability range an ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2019

On the Influence of Bias-Correction on Distributed Stochastic Optimization

Ali H. Sayed, Bicheng Ying, Kun Yuan, Sulaiman A S A E Alghunaim

Various bias-correction methods such as EXTRA, gradient tracking methods, and exact diffusion have been proposed recently to solve distributed deterministic optimization problems. These methods employ constant step-sizes and converge linearly to the exact ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

A Linearly Convergent Proximal Gradient Algorithm for Decentralized Optimization

Ali H. Sayed, Kun Yuan, Sulaiman A S A E Alghunaim

Decentralized optimization is a powerful paradigm that finds applications in engineering and learning design. This work studies decentralized composite optimization problems with non-smooth regularization terms. Most existing gradient-based proximal decent ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2019

Distributed Value-Function Learning with Linear Convergence Rates

Ali H. Sayed, Kun Yuan, Lucas Cesar Eduardo Cassano

In this paper we develop a fully decentralized algorithm for policy evaluation with off-policy learning and linear function approximation. The proposed algorithm is of the variance reduced kind and achieves linear convergence with O(1) memory requirements. ...

IEEE2019

Stochastic Learning Under Random Reshuffling With Constant Step-Sizes

Ali H. Sayed, Stefan Vlaski, Kun Yuan, Bicheng Ying

In empirical risk optimization, it has been observed that stochastic gradient implementations that rely on random reshuffling of the data achieve better performance than implementations that rely on sampling the data uniformly. Recent works have pursued ju ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2019

Decentralized Proximal Gradient Algorithms With Linear Convergence Rates

Ali H. Sayed, Kun Yuan, Sulaiman A S A E Alghunaim

This article studies a class of nonsmooth decentralized multiagent optimization problems where the agents aim at minimizing a sum of local strongly-convex smooth components plus a common nonsmooth term. We propose a general primal-dual algorithmic framewor ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

On the Performance of Exact Diffusion over Adaptive Networks

Ali H. Sayed, Bicheng Ying, Kun Yuan, Sulaiman A S A E Alghunaim

Various bias-correction methods such as EXTRA, DIGing, and exact diffusion have been proposed recently to solve distributed deterministic optimization problems. These methods employ constant step-sizes and converge linearly to the exact solution under prop ...

IEEE2019

Dynamic Average Diffusion With Randomized Coordinate Updates

Ali H. Sayed, Kun Yuan, Bicheng Ying

This work derives and analyzes an online learning strategy for tracking the average of time-varying distributed signals by relying on randomized coordinate-descent updates. During each iteration, each agent selects or observes a random entry of the observa ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2019

Variance-Reduced Stochastic Learning by Networked Agents Under Random Reshuffling

Ali H. Sayed, Kun Yuan, Bicheng Ying

This paper develops a distributed variance-reduced strategy for a collection of interacting agents that are connected by a graph topology. The resulting diffusion-AVRG (where AVRG stands for "amortized variance-reduced gradient") algorithm is shown to have ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2019