Publications associées à Momentum-based Gradient Methods in Multi-Objective Recommendation

Publications associées (16)

Robust Collaborative Learning with Linear Gradient Overhead

Rachid Guerraoui, Nirupam Gupta, Rafaël Benjamin Pinot, Le Nguyen Hoang, Sadegh Farhadkhani, John Stephan

Collaborative learning algorithms, such as distributed SGD (or D-SGD), are prone to faulty machines that may deviate from their prescribed algorithm because of software or hardware bugs, poisoned data or malicious behaviors. While many solutions have been ...

PLMR2023

Saddle-to-Saddle Dynamics in Diagonal Linear Networks

Nicolas Henri Bernard Flammarion, Scott William Pesme

In this paper we fully describe the trajectory of gradient flow over diagonal linear networks in the limit of vanishing initialisation. We show that the limiting flow successively jumps from a saddle of the training loss to another until reaching the minim ...

2023

Tensor approximation of the self-diffusion matrix of tagged particle processes

Christoph Max Strössner

The objective of this paper is to investigate a new numerical method for the approximation of the self-diffusion matrix of a tagged particle process defined on a grid. While standard numerical methods make use of long-time averages of empirical means of de ...

ACADEMIC PRESS INC ELSEVIER SCIENCE2023

Democratizing Machine Learning

Nirupam Gupta, Rafaël Benjamin Pinot, Alexandre David Olivier Maurer

The increasing prevalence of personal devices motivates the design of algorithms that can leverage their computing power, together with the data they generate, in order to build privacy-preserving and effective machine learning models. However, traditional ...

IEEE2022

Data-Driven Control and Its Applications to Microgrids

Alireza Karimi, Seyed Sohail Madani

Renewable energy resources have an increasing share of the electrical power, and the concept of distributed generation and storage units is about to completely change the basic architecture and control algorithms of the electric power grids. In this tutori ...

IEEE2022

On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them

Sabine Süsstrunk, Mathieu Salzmann, Tao Lin, Chen Liu

We analyze the influence of adversarial training on the loss landscape of machine learning models. To this end, we first provide analytical studies of the properties of adversarial loss functions under different adversarial budgets. We then demonstrate tha ...

2021

Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithms

Volkan Cevher, Junhong Lin

We study generalization properties of distributed algorithms in the setting of nonparametric regression over a reproducing kernel Hilbert space (RKHS). We first investigate distributed stochastic gradient methods (SGM), with mini-batches and multi-passes o ...

2020

Conditional gradient methods for stochastically constrained convex minimization

Volkan Cevher, Ya-Ping Hsieh, Ahmet Alacaoglu, Maria-Luiza Vladarean

We propose two novel conditional gradient-based methods for solving structured stochastic convex optimization problems with a large number of linear constraints. Instances of this template naturally arise from SDP-relaxations of combinatorial problems, whi ...

2020

Polynomial Escape-Time From Saddle Points In Distributed Non-Convex Optimization

Ali H. Sayed, Stefan Vlaski

The diffusion strategy for distributed learning from streaming data employs local stochastic gradient updates along with exchange of iterates over neighborhoods. In this work we establish that agents cluster around a network centroid in the mean-fourth sen ...

IEEE2019

Jamming transition as a paradigm to understand the loss landscape of deep neural networks

Matthieu Wyart, Mario Geiger, Stefano Spigler, Levent Dogus Sagun

Deep learning has been immensely successful at a variety of tasks, ranging from classification to artificial intelligence. Learning corresponds to fitting training data, which is implemented by descending a very high-dimensional loss function. Understandin ...

AMER PHYSICAL SOC2019

Chemical machine learning with kernels: The impact of loss functions

Volkan Cevher, Quoc Tran Dinh, Sandip De, Junhong Lin, Van Quang Nguyen

Machine learning promises to accelerate materials discovery by allowing computational efficient property predictions from a small number of reference calculations. As a result, the literature has spent a considerable effort in designing representations tha ...

2019

Dynamic Clustering and Control for Large Scale Urban Networks

Mohammadreza Saeedmanesh

Traffic congestion is a significant issue in all urban areas with concentration of activities for various city topologies and distribution of population and land use around the world. Developing realistic models that are able to replicate congestion spread ...

EPFL2018

Learning with Structured Sparsity: From Discrete to Convex and Back

Marwa El Halabi

In modern-data analysis applications, the abundance of data makes extracting meaningful information from it challenging, in terms of computation, storage, and interpretability. In this setting, exploiting sparsity in data has been essential to the developm ...

EPFL2018

A randomised primal-dual algorithm for distributed radio-interferometric imaging

Yves Wiaux, Jason Douglas McEwen, Rafael Eduardo Carrillo Rangel

Next generation radio telescopes, like the Square Kilometre Array, will acquire an unprecedented amount of data for radio astronomy. The development of fast, parallelisable or distributed algorithms for handling such large-scale data sets is of prime impor ...

Ieee2016

Use of Transient Measurements for the Optimization of Steady-State Performance via Modifier Adaptation

Dominique Bonvin, Grégory François

Real-time optimization (RTO) methods use measurements to offset the effect of uncertainty and drive the plant to optimality. RTO schemes differ in the way measurements are incorporated in the optimization framework. Explicit RTO schemes solve a static opti ...

American Chemical Society2014

Global mixed-integer dynamic optimization

Benoît Chachuat

Recent advances in process synthesis, design, operations, and control have created an increasing demand for efficient numerical algorithms for optimizing a dynamic system coupled with discrete decisions; these problems are termed mixed-integer dynamic opti ...

2005