Publication

Cover: A Cluster-Based Variance Reduced Method For Online Learning

Publications associées (31)

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

Volkan Cevher, Alp Yurtsever, Maria-Luiza Vladarean

We propose a stochastic conditional gradient method (CGM) for minimizing convex finitesum objectives formed as a sum of smooth and non-smooth terms. Existing CGM variants for this template either suffer from slow convergence rates, or require carefully inc ...

2022

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Saber Salehkaleybar, Negar Kiyavash

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Florent Gérard Krzakala, Lenka Zdeborová, Bruno Loureiro, Bruno Loureiro, Ludovic Théo Stephan

Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achieve global convergence under gradient descent. The picture can be radically different for narrow net-works, which tend to get stuck in badly-generalizing loca ...

2022

Memory of Motion for Initializing Optimization in Robotics

Teguh Santoso Lembono

Many robotics problems are formulated as optimization problems. However, most optimization solvers in robotics are locally optimal and the performance depends a lot on the initial guess. For challenging problems, the solver will often get stuck at poor loc ...

EPFL2022

Gradient-Based Learning of Discrete Structured Measurement Operators for Signal Recovery

Jonathan Paul Sauder

Countless signal processing applications include the reconstruction of signals from few indirect linear measurements. The design of effective measurement operators is typically constrained by the underlying hardware and physics, posing a challenging and of ...

2022

The DESI N-body Simulation Project - II. Suppressing sample variance with fast simulations

Yu Yu, Andrei Variu, Cheng Zhao, Hanyu Zhang

Dark Energy Spectroscopic Instrument (DESI) will construct a large and precise three-dimensional map of our Universe. The survey effective volume reaches similar to 20 h(-3) Gpc(3). It is a great challenge to prepare high-resolution simulations with a much ...

OXFORD UNIV PRESS2022

Discriminative clustering with representation learning with any ratio of labeled to unlabeled data

We present a discriminative clustering approach in which the feature representation can be learned from data and moreover leverage labeled data. Representation learning can give a similarity-based clustering method the ability to automatically adapt to an ...

2022

Educational Technologies for Vocational Education. Experiences as Digital Clay

Pierre Dillenbourg, Richard Lee Davis

This book summarises 15 years of research (2007-2021) on the exploitation of digital technologies for VET. In other words, our activities started the same year as the first iPhone was launched! Four research groups formed the leading house on the topic, na ...

EPFL2022

Masked Training of Neural Networks with Partial Gradients

Martin Jaggi, Sebastian Urban Stich, Amirkeivan Mohtashami

State-of-the-art training algorithms for deep learning models are based on stochastic gradient descent (SGD). Recently, many variations have been explored: perturbing parameters for better accuracy (such as in Extra-gradient), limiting SGD updates to a sub ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

System and method for privacy-preserving distributed training of neural network models on distributed datasets

Jean-Pierre Hubaux, Jean-Philippe Léonard Bossuat, David Jules Froelicher, Joao André Gomes de Sá e Sousa, Juan Ramón Troncoso-Pastoriza, Sinem Sav, Apostolos Pyrgelis

A computer-implemented method and a distributed computer system (100) for privacy- preserving distributed training of a global neural network model on distributed datasets (DS1 to DSn). The system has a plurality of data providers (DP1 to DPn) being commun ...

2022

Momentum-Based Policy Gradient with Second-Order Information

Patrick Thiran, Saber Salehkaleybar, Negar Kiyavash, Mohammadsadegh Khorasani

Variance-reduced gradient estimators for policy gradient methods have been one of the main focus of research in the reinforcement learning in recent years as they allow acceleration of the estimation process. We propose a variance-reduced policy-gradient m ...

2022

Publications associées (31)

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

Volkan Cevher, Alp Yurtsever, Maria-Luiza Vladarean

2022

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Saber Salehkaleybar, Negar Kiyavash

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022