Sebastian Urban Stich

This person has left EPFL

About
Privacy
Disclaimer

Graph Chatbot

Related publications (27)

Page 2 of 2

SCAFFOLD Stochastic Controlled Averaging for Federated Learning

Sebastian Urban Stich, Sai Praneeth Reddy Karimireddy

Federated Averaging (FEDAVG) has emerged as the algorithm of choice for federated learning due to its simplicity and low communication cost. However, in spite of recent research efforts, its performance is not fully understood. We obtain tight convergence ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2020

Extrapolation for Large-batch Training in Deep Learning

Martin Jaggi, Sebastian Urban Stich, Tao Lin

Deep learning networks are typically trained by Stochastic Gradient Descent (SGD) methods that iteratively improve the model parameters by estimating a gradient on a very small fraction of the training data. A major roadblock faced when increasing the batc ...

2020

Safe Adaptive Importance Sampling

Martin Jaggi, Sebastian Urban Stich

Importance sampling has become an indispensable strategy to speed up optimization algorithms for large-scale applications. Improved adaptive variants - using importance values defined by the complete gradient information which changes during optimization - ...

2017

Advances and Open Problems in Federated Learning

Martin Jaggi, Ayfer Özgür Aydin, Sebastian Urban Stich, Florian Tramèr, Lie He

Federated learning (FL) is a machine learning setting where many clients (e.g., mobile devices or whole organizations) collaboratively train a model under the orchestration of a central server (e.g., service provider), while keeping the training data decen ...

NOW PUBLISHERS INC2021

Stochastic Variance Reduced Gradient Optimization of Generative Adversarial Networks

François Fleuret, Martin Jaggi, Sebastian Urban Stich, Tatjana Chavdarova

2018

Dynamic Model Pruning with Feedback

Martin Jaggi, Sebastian Urban Stich, Tao Lin, Luis Felipe Barba Flores

Deep neural networks often have millions of parameters. This can hinder their deployment to low-end devices, not only due to high memory requirements but also because of increased latency at inference. We propose a novel model compression method that gener ...

2020

Don't Use Large Mini-Batches, Use Local SGD

Martin Jaggi, Sebastian Urban Stich, Tao Lin

Mini-batch stochastic gradient methods (SGD) are state of the art for distributed training of deep neural networks. Drastic increases in the mini-batch sizes have lead to key efficiency and scalability gains in recent years. However, progress faces a major ...

2019