Recherche

Afficher tous les résultats pour

Accueil

Personne

Martin Jaggi

À propos
Confidentialité
Mentions légales

Graph Chatbot

Publications associées (76)

Page 2 sur 4

Decentralized deep learning with arbitrary communication compression

Martin Jaggi, Sebastian Urban Stich, Tao Lin, Anastasiia Koloskova

Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks, as well as for efficient scaling to large compute clusters. As current approaches are limited by network bandwidth, we propose t ...

2019

Beyond Spectral Gap: The Role of the Topology in Decentralized Learning

Martin Jaggi, Thijs Vogels, Hadrien Hendrikx

In data-parallel optimization of machine learning models, workers collaborate to improve their estimates of the model: more accurate gradients allow them to use larger learning rates and optimize faster. In the decentralized setting, in which workers commu ...

Microtome Publishing2023

Modular Clinical Decision Support Networks (MoDN)-Updatable, interpretable, and portable predictions for evolving clinical environments

Cécile Claire Trottet, Martin Jaggi, Mary-Anne Hartley, Thijs Vogels

Clinical Decision Support Systems (CDSS) have the potential to improve and standardise care with probabilistic guidance. However, many CDSS deploy static, generic rule-based logic, resulting in inequitably distributed accuracy and inconsistent performance ...

PUBLIC LIBRARY SCIENCE2023

Masked Training of Neural Networks with Partial Gradients

Martin Jaggi, Sebastian Urban Stich, Amirkeivan Mohtashami

State-of-the-art training algorithms for deep learning models are based on stochastic gradient descent (SGD). Recently, many variations have been explored: perturbing parameters for better accuracy (such as in Extra-gradient), limiting SGD updates to a sub ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

Unsupervised Scalable Representation Learning for Multivariate Time Series

Martin Jaggi, Aymeric Daphnis Kévin Dieuleveut

Time series constitute a challenging data type for machine learning algorithms, due to their highly variable lengths and sparse labeling in practice. In this paper, we tackle this challenge by proposing an unsupervised method to learn universal embeddings ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2019

Faster Parallel Training of Word Embeddings

Martin Jaggi

Word embeddings have gained increasing popularity in the recent years due to the Word2vec library and its extension fastText that uses subword information. In this paper, we aim at improving the execution speed of fastText training on homogeneous multi- an ...

IEEE COMPUTER SOC2021

Learning Aerial Image Segmentation from Online Maps

Martin Jaggi, Aurélien Lucchi

This study deals with semantic segmentation of high-resolution (aerial) images where a semantic class label is assigned to each pixel via supervised classification as a basis for automatic map generation. Recently, deep convolutional neural networks (CNNs) ...

Institute of Electrical and Electronics Engineers2017

Model Fusion via Optimal Transport

Martin Jaggi, Sidak Pal Singh

Combining different models is a widely used paradigm in machine learning applications. While the most common approach is to form an ensemble of models and average their individual predictions, this approach is often rendered infeasible by given resource co ...

2020

A Linearly Convergent Algorithm for Decentralized Optimization: Sending Less Bits for Free!

Martin Jaggi, Anastasiia Koloskova

Decentralized optimization methods enable on-device training of machine learning models without a central coordinator. In many scenarios communication between devices is energy demanding and time consuming and forms the bottleneck of the entire system. We ...

MICROTOME PUBLISHING2020

Faster Coordinate Descent via Adaptive Importance Sampling

Volkan Cevher, Martin Jaggi

Coordinate descent methods employ random partial updates of decision variables in order to solve huge-scale convex optimization problems. In this work, we introduce new adaptive rules for the random selection of their updates. By adaptive, we mean that our ...

PMLR2017

Consensus Control for Decentralized Deep Learning

Martin Jaggi, Sebastian Urban Stich, Tao Lin, Anastasiia Koloskova

Decentralized training of deep learning models enables on-device learning over networks, as well as efficient scaling to large compute clusters. Experiments in earlier works reveal that, even in a data-center setup, decentralized training often suffers fro ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Interpreting Language Models Through Knowledge Graph Extraction

Martin Jaggi, Vinitra Swamy, Angeliki Romanou

Transformer-based language models trained on large text corpora have enjoyed immense popularity in the natural language processing community and are commonly used as a starting point for downstream tasks. While these models are undeniably useful, it is a c ...

2021

Generating Steganographic Text with LSTMs

Martin Jaggi, Aikaterini Argyraki

Motivated by concerns for user privacy, we design a steganographic system ("stegosystem") that enables two users to exchange encrypted messages without an adversary detecting that such an exchange is taking place. We propose a new linguistic stegosystem ba ...

2017

Attention with Markov: A Curious Case of Single-layer Transformers

Michael Christoph Gastpar, Martin Jaggi, Marco Bondaschi, Adway Girish, Ashok Vardhan Makkuva

Attention-based transformers have achieved tremendous success across a variety of disciplines including natural languages. To deepen our understanding of their sequential modeling capabilities, there is a growing interest in using Markov input processes to ...

ICLR2025

Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data

Martin Jaggi, Sebastian Urban Stich, Tao Lin, Sai Praneeth Reddy Karimireddy

Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks. In realistic learning scenarios, the presence of heterogeneity across different clients' local datasets poses an optimization ch ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Error Feedback Fixes SignSGD and other Gradient Compression Schemes

Martin Jaggi, Sebastian Urban Stich, Sai Praneeth Reddy Karimireddy, Quentin Rebjock

Sign-based algorithms (e.g. signSGD) have been proposed as a biased gradient compression technique to alleviate the communication bottleneck in training large neural networks across multiple workers. We show simple convex counter-examples where signSGD doe ...

PMLR2019

Distributed Optimization with Arbitrary Local Solvers

Martin Jaggi

With the growth of data and necessity for distributed optimization methods, solvers that work well on a single machine must be re-designed to leverage distributed computation. Recent work in this area has been limited by focusing heavily on developing high ...

Taylor & Francis Ltd2017

An accelerated communication-efficient primal-dual optimization framework for structured machine learning

Martin Jaggi

Distributed optimization algorithms are essential for training machine learning models on very large-scale datasets. However, they often suffer from communication bottlenecks. Confronting this issue, a communication-efficient primal-dual coordinate ascent ...

2021

Approximate Steepest Coordinate Descent

Martin Jaggi, Sebastian Urban Stich

We propose a new selection rule for the coordinate selection in coordinate descent methods for huge-scale optimization. The efficiency of this novel scheme is provably better than the efficiency of uniformly random selection, and can reach the efficiency o ...

PMLR2017

Decoupling Backpropagation using Constrained Optimization Methods

Martin Jaggi, Johanni Michael Brea

We propose BlockProp, a neural network training algorithm. Unlike backpropagation, it does not rely on direct top-to-bottom propagation of an error signal. Rather, by interpreting backpropagation as a constrained optimization problem we split the neural ne ...

2018