Recherche

Afficher tous les résultats pour

Accueil

Personne

Martin Jaggi

À propos
Confidentialité
Mentions légales

Graph Chatbot

Publications associées (76)

Page 3 sur 4

Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication

Martin Jaggi, Sebastian Urban Stich, Anastasiia Koloskova

We consider decentralized stochastic optimization with the objective function (e.g. data samples for machine learning task) being distributed over n machines that can only communicate to their neighbors on a fixed communication graph. To reduce the communi ...

2019

Context Mover's Distance & Barycenters: Optimal Transport of Contexts for Building Representations

Sidak Pal Singh, Andreas Hug, Martin Jaggi, Aymeric Daphnis Kévin Dieuleveut

We present a framework for building unsupervised representations of entities and their compositions, where each entity is viewed as a probability distribution rather than a vector embedding. In particular, this distribution is supported over the contexts w ...

ADDISON-WESLEY PUBL CO2020

Semantic Perturbations with Normalizing Flows for Improved Generalization

Martin Jaggi, Sebastian Urban Stich, Tatjana Chavdarova

Data augmentation is a widely adopted technique for avoiding overfitting when training deep neural networks. However, this approach requires domain-specific knowledge and is often limited to a fixed set of hard-coded transformations. Recently, several work ...

IEEE2021

A Unified Theory of Decentralized SGD with Changing Topology and Local Updates

Martin Jaggi, Sebastian Urban Stich, Anastasiia Koloskova

Decentralized stochastic optimization methods have gained a lot of attention recently, mainly because of their cheap per iteration cost, data locality, and their communication-efficiency. In this paper we introduce a unified convergence analysis that cover ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2020

Adaptive balancing of gradient and update computation times using global geometry and approximate subproblems

Martin Jaggi, Sebastian Urban Stich, Sai Praneeth Reddy Karimireddy

2018

SIMSUM: Document-level Text Simplification via Simultaneous Summarization

Xinyu Zhou, Martin Jaggi

Document-level text simplification is a specific type of simplification which involves simplifying documents consisting of several sentences by rewriting them into fewer or more sentences. In this paper, we propose a new two-stage framework SIMSUM for auto ...

Assoc Computational Linguistics-Acl2023

Overcoming Multi-model Forgetting

Anthony Christopher Davison, Mathieu Salzmann, Martin Jaggi, Claudiu-Cristian Musat, Kaicheng Yu

We identify a phenomenon, which we refer to as multi-model forgetting, that occurs when sequentially training multiple deep networks with partially-shared parameters; the performance of previously-trained models degrades as one optimizes a subsequent one, ...

JMLR2019

Practical Low-Rank Communication Compression in Decentralized Deep Learning

Martin Jaggi, Sai Praneeth Reddy Karimireddy, Thijs Vogels

Lossy gradient compression has become a practical tool to overcome the communication bottleneck in centrally coordinated distributed training of machine learning models. However, algorithms for decentralized training with compressed communication over arbi ...

2020

SKILL: Structured Knowledge Infusion for Large Language Models

Martin Jaggi

Large language models (LLMs) have demonstrated human-level performance on a vast spectrum of natural language tasks. However, it is largely unexplored whether they can better internalize knowledge from a structured data, such as a knowledge graph, or from ...

ASSOC COMPUTATIONAL LINGUISTICS-ACL2022

Sparsified SGD with Memory

Martin Jaggi, Sebastian Urban Stich, Jean-Baptiste Francis Marie Juliette Cordonnier

Huge scale machine learning problems are nowadays tackled by distributed optimization algorithms, i.e. algorithms that leverage the compute power of many devices for training. The communication overhead is a key bottleneck that hinders perfect scalability. ...

2018

CoCoA: A General Framework for Communication-Efficient Distributed Optimization

Martin Jaggi

The scale of modern datasets necessitates the development of efficient distributed optimization methods for machine learning. We present a general-purpose framework for distributed computing environments, CoCoA, that has an efficient communication scheme a ...

2018

Weight Erosion: An Update Aggregation Scheme for Personalized Collaborative Machine Learning

Martin Jaggi, Sai Praneeth Reddy Karimireddy, Mary-Anne Hartley, Felix Hans Michel Grimberg

Background. In medicine and other applications, the copying and sharing of data is impractical for a range of well-considered reasons. With federated learning (FL) techniques, machine learning models can be trained on data spread across several locations w ...

Springer, Cham2020

The Privacy Power of Correlated Noise in Decentralized Learning

Rachid Guerraoui, Martin Jaggi, Youssef Allouah, Anastasiia Koloskova

Decentralized learning is appealing as it enables the scalable usage of large amounts of distributed data and resources (without resorting to any central entity), while promoting privacy since every user minimizes the direct exposure of their data. Yet, wi ...

PMLR2024

Extrapolation for Large-batch Training in Deep Learning

Martin Jaggi, Sebastian Urban Stich, Tao Lin

Deep learning networks are typically trained by Stochastic Gradient Descent (SGD) methods that iteratively improve the model parameters by estimating a gradient on a very small fraction of the training data. A major roadblock faced when increasing the batc ...

2020

Safe Adaptive Importance Sampling

Martin Jaggi, Sebastian Urban Stich

Importance sampling has become an indispensable strategy to speed up optimization algorithms for large-scale applications. Improved adaptive variants - using importance values defined by the complete gradient information which changes during optimization - ...

2017

Automated Essay Scoring in Foreign Language Students Based on Deep Contextualised Word Representations

Martin Jaggi, Bojana Rankovic

We introduce a method for automated grading of handwritten essays written by foreign language learners of French. The handwriting recognition system allows digitising the essays for further processing and functions at a low character error rate. The transc ...

2020

Advances and Open Problems in Federated Learning

Martin Jaggi, Ayfer Özgür Aydin, Sebastian Urban Stich, Florian Tramèr, Lie He

Federated learning (FL) is a machine learning setting where many clients (e.g., mobile devices or whole organizations) collaboratively train a model under the orchestration of a central server (e.g., service provider), while keeping the training data decen ...

NOW PUBLISHERS INC2021

Open-Vocabulary Keyword Spotting with Audio and Text Embeddings

Martin Jaggi

Keyword Spotting (KWS) systems allow detecting a set of spoken (pre-defined) keywords. Open-vocabulary KWS systems search for the keywords in the set of word hypotheses generated by an automatic speech recognition (ASR) system which is computationally expe ...

2019

Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features

Martin Jaggi, Matteo Pagliardini, Prakhar Gupta

The recent tremendous success of unsupervised word embeddings in a multitude of applications raises the obvious question if similar methods could be derived to improve embeddings (i.e. semantic representations) of word sequences as well. We present a simpl ...

2017

Better Word Embeddings by Disentangling Contextual n-Gram Information

Martin Jaggi, Matteo Pagliardini, Prakhar Gupta

Pre-trained word vectors are ubiquitous in Natural Language Processing applications. In this paper, we show how training word embeddings jointly with bigram and even trigram embeddings, results in improved unigram embeddings. We claim that training word em ...

2019