Publications de Martin Jaggi | EPFL Graph Search

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

We present a framework for building unsupervised representations of entities and their compositions, where each entity is viewed as a probability distribution rather than a vector embedding. In particular, this distribution is supported over the contexts w ...

Data augmentation is a widely adopted technique for avoiding overfitting when training deep neural networks. However, this approach requires domain-specific knowledge and is often limited to a fixed set of hard-coded transformations. Recently, several work ...

Decentralized stochastic optimization methods have gained a lot of attention recently, mainly because of their cheap per iteration cost, data locality, and their communication-efficiency. In this paper we introduce a unified convergence analysis that cover ...

Document-level text simplification is a specific type of simplification which involves simplifying documents consisting of several sentences by rewriting them into fewer or more sentences. In this paper, we propose a new two-stage framework SIMSUM for auto ...

We identify a phenomenon, which we refer to as multi-model forgetting, that occurs when sequentially training multiple deep networks with partially-shared parameters; the performance of previously-trained models degrades as one optimizes a subsequent one, ...

Lossy gradient compression has become a practical tool to overcome the communication bottleneck in centrally coordinated distributed training of machine learning models. However, algorithms for decentralized training with compressed communication over arbi ...

Large language models (LLMs) have demonstrated human-level performance on a vast spectrum of natural language tasks. However, it is largely unexplored whether they can better internalize knowledge from a structured data, such as a knowledge graph, or from ...

Huge scale machine learning problems are nowadays tackled by distributed optimization algorithms, i.e. algorithms that leverage the compute power of many devices for training. The communication overhead is a key bottleneck that hinders perfect scalability. ...

Background. In medicine and other applications, the copying and sharing of data is impractical for a range of well-considered reasons. With federated learning (FL) techniques, machine learning models can be trained on data spread across several locations w ...

The scale of modern datasets necessitates the development of efficient distributed optimization methods for machine learning. We present a general-purpose framework for distributed computing environments, CoCoA, that has an efficient communication scheme a ...

Decentralized learning is appealing as it enables the scalable usage of large amounts of distributed data and resources (without resorting to any central entity), while promoting privacy since every user minimizes the direct exposure of their data. Yet, wi ...

Deep learning networks are typically trained by Stochastic Gradient Descent (SGD) methods that iteratively improve the model parameters by estimating a gradient on a very small fraction of the training data. A major roadblock faced when increasing the batc ...

Importance sampling has become an indispensable strategy to speed up optimization algorithms for large-scale applications. Improved adaptive variants - using importance values defined by the complete gradient information which changes during optimization - ...

We introduce a method for automated grading of handwritten essays written by foreign language learners of French. The handwriting recognition system allows digitising the essays for further processing and functions at a low character error rate. The transc ...

Federated learning (FL) is a machine learning setting where many clients (e.g., mobile devices or whole organizations) collaboratively train a model under the orchestration of a central server (e.g., service provider), while keeping the training data decen ...

Keyword Spotting (KWS) systems allow detecting a set of spoken (pre-defined) keywords. Open-vocabulary KWS systems search for the keywords in the set of word hypotheses generated by an automatic speech recognition (ASR) system which is computationally expe ...

Recent trends of incorporating attention mechanisms in vision have led re- searchers to reconsider the supremacy of convolutional layers as a primary build- ing block. Beyond helping CNNs to handle long-range dependencies, Ramachandran et al. (2019) showed ...

Structured constraints in Machine Learning have recently brought the Frank-Wolfe (FW) family of algorithms back into the spotlight. While the classical FW algorithm has poor local convergence properties, Away-steps FW and Pairwise FW have emerged as improv ...

Deep neural networks often have millions of parameters. This can hinder their deployment to low-end devices, not only due to high memory requirements but also because of increased latency at inference. We propose a novel model compression method that gener ...

Mini-batch stochastic gradient methods (SGD) are state of the art for distributed training of deep neural networks. Drastic increases in the mini-batch sizes have lead to key efficiency and scalability gains in recent years. However, progress faces a major ...

Martin Jaggi

Chattez avec Graph Search

Context Mover's Distance & Barycenters: Optimal Transport of Contexts for Building Representations

Semantic Perturbations with Normalizing Flows for Improved Generalization

A Unified Theory of Decentralized SGD with Changing Topology and Local Updates

SIMSUM: Document-level Text Simplification via Simultaneous Summarization

Overcoming Multi-model Forgetting

Practical Low-Rank Communication Compression in Decentralized Deep Learning

SKILL: Structured Knowledge Infusion for Large Language Models

Sparsified SGD with Memory

Weight Erosion: An Update Aggregation Scheme for Personalized Collaborative Machine Learning

CoCoA: A General Framework for Communication-Efficient Distributed Optimization

The Privacy Power of Correlated Noise in Decentralized Learning

Extrapolation for Large-batch Training in Deep Learning

Safe Adaptive Importance Sampling

Automated Essay Scoring in Foreign Language Students Based on Deep Contextualised Word Representations

Advances and Open Problems in Federated Learning

Open-Vocabulary Keyword Spotting with Audio and Text Embeddings

On the Relationship between Self-Attention and Convolutional Layers

Linearly Convergent Frank-Wolfe with Backtracking Line-Search

Dynamic Model Pruning with Feedback

Don't Use Large Mini-Batches, Use Local SGD