Publications associées à Collaborative Learning in the Jungle (Decentralized, Byzantine, Heterogeneous, Asynchronous and Nonconvex Learning)

EdgeAI-Aware Design of In-Memory Computing Architectures

Driven by the demand for real-time processing and the need to minimize latency in AI algorithms, edge computing has experienced remarkable progress. Decision-making AI applications stand out for their heavy reliance on data-centric operations, predominantl ...

EPFL2024

COMMUNICATION LOWER BOUNDS AND OPTIMAL ALGORITHMS FOR MULTIPLE TENSOR-TIMES-MATRIX COMPUTATION

Laura Grigori

Multiple tensor-times-matrix (Multi-TTM) is a key computation in algorithms for computing and operating with the Tucker tensor decomposition, which is frequently used in multidimensional data analysis. We establish communication lower bounds that determine ...

Siam Publications2024

Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneity

Rachid Guerraoui, John Stephan, Sadegh Farhadkhani, Nirupam Gupta, Rafaël Benjamin Pinot, Youssef Allouah

Byzantine machine learning (ML) aims to ensure the resilience of distributed learning algorithms to misbehaving (or Byzantine) machines. Although this problem received significant attention, prior works often assume the data held by the machines to be homo ...

PMLR2023

Diversity of radial spin textures in chiral materials

Oleg Yazyev, Alberto Crepaldi, Daniel Gosalbez Martinez

We introduce a classification of the radial spin textures in momentum space that emerge at the high-symmetry points in crystals characterized by nonpolar chiral point groups (D2, D3, D4, D6, T, O). Based on the symmetry constraints imposed by these point g ...

Amer Physical Soc2023

Get the Best of Both Worlds: Improving Accuracy and Transferability by Grassmann Class Representation

Zhigang Li, Haoqi Wang

We generalize the class vectors found in neural networks to linear subspaces (i.e., points in the Grassmann manifold) and show that the Grassmann Class Representation (GCR) enables simultaneous improvement in accuracy and feature transferability. In GCR, e ...

Ieee Computer Soc2023

Byzantine Machine Learning Made Easy By Resilient Averaging of Momentums

Rachid Guerraoui, John Stephan, Sadegh Farhadkhani, Nirupam Gupta, Rafaël Benjamin Pinot

Byzantine resilience emerged as a prominent topic within the distributed machine learning community. Essentially, the goal is to enhance distributed optimization algorithms, such as distributed SGD, in a way that guarantees convergence despite the presence ...

PMLR2022

An Equivalence Between Data Poisoning and Byzantine Gradient Attacks

Rachid Guerraoui, Le Nguyen Hoang, Sadegh Farhadkhani, Oscar Jean Olivier Villemaud

To study the resilience of distributed learning, the “Byzantine" literature considers a strong threat model where workers can report arbitrary gradients to the parameter server. Whereas this model helped obtain several fundamental results, it has sometimes ...

PMLR2022

Linear and nonlinear substructured Restricted Additive Schwarz iterations and preconditioning

Tommaso Vanzan

Iterative substructuring Domain Decomposition (DD) methods have been extensively studied, and they are usually associated with nonoverlapping decompositions. It is less known that classical overlapping DD methods can also be formulated in substructured for ...

2022

Dalton: Learned Partitioning for Distributed Data Streams

Anastasia Ailamaki, Ioannis Mytilinis, Eleni Zapridou

To sustain the input rate of high-throughput streams, modern stream processing systems rely on parallel execution. However, skewed data yield imbalanced load assignments and create stragglers that hinder scalability. Deciding on a static partitioning for a ...

2022

Score Matching Enables Causal Discovery of Nonlinear Additive Noise Models

Volkan Cevher, Paul Thierry Yves Rolland

This paper demonstrates how to recover causal graphs from the score of the data distribution in non-linear additive (Gaussian) noise models. Using score matching algorithms as a building block, we show how to design a new generation of scalable causal disc ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

Fabrication and Characterization of High Aspect Ratio Amorphous Silicon Based Microchannel Plates

Christophe Ballif, Nicolas Würsch, Janina Christine Isabelle Löffler, Mohammad Beygi, Samira Alexandra Frey

This contribution focuses on the fabrication and characterization of microchannel plates made of hydrogenated amorphous silicon (AMCPs). Flexible fabrication processes and the semi-conducting nature of amorphous silicon could give these detectors the advan ...

2021

GARFIELD: System Support for Byzantine Machine Learning (Regular Paper)

Rachid Guerraoui, Sébastien Louis Alexandre Rouault, Arsany Hany Abdelmessih Guirguis

We present GARFIELD, a library to transparently make machine learning (ML) applications, initially built with popular (but fragile) frameworks, e.g., TensorFlow and PyTorch, Byzantine–resilient. GARFIELD relies on a novel object–oriented design, reducing t ...

IEEE2021

Further results on latent discourse models and word embeddings

Youssef Allouah

We discuss some properties of generative models for word embeddings. Namely, (Arora et al., 2016) proposed a latent discourse model implying the concentration of the partition function of the word vectors. This concentration phenomenon led to an asymptotic ...

Microtome Publishing2021

Deep Domain Adaptation in Earth Observation

Devis Tuia, Nicolas Courty, Benjamin Alexander Kellenberger

When applied to new datasets, acquired at different time moments, with different sensors or under different acquisition conditions, deep learning models might fail spectacularly. This is because they have learned from the data distribution observed during ...

Wiley2021

Rational Agreement in the Presence of Crash Faults

Vincent Gramoli, Alejandro Ranchal Pedrosa

Blockchain systems need to solve consensus despite the presence of rational users and failures. The notion of (k, t)-robustness is key to derive impossibility results with k rational players and t faulty players. However, these t faulty players are always ...

IEEE2021

Coarse graining and large-N behavior of the d-dimensional N-clock model

Matthias Ruf, Matthias Benjamin Ruf

We study the asymptotic behavior of the N-clock model, a nearest neighbors ferromagnetic spin model on the d-dimensional cubic epsilon-lattice in which the spin field is constrained to take values in a discretization S-N of the unit circle S-1 consisting o ...

2021

Locally supported tangential vector, n-vector, and tensor fields

Christopher Brandt

We introduce a construction of subspaces of the spaces of tangential vector, n-vector, and tensor fields on surfaces. The resulting subspaces can be used as the basis of fast approximation algorithms for design and processing problems that involve tangenti ...

2020

Proximity Results and Faster Algorithms for Integer Programming Using the Steinitz Lemma

Friedrich Eisenbrand

We consider integer programming problems in standard form max{c(T)x : Ax = b, x >= 0, x is an element of Z(n)} where A is an element of Z(mxn), b is an element of Z(m), and c is an element of Z(n). We show that such an integer program can be solved in time ...

ASSOC COMPUTING MACHINERY2020

Private and Secure Distributed Learning

Georgios Damaskinos

The ever-growing number of edge devices (e.g., smartphones) and the exploding volume of sensitive data they produce, call for distributed machine learning techniques that are privacy-preserving. Given the increasing computing capabilities of modern edge de ...

EPFL2020

ESSOP: Efficient and Scalable Stochastic Outer Product Architecture for Deep Learning

Irem Boybat Kara, Abu Sebastian, Evangelos Eleftheriou

Deep neural networks (DNNs) have surpassed human-level accuracy in a variety of cognitive tasks but at the cost of significant memory/time requirements in DNN training. This limits their deployment in energy and memory limited applications that require rea ...

IEEE2020