Publications associées à Collaborative Learning in the Jungle (Decentralized, Byzantine, Heterogeneous, Asynchronous and Nonconvex Learning)

Publications associées (33)

Roger F. Harrington and the Method of Moments: Part 2: Electrodynamics

The method of moments (MOM), as introduced by Roger F. Harrington more than 50 years ago, is reviewed in the context of the classic potential integral equation (IE) formulations applied to both electrostatic (part 1) and electrodynamic or full-wave problem ...

Ieee-Inst Electrical Electronics Engineers Inc2024

EdgeAI-Aware Design of In-Memory Computing Architectures

Marco Antonio Rios

Driven by the demand for real-time processing and the need to minimize latency in AI algorithms, edge computing has experienced remarkable progress. Decision-making AI applications stand out for their heavy reliance on data-centric operations, predominantl ...

EPFL2024

COMMUNICATION LOWER BOUNDS AND OPTIMAL ALGORITHMS FOR MULTIPLE TENSOR-TIMES-MATRIX COMPUTATION

Laura Grigori

Multiple tensor-times-matrix (Multi-TTM) is a key computation in algorithms for computing and operating with the Tucker tensor decomposition, which is frequently used in multidimensional data analysis. We establish communication lower bounds that determine ...

Siam Publications2024

Which Coupled is Best Coupled? An Exploration of AIMC Tile Interfaces and Load Balancing for CNNs

David Atienza Alonso, Giovanni Ansaloni, Irem Boybat Kara, Joshua Alexander Harrison Klein

Due to stringent energy and performance constraints, edge AI computing often employs heterogeneous systems that utilize both general-purpose CPUs and accelerators. Analog in-memory computing (AIMC) is a well-known AI inference solution that overcomes compu ...

Institute of Electrical and Electronics Engineers (IEEE)2024

Generative models for synthetic data generation: application to pharmacokinetic/pharmacodynamic data

Yulun Jiang

The generation of synthetic patient data that reflect the statistical properties of real data plays a fundamental role in today’s world because of its potential to (i) be enable proprietary data access for statistical and research purposes and (ii) increas ...

2024

Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneity

Rachid Guerraoui, Nirupam Gupta, Rafaël Benjamin Pinot, Youssef Allouah, Sadegh Farhadkhani, John Stephan

Byzantine machine learning (ML) aims to ensure the resilience of distributed learning algorithms to misbehaving (or Byzantine) machines. Although this problem received significant attention, prior works often assume the data held by the machines to be homo ...

PMLR2023

Diversity of radial spin textures in chiral materials

Oleg Yazyev, Alberto Crepaldi, Daniel Gosalbez Martinez

We introduce a classification of the radial spin textures in momentum space that emerge at the high-symmetry points in crystals characterized by nonpolar chiral point groups (D2, D3, D4, D6, T, O). Based on the symmetry constraints imposed by these point g ...

Amer Physical Soc2023

Robust Training and Verification of Deep Neural Networks

Fabian Ricardo Latorre Gomez

According to the proposed Artificial Intelligence Act by the European Comission (expected to pass at the end of 2023), the class of High-Risk AI Systems (Title III) comprises several important applications of Deep Learning like autonomous driving vehicles ...

EPFL2023

Get the Best of Both Worlds: Improving Accuracy and Transferability by Grassmann Class Representation

Haoqi Wang

We generalize the class vectors found in neural networks to linear subspaces (i.e., points in the Grassmann manifold) and show that the Grassmann Class Representation (GCR) enables simultaneous improvement in accuracy and feature transferability. In GCR, e ...

Ieee Computer Soc2023

Byzantine Machine Learning Made Easy By Resilient Averaging of Momentums

Rachid Guerraoui, Nirupam Gupta, Rafaël Benjamin Pinot, Sadegh Farhadkhani, John Stephan

Byzantine resilience emerged as a prominent topic within the distributed machine learning community. Essentially, the goal is to enhance distributed optimization algorithms, such as distributed SGD, in a way that guarantees convergence despite the presence ...

PMLR2022

An Equivalence Between Data Poisoning and Byzantine Gradient Attacks

Rachid Guerraoui, Le Nguyen Hoang, Sadegh Farhadkhani, Oscar Jean Olivier Villemaud

To study the resilience of distributed learning, the “Byzantine" literature considers a strong threat model where workers can report arbitrary gradients to the parameter server. Whereas this model helped obtain several fundamental results, it has sometimes ...

PMLR2022

Locally supported tangential vector, n-vector, and tensor fields

Christopher Brandt

We introduce a construction of subspaces of the spaces of tangential vector, n-vector, and tensor fields on surfaces. The resulting subspaces can be used as the basis of fast approximation algorithms for design and processing problems that involve tangenti ...

2020

Proximity Results and Faster Algorithms for Integer Programming Using the Steinitz Lemma

Friedrich Eisenbrand

We consider integer programming problems in standard form max{c(T)x : Ax = b, x >= 0, x is an element of Z(n)} where A is an element of Z(mxn), b is an element of Z(m), and c is an element of Z(n). We show that such an integer program can be solved in time ...

ASSOC COMPUTING MACHINERY2020

Linear and nonlinear substructured Restricted Additive Schwarz iterations and preconditioning

Tommaso Vanzan

Iterative substructuring Domain Decomposition (DD) methods have been extensively studied, and they are usually associated with nonoverlapping decompositions. It is less known that classical overlapping DD methods can also be formulated in substructured for ...

2022

Dalton: Learned Partitioning for Distributed Data Streams

Anastasia Ailamaki, Ioannis Mytilinis, Eleni Zapridou

To sustain the input rate of high-throughput streams, modern stream processing systems rely on parallel execution. However, skewed data yield imbalanced load assignments and create stragglers that hinder scalability. Deciding on a static partitioning for a ...

2022

Private and Secure Distributed Learning

Georgios Damaskinos

The ever-growing number of edge devices (e.g., smartphones) and the exploding volume of sensitive data they produce, call for distributed machine learning techniques that are privacy-preserving. Given the increasing computing capabilities of modern edge de ...

EPFL2020

ESSOP: Efficient and Scalable Stochastic Outer Product Architecture for Deep Learning

Irem Boybat Kara, Abu Sebastian

Deep neural networks (DNNs) have surpassed human-level accuracy in a variety of cognitive tasks but at the cost of significant memory/time requirements in DNN training. This limits their deployment in energy and memory limited applications that require rea ...

IEEE2020

Score Matching Enables Causal Discovery of Nonlinear Additive Noise Models

Volkan Cevher, Paul Thierry Yves Rolland

This paper demonstrates how to recover causal graphs from the score of the data distribution in non-linear additive (Gaussian) noise models. Using score matching algorithms as a building block, we show how to design a new generation of scalable causal disc ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

Scalable Byzantine Reliable Broadcast

Rachid Guerraoui, Matej Pavlovic, Dragos-Adrian Seredinschi, Matteo Monti

Byzantine reliable broadcast is a powerful primitive that allows a set of processes to agree on a message from a designated sender, even if some processes (including the sender) are Byzantine. Existing broadcast protocols for this setting scale poorly, as ...

Schloss Dagstuhl--Leibniz-Zentrum fer Informatik2019

Observation of tau(-) -> pi(-)nu(tau)e(+)e(-) and search for tau(-) -> pi(-)nu(tau)mu(+)mu(-)

Mingkui Wang, Olivier Schneider, Chao Wang, Yiming Li, Sun Hee Kim, Ke Huang

We present the first measurements of branching fractions of rare tau-lepton decays, tau(-) -> pi(-)nu(tau)l(+)l(-) (l = e or mu), using a data sample corresponding to 562 fb(-1) collected at a center-of-mass energy of 10.58 GeV with the Belle detector at t ...

American Physical Society2019