Collaborative Learning in the Jungle (Decentralized, Byzantine, Heterogeneous, Asynchronous and Nonconvex Learning)

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

The method of moments (MOM), as introduced by Roger F. Harrington more than 50 years ago, is reviewed in the context of the classic potential integral equation (IE) formulations applied to both electrostatic (part 1) and electrodynamic or full-wave problem ...

Driven by the demand for real-time processing and the need to minimize latency in AI algorithms, edge computing has experienced remarkable progress. Decision-making AI applications stand out for their heavy reliance on data-centric operations, predominantl ...

Multiple tensor-times-matrix (Multi-TTM) is a key computation in algorithms for computing and operating with the Tucker tensor decomposition, which is frequently used in multidimensional data analysis. We establish communication lower bounds that determine ...

Due to stringent energy and performance constraints, edge AI computing often employs heterogeneous systems that utilize both general-purpose CPUs and accelerators. Analog in-memory computing (AIMC) is a well-known AI inference solution that overcomes compu ...

The generation of synthetic patient data that reflect the statistical properties of real data plays a fundamental role in today’s world because of its potential to (i) be enable proprietary data access for statistical and research purposes and (ii) increas ...

Byzantine machine learning (ML) aims to ensure the resilience of distributed learning algorithms to misbehaving (or Byzantine) machines. Although this problem received significant attention, prior works often assume the data held by the machines to be homo ...

We introduce a classification of the radial spin textures in momentum space that emerge at the high-symmetry points in crystals characterized by nonpolar chiral point groups (D2, D3, D4, D6, T, O). Based on the symmetry constraints imposed by these point g ...

According to the proposed Artificial Intelligence Act by the European Comission (expected to pass at the end of 2023), the class of High-Risk AI Systems (Title III) comprises several important applications of Deep Learning like autonomous driving vehicles ...

We generalize the class vectors found in neural networks to linear subspaces (i.e., points in the Grassmann manifold) and show that the Grassmann Class Representation (GCR) enables simultaneous improvement in accuracy and feature transferability. In GCR, e ...

Byzantine resilience emerged as a prominent topic within the distributed machine learning community. Essentially, the goal is to enhance distributed optimization algorithms, such as distributed SGD, in a way that guarantees convergence despite the presence ...

To study the resilience of distributed learning, the “Byzantine" literature considers a strong threat model where workers can report arbitrary gradients to the parameter server. Whereas this model helped obtain several fundamental results, it has sometimes ...

We introduce a construction of subspaces of the spaces of tangential vector, n-vector, and tensor fields on surfaces. The resulting subspaces can be used as the basis of fast approximation algorithms for design and processing problems that involve tangenti ...

We consider integer programming problems in standard form max{c(T)x : Ax = b, x >= 0, x is an element of Z(n)} where A is an element of Z(mxn), b is an element of Z(m), and c is an element of Z(n). We show that such an integer program can be solved in time ...

Iterative substructuring Domain Decomposition (DD) methods have been extensively studied, and they are usually associated with nonoverlapping decompositions. It is less known that classical overlapping DD methods can also be formulated in substructured for ...

To sustain the input rate of high-throughput streams, modern stream processing systems rely on parallel execution. However, skewed data yield imbalanced load assignments and create stragglers that hinder scalability. Deciding on a static partitioning for a ...

The ever-growing number of edge devices (e.g., smartphones) and the exploding volume of sensitive data they produce, call for distributed machine learning techniques that are privacy-preserving. Given the increasing computing capabilities of modern edge de ...

Deep neural networks (DNNs) have surpassed human-level accuracy in a variety of cognitive tasks but at the cost of significant memory/time requirements in DNN training. This limits their deployment in energy and memory limited applications that require rea ...

This paper demonstrates how to recover causal graphs from the score of the data distribution in non-linear additive (Gaussian) noise models. Using score matching algorithms as a building block, we show how to design a new generation of scalable causal disc ...

Byzantine reliable broadcast is a powerful primitive that allows a set of processes to agree on a message from a designated sender, even if some processes (including the sender) are Byzantine. Existing broadcast protocols for this setting scale poorly, as ...

We present the first measurements of branching fractions of rare tau-lepton decays, tau(-) -> pi(-)nu(tau)l(+)l(-) (l = e or mu), using a data sample corresponding to 562 fb(-1) collected at a center-of-mass energy of 10.58 GeV with the Belle detector at t ...

Collaborative Learning in the Jungle (Decentralized, Byzantine, Heterogeneous, Asynchronous and Nonconvex Learning)

Chat with Graph Search

Roger F. Harrington and the Method of Moments: Part 2: Electrodynamics

EdgeAI-Aware Design of In-Memory Computing Architectures

COMMUNICATION LOWER BOUNDS AND OPTIMAL ALGORITHMS FOR MULTIPLE TENSOR-TIMES-MATRIX COMPUTATION

Which Coupled is Best Coupled? An Exploration of AIMC Tile Interfaces and Load Balancing for CNNs

Generative models for synthetic data generation: application to pharmacokinetic/pharmacodynamic data

Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneity

Diversity of radial spin textures in chiral materials

Robust Training and Verification of Deep Neural Networks

Get the Best of Both Worlds: Improving Accuracy and Transferability by Grassmann Class Representation

Byzantine Machine Learning Made Easy By Resilient Averaging of Momentums

An Equivalence Between Data Poisoning and Byzantine Gradient Attacks

Locally supported tangential vector, n-vector, and tensor fields

Proximity Results and Faster Algorithms for Integer Programming Using the Steinitz Lemma

Linear and nonlinear substructured Restricted Additive Schwarz iterations and preconditioning

Dalton: Learned Partitioning for Distributed Data Streams

Private and Secure Distributed Learning

ESSOP: Efficient and Scalable Stochastic Outer Product Architecture for Deep Learning

Score Matching Enables Causal Discovery of Nonlinear Additive Noise Models

Scalable Byzantine Reliable Broadcast

Observation of tau(-) -> pi(-)nu(tau)e(+)e(-) and search for tau(-) -> pi(-)nu(tau)mu(+)mu(-)