Publications related to Robust Collaborative Learning with Linear Gradient Overhead

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Distributed learning is the key for enabling training of modern large-scale machine learning models, through parallelising the learning process. Collaborative learning is essential for learning from privacy-sensitive data that is distributed across various ...

EPFL2024

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Scott William Pesme, Nicolas Henri Bernard Flammarion, Hristo Georgiev Papazov

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

2024

Understanding generalization and robustness in modern deep learning

Maksym Andriushchenko

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

EPFL2024

On the Generalization of Stochastic Gradient Descent with Momentum

Volkan Cevher, Kimon Antonakopoulos

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...

Microtome Publishing2024

Robust NAS under adversarial training: benchmark, theory, and beyond

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Yongtao Wu

Recent developments in neural architecture search (NAS) emphasize the significance of considering robust architectures against malicious data. However, there is a notable absence of benchmark evaluations and theoretical guarantees for searching these robus ...

2024

Global existence for perturbations of the 2D stochastic Navier-Stokes equations with space-time white noise

Martin Hairer

We prove global in time well-posedness for perturbations of the 2D stochastic Navier-Stokes equations partial derivative( t)u + u center dot del u = Delta u - del p + sigma + xi, u(0, center dot ) = u(0),div (u) = 0, driven by additive space-time white noi ...

Springernature2024

Biobased surfactant

Jeremy Luterbacher, Stefania Bertella, Anastasiia Komarova, Songlan Sun

The present invention relates to a compound of the general formula (I), and (II)wherein one of R11 and R12 is hydrogen and the other is -CH2-R70, and one of R13 and R14 is hydrogen and the other is -CH2-R71, or wherein one of R11 and R12 is hydrogen and th ...

2024

Spectral Estimators for High-Dimensional Matrix Inference

Farzad Pourkamali

A key challenge across many disciplines is to extract meaningful information from data which is often obscured by noise. These datasets are typically represented as large matrices. Given the current trend of ever-increasing data volumes, with datasets grow ...

EPFL2024

A ride time-oriented scheduling algorithm for dial-a-ride problems

Nikolaos Geroliminis, Claudia Bongiovanni, Mor Kaspi

This paper offers a new algorithm to efficiently optimize scheduling decisions for dial-a-ride problems (DARPs), including problem variants considering electric and autonomous vehicles (e-ADARPs). The scheduling heuristic, based on linear programming theor ...

Pergamon-Elsevier Science Ltd2024

Critical degeneracy in a nonlinear hyperbolic equation can produce atypical instability

Charles Stuart

This paper deals with the initial value problem for a semilinear wave equation on a bounded domain and solutions are required to vanish on the boundary of this domain. The essential feature of the situation considered here is that the ellipticity of the sp ...

2024

Robust machine learning for neuroscientific inference

Steffen Schneider

Modern neuroscience research is generating increasingly large datasets, from recording thousands of neurons over long timescales to behavioral recordings of animals spanning weeks, months, or even years. Despite a great variety in recording setups and expe ...

EPFL2024

Biobased surfactant

Jeremy Luterbacher, Stefania Bertella, Anastasiia Komarova, Songlan Sun

The present invention relates to a compound of the general formula (I), (II) and (III), more specifically of formula (Ia), (Ib), (Ic)wherein R11 and R12 or R21 and R22 or R31 and R32 are both hydrogen or form together with CHR50 a cyclic moiety or one of R ...

2024

Experimental observation of topological transition in linear and nonlinear parametric oscillators

Romain Christophe Rémy Fleury, Benjamin Apffel

Parametric oscillators are examples of externally driven systems that can exhibit two stable states with opposite phase depending on the initial conditions. In this work, we propose to study what happens when the external forcing is perturbed by a continuo ...

2024

Extensions of Peer Prediction Incentive Mechanisms

Adam Julian Richardson

As large, data-driven artificial intelligence models become ubiquitous, guaranteeing high data quality is imperative for constructing models. Crowdsourcing, community sensing, and data filtering have long been the standard approaches to guaranteeing or imp ...

EPFL2024

Revisiting Character-level Adversarial Attacks for Language Models

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Yongtao Wu, Elias Abad Rocamora

Adversarial attacks in Natural Language Processing apply perturbations in the character or token levels. Token-level attacks, gaining prominence for their use of gradient-based methods, are susceptible to altering sentence semantics, leading to invalid adv ...

2024

Towards Trustworthy Deep Learning for Image Reconstruction

Alexis Marie Frederic Goujon

The remarkable ability of deep learning (DL) models to approximate high-dimensional functions from samples has sparked a revolution across numerous scientific and industrial domains that cannot be overemphasized. In sensitive applications, the good perform ...

EPFL2024

A graph convolutional autoencoder approach to model order reduction for parametrized PDEs

Jan Sickmann Hesthaven, Federico Pichi

The present work proposes a framework for nonlinear model order reduction based on a Graph Convolutional Autoencoder (GCA-ROM). In the reduced order modeling (ROM) context, one is interested in obtaining real -time and many-query evaluations of parametric ...

Academic Press Inc Elsevier Science2024

Gibbs sampling the posterior of neural networks

Giovanni Piccioli, Lenka Zdeborová, Emanuele Troiani

In this paper, we study sampling from a posterior derived from a neural network. We propose a new probabilistic model consisting of adding noise at every pre- and post-activation in the network, arguing that the resulting posterior can be sampled using an ...

Iop Publishing Ltd2024

Data set for control of Ge island coalescence for the formation of nanowires on silicon

Alok Rudra, Anna Fontcuberta i Morral, Sara Marti Sanchez, Santhanu Panikar Ramanandan, Joel René Sapera, Vladimir Dubrovskii

This document contains all the data and the details of the analysis used in the manuscript titled " Control of Ge island coalescence for the formation of nanowires on silicon." https://doi.org/10.1039/D3NH00573A ...

EPFL Infoscience2024

Adaptive projected variational quantum dynamics

Giuseppe Carleo, Stefano Barison, David Linteau

We propose an adaptive quantum algorithm to prepare accurate variational time evolved wave functions. The method is based on the projected variational quantum dynamics (pVQD) algorithm, that performs a global optimization with linear scaling in the number ...

Amer Physical Soc2024