Publication

Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity

Related publications (31)

Revisiting Character-level Adversarial Attacks for Language Models

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Yongtao Wu, Elias Abad Rocamora

Adversarial attacks in Natural Language Processing apply perturbations in the character or token levels. Token-level attacks, gaining prominence for their use of gradient-based methods, are susceptible to altering sentence semantics, leading to invalid adv ...

2024

Scalable constrained optimization

Maria-Luiza Vladarean

Modern optimization is tasked with handling applications of increasingly large scale, chiefly due to the massive amounts of widely available data and the ever-growing reach of Machine Learning. Consequently, this area of research is under steady pressure t ...

EPFL2024

Robust machine learning for neuroscientific inference

Steffen Schneider

Modern neuroscience research is generating increasingly large datasets, from recording thousands of neurons over long timescales to behavioral recordings of animals spanning weeks, months, or even years. Despite a great variety in recording setups and expe ...

EPFL2024

The Halos and Environments of Nearby Galaxies (HERON) survey IV. Complexity in the boxy galaxies NGC720 and NGC2768

Oliver Müller

The shapes of galaxies, their outer regions in particular, are important guideposts to their formation and evolution. In this work, we report on the discovery of strongly box-shaped morphologies of the otherwise well-studied elliptical and lenticular galax ...

Edp Sciences S A2024

Long live(d) CsPbBr3 superlattices: colloidal atomic layer deposition for structural stability

Raffaella Buonsanti, Alexander Nicolas Chen, Philippe Benjamin Green, Victoria Lapointe

Superlattice formation afforded by metal halide perovskite nanocrystals has been a phenomenon of interest due to the high structural order induced in these self-assemblies, an order that is influenced by the surface chemistry and particle morphology of the ...

Royal Soc Chemistry2024

A Combination Technique for Optimal Control Problems Constrained by Random PDEs

Fabio Nobile, Tommaso Vanzan

We present a combination technique based on mixed differences of both spatial approximations and quadrature formulae for the stochastic variables to solve efficiently a class of optimal control problems (OCPs) constrained by random partial differential equ ...

2024

Robust Collaborative Learning with Linear Gradient Overhead

Rachid Guerraoui, Le Nguyen Hoang, John Stephan, Sadegh Farhadkhani, Nirupam Gupta, Rafaël Benjamin Pinot

Collaborative learning algorithms, such as distributed SGD (or D-SGD), are prone to faulty machines that may deviate from their prescribed algorithm because of software or hardware bugs, poisoned data or malicious behaviors. While many solutions have been ...

PLMR2023

Functional data analysis with rough sample paths?

Victor Panaretos, Neda Mohammadi Jouzdani

Functional data are typically modeled as sample paths of smooth stochastic processes in order to mitigate the fact that they are often observed discretely and noisily, occasionally irregularly and sparsely. The smoothness assumption is imposed to allow for ...

TAYLOR & FRANCIS LTD2024

Incremental Learning in Diagonal Linear Networks

Raphaël Jean Berthier

Diagonal linear networks (DLNs) are a toy simplification of artificial neural networks; they consist in a quadratic reparametrization of linear regression inducing a sparse implicit regularization. In this paper, we describe the trajectory of the gradient ...

Microtome Publishing2023

Universal and adaptive methods for robust stochastic optimization

Ali Kavis

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

EPFL2023

Timing and spatial selection bias in rapid extreme event attribution

Anthony Christopher Davison, Ophélia Mireille Anna Miralles

Selection bias may arise when data have been chosen in a way that subsequent analysis does not account for. Such bias can arise in climate event attribution studies that are performed rapidly after a devastating "trigger event'', whose occurrence correspon ...

ELSEVIER2023