Publication

Online Robust Regression via SGD on the l1 loss

Publications associées (31)

(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability

Scott William Pesme, Nicolas Henri Bernard Flammarion, Mathieu Even

In this paper, we investigate the impact of stochasticity and large stepsizes on the implicit regularisation of gradient descent (GD) and stochastic gradient descent (SGD) over diagonal linear networks. We prove the convergence of GD and SGD with macroscop ...

2023

The statistical complexity of early-stopped mirror descent

Varun Kanade, Tomas Vaskevicius

Recently there has been a surge of interest in understanding implicit regularization properties of iterative gradient-based optimization algorithms. In this paper, we study the statistical guarantees on the excess risk achieved by early-stopped unconstrain ...

Oxford Univ Press2023

Phenomenological theory of variational quantum ground-state preparation

Ivano Tavernelli, Giuseppe Carleo

The variational approach is a cornerstone of computational physics, considering both conventional and quantum computing computational platforms. The variational quantum eigensolver algorithm aims to prepare the ground state of a Hamiltonian exploiting para ...

Amer Physical Soc2023

Deep Learning Generalization with Limited and Noisy Labels

Mahsa Forouzesh

Deep neural networks have become ubiquitous in today's technological landscape, finding their way in a vast array of applications. Deep supervised learning, which relies on large labeled datasets, has been particularly successful in areas such as image cla ...

EPFL2023

BigNeuron: a resource to benchmark and predict performance of algorithms for automated tracing of neurons in light microscopy datasets

Pascal Fua, Sean Lewis Hill, Amos Sironi, Jiancheng Yang, Jian Zhou, Siqi Liu, Xiang Li, Jie Zhou

BigNeuron is an open community bench-testing platform with the goal of setting open standards for accurate and fast automatic neuron tracing. We gathered a diverse set of image volumes across several species that is representative of the data obtained in m ...

2023

Fast Adversarial Training With Adaptive Step Size

Mathieu Salzmann, Sabine Süsstrunk, Jue Wang, Chen Liu, Yong Zhang, Zhuoyi Huang

While adversarial training and its variants have shown to be the most effective algorithms to defend against adversarial attacks, their extremely slow training process makes it hard to scale to large datasets like ImageNet. The key idea of recent works to ...

Ieee-Inst Electrical Electronics Engineers Inc2023

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Mehmet Fatih Sahin

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

EPFL2023

Regularization Techniques for Low-Resource Machine Translation

Alejandro Ramírez Atrio

Neural machine translation (MT) and text generation have recently reached very high levels of quality. However, both areas share a problem: in order to reach these levels, they require massive amounts of data. When this is not present, they lack generaliza ...

EPFL2023

Robust discrete choice models with t-distributed kernel errors

Michel Bierlaire, Prateek Bansal, Thomas Gasos

Outliers in discrete choice response data may result from misclassification and misreporting of the response variable and from choice behaviour that is inconsistent with modelling assumptions (e.g. random utility maximisation). In the presence of outliers, ...

2023

Byzantine Fault-Tolerance in Federated Local SGD Under 2f-Redundancy

Nirupam Gupta

In this article, we study the problem of Byzantine fault-tolerance in a federated optimization setting, where there is a group of agents communicating with a centralized coordinator. We allow up to

f

Byzantine-faulty agents, which may not follow a prescr ...

Ieee-Inst Electrical Electronics Engineers Inc2023

Robust Collaborative Learning with Linear Gradient Overhead

Rachid Guerraoui, Le Nguyen Hoang, John Stephan, Sadegh Farhadkhani, Nirupam Gupta, Rafaël Benjamin Pinot

Collaborative learning algorithms, such as distributed SGD (or D-SGD), are prone to faulty machines that may deviate from their prescribed algorithm because of software or hardware bugs, poisoned data or malicious behaviors. While many solutions have been ...

PLMR2023

Publications associées (31)

BigNeuron: a resource to benchmark and predict performance of algorithms for automated tracing of neurons in light microscopy datasets

Pascal Fua, Sean Lewis Hill, Amos Sironi, Jiancheng Yang, Jian Zhou, Siqi Liu, Xiang Li, Jie Zhou

2023

f

Byzantine-faulty agents, which may not follow a prescr ...

Ieee-Inst Electrical Electronics Engineers Inc2023

Robust Collaborative Learning with Linear Gradient Overhead

Rachid Guerraoui, Le Nguyen Hoang, John Stephan, Sadegh Farhadkhani, Nirupam Gupta, Rafaël Benjamin Pinot

PLMR2023