Publications associées à Wasserstein Adversarial Regularization for learning with label noise

Efficient local linearity regularization to overcome catastrophic overfitting

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Elias Abad Rocamora

Catastrophic overfitting (CO) in single-step adversarial training (AT) results in abrupt drops in the adversarial test accuracy (even down to 0%). For models trained with multi-step AT, it has been observed that the loss function behaves locally linearly w ...

2024

Understanding generalization and robustness in modern deep learning

Maksym Andriushchenko

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

EPFL2024

Statistical Inference for Inverse Problems: From Sparsity-Based Methods to Neural Networks

Pakshal Narendra Bohra

In inverse problems, the task is to reconstruct an unknown signal from its possibly noise-corrupted measurements. Penalized-likelihood-based estimation and Bayesian estimation are two powerful statistical paradigms for the resolution of such problems. They ...

EPFL2024

Deep Learning Generalization with Limited and Noisy Labels

Mahsa Forouzesh

Deep neural networks have become ubiquitous in today's technological landscape, finding their way in a vast array of applications. Deep supervised learning, which relies on large labeled datasets, has been particularly successful in areas such as image cla ...

EPFL2023

Bayes-optimal Learning of Deep Random Networks of Extensive-width

Hugo Chao Cui, Florent Gérard Krzakala, Lenka Zdeborová

We consider the problem of learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights. We consider the asymptotic limit where the number of samples, the input dimension and the network width ...

2023

From Kernel Methods to Neural Networks: A Unifying Variational Formulation

Michaël Unser

The minimization of a data-fidelity term and an additive regularization functional gives rise to a powerful framework for supervised learning. In this paper, we present a unifying regularization functional that depends on an operator L\documentclass[12pt]{ ...

Springer2023

Regularization Techniques for Low-Resource Machine Translation

Alejandro Ramírez Atrio

Neural machine translation (MT) and text generation have recently reached very high levels of quality. However, both areas share a problem: in order to reach these levels, they require massive amounts of data. When this is not present, they lack generaliza ...

EPFL2023

A Neural-Network-Based Convex Regularizer for Inverse Problems

Michaël Unser, Stanislas Ducotterd, Pakshal Narendra Bohra, Alexis Marie Frederic Goujon, Sebastian Jonas Neumayer

The emergence of deep-learning-based methods to solve image-reconstruction problems has enabled a significant increase in quality. Unfortunately, these new methods often lack reliability and explainability, and there is a growing interest to address these ...

2023

Adversarial Training Should Be Cast As a Non-Zero-Sum Game

Seyed Hamed Hassani, Volkan Cevher, Fabian Ricardo Latorre Gomez

One prominent approach toward resolving the adversarial vulnerability of deep neural networks is the two-player zero-sum paradigm of adversarial training, in which predictors are trained against adversarially-chosen perturbations of data. Despite the promi ...

2023

Robust Training and Verification of Deep Neural Networks

Fabian Ricardo Latorre Gomez

According to the proposed Artificial Intelligence Act by the European Comission (expected to pass at the end of 2023), the class of High-Risk AI Systems (Title III) comprises several important applications of Deep Learning like autonomous driving vehicles ...

EPFL2023

Regularization of polynomial networks for image recognition

Volkan Cevher, Bohan Wang, Grigorios Chrysos

Deep Neural Networks (DNNs) have obtained impressive performance across tasks, however they still remain as black boxes, e.g., hard to theoretically analyze. At the same time, Polynomial Networks (PNs) have emerged as an alternative method with a promising ...

2023

Controlled physics-informed data generation for deep learning-based remaining useful life prediction under unseen operation conditions

Jian Zhou, Olga Fink

Limited availability of representative time-to-failure (TTF) trajectories either limits the performance of deep learning (DL)-based approaches on remaining useful life (RUL) prediction in practice or even precludes their application. Generating synthetic d ...

2023

Regularization of polynomial networks for image recognition

Volkan Cevher, Bohan Wang, Grigorios Chrysos

Deep Neural Networks (DNNs) have obtained impressive performance across tasks, however they still remain as black boxes, e.g., hard to theoretically analyze. At the same time, Polynomial Networks (PNs) have emerged as an alternative method with a promising ...

Ieee Computer Soc2023

The statistical complexity of early-stopped mirror descent

Varun Kanade, Tomas Vaskevicius

Recently there has been a surge of interest in understanding implicit regularization properties of iterative gradient-based optimization algorithms. In this paper, we study the statistical guarantees on the excess risk achieved by early-stopped unconstrain ...

Oxford Univ Press2023

Combining Model Driven and Data Driven Approaches for Inverse Problems in Parameter Estimation and Image Reconstruction: From Modelling to Validation

Thomas Yu

Machine learning has become the state of the art for the solution of the diverse inverse problems arising from computer vision and medical imaging, e.g. denoising, super-resolution, de-blurring, reconstruction from scanner data, quantitative magnetic reson ...

EPFL2022

Role of stochastic noise and generalization error in the time propagation of neural-network quantum states

Giuseppe Carleo

Neural-network quantum states (NQS) have been shown to be a suitable variational ansatz to simulate out-of-equilibrium dynamics in two-dimensional systems using timedependent variational Monte Carlo (t-VMC). In particular, stable and accurate time propagat ...

SCIPOST FOUNDATION2022

Predicting in Uncertain Environments: Methods for Robust Machine Learning

Paul Thierry Yves Rolland

One of the main goal of Artificial Intelligence is to develop models capable of providing valuable predictions in real-world environments. In particular, Machine Learning (ML) seeks to design such models by learning from examples coming from this same envi ...

EPFL2022

Decentralized Semi-supervised Learning over Multitask Graphs

Ali H. Sayed, Roula Nassif, Elsa Rizk

In network semi-supervised learning problems, only a subset of the network nodes is able to access the data labeling. This paper formulates a decentralized optimization problem where agents have individual decision rules to estimate, subject to the conditi ...

IEEE2022

Towards Verifiable, Generalizable and Efficient Robust Deep Neural Networks

Chen Liu

In the last decade, deep neural networks have achieved tremendous success in many fields of machine learning. However, they are shown vulnerable against adversarial attacks: well-designed, yet imperceptible, perturbations can make the state-of-the-art deep ...

EPFL2022

Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problems

Volkan Cevher, Thomas Michaelsen Pethick

This paper introduces a new extragradient-type algorithm for a class of nonconvex-nonconcave minimax problems. It is well-known that finding a local solution for general minimax problems is computationally intractable. This observation has recently motivat ...

2022