Publications related to A Sequential Topic Model for Mining Recurrent Activities from Long Term Video Logs

Statistical Inference for Inverse Problems: From Sparsity-Based Methods to Neural Networks

In inverse problems, the task is to reconstruct an unknown signal from its possibly noise-corrupted measurements. Penalized-likelihood-based estimation and Bayesian estimation are two powerful statistical paradigms for the resolution of such problems. They ...

EPFL2024

Robust machine learning for neuroscientific inference

Steffen Schneider

Modern neuroscience research is generating increasingly large datasets, from recording thousands of neurons over long timescales to behavioral recordings of animals spanning weeks, months, or even years. Despite a great variety in recording setups and expe ...

EPFL2024

High-Dimensional Kernel Methods under Covariate Shift: Data-Dependent Implicit Regularization

Volkan Cevher, Fanghui Liu

This paper studies kernel ridge regression in high dimensions under covariate shifts and analyzes the role of importance re-weighting. We first derive the asymptotic expansion of high dimensional kernels under covariate shifts. By a bias-variance decomposi ...

2024

Efficient local linearity regularization to overcome catastrophic overfitting

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Elias Abad Rocamora

Catastrophic overfitting (CO) in single-step adversarial training (AT) results in abrupt drops in the adversarial test accuracy (even down to 0%). For models trained with multi-step AT, it has been observed that the loss function behaves locally linearly w ...

2024

Understanding generalization and robustness in modern deep learning

Maksym Andriushchenko

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

EPFL2024

Quantifying the Unknown: Data-Driven Approaches and Applications in Energy Systems

Paul Scharnhorst

In light of the challenges posed by climate change and the goals of the Paris Agreement, electricity generation is shifting to a more renewable and decentralized pattern, while the operation of systems like buildings is increasingly electrified. This calls ...

EPFL2024

Towards Trustworthy Deep Learning for Image Reconstruction

Alexis Marie Frederic Goujon

The remarkable ability of deep learning (DL) models to approximate high-dimensional functions from samples has sparked a revolution across numerous scientific and industrial domains that cannot be overemphasized. In sensitive applications, the good perform ...

EPFL2024

Robust Outlier Rejection for 3D Registration with Variational Bayes

Mathieu Salzmann, Zheng Dang, Zhen Wei, Jiancheng Yang, Haobo Jiang

Learning-based outlier (mismatched correspondence) rejection for robust 3D registration generally formulates the outlier removal as an inlier/outlier classification problem. The core for this to be successful is to learn the discriminative inlier/outlier f ...

Ieee Computer Soc2023

An Ordinal Latent Variable Model of Conflict Intensity

Robert West

Measuring the intensity of events is crucial for monitoring and tracking armed conflict. Advances in automated event extraction have yielded massive data sets of '' who did what to whom '' micro-records that enable datadriven approaches to monitoring confl ...

Assoc Computational Linguistics-Acl2023

Bayes-optimal Learning of Deep Random Networks of Extensive-width

Hugo Chao Cui, Florent Gérard Krzakala, Lenka Zdeborová

We consider the problem of learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights. We consider the asymptotic limit where the number of samples, the input dimension and the network width ...

2023

Automatic Parameterization for Aerodynamic Shape Optimization via Deep Geometric Learning

Pascal Fua, Zhen Wei

We propose two deep learning models that fully automate shape parameterization for aerodynamic shape optimization. Both models are optimized to parameterize via deep geometric learning to embed human prior knowledge into learned geometric patterns, elimina ...

2023

A Neural-Network-Based Convex Regularizer for Inverse Problems

Michaël Unser, Stanislas Ducotterd, Pakshal Narendra Bohra, Alexis Marie Frederic Goujon, Sebastian Jonas Neumayer

The emergence of deep-learning-based methods to solve image-reconstruction problems has enabled a significant increase in quality. Unfortunately, these new methods often lack reliability and explainability, and there is a growing interest to address these ...

2023

Regularization of polynomial networks for image recognition

Volkan Cevher, Bohan Wang, Grigorios Chrysos

Deep Neural Networks (DNNs) have obtained impressive performance across tasks, however they still remain as black boxes, e.g., hard to theoretically analyze. At the same time, Polynomial Networks (PNs) have emerged as an alternative method with a promising ...

2023

Regularization of polynomial networks for image recognition

Volkan Cevher, Bohan Wang, Grigorios Chrysos

Deep Neural Networks (DNNs) have obtained impressive performance across tasks, however they still remain as black boxes, e.g., hard to theoretically analyze. At the same time, Polynomial Networks (PNs) have emerged as an alternative method with a promising ...

Ieee Computer Soc2023

Distribution Inference Risks: Identifying and Mitigating Sources of Leakage

Léo Nicolas René Meynent, Maxime Jean Julien Peyrard, Robert West, Valentin Hartmann

A large body of work shows that machine learning (ML) models can leak sensitive or confidential information about their training data. Recently, leakage due to distribution inference (or property inference) attacks is gaining attention. In this attack, the ...

IEEE COMPUTER SOC2023

Deep Learning Generalization with Limited and Noisy Labels

Mahsa Forouzesh

Deep neural networks have become ubiquitous in today's technological landscape, finding their way in a vast array of applications. Deep supervised learning, which relies on large labeled datasets, has been particularly successful in areas such as image cla ...

EPFL2023

Regularization Techniques for Low-Resource Machine Translation

Alejandro Ramírez Atrio

Neural machine translation (MT) and text generation have recently reached very high levels of quality. However, both areas share a problem: in order to reach these levels, they require massive amounts of data. When this is not present, they lack generaliza ...

EPFL2023

Fundamental Limits in Statistical Learning Problems: Block Models and Neural Networks

Elisabetta Cornacchia

This thesis focuses on two selected learning problems: 1) statistical inference on graphs models, and, 2) gradient descent on neural networks, with the common objective of defining and analysing the measures that characterize the fundamental limits. In th ...

EPFL2023

(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability

Scott William Pesme, Nicolas Henri Bernard Flammarion, Mathieu Even

In this paper, we investigate the impact of stochasticity and large stepsizes on the implicit regularisation of gradient descent (GD) and stochastic gradient descent (SGD) over diagonal linear networks. We prove the convergence of GD and SGD with macroscop ...

2023

Analysis of a Class of Multilevel Markov Chain Monte Carlo Algorithms Based on Independent Metropolis–Hastings

Fabio Nobile, Juan Pablo Madrigal Cianci

In this work, we present, analyze, and implement a class of multilevel Markov chain Monte Carlo(ML-MCMC) algorithms based on independent Metropolis--Hastings proposals for Bayesian inverse problems. In this context, the likelihood function involves solving ...

2023