Publications related to On the Generalization of Stochastic Gradient Descent with Momentum

Related publications (50)

Overcoming Distribution Shifts and Imbalance Challenges in Representation Learning for Deep Regression Models

Deep learning has shown remarkable potential for industrial applications, particularly in predictive maintenance and condition monitoring. A key challenge in this domain is prognostics, specifically predicting the Remaining Useful Life (RUL) of assets - a ...

EPFL2025

Exploiting Representation Similarities in Self-Supervised Learning for Vision Tasks

Thomas Grégoire Stegmüller

Recent progress in computer vision has been driven by a simple yet transformative observation: the potential of 30-year-old neural networks can be unlocked by massively scaling their parameters and the amount of human-labeled training data. This breakthrou ...

EPFL2025

Scalable kernel logistic regression with Nyström approximation: Theoretical analysis and application to discrete choice modelling

Michel Bierlaire

The application of kernel-based Machine Learning (ML) techniques to discrete choice modelling using large datasets often faces challenges due to memory requirements and the considerable number of parameters involved in these models. This complexity hampers ...

2025

Boosting Resource-Constrained Federated Learning Systems with Guessed Updates

Mohamed Yassine Boukhari, Anne-Marie Kermarrec, Rafael Pereira Pires, Rishi Sharma, Akash Balasaheb Dhasade

Federated learning (FL) enables a set of client devices to collaboratively train a model without sharing raw data. This process, though, operates under the constrained computation and communication resources of edge devices. These constraints combined with ...

Institute of Electrical and Electronics Engineers (IEEE)2025

Weighting Pseudo-labels via High-Activation Feature Index Similarity and Object Detection for Semi-supervised Segmentation

Minh Hieu Lê

Semi-supervised semantic segmentation methods leverage unlabeled data by pseudo-labeling them. Thus the success of these methods hinges on the reliability of the pseudo-labels. Existing methods mostly choose high-confidence pixels in an effort to avoid err ...

Springer Science and Business Media Deutschland GmbH2025

Generalization of Noisy SGD in Unbounded Non-convex Settings

Volkan Cevher, Leello Tadesse Dadi

We study the generalization of iterative noisy gradient schemes on smooth non-convex losses. Formally, we establish time-independent information theoretic generalization bounds for Stochastic Gradient Langevin Dynamics (SGLD) that do not diverge as the ite ...

2025

Learning to Reason with Neural Networks: Principles and Methods

Aryo Lotfi Jandaghi

This thesis focuses on understanding and improving the reasoning capabilities of neural networks. It develops theoretical results and empirical analyses to uncover reasoning potential and limitations, leveraging these insights to guide the design of improv ...

EPFL2025

K-space physics-informed neural network (k-PINN) for compressed spectral mapping and efficient inversion of vibrations in thin composite laminates

Olga Fink

The vibrational response of structural components carries valuable information about their underlying mechanical properties, health status and operational conditions. This underscores the need for the development of efficient physics-based inversion algori ...

Elsevier BV2025

Classifier-free diffusion-based weakly-supervised approach for health indicator derivation in rotating machines: Advancing early fault detection and condition monitoring

Olga Fink, Gaëtan Michel Frusque

Deriving health indicators of rotating machines is crucial for their maintenance. However, this process is challenging for the prevalent adopted intelligent methods since they may take the whole data distributions, not only introducing noise interference b ...

2025

A comparative analysis of gene expression profiling by statistical and machine learning approaches

Anaïs Betsabeh Haget, Maria Boulougouri

Motivation Many machine learning (ML) models developed to classify phenotype from gene expression data provide interpretations for their decisions, with the aim of understanding biological processes. For many models, including neural networks, interpretati ...

OXFORD UNIV PRESS2025

Error assessment of an adaptive finite elements-neural networks method for an elliptic parametric PDE

Maude Girardin, Marco Picasso, Alexandre Caboussat

We present a finite elements-neural network approach for the numerical approximation of parametric partial differential equations. The algorithm generates training data from finite element simulations, and uses a data -driven (supervised) feedforward neura ...

Elsevier Science Sa2024

Fair Decentralized Learning

Anne-Marie Kermarrec, Rishi Sharma, Marinus Abraham de Vos, Sayan Biswas

Decentralized learning (DL) is an emerging approach that enables nodes to collaboratively train a machine learning model without sharing raw data. In many application domains, such as healthcare, this approach faces challenges due to the high level of hete ...

IEEE2025

Machine Learning for Modeling Stock Returns

Teng Andrea Xu

Throughout history, the pace of knowledge and information sharing has evolved into an unthinkable speed and media. At the end of the XVII century, in Europe, the ideas that would shape the "Age of Enlightenment" were slowly being developed in coffeehouses, ...

EPFL2024

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Scott William Pesme

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

Harnessing Increased Client Participation with Cohort-Parallel Federated Learning

Anne-Marie Kermarrec, Rafael Pereira Pires, Marinus Abraham de Vos, Akash Balasaheb Dhasade, Tuan Anh Nguyen

Federated learning (FL) is a machine learning approach where nodes collaboratively train a global model. As more nodes participate in a round of FL, the effectiveness of individual model updates by nodes also diminishes. In this study, we increase the effe ...

ACM2025

Using automated design appraisal to model building-specific devaluation risk due to land-use change

Adam Robert Swietek

Uncertainty of spillover effects – including property devaluation - from proposed land-use change elicits opposition to local development. This hinders cities’ ability to implement land-use policy aimed at housing affordability and environmental sustainabi ...

2024

Sequential Reasoning with Socially Caused Beliefs

Mert Kayaalp

Machine learning and artificial intelligence methods have achieved remarkable success, matching and even surpassing human capabilities in various complex tasks. However, many demonstrations have generally neglected a critical part of the intelligence that ...

EPFL2025

SaliencyMix plus : Noise-Minimized Image Mixing Method With Saliency Map in Data Augmentation

Zhixiong Jin

Data augmentation is vital in deep learning for enhancing model robustness by artificially expanding training datasets. However, advanced methods like CutMix blend images and assign labels based on pixel ratios, often introducing label noise by neglecting ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2025

On the Perspectives of Image-to-Lidar Constraints in Dynamic Network Optimisation

Thibaut Stoltz, Jan Skaloud, Davide Antonio Cucci, Laurent Valentin Jospin, Kyriaki Mouzakidou

The evolution of airborne mapping witnesses the introduction of hybrid lidar-camera systems to enhance data collection, i.e. to obtain simultaneously high-density point-cloud and texture. Yet, the common adjustment of both optical data streams is a nontriv ...

International Society for Photogrammetry and Remote Sensing2025

Machine Learning Security Against Data Poisoning: Are We There Yet?

Kathrin Grosse

Poisoning attacks compromise the training data utilized to train machine learning (ML) models, diminishing their overall performance, manipulating predictions on specific test samples, and implanting backdoors. This article thoughtfully explores these atta ...

Ieee Computer Soc2024