Publications related to Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification*

Related publications (26)

Boosting Resource-Constrained Federated Learning Systems with Guessed Updates

Mohamed Yassine Boukhari, Anne-Marie Kermarrec, Rafael Pereira Pires, Rishi Sharma, Akash Balasaheb Dhasade

Federated learning (FL) enables a set of client devices to collaboratively train a model without sharing raw data. This process, though, operates under the constrained computation and communication resources of edge devices. These constraints combined with ...

Institute of Electrical and Electronics Engineers (IEEE)2025

Overcoming Distribution Shifts and Imbalance Challenges in Representation Learning for Deep Regression Models

Ismail Nejjar

Deep learning has shown remarkable potential for industrial applications, particularly in predictive maintenance and condition monitoring. A key challenge in this domain is prognostics, specifically predicting the Remaining Useful Life (RUL) of assets - a ...

EPFL2025

Sequential Reasoning with Socially Caused Beliefs

Mert Kayaalp

Machine learning and artificial intelligence methods have achieved remarkable success, matching and even surpassing human capabilities in various complex tasks. However, many demonstrations have generally neglected a critical part of the intelligence that ...

EPFL2025

Scalable kernel logistic regression with Nyström approximation: Theoretical analysis and application to discrete choice modelling

Michel Bierlaire

The application of kernel-based Machine Learning (ML) techniques to discrete choice modelling using large datasets often faces challenges due to memory requirements and the considerable number of parameters involved in these models. This complexity hampers ...

2025

A learning-based approach to stochastic optimal control under reach-avoid constraint

Maryam Kamgarpour, Tingting Ni

We develop a model-free approach to optimally control stochastic, Markovian systems subject to a reach-avoid constraint. Specifically, the state trajectory must remain within a safe set while reaching a target set within a finite time horizon. Due to the t ...

Association for Computing Machinery2025

K-space physics-informed neural network (k-PINN) for compressed spectral mapping and efficient inversion of vibrations in thin composite laminates

Olga Fink

The vibrational response of structural components carries valuable information about their underlying mechanical properties, health status and operational conditions. This underscores the need for the development of efficient physics-based inversion algori ...

Elsevier BV2025

Learning to Reason with Neural Networks: Principles and Methods

Aryo Lotfi Jandaghi

This thesis focuses on understanding and improving the reasoning capabilities of neural networks. It develops theoretical results and empirical analyses to uncover reasoning potential and limitations, leveraging these insights to guide the design of improv ...

EPFL2025

On the Perspectives of Image-to-Lidar Constraints in Dynamic Network Optimisation

Thibaut Stoltz, Jan Skaloud, Davide Antonio Cucci, Laurent Valentin Jospin, Kyriaki Mouzakidou

The evolution of airborne mapping witnesses the introduction of hybrid lidar-camera systems to enhance data collection, i.e. to obtain simultaneously high-density point-cloud and texture. Yet, the common adjustment of both optical data streams is a nontriv ...

International Society for Photogrammetry and Remote Sensing2025

Generalization of Noisy SGD in Unbounded Non-convex Settings

Volkan Cevher, Leello Tadesse Dadi

We study the generalization of iterative noisy gradient schemes on smooth non-convex losses. Formally, we establish time-independent information theoretic generalization bounds for Stochastic Gradient Langevin Dynamics (SGLD) that do not diverge as the ite ...

2025

On the Generalization of Stochastic Gradient Descent with Momentum

Volkan Cevher, Kimon Antonakopoulos

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...

Microtome Publishing2024

Understanding generalization and robustness in modern deep learning

Maksym Andriushchenko

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

EPFL2024

Electronic Excited States from Physically Constrained Machine Learning

Michele Ceriotti, Jigyasa Nigam, Divya Suman

Data-driven techniques are increasingly used to replace electronic-structure calculations of matter. In this context, a relevant question is whether machine learning (ML) should be applied directly to predict the desired properties or combined explicitly w ...

2024

Environmental and geographical factors influencing the spread of SARS-CoV-2 over 2 years: a fine-scale spatiotemporal analysis

Stéphane Joost, Anaïs Laurence Ladoy, David Nicolas De Ridder

Introduction Since its emergence in late 2019, the SARS-CoV-2 virus has led to a global health crisis, affecting millions and reshaping societies and economies worldwide. Investigating the determinants of SARS-CoV-2 diffusion and their spatiotemporal dynam ...

FRONTIERS MEDIA SA2024

Asymptotics of feature learning in two-layer networks after one gradient-step

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Hugo Chao Cui, Yatin Dandi, Luca Pesce, Yue Lu

In this manuscript, we investigate the problem of how two-layer neural networks learn features from data, and improve over the kernel regime, after being trained with a single gradient descent step. Leveraging the insight from (Ba et al., 2022), we model t ...

2024

Using Motion Cues to Supervise Single-Frame Body Pose and Shape Estimation in Low Data Regimes

Pascal Fua, Mathieu Salzmann, Andrey Davydov

When enough annotated training data is available, supervised deep-learning. algorithms excel at estimating human body pose and shape using a single camera. The effects of too little such data being available can be mitigated by using other information sour ...

Journal of Machine Learning Research Inc.2024

On the Utility of Speech and Audio Foundation Models for Marmoset Call Analysis

Mathew Magimai Doss, Eklavya Sarkar

Marmoset monkeys encode vital information in their calls and serve as a surrogate model for neuro-biologists to understand the evolutionary origins of human vocal communication. Traditionally analyzed with signal processing-based features, recent approache ...

2024

Semi-Discrete Optimal Transport: Hardness, Regularization and Numerical Solution

Daniel Kuhn, Soroosh Shafieezadeh Abadeh, Bahar Taskesen

Semi-discrete optimal transport problems, which evaluate the Wasserstein distance between a discrete and a generic (possibly non-discrete) probability measure, are believed to be computationally hard. Even though such problems are ubiquitous in statistics, ...

2023

Byzantine Machine Learning: A Primer

Rachid Guerraoui, Nirupam Gupta, Rafaël Benjamin Pinot

The problem of Byzantine resilience in distributed machine learning, a.k.a., Byzantine machine learning, consists in designing distributed algorithms that can train an accurate model despite the presence of Byzantine nodes, i.e., nodes with corrupt data or ...

2023

A Theory of Finite-Width Neural Networks: Generalization, Scaling Laws, and the Loss Landscape

Berfin Simsek

Deep learning has achieved remarkable success in various challenging tasks such as generating images from natural language or engaging in lengthy conversations with humans. The success in practice stems from the ability to successfully train massive neural ...

EPFL2023

Twisted chiral superconductivity in photodoped frustrated Mott insulators

Jiajun Li, Markus Müller, Philipp Georg Werner Seeberger

Recent advances in ultrafast pump-probe spectroscopy provide access to hidden phases of correlated matter, including light-induced superconducting states. The theoretical understanding of these nonequilibrium phases remains limited, particularly for correl ...

AMER PHYSICAL SOC2023