Publications associées à Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem

Publications associées (51)

MARBLE: interpretable representations of neural population dynamics using geometric deep learning

The dynamics of neuron populations commonly evolve on low-dimensional manifolds. Thus, we need methods that learn the dynamical processes over neural manifolds to infer interpretable and consistent latent representations. We introduce a representation lear ...

Nature Publishing Group2025

Phase Retrieval: From Computational Imaging to Machine Learning: A tutorial

Michaël Unser, Thanh-An Michel Pham, Jonathan Yuelin Dong

Phase retrieval consists in the recovery of a complex-valued signal from intensity-only measurements. As it pervades a broad variety of applications, many researchers have striven to develop phase-retrieval algorithms. Classical approaches involve techniqu ...

2023

Teaching oxidation states to neural networks

Nicola Marzari

While the accurate description of redox reactions remains a challenge for first-principles calculations, it has been shown that extended Hubbard functionals (DFT+U+V) can provide a reliable approach, mitigating self-interaction errors, in materials with st ...

2025

Stable Nonconvex-Nonconcave Training via Linear Interpolation

Volkan Cevher, Thomas Michaelsen Pethick, Wanyun Xie

This paper presents a theoretical analysis of linear interpolation as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss lan ...

2023

Foundation neural-networks quantum states as a unified Ansatz for multiple hamiltonians

Luciano Loris Viteritti, Giuseppe Carleo

Foundation models are highly versatile neural-network architectures capable of processing different data types, such as text and images, and generalizing across various tasks like classification and generation. Inspired by this success, we propose Foundati ...

2025

Training Efficient Controllers via Analytic Policy Gradient

Dario Floreano, Davide Scaramuzza, Valentin Wüest

Control design for robotic systems is complex and often requires solving an optimization to follow a trajectory accurately. Online optimization approaches like Model Predictive Control (MPC) have been shown to achieve great tracking performance, but requir ...

New York2023

Transferability of Learnt Speech Representations for Decoding Non-Human Vocal Communication

Eklavya Sarkar

Humans and animals both use acoustic signals for vocal communication. The advent of self-supervised learning (SSL) has enabled neural networks to learn robust and general feature representations through the intrinsic acoustic structure of input signals, wi ...

EPFL2025

An optimal preconditioned FFT-accelerated finite element solver for homogenization

Till Junge, Ali Falsafi

We generalize and provide a linear algebra-based perspective on a finite element (FE) ho-mogenization scheme, pioneered by Schneider et al. (2017)[1] and Leuschner and Fritzen (2018)[2]. The efficiency of the scheme is based on a preconditioned, well-scale ...

ELSEVIER SCIENCE INC2023

Hybrid Decentralized Optimization: Leveraging Both First- and Zeroth-Order Optimizers for Faster Convergence

Matin Ansaripour

Distributed optimization is the standard way of speeding up machine learning training, and most of the research in the area focuses on distributed first-order, gradient-based methods. Yet, there are settings where some computationally-bounded nodes may not ...

Association for the Advancement of Artificial Intelligence2025

Phase Reconstruction of Low-Energy Electron Holograms of Individual Proteins

Klaus Kern, Stephan Rauschenbach, Sven Alexander Szilagyi, Hannah Julia Ochner

Low-energy electron holography (LEEH) is one of the few techniques capable of imaging large and complex three-dimensional molecules, such as proteins, on the single molecule level at subnanometer resolution. During the imaging process, the structural infor ...

AMER CHEMICAL SOC2022

Musculoskeletal motor control with reinforcement learning

Alberto Silvio Chiappa

Animals, including humans, interact with the external environment primarily through motion. Replicating their motor control skills in artificial embodied agents is a major objective of artificial intelligence research. This thesis presents a collection of ...

EPFL2025

Intelligent Soft Robotic System for Sensing and Recognition via Triboelectric‐Based Multi‐Sensor Fusion

Xin Chen

Triboelectric nanogenerators (TENGs) serve as a key technology in flexible electronic devices, especially tactile sensors, enabling flexible robotic manipulators to effectively interact with their external environment. Persistent challenges include inadequ ...

Wiley2025

Masked Training of Neural Networks with Partial Gradients

Martin Jaggi, Sebastian Urban Stich, Amirkeivan Mohtashami

State-of-the-art training algorithms for deep learning models are based on stochastic gradient descent (SGD). Recently, many variations have been explored: perturbing parameters for better accuracy (such as in Extra-gradient), limiting SGD updates to a sub ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

On the Double Descent of Random Features Models Trained with SGD

Volkan Cevher, Fanghui Liu

We study generalization properties of random features (RF) regression in high dimensions optimized by stochastic gradient descent (SGD) in under-/overparameterized regime. In this work, we derive precise non-asymptotic error bounds of RF regression under b ...

2022

TimeGPT in load forecasting: A large time series model perspective

Fernando Porté Agel, Jiannong Fang, Wenlong Liao

Machine learning models have made significant progress in load forecasting, but their forecast accuracy is limited in cases where historical load data is scarce. Inspired by the outstanding performance of large language models (LLMs) in computer vision and ...

2025

Ensemble learning for portfolio valuation and risk management

Damir Filipovic, Lotfi Boudabsa

We introduce an ensemble learning method for dynamic portfolio valuation and risk management building on regression trees. We learn the dynamic value process of a derivative portfolio from a finite sample of its cumulative cash flow. The estimator is given ...

ROUTLEDGE JOURNALS, TAYLOR & FRANCIS LTD2025

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Ludovic Théo Stephan, Rodrigo Soares Veiga

Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achieve global convergence under gradient descent. The picture can be radically different for narrow net-works, which tend to get stuck in badly-generalizing loca ...

2022

Beyond Pixels: Semi-supervised Semantic Segmentation with a Multi-scale Patch-Based Multi-label Classifier

Minh Hieu Lê

Incorporating pixel contextual information is critical for accurate segmentation. In this paper, we show that an effective way to incorporate contextual information is through a patch-based classifier. This patch classifier is trained to identify classes p ...

Springer Science and Business Media Deutschland GmbH2025

Towards Understanding Sharpness-Aware Minimization

Nicolas Henri Bernard Flammarion, Maksym Andriushchenko

Sharpness-Aware Minimization (SAM) is a recent training method that relies on worst-case weight perturbations which significantly improves generalization in various settings. We argue that the existing justifications for the success of SAM which are based ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

Iterative pre-conditioning for expediting the distributed gradient-descent method: The case of linear least-squares problem

Nirupam Gupta

This paper considers the multi-agent linear least-squares problem in a server-agent network architecture. The system comprises multiple agents, each with a set of local data points. The agents are connected to a server, and there is no inter-agent communic ...

PERGAMON-ELSEVIER SCIENCE LTD2022