Publication

Comparing Langevin Dynamics and Stochastic Gradient Flow in the Weak Features Model

2026

Abstract

We compare the test risk of two continuous-time stochastic optimization dynamics within the weak features model originally introduced by Breiman and Freedman. The first dynamics corresponds to the continuous-time limit of stochastic gradient descent with single-sample updates, while the second is overdamped Langevin dynamics, obtained by adding isotropic Brownian noise to gradient flow. Within this framework, the test risk can be computed explicitly. We analyze and contrast the behavior of the two dynamics as functions of time and model parameters, with particular emphasis on their impact on the double descent phenomenon known to arise in the deterministic setting. Our analytical results are validated by numerical simulations of the corresponding discrete-time algorithms, showing good agreement.

Official source

https://infoscience.epfl.ch/entities/publication/3bd4b58b-e4ad-4208-bd18-22238e62b472

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

2026

Abstract

Official source

https://infoscience.epfl.ch/entities/publication/3bd4b58b-e4ad-4208-bd18-22238e62b472

About this result

Related concepts (32)

Stochastic gradient descent

Stochastic gradient descent (often abbreviated SGD) is an iterative method for optimizing an objective function with suitable smoothness properties (e.g. differentiable or subdifferentiable). It can be regarded as a stochastic approximation of gradient descent optimization, since it replaces the actual gradient (calculated from the entire data set) by an estimate thereof (calculated from a randomly selected subset of the data).

Gradient descent

In mathematics, gradient descent (also often called steepest descent) is a iterative optimization algorithm for finding a local minimum of a differentiable function. The idea is to take repeated steps in the opposite direction of the gradient (or approximate gradient) of the function at the current point, because this is the direction of steepest descent. Conversely, stepping in the direction of the gradient will lead to a local maximum of that function; the procedure is then known as gradient ascent.

Molecular dynamics

Molecular dynamics (MD) is a computer simulation method for analyzing the physical movements of atoms and molecules. The atoms and molecules are allowed to interact for a fixed period of time, giving a view of the dynamic "evolution" of the system. In the most common version, the trajectories of atoms and molecules are determined by numerically solving Newton's equations of motion for a system of interacting particles, where forces between the particles and their potential energies are often calculated using interatomic potentials or molecular mechanical force fields.

Related publications (32)

Comparing Langevin Dynamics and Stochastic Gradient Flow in the Weak Features Model

Chat with Graph Search

The semi-explicit nonsmooth Newmark-β time integrator for robust unilateral contact in dynamic fragmentation simulations

Training Dynamics of Gradient Methods on Simple Neural Architectures

Optimization in Modern Machine Learning: Steepest Descent Theory and Trustworthy Models