Comparing Langevin Dynamics and Stochastic Gradient Flow in the Weak Features Model

We compare the test risk of two continuous-time stochastic optimization dynamics within the weak features model originally introduced by Breiman and Freedman. The first dynamics corresponds to the continuous-time limit of stochastic gradient descent with single-sample updates, while the second is overdamped Langevin dynamics, obtained by adding isotropic Brownian noise to gradient flow. Within this framework, the test risk can be computed explicitly. We analyze and contrast the behavior of the two dynamics as functions of time and model parameters, with particular emphasis on their impact on the double descent phenomenon known to arise in the deterministic setting. Our analytical results are validated by numerical simulations of the corresponding discrete-time algorithms, showing good agreement.