On the Generalization of Stochastic Gradient Descent with Momentum

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In computational hydraulics models, predicting bed topography and bedload transport with sufficient accuracy remains a significant challenge. An accurate assessment of a river's sediment transport rate necessitates a prior understanding of its bed topograp ...

Low-light photography produces images with low signal-to-noise ratios due to limited photons. In such conditions, common approximations like the Gaussian noise model fall short, and many denoising techniques fail to remove noise effectively. Although deep- ...

This paper presents a comparison of machine learning (ML) methods used for three-dimensional localization of partial discharges (PD) in a power transformer tank. The study examines ML and deep learning (DL) methods, ranging from support vector machines (SV ...

Earth scientists study a variety of problems with remote sensing data, but they most often consider them in isolation from each other, which limits information flows across disciplines. In this work, we present METEOR, a meta-learning methodology for Earth ...

Utilization of small molecules as passivation materials for perovskite solar cells (PSCs) has gained significant attention recently, with hundreds of small molecules demonstrating passivation effects. In this study, a high-accuracy machine learning model i ...

Computing servers have played a key role in developing and processing emerging compute-intensive applications in recent years. Consolidating multiple virtual machines (VMs) inside one server to run various applications introduces severe competence for limi ...

The ability to reason, plan and solve highly abstract problems is a hallmark of human intelligence. Recent advancements in artificial intelligence, propelled by deep neural networks, have revolutionized disciplines like computer vision and natural language ...

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

Digital twins are virtual models of physical objects or systems that enable real-time monitoring and analysis. In the ﬁeld of stone masonry buildings, digital twins can be used to assess damage, predict maintenance needs, and opti- mize building performanc ...

Statistical (machine-learning, ML) models are more and more often used in computational chemistry as a substitute to more expensive ab initio and parametrizable methods. While the ML algorithms are capable of learning physical laws implicitly from data, ad ...

In this manuscript, we investigate the problem of how two-layer neural networks learn features from data, and improve over the kernel regime, after being trained with a single gradient descent step. Leveraging the insight from (Ba et al., 2022), we model t ...

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

Machine learning has provided a means to accelerate early-stage drug discovery by combining molecule generation and filtering steps in a single architecture that leverages the experience and design preferences of medicinal chemists. However, designing mach ...

In the past few years, Machine Learning (ML) techniques have ushered in a paradigm shift, allowing the harnessing of ever more abundant sources of data to automate complex tasks. The technical workhorse behind these important breakthroughs arguably lies in ...

The performance of machine learning algorithms is conditioned by the availability of training datasets, which is especially true for the field of nondestructive evaluation. Here we propose one reconfigurable specimen instead of numerous reference specimens ...

Magnonics is a research field that has gained an increasing interest in both the fundamental and applied sciences in recent years. This field aims to explore and functionalize collective spin excitations in magnetically ordered materials for modern informa ...

Machine learning (ML) enables artificial intelligent (AI) agents to learn autonomously from data obtained from their environment to perform tasks. Modern ML systems have proven to be extremely effective, reaching or even exceeding human intelligence. Altho ...

Modern optimization is tasked with handling applications of increasingly large scale, chiefly due to the massive amounts of widely available data and the ever-growing reach of Machine Learning. Consequently, this area of research is under steady pressure t ...

Herein, machine learning (ML) models using multiple linear regression (MLR), support vector regression (SVR), random forest (RF) and artificial neural network (ANN) are developed and compared to predict the output features viz. specific capacitance (Csp), ...

In the rapidly evolving landscape of machine learning research, neural networks stand out with their ever-expanding number of parameters and reliance on increasingly large datasets. The financial cost and computational resources required for the training p ...

On the Generalization of Stochastic Gradient Descent with Momentum

Chat with Graph Search

Stochastic Simulations and Machine Learning Modeling to Predict Bedload Transport and Bed Topography

Dark Noise Diffusion: Noise Synthesis for Low-Light Image Denoising

Partial discharge localization in power transformer tanks using machine learning methods

Meta-learning to address diverse Earth observation problems across resolutions

Machine Learning for Screening Small Molecules as Passivation Materials for Enhanced Perovskite Solar Cells

CloudProphet: A Machine Learning-Based Performance Prediction for Public Clouds

Infusing structured knowledge priors in neural models for sample-efficient symbolic reasoning

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Digital Twins of Stone Masonry Buildings for Damage Assessment

Encoding quantum-chemical knowledge into machine-learning models of complex molecular properties

Asymptotics of feature learning in two-layer networks after one gradient-step

Understanding generalization and robustness in modern deep learning

Machine learning-aided generative molecular design

Topics in statistical physics of high-dimensional machine learning

Reduced Training Data for Laser Ultrasound Signal Interpretation by Neural Networks

The 2024 Magnonics Roadmap

Few-shot Learning for Efficient and Effective Machine Learning Model Adaptation

Scalable constrained optimization

Machine learning models for prediction of electrochemical properties in supercapacitor electrodes using MXene and graphene nanoplatelets

Random matrix methods for high-dimensional machine learning models