On the Generalization of Stochastic Gradient Descent with Momentum

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

Deep learning has shown remarkable potential for industrial applications, particularly in predictive maintenance and condition monitoring. A key challenge in this domain is prognostics, specifically predicting the Remaining Useful Life (RUL) of assets - a ...

Recent progress in computer vision has been driven by a simple yet transformative observation: the potential of 30-year-old neural networks can be unlocked by massively scaling their parameters and the amount of human-labeled training data. This breakthrou ...

The application of kernel-based Machine Learning (ML) techniques to discrete choice modelling using large datasets often faces challenges due to memory requirements and the considerable number of parameters involved in these models. This complexity hampers ...

Semi-supervised semantic segmentation methods leverage unlabeled data by pseudo-labeling them. Thus the success of these methods hinges on the reliability of the pseudo-labels. Existing methods mostly choose high-confidence pixels in an effort to avoid err ...

We study the generalization of iterative noisy gradient schemes on smooth non-convex losses. Formally, we establish time-independent information theoretic generalization bounds for Stochastic Gradient Langevin Dynamics (SGLD) that do not diverge as the ite ...

This thesis focuses on understanding and improving the reasoning capabilities of neural networks. It develops theoretical results and empirical analyses to uncover reasoning potential and limitations, leveraging these insights to guide the design of improv ...

Deriving health indicators of rotating machines is crucial for their maintenance. However, this process is challenging for the prevalent adopted intelligent methods since they may take the whole data distributions, not only introducing noise interference b ...

Motivation Many machine learning (ML) models developed to classify phenotype from gene expression data provide interpretations for their decisions, with the aim of understanding biological processes. For many models, including neural networks, interpretati ...

We present a finite elements-neural network approach for the numerical approximation of parametric partial differential equations. The algorithm generates training data from finite element simulations, and uses a data -driven (supervised) feedforward neura ...

Decentralized learning (DL) is an emerging approach that enables nodes to collaboratively train a machine learning model without sharing raw data. In many application domains, such as healthcare, this approach faces challenges due to the high level of hete ...

Throughout history, the pace of knowledge and information sharing has evolved into an unthinkable speed and media. At the end of the XVII century, in Europe, the ideas that would shape the "Age of Enlightenment" were slowly being developed in coffeehouses, ...

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

Federated learning (FL) is a machine learning approach where nodes collaboratively train a global model. As more nodes participate in a round of FL, the effectiveness of individual model updates by nodes also diminishes. In this study, we increase the effe ...

Uncertainty of spillover effects – including property devaluation - from proposed land-use change elicits opposition to local development. This hinders cities’ ability to implement land-use policy aimed at housing affordability and environmental sustainabi ...

Machine learning and artificial intelligence methods have achieved remarkable success, matching and even surpassing human capabilities in various complex tasks. However, many demonstrations have generally neglected a critical part of the intelligence that ...

Data augmentation is vital in deep learning for enhancing model robustness by artificially expanding training datasets. However, advanced methods like CutMix blend images and assign labels based on pixel ratios, often introducing label noise by neglecting ...

The evolution of airborne mapping witnesses the introduction of hybrid lidar-camera systems to enhance data collection, i.e. to obtain simultaneously high-density point-cloud and texture. Yet, the common adjustment of both optical data streams is a nontriv ...

Poisoning attacks compromise the training data utilized to train machine learning (ML) models, diminishing their overall performance, manipulating predictions on specific test samples, and implanting backdoors. This article thoughtfully explores these atta ...

When a plasma disrupts in a tokamak, significant heat and electromagnetic loads are deposited onto the surrounding device components. These forces scale with plasma current and magnetic field strength, making disruptions one of the key challenges for futur ...

We develop a model-free approach to optimally control stochastic, Markovian systems subject to a reach-avoid constraint. Specifically, the state trajectory must remain within a safe set while reaching a target set within a finite time horizon. Due to the t ...

On the Generalization of Stochastic Gradient Descent with Momentum

Chattez avec Graph Search

Overcoming Distribution Shifts and Imbalance Challenges in Representation Learning for Deep Regression Models

Exploiting Representation Similarities in Self-Supervised Learning for Vision Tasks

Scalable kernel logistic regression with Nyström approximation: Theoretical analysis and application to discrete choice modelling

Weighting Pseudo-labels via High-Activation Feature Index Similarity and Object Detection for Semi-supervised Segmentation

Generalization of Noisy SGD in Unbounded Non-convex Settings

Learning to Reason with Neural Networks: Principles and Methods

Classifier-free diffusion-based weakly-supervised approach for health indicator derivation in rotating machines: Advancing early fault detection and condition monitoring

A comparative analysis of gene expression profiling by statistical and machine learning approaches

Error assessment of an adaptive finite elements-neural networks method for an elliptic parametric PDE

Fair Decentralized Learning

Machine Learning for Modeling Stock Returns

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Harnessing Increased Client Participation with Cohort-Parallel Federated Learning

Using automated design appraisal to model building-specific devaluation risk due to land-use change

Sequential Reasoning with Socially Caused Beliefs

SaliencyMix plus : Noise-Minimized Image Mixing Method With Saliency Map in Data Augmentation

On the Perspectives of Image-to-Lidar Constraints in Dynamic Network Optimisation

Machine Learning Security Against Data Poisoning: Are We There Yet?

Plasma state monitoring and disruption characterization using multimodal VAEs

A learning-based approach to stochastic optimal control under reach-avoid constraint