Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithms

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

The Birman-Williams theorem gives a connection between the collection of unstable periodic orbits (UPOs) contained within a chaotic attractor and the topology of that attractor, for three-dimensional systems. In certain cases, the fractal dimension of a ch ...

This study integrates deep learning and geospatial analysis to enhance soil loss estimation in the Moulouya Watershed, a region prone to erosion due to diverse topography and climatic conditions. Traditional models like the Universal Soil Loss Equation (US ...

Decision-making permeates every aspect of human and societal development, from individuals' daily choices to the complex decisions made by communities and institutions. Central to effective decision-making is the discipline of optimization, which seeks th ...

Context. Imaging in radio interferometry requires solving an ill-posed noisy inverse problem, for which the most adopted algorithm is the original CLEAN algorithm and its variants. Alternative explicit optimization methods have gained increasing attention, ...

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

Many challenges of modern graph processing stem from the fact that graphs have become massive and change frequently. Numerous techniques have been developed to address those issues, with graph cut sparsification offering to replace original graphs with th ...

Machine learning (ML) enables artificial intelligent (AI) agents to learn autonomously from data obtained from their environment to perform tasks. Modern ML systems have proven to be extremely effective, reaching or even exceeding human intelligence. Altho ...

To retrieve Surface Solar Radiation (SSR) from satellite images, a baseline reflectance of the observed ground in clear-sky conditions, also called background reflectance, is necessary to distinguish atmospheric absorption and scattering effects from surfa ...

This paper studies kernel ridge regression in high dimensions under covariate shifts and analyzes the role of importance re-weighting. We first derive the asymptotic expansion of high dimensional kernels under covariate shifts. By a bias-variance decomposi ...

Estimating ecosystem-atmosphere fluxes such as evapotranspiration (ET) in a robust manner and at a global scale remains a challenge. Methods based on machine learning (ML) have shown promising results in achieving such upscaling, providing a complementary ...

The remarkable ability of deep learning (DL) models to approximate high-dimensional functions from samples has sparked a revolution across numerous scientific and industrial domains that cannot be overemphasized. In sensitive applications, the good perform ...

Jointly estimating hand and object shape facilitates the grasping task in human-to-robot handovers. Relying on handcrafted prior knowledge about the geometric structure of the object fails when generalising to unseen objects, and depth sensors fail to dete ...

Throughout history, the pace of knowledge and information sharing has evolved into an unthinkable speed and media. At the end of the XVII century, in Europe, the ideas that would shape the "Age of Enlightenment" were slowly being developed in coffeehouses, ...

Whole-brain proton magnetic resonance spectroscopic imaging (H-1-MRSI) is a non-invasive technique for assessing neurochemical distribution in the brain, offering valuable insights into brain functions and neural diseases. It greatly benefits from the impr ...

Adversarial attacks in Natural Language Processing apply perturbations in the character or token levels. Token-level attacks, gaining prominence for their use of gradient-based methods, are susceptible to altering sentence semantics, leading to invalid adv ...

Slope deformation poses significant risks to infrastructure, ecosystems, and human safety, making early and accurate predictions essential for mitigating slope failures and landslides. In this study, we propose a novel approach that integrates a fractional ...

In inverse problems, the task is to reconstruct an unknown signal from its possibly noise-corrupted measurements. Penalized-likelihood-based estimation and Bayesian estimation are two powerful statistical paradigms for the resolution of such problems. They ...

Scientific Machine Learning (SciML) is a recently emerged research field which combines physics-based and data-driven models for the numerical approximation of differential problems. Physics-based models rely on the physical understanding of the problem at ...

We propose a local, non -intrusive model order reduction technique to accurately approximate the solution of coupled multi -component parametrized systems governed by partial differential equations. Our approach is based on the approximation of the boundar ...

Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithms

Chat with Graph Search

Deep Learning Theory Through the Lens of Diagonal Linear Networks

The topology of a chaotic attractor in the Kuramoto-Sivashinsky equation

Application of Deep Learning and Geospatial Analysis in Soil Loss Risk in the Moulouya Watershed, Morocco

Reliable data-driven decision-making through optimal transport

PolyCLEAN: Atomic optimization for super-resolution imaging and uncertainty estimation in radio interferometry

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Graph sparsification, sketching and adaptivity

Few-shot Learning for Efficient and Effective Machine Learning Model Adaptation

Retrieval of Surface Solar Radiation through Implicit Albedo Recovery from Temporal Context

High-Dimensional Kernel Methods under Covariate Shift: Data-Dependent Implicit Regularization

On the added value of sequential deep learning for the upscaling of evapotranspiration

Towards Trustworthy Deep Learning for Image Reconstruction

Stereo Hand-Object Reconstruction for Human-to-Robot Handover

Machine Learning for Modeling Stock Returns

Rosette Spectroscopic Imaging for Whole-Brain Slab Metabolite Mapping at 7T: Acceleration Potential and Reproducibility

Revisiting Character-level Adversarial Attacks for Language Models

Slope Deformation Prediction Combining Particle Swarm Optimization-Based Fractional-Order Grey Model and K-Means Clustering

Statistical Inference for Inverse Problems: From Sparsity-Based Methods to Neural Networks

Combining physics-based and data-driven models: advancing the frontiers of research with scientific machine learning

Model reduction of coupled systems based on non-intrusive approximations of the boundary response maps