Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

Genetic variation plays a crucial role in human disease susceptibility and molecular pathophysiology. This thesis explores genetic determinants of disease using advanced computational approaches, including statistical modeling, network analysis, and deep l ...

The growing deployment of low-cost, distributed sensor networks in environmental and biomedical domains has enabled continuous, large-scale health monitoring. However, these systems often face challenges related to degraded data quality caused by sensor dr ...

We consider the numerical approximation of an optimal control problem for an elliptic Partial Differential Equation (PDE) with random coefficients. Specifically, the control function is a deterministic, distributed forcing term that minimizes the expected ...

This thesis investigates methods to construct more capable models more efficiently, focusing on two aspects: improved architectures and optimization. We examine principled architectural modifications that reduce computational costs or introduce features fo ...

We introduce the "continuized" Nesterov acceleration, a close variant of Nesterov acceleration whose variables are indexed by a continuous time parameter. The two variables continuously mix following a linear ordinary differential equation and take gradien ...

Understanding the implicit bias of training algorithms is of crucial importance in order to explain the success of overparametrised neural networks. In this paper, we study the dynamics of stochastic gradient descent over diagonal linear networks through i ...

Modern machine learning architectures distinguish servers and workers. Typically, a d-dimensional model is hosted by a server and trained by n workers, using a distributed stochastic gradient descent (SGD) optimization scheme. At each SGD step, the goal is ...

Inverse reconstruction from images is a central problem in many scientific and engineering disciplines. Recent progress on differentiable rendering has led to methods that can efficiently differentiate the full process of image formation with respect to mi ...

In this dissertation, we propose gradient-based methods for characterizing model behaviour for the purposes of knowledge transfer and post-hoc model interpretation. Broadly, gradients capture the variation of some output feature of the model upon unit vari ...

This paper develops a methodology for regret minimization with stochastic first-order oracle feedback in online, constrained, non-smooth, non-convex problems. In this setting, the minimization of external regret is beyond reach for first-order methods, and ...

Driven by the need to solve increasingly complex optimization problems in signal processing and machine learning, there has been increasing interest in understanding the behavior of gradient-descent algorithms in non-convex environments. Most available wor ...

In the past few years, Machine Learning (ML) techniques have ushered in a paradigm shift, allowing the harnessing of ever more abundant sources of data to automate complex tasks. The technical workhorse behind these important breakthroughs arguably lies in ...

Here we provide the synthetic spindle datasets of our article "Task-driven neural network models predict neural dynamics of proprioception". It contains the synthetic generated training dataset of simulated muscle spindles during arm passive movements gene ...

This paper proposes a shape optimization algorithm based on the principles of Isogeometric Analysis (IGA) in which the parameterization of the geometry enters the problem formulation as an additional PDE-constraint. Inspired by the isoparametric principle ...

The environment where we live and recreate can have a significant effect on our well-being. More beautiful landscapes have considerable benefits to both health and quality of life. When we chose where to live or our next holiday destination, we do so accor ...

Goal: In patients with coronary artery disease, the prediction of future cardiac events such as myocardial infarction (MI) remains a major challenge. In this work, we propose a novel anatomy-informed multimodal deep learning framework to predict future MI ...

We propose a metric for evaluating the generalization ability of deep neural networks trained with mini-batch gradient descent. Our metric, called gradient disparity, is the l2 norm distance between the gradient vectors of two mini-batches drawn from the t ...

We present a chance-constrained model predictive control (MPC) framework under Gaussian mixture model (GMM) uncertainty. Specifically, we consider the uncertainty that arises from predicting future behaviors of moving obstacles, which may exhibit multiple ...

We analyze in a closed form the learning dynamics of stochastic gradient descent (SGD) for a single layer neural network classifying a high-dimensional Gaussian mixture where each cluster is assigned one of two labels. This problem provides a prototype of ...

Probing the architecture of neuronal circuits and the principles that underlie their functional organization remains an important challenge of modern neurosciences. This holds true, in particular, for the inference of neuronal connectivity from large-scale ...

Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem

Chattez avec Graph Search

Learning the language of the genome: AI- and data-driven decoding of genetic variants and disease mechanisms

Efficient Unsupervised Domain Adaptation Regression for Spatial-Temporal Sensor Fusion

Complexity analysis of stochastic gradient methods for PDE-constrained optimal control problems with uncertain parameters

Enhanced Architectures and Optimization Methods for Efficient Language Modeling

A Continuized View on Nesterov Acceleration

Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity

AKSEL: Fast Byzantine SGD

Large Steps in Inverse Rendering of Geometry

Gradient-based Methods for Deep Model Interpretability

Regret Minimization in Stochastic Non-Convex Learning via a Proximal-Gradient Approach

Distributed Learning in Non-Convex Environments-Part I: Agreement at a Linear Rate

Topics in statistical physics of high-dimensional machine learning

Task-driven neural network models predict neural dynamics of proprioception: Synthetic muscle spindle datasets

The role of PDE-based parameterization techniques in gradient-based IGA shape optimization applications

On the relation between landscape beauty and land cover: A case study in the U.K. at Sentinel-2 resolution with interpretable AI

Anatomy-Informed Multimodal Learning for Myocardial Infarction Prediction

Disparity Between Batches as a Signal for Early Stopping

Recursively Feasible Chance-Constrained Model Predictive Control Under Gaussian Mixture Model Uncertainty

Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification

Ensemble learning and ground-truth validation of synaptic connectivity inferred from spike trains