Adaptive Gradient Descent without Descent

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Modern optimization is tasked with handling applications of increasingly large scale, chiefly due to the massive amounts of widely available data and the ever-growing reach of Machine Learning. Consequently, this area of research is under steady pressure t ...

In light of the challenges posed by climate change and the goals of the Paris Agreement, electricity generation is shifting to a more renewable and decentralized pattern, while the operation of systems like buildings is increasingly electrified. This calls ...

Distributed learning is the key for enabling training of modern large-scale machine learning models, through parallelising the learning process. Collaborative learning is essential for learning from privacy-sensitive data that is distributed across various ...

Driven by the need for more efficient and seamless integration of physical models and data, physics -informed neural networks (PINNs) have seen a surge of interest in recent years. However, ensuring the reliability of their convergence and accuracy remains ...

Hyperbolic lattices are a new type of synthetic materials based on regular tessellations in non-Euclidean spaces with constant negative curvature. While so far, there has been several theoretical investigations of hyperbolic topological media, experimental ...

Herein, machine learning (ML) models using multiple linear regression (MLR), support vector regression (SVR), random forest (RF) and artificial neural network (ANN) are developed and compared to predict the output features viz. specific capacitance (Csp), ...

Given a sequence of functions

f_1,\ldots,f_n

with

f_i:\mathcal{D}\mapsto \mathbb{R}

, finite-sum minimization seeks a point

{x}^\star \in \mathcal{D}

minimizing

\sum_{j=1}^nf_j(x)/n

. In this work, we propose a key twist into the finite-sum minimizat ...

Molecular quantum dynamics simulations are essential for understanding many fundamental phenomena in physics and chemistry. They often require solving the time-dependent Schrödinger equation for molecular nuclei, which is challenging even for medium-sized ...

. We study very weak solutions to scalar Euler-Lagrange equations associated with quadratic convex functionals. We investigate whether W1,1 solutions are necessarily W 1,2 Nash and Schauder applicable. We answer this question positively for a suitable clas ...

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...

Simplicial Kuramoto models have emerged as a diverse and intriguing class of models describing oscillators on simplices rather than nodes. In this paper, we present a unified framework to describe different variants of these models, categorized into three ...

This paper develops a fast algorithm for computing the equilibrium assignment with the perturbed utility route choice (PURC) model. Without compromise, this allows the significant advantages of the PURC model to be used in large-scale applications. We form ...

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

A method for optimizing at least one of a geometry, an implantation procedure, and/or stimulation protocol of one or more electrodes for an electrical stimulation of a target structure in a nervous system of a living being by a computer device, the method ...

We present a combination technique based on mixed differences of both spatial approximations and quadrature formulae for the stochastic variables to solve efficiently a class of optimal control problems (OCPs) constrained by random partial differential equ ...

Orthogonal group synchronization is the problem of estimating n elements Z(1),& mldr;,Z(n) from the rxr orthogonal group given some relative measurements R-ij approximate to Z(i)Z(j)(-1). The least-squares formulation is nonconvex. To avoid its local minim ...

We introduce robust principal component analysis from a data matrix in which the entries of its columns have been corrupted by permutations, termed Unlabeled Principal Component Analysis (UPCA). Using algebraic geometry, we establish that UPCA is a well-de ...

Catastrophic overfitting (CO) in single-step adversarial training (AT) results in abrupt drops in the adversarial test accuracy (even down to 0%). For models trained with multi-step AT, it has been observed that the loss function behaves locally linearly w ...

The integration of the new plasma position reflectometer in the RFX-mod2 experiment (the upgraded version of the previous RFX-mod that operated until 2015) is presented in this contribution. Particular attention has been devoted to the high field side subs ...

Adaptive Gradient Descent without Descent

Chat with Graph Search

Scalable constrained optimization

Quantifying the Unknown: Data-Driven Approaches and Applications in Energy Systems

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Residual-based attention in physics-informed neural networks

Anomalous and Chern topological waves in hyperbolic networks

Machine learning models for prediction of electrochemical properties in supercapacitor electrodes using MXene and graphene nanoplatelets

Efficient Continual Finite-Sum Minimization

High-order geometric integrators for the variational Gaussian wavepacket dynamics and application to vibronic spectra at finite temperature

WILD SOLUTIONS TO SCALAR EULER-LAGRANGE EQUATIONS

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

On the Generalization of Stochastic Gradient Descent with Momentum

A unified framework for Simplicial Kuramoto models

Perturbed Utility Stochastic Traffic Assignment

Understanding generalization and robustness in modern deep learning

A method and system for optimizing design, implantation, and stimulation protocols of electrodes stimulating the nervous system

A Combination Technique for Optimal Control Problems Constrained by Random PDEs

BENIGN LANDSCAPES OF LOW-DIMENSIONAL RELAXATIONS FOR ORTHOGONAL SYNCHRONIZATION ON GENERAL GRAPHS

Unlabeled Principal Component Analysis and Matrix Completion

Efficient local linearity regularization to overcome catastrophic overfitting

Technology challenges and integration of the plasma position reflectometer in RFX-mod2