Newton's method in optimization

Applied sciences
Information engineering
Machine learning
Artificial neural networks

About
Privacy
Disclaimer

Graph Chatbot

Related publications (30)

Page 2 of 2

Pareto-like sequential sampling heuristic for global optimisation

Katrin Beyer, Mahmoud S. M. Shaqfa

In this paper, we propose a simple global optimisation algorithm inspired by Pareto’s principle. This algorithm samples most of its solutions within prominent search domains and is equipped with a self-adaptive mechanism to control the dynamic tightening o ...

2021

Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval

Lenka Zdeborová, Florent Gérard Krzakala

Despite the widespread use of gradient-based algorithms for optimizing high-dimensional non-convex functions, understanding their ability of finding good minima instead of being trapped in spurious ones remains to a large extent an open problem. Here we fo ...

Curran Associates, Inc.2020

Adaptive Gradient Descent without Descent

We present a strikingly simple proof that two rules are sufficient to automate gradient descent: 1) don’t increase the stepsize too fast and 2) don’t overstep the local curvature. No need for functional values, no line search, no information about the func ...

2020

Adaptation and Learning Over Networks Under Subspace Constraints-Part I: Stability Analysis

Ali H. Sayed, Stefan Vlaski, Roula Nassif

This paper considers optimization problems over networks where agents have individual objectives to meet, or individual parameter vectors to estimate, subject to subspace constraints that require the objectives across the network to lie in low-dimensional ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Compress-and-restart block Krylov subspace methods for Sylvester matrix equations

Daniel Kressner, Stefano Massei, Kathryn Dianne Lund

Block Krylov subspace methods (KSMs) comprise building blocks in many state-of-the-art solvers for large-scale matrix equations as they arise, for example, from the discretization of partial differential equations. While extended and rational block Krylov ...

2020

On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems

Volkan Cevher, Ali Kavis, Shaul Nadav Hallak

This paper analyzes the trajectories of stochastic gradient descent (SGD) to help understand the algorithm’s convergence properties in non-convex problems. We first show that the sequence of iterates generated by SGD remains bounded and converges with prob ...

2020

The Error-Feedback Framework: Better Rates for SGD with Delayed Gradients and Compressed Updates

Sebastian Urban Stich, Sai Praneeth Reddy Karimireddy

We analyze (stochastic) gradient descent (SGD) with delayed updates on smooth quasi-convex and non-convex functions and derive concise, non-asymptotic, convergence rates. We show that the rate of convergence in all cases consists of two terms: (i) a stocha ...

Microtome Publishing2020

On the choice of metric in gradient-based theories of brain function

Wulfram Gerstner, Johanni Michael Brea, Jean-Pascal Théodor Pfister

The idea that the brain functions so as to minimize certain costs pervades theoretical neuroscience. Because a cost function by itself does not predict how the brain finds its minima, additional assumptions about the optimization method need to be made to ...

2020

Allocation of Active Power Reserve from Active Distribution Networks Using a Cost-Benefit Approach: Application to Swissgrid Network

Sidi-Rachid Cherkaoui, Fabrizio Sossan, Mokhtar Bozorg, Mohsen Kalantar Neyestanaki

The flexibility of distributed energy resources (DERs) accommodated in active distribution networks (ADNs) can be aggregated and then used to provide ancillary services to the transmission system. In this context, this paper presents a linear optimization ...

IEEE2019

Inverse Problems and Optimal Power Flow in Active Distribution Networks

Cong Wang

During the last decade, distribution networks have experienced essential changes driven by the integration of renewable-energy sources, batteries, electric-vehicle charging stations, etc. This results in not only opportunities, but also operational problem ...

EPFL2019