Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

We address black-box convex optimization problems, where the objective and constraint functions are not explicitly known but can be sampled within the feasible set. The challenge is thus to generate a sequence of feasible points converging towards an optim ...

In Process Systems Engineering, computationally-demanding models are frequent and plentiful. Handling such complexity in an optimization framework in a fast and reliable way is essential, not only for generating meaningful solutions but also for providing ...

Variance-reduced gradient estimators for policy gradient methods have been one of the main focus of research in the reinforcement learning in recent years as they allow acceleration of the estimation process. We propose a variance-reduced policy-gradient m ...

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins. All these successful methods depend on a gradient-based learning algorithm to train a model on massive ...

Integrating functions on discrete domains into neural networks is key to developing their capability to reason about discrete objects. But, discrete domains are (I) not naturally amenable to gradient-based optimization, and (II) incompatible with deep lear ...

This work presents a new computational optimization framework for the robust control of parks of Wave Energy Converters (WEC) in irregular waves. The power of WEC parks is maximized with respect to the individual control damping and stiffness coefficients ...

The progress towards intelligent systems and digitalization relies heavily on the use of automation technology. However, the growing diversity of control objects presents significant challenges for traditional control approaches, as they are highly depende ...

Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achieve global convergence under gradient descent. The picture can be radically different for narrow net-works, which tend to get stuck in badly-generalizing loca ...

Soft Magnetic Composites (SMCs) demonstrate excellent potentials for electromagnetic applications. The isotropy is one critical feature in designing the composite. This work presents a shape optimization algorithm for SMCs subject to isotropy and volume co ...

This work aims to study the effects of wind uncertainties in civil engineering structural design. Optimising the design of a structure for safety or operability without factoring in these uncertainties can result in a design that is not robust to these per ...

Many robotics problems are formulated as optimization problems. However, most optimization solvers in robotics are locally optimal and the performance depends a lot on the initial guess. For challenging problems, the solver will often get stuck at poor loc ...

This paper considers the multi-agent linear least-squares problem in a server-agent network architecture. The system comprises multiple agents, each with a set of local data points. The agents are connected to a server, and there is no inter-agent communic ...

In this thesis, we propose model order reduction techniques for high-dimensional PDEs that preserve structures of the original problems and develop a closure modeling framework leveraging the Mori-Zwanzig formalism and recurrent neural networks. Since high ...

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval

Chat with Graph Search

Safe Zeroth-Order Convex Optimization Using Quadratic Local Approximations

A combined genetic algorithm and active learning approach to build and test surrogate models in Process Systems Engineering

Momentum-Based Policy Gradient with Second-Order Information

Transformer Models for Vision

Neural Set Function Extensions: Learning with Discrete Functions in High Dimensions

Robust optimization of control parameters for WEC arrays using stochastic methods

Data-driven Methods for Control: from Linear to Lifting

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Level-set-based Shape Optimization on Soft Magnetic Composites with Isotropy Constraint

Multi-Level Monte Carlo Methods for Uncertainty Quantification and Risk-Averse Optimisation

Memory of Motion for Initializing Optimization in Robotics

Iterative pre-conditioning for expediting the distributed gradient-descent method: The case of linear least-squares problem

Structure-preserving approaches and data-driven closure modeling for model order reduction

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function