Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Bayesian Optimization (BO) is typically used to optimize an unknown function f that is noisy and costly to evaluate, by exploiting an acquisition function that must be maximized at each optimization step. Even if provably asymptotically optimal BO algorith ...

Distributed learning is the key for enabling training of modern large-scale machine learning models, through parallelising the learning process. Collaborative learning is essential for learning from privacy-sensitive data that is distributed across various ...

Modern optimization is tasked with handling applications of increasingly large scale, chiefly due to the massive amounts of widely available data and the ever-growing reach of Machine Learning. Consequently, this area of research is under steady pressure t ...

In the past few years, Machine Learning (ML) techniques have ushered in a paradigm shift, allowing the harnessing of ever more abundant sources of data to automate complex tasks. The technical workhorse behind these important breakthroughs arguably lies in ...

Orthogonal group synchronization is the problem of estimating n elements Z(1),& mldr;,Z(n) from the rxr orthogonal group given some relative measurements R-ij approximate to Z(i)Z(j)(-1). The least-squares formulation is nonconvex. To avoid its local minim ...

In this paper, we present a new parameterization and optimization procedure for minimizing the weight of ribbed plates. The primary goal is to reduce embodied CO2 in concrete floors as part of the effort to diminish the carbon footprint of the construction ...

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

We develop new tools to study landscapes in nonconvex optimization. Given one optimization problem, we pair it with another by smoothly parametrizing the domain. This is either for practical purposes (e.g., to use smooth optimization algorithms with good g ...

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

In the field of choice modeling, the availability of ever-larger datasets has the potential to significantly expand our understanding of human behavior, but this prospect is limited by the poor scalability of discrete choice models (DCMs): as sample sizes ...

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

According to the proposed Artificial Intelligence Act by the European Comission (expected to pass at the end of 2023), the class of High-Risk AI Systems (Title III) comprises several important applications of Deep Learning like autonomous driving vehicles ...

Recently there has been a surge of interest in understanding implicit regularization properties of iterative gradient-based optimization algorithms. In this paper, we study the statistical guarantees on the excess risk achieved by early-stopped unconstrain ...

A linear dynamical model for the development of the turbulent energy cascade was introduced in Apolina ' rio et al. (J. Stat. Phys., 186, 15 (2022)). This partial differential equation, randomly stirred by a forcing term which is smooth in space and delta- ...

Programming intelligent robots requires robust controllers that can achieve desired tasks while adapting to the changes in the task and the environment. In this thesis, we address the challenges in designing such adaptive and anticipatory feedback controll ...

We develop a principled approach to end-to-end learning in stochastic optimization. First, we show that the standard end-to-end learning algorithm admits a Bayesian interpretation and trains a posterior Bayes action map. Building on the insights of this an ...

The decentralisation and unpredictability of new renewable energy sources require rethinking our energy system. Data-driven approaches, such as reinforcement learning (RL), have emerged as new control strategies for operating these systems, but they have n ...

In this thesis, we address the complex issue of collision avoidance in the joint space of robots. Avoiding collisions with both the robot's own body parts and obstacles in the environment is a critical constraint in motion planning and is crucial for ensur ...

Control design for robotic systems is complex and often requires solving an optimization to follow a trajectory accurately. Online optimization approaches like Model Predictive Control (MPC) have been shown to achieve great tracking performance, but requir ...

The load tracking performance of combined cooling, heating, and power multi-energy system (CCHP-MES) greatly depends on the equipment capacity configuration. And the frequent fluctuations in the source-load uncertainty puts higher demands on the load track ...

Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval

Chat with Graph Search

Relaxing the Additivity Constraints in Decentralized No-Regret High-Dimensional Bayesian Optimization

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Scalable constrained optimization

Topics in statistical physics of high-dimensional machine learning

BENIGN LANDSCAPES OF LOW-DIMENSIONAL RELAXATIONS FOR ORTHOGONAL SYNCHRONIZATION ON GENERAL GRAPHS

Mesh-based topology, shape and sizing optimization of ribbed plates

Deep Learning Theory Through the Lens of Diagonal Linear Networks

The effect of smooth parametrizations on nonconvex optimization landscapes

Universal and adaptive methods for robust stochastic optimization

Stochastic adaptive resampling for the estimation of discrete choice models

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Robust Training and Verification of Deep Neural Networks

The statistical complexity of early-stopped mirror descent

Space-time statistics of a linear dynamical energy cascade model dagger

Learning and optimization of anticipatory feedback controllers for robot manipulation

End-to-End Learning for Stochastic Optimization: A Bayesian Perspective

Reinforcement Learning for Joint Design and Control of Battery-PV Systems

Implicit Distance Functions: Learning and Applications in Robotics

Training Efficient Controllers via Analytic Policy Gradient

Stochastic-robust planning optimization method based on tracking-economy extreme scenario tradeoff for CCHP multi-energy system