Publications related to Comparison of Gradient Estimation Methods for Real-time Optimization

Related publications (32)

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Distributed learning is the key for enabling training of modern large-scale machine learning models, through parallelising the learning process. Collaborative learning is essential for learning from privacy-sensitive data that is distributed across various ...

EPFL2024

Mesh-based topology, shape and sizing optimization of ribbed plates

Ahmad Abdulrahman Ahmad Majdouba

In this paper, we present a new parameterization and optimization procedure for minimizing the weight of ribbed plates. The primary goal is to reduce embodied CO2 in concrete floors as part of the effort to diminish the carbon footprint of the construction ...

Springer2024

Scalable constrained optimization

Maria-Luiza Vladarean

Modern optimization is tasked with handling applications of increasingly large scale, chiefly due to the massive amounts of widely available data and the ever-growing reach of Machine Learning. Consequently, this area of research is under steady pressure t ...

EPFL2024

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Nicolas Henri Bernard Flammarion, Scott William Pesme, Hristo Georgiev Papazov

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

2024

BENIGN LANDSCAPES OF LOW-DIMENSIONAL RELAXATIONS FOR ORTHOGONAL SYNCHRONIZATION ON GENERAL GRAPHS

Nicolas Boumal

Orthogonal group synchronization is the problem of estimating n elements Z(1),& mldr;,Z(n) from the rxr orthogonal group given some relative measurements R-ij approximate to Z(i)Z(j)(-1). The least-squares formulation is nonconvex. To avoid its local minim ...

Siam Publications2024

The effect of smooth parametrizations on nonconvex optimization landscapes

Nicolas Boumal

We develop new tools to study landscapes in nonconvex optimization. Given one optimization problem, we pair it with another by smoothly parametrizing the domain. This is either for practical purposes (e.g., to use smooth optimization algorithms with good g ...

Springer Heidelberg2024

Understanding generalization and robustness in modern deep learning

Maksym Andriushchenko

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

EPFL2024

A method and system for optimizing design, implantation, and stimulation protocols of electrodes stimulating the nervous system

Silvestro Micera, Simone Romeni

A method for optimizing at least one of a geometry, an implantation procedure, and/or stimulation protocol of one or more electrodes for an electrical stimulation of a target structure in a nervous system of a living being by a computer device, the method ...

2024

On the Generalization of Stochastic Gradient Descent with Momentum

Volkan Cevher, Kimon Antonakopoulos

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...

Microtome Publishing2024

Contemporary Logic Synthesis: with an Application to AQFP Circuit Optimization

Siang-Yun Lee

Electronic devices play an irreplaceable role in our lives. With the tightening time to market, exploding demand for computing power, and continuous desire for smaller, faster, less energy-consuming, and lower-cost chips, computer-aided design for electron ...

EPFL2024

Residual-based attention in physics-informed neural networks

Nikolaos Stergiopoulos, Sokratis Anagnostopoulos

Driven by the need for more efficient and seamless integration of physical models and data, physics -informed neural networks (PINNs) have seen a surge of interest in recent years. However, ensuring the reliability of their convergence and accuracy remains ...

Elsevier Science Sa2024

A ride time-oriented scheduling algorithm for dial-a-ride problems

Nikolaos Geroliminis, Claudia Bongiovanni, Mor Kaspi

This paper offers a new algorithm to efficiently optimize scheduling decisions for dial-a-ride problems (DARPs), including problem variants considering electric and autonomous vehicles (e-ADARPs). The scheduling heuristic, based on linear programming theor ...

Pergamon-Elsevier Science Ltd2024

Universal and adaptive methods for robust stochastic optimization

Ali Kavis

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

EPFL2023

Training Efficient Controllers via Analytic Policy Gradient

Dario Floreano, Davide Scaramuzza, Valentin Wüest

Control design for robotic systems is complex and often requires solving an optimization to follow a trajectory accurately. Online optimization approaches like Model Predictive Control (MPC) have been shown to achieve great tracking performance, but requir ...

New York2023

Stochastic adaptive resampling for the estimation of discrete choice models

Michel Bierlaire, Matthieu Marie Cochon de Lapparent, Nicola Marco Ortelli

In the field of choice modeling, the availability of ever-larger datasets has the potential to significantly expand our understanding of human behavior, but this prospect is limited by the poor scalability of discrete choice models (DCMs): as sample sizes ...

2023

Robust optimization of control parameters for WEC arrays using stochastic methods

Tommaso Vanzan

This work presents a new computational optimization framework for the robust control of parks of Wave Energy Converters (WEC) in irregular waves. The power of WEC parks is maximized with respect to the individual control damping and stiffness coefficients ...

Journal of Computational Physics2023

A comprehensive review of digital twin-part 2: roles of uncertainty quantification and optimization, a battery digital twin, and perspectives

Olga Fink

As an emerging technology in the era of Industry 4.0, digital twin is gaining unprecedented attention because of its promise to further optimize process design, quality control, health monitoring, decision- and policy-making, and more, by comprehensively m ...

SPRINGER2023

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Mehmet Fatih Sahin

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

EPFL2023

Probabilistic methods for neural combinatorial optimization

Nikolaos Karalias

The monumental progress in the development of machine learning models has led to a plethora of applications with transformative effects in engineering and science. This has also turned the attention of the research community towards the pursuit of construc ...

EPFL2023

Quality-Constrained Encoding Optimization for Omnidirectional Video Streaming

Pascal Frossard, Roberto Gerson De Albuquerque Azevedo

Omnidirectional video streaming is usually implemented based on the representations of tiles, where the tiles are obtained by splitting the video frame into several rectangular areas and each tile is converted into multiple representations with different r ...

Ieee-Inst Electrical Electronics Engineers Inc2023