Publications related to Smooth Primal-Dual Coordinate Descent Algorithms for Nonsmooth Convex Optimization

Related publications (36)

How Gradient Descent Balances Features: A Dynamical Analysis For Two-Layer Neural Networks

This paper investigates the fundamental regression task of learning k neurons (a.k.a. teachers) from Gaussian input, using two-layer ReLU neural networks with width m (a.k.a. students) and m, k = O(1), trained via gradient descent under proper initializati ...

2025

Foundations of a New Learning Paradigm in AI Grounded in the Principles of Evolutionary Developmental Biology

Zeki Doruk Erden

Contemporary AI faces important limitations that remain unresolved despite significant successes. Chief among these are the inability to acquire new knowledge without destroying old, the incomprehensible and nonengineerable nature of internal representatio ...

EPFL2025

Data Downloaded via Parachute from a NASA Super-Pressure Balloon

David Richard Harvey

In April 2023, the superBIT telescope was lifted to the Earth's stratosphere by a helium-filled super-pressure balloon to acquire astronomical imaging from above (99.5% of) the Earth's atmosphere. It was launched from New Zealand and then, for 40 days, cir ...

MDPI2023

Universal and adaptive methods for robust stochastic optimization

Ali Kavis

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

EPFL2023

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Mehmet Fatih Sahin

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

EPFL2023

A Study on Gradient-based Meta-learning for Robust Deep Digital Twins

Olga Fink, Raffael Pascal Theiler, Michele Viscione

Deep-learning-based digital twins (DDT) are a promising tool for data-driven system health management because they can be trained directly on operational data. A major challenge for efficient training however is that industrial datasets remain unlabeled. T ...

Research Publishing2023

Safe Zeroth-Order Convex Optimization Using Quadratic Local Approximations

Giancarlo Ferrari Trecate, Yuning Jiang, Maryam Kamgarpour, Baiwei Guo

We address black-box convex optimization problems, where the objective and constraint functions are not explicitly known but can be sampled within the feasible set. The challenge is thus to generate a sequence of feasible points converging towards an optim ...

IEEE2023

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Negar Kiyavash, Saber Salehkaleybar

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

Iterative pre-conditioning for expediting the distributed gradient-descent method: The case of linear least-squares problem

Nirupam Gupta

This paper considers the multi-agent linear least-squares problem in a server-agent network architecture. The system comprises multiple agents, each with a set of local data points. The agents are connected to a server, and there is no inter-agent communic ...

PERGAMON-ELSEVIER SCIENCE LTD2022

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

Volkan Cevher, Alp Yurtsever, Maria-Luiza Vladarean

We propose a stochastic conditional gradient method (CGM) for minimizing convex finitesum objectives formed as a sum of smooth and non-smooth terms. Existing CGM variants for this template either suffer from slow convergence rates, or require carefully inc ...

2022

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Scott William Pesme

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Nicolas Henri Bernard Flammarion, Scott William Pesme, Hristo Georgiev Papazov

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

2024

On the Generalization of Stochastic Gradient Descent with Momentum

Volkan Cevher, Kimon Antonakopoulos

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...

Microtome Publishing2024

Stop Wasting my FLOPS: Improving the Efficiency of Deep Learning Models

Angelos Katharopoulos

Deep neural networks have completely revolutionized the field of machine learning by achieving state-of-the-art results on various tasks ranging from computer vision to protein folding. However, their application is hindered by their large computational an ...

EPFL2022

Scalable constrained optimization

Maria-Luiza Vladarean

Modern optimization is tasked with handling applications of increasingly large scale, chiefly due to the massive amounts of widely available data and the ever-growing reach of Machine Learning. Consequently, this area of research is under steady pressure t ...

EPFL2024

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Volkan Cevher, Ali Kavis, Kimon Antonakopoulos, Leello Tadesse Dadi, Efstratios Panteleimon Skoulakis

We propose an adaptive variance-reduction method, called AdaSpider, for minimization of L-smooth, non-convex functions with a finite-sum structure. In essence, AdaSpider combines an AdaGrad-inspired [Duchi et al., 2011, McMahan & Streeter, 2010], but a fai ...

2022

Seeking the new, learning from the unexpected: Computational models of surprise and novelty in the brain

Alireza Modirshanechi

Human babies have a natural desire to interact with new toys and objects, through which they learn how the world around them works, e.g., that glass shatters when dropped, but a rubber ball does not. When their predictions are proven incorrect, such as whe ...

EPFL2024

Level-set-based Shape Optimization on Soft Magnetic Composites with Isotropy Constraint

Yves Perriard, Adrien Jean-Michel Thabuis, Xiaotao Ren

Soft Magnetic Composites (SMCs) demonstrate excellent potentials for electromagnetic applications. The isotropy is one critical feature in designing the composite. This work presents a shape optimization algorithm for SMCs subject to isotropy and volume co ...

Institute of Electrical and Electronics Engineers2022

Inverse Design of Singlet-Fission Materials with Uncertainty-Controlled Genetic Optimization

Rubén Laplaza Solanas, Jacob Terence Blaskovits

Singlet fission has shown potential for boosting the efficiency of solar cells, but the scarcity of suitable molecular materials hinders its implementation. We introduce an uncertainty-controlled genetic algorithm (ucGA) based on ensemble machine learning ...

Wiley2024

On The Convergence Of Stochastic Primal-Dual Hybrid Gradient

Volkan Cevher, Ahmet Alacaoglu

In this paper, we analyze the recently proposed stochastic primal-dual hybrid gradient (SPDHG) algorithm and provide new theoretical results. In particular, we prove almost sure convergence of the iterates to a solution with convexity and linear convergenc ...

SIAM PUBLICATIONS2022