Publications de Kimon Antonakopoulos

Publications associées (12)

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Volkan Cevher, Ali Kavis, Kimon Antonakopoulos, Leello Tadesse Dadi, Efstratios Panteleimon Skoulakis

We propose an adaptive variance-reduction method, called AdaSpider, for minimization of L-smooth, non-convex functions with a finite-sum structure. In essence, AdaSpider combines an AdaGrad-inspired [Duchi et al., 2011, McMahan & Streeter, 2010], but a fai ...

2022

Sifting through the Noise: Universal First-Order Methods for Stochastic Variational Inequalities

Volkan Cevher, Ali Kavis, Kimon Antonakopoulos, Thomas Michaelsen Pethick

We examine a flexible algorithmic framework for solving monotone variational inequalities in the presence of randomness and uncertainty. The proposed template encompasses a wide range of popular first-order methods, including dual averaging, dual extrapola ...

2021

ADAGRAD Avoids Saddle Points

Kimon Antonakopoulos

Adaptive first-order methods in optimization are prominent in machine learning and data science owing to their ability to automatically adapt to the landscape of the function being optimized. However, their convergence guarantees are typically stated in te ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

On the Generalization of Stochastic Gradient Descent with Momentum

Volkan Cevher, Kimon Antonakopoulos

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...

Microtome Publishing2024

Improving SAM Requires Rethinking its Optimization Formulation

Volkan Cevher, Kimon Antonakopoulos, Fabian Ricardo Latorre Gomez, Thomas Michaelsen Pethick, Wanyun Xie

This paper rethinks Sharpness-Aware Minimization (SAM), which is originally formulated as a zero-sum game where the weights of a network and a bounded perturbation try to minimize/maximize, respectively, the same differentiable loss. We argue that SAM shou ...

2024

Distributed Extra-Gradient With Optimal Complexity And Communication Guarantees

Volkan Cevher, Kimon Antonakopoulos, Igor Krawczuk

We consider monotone variational inequality (VI) problems in multi-GPU settings where multiple processors/workers/clients have access to local stochastic dual vectors. This setting includes a broad range of important problems from distributed convex minimi ...

2023

Layer-wise Quantization for Quantized Optimistic Dual Averaging

Volkan Cevher, Dan Alistarh, Kimon Antonakopoulos

Modern deep neural networks exhibit heterogeneity across numerous layers of various types such as residuals, multi-head attention, etc., due to varying structures (dimensions, activation functions, etc.), distinct representation characteristics, which impa ...

2025

UNDERGRAD: A Universal Black-Box Optimization Method with Almost Dimension-Free Convergence Rate Guarantees

Volkan Cevher, Kimon Antonakopoulos

Universal methods for optimization are designed to achieve theoretically optimal convergence rates without any prior knowledge of the problem’s regularity parameters or the accurarcy of the gradient oracle employed by the optimizer. In this regard, existin ...

2022

Adaptive Bilevel Optimization

Volkan Cevher, Kimon Antonakopoulos, Luca Viano

We propose a new adaptive optimization algorithm based on mirror descent for a class of possibly non-convex smooth bilevel optimization problems. The bilevel optimization template is broadly applicable in machine learning as it features two coupled problem ...

Association for Computing Machinery (ACM)2025

Training Deep Learning Models with Norm-Constrained LMOs

Volkan Cevher, Kimon Antonakopoulos, Thomas Michaelsen Pethick, Zhenyu Zhu, Wanyun Xie

In this work, we study optimization methods that leverage the linear minimization oracle (lmo) over a norm-ball. We propose a new stochastic family of algorithms that uses the lmo to adapt to the geometry of the problem and, perhaps surprisingly, show that ...

2025

No-regret learning in games with noisy feedback: Faster rates and adaptivity via learning rate separation

Volkan Cevher, Ya-Ping Hsieh, Kimon Antonakopoulos

We examine the problem of regret minimization when the learner is involved in a continuous game with other optimizing agents: in this case, if all players follow a no-regret algorithm, it is possible to achieve significantly lower regret relative to fully ...

2022

Universal Gradient Methods for Stochastic Convex Optimization

Volkan Cevher, Ali Kavis, Kimon Antonakopoulos, Yongtao Wu

We develop universal gradient methods for Stochastic Convex Optimization (SCO). Our algorithms automatically adapt not only to the oracle's noise but also to the Hölder smoothness of the objective function without a priori knowledge of the particular setti ...

2024