Publications related to Universal Gradient Methods for Stochastic Convex Optimization

Related publications (25)

Boosting Resource-Constrained Federated Learning Systems with Guessed Updates

Mohamed Yassine Boukhari, Anne-Marie Kermarrec, Rafael Pereira Pires, Rishi Sharma, Akash Balasaheb Dhasade

Federated learning (FL) enables a set of client devices to collaboratively train a model without sharing raw data. This process, though, operates under the constrained computation and communication resources of edge devices. These constraints combined with ...

Institute of Electrical and Electronics Engineers (IEEE)2025

Scalable kernel logistic regression with Nyström approximation: Theoretical analysis and application to discrete choice modelling

Michel Bierlaire

The application of kernel-based Machine Learning (ML) techniques to discrete choice modelling using large datasets often faces challenges due to memory requirements and the considerable number of parameters involved in these models. This complexity hampers ...

2025

On the Perspectives of Image-to-Lidar Constraints in Dynamic Network Optimisation

Thibaut Stoltz, Jan Skaloud, Davide Antonio Cucci, Laurent Valentin Jospin, Kyriaki Mouzakidou

The evolution of airborne mapping witnesses the introduction of hybrid lidar-camera systems to enhance data collection, i.e. to obtain simultaneously high-density point-cloud and texture. Yet, the common adjustment of both optical data streams is a nontriv ...

International Society for Photogrammetry and Remote Sensing2025

Understanding generalization and robustness in modern deep learning

Maksym Andriushchenko

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

EPFL2024

Electronic Excited States from Physically Constrained Machine Learning

Michele Ceriotti, Jigyasa Nigam, Divya Suman

Data-driven techniques are increasingly used to replace electronic-structure calculations of matter. In this context, a relevant question is whether machine learning (ML) should be applied directly to predict the desired properties or combined explicitly w ...

2024

A Theory of Finite-Width Neural Networks: Generalization, Scaling Laws, and the Loss Landscape

Berfin Simsek

Deep learning has achieved remarkable success in various challenging tasks such as generating images from natural language or engaging in lengthy conversations with humans. The success in practice stems from the ability to successfully train massive neural ...

EPFL2023

Semi-Discrete Optimal Transport: Hardness, Regularization and Numerical Solution

Daniel Kuhn, Soroosh Shafieezadeh Abadeh, Bahar Taskesen

Semi-discrete optimal transport problems, which evaluate the Wasserstein distance between a discrete and a generic (possibly non-discrete) probability measure, are believed to be computationally hard. Even though such problems are ubiquitous in statistics, ...

2023

Beyond worst-case analysis, with or without predictions

Andreas Maggiori

In this thesis we design online combinatorial optimization algorithms for beyond worst-case analysis settings. In the first part, we discuss the online matching problem and prove that, in the edge arrival model, no online algorithm can achieve a competiti ...

EPFL2023

Towards Verifiable, Generalizable and Efficient Robust Deep Neural Networks

Chen Liu

In the last decade, deep neural networks have achieved tremendous success in many fields of machine learning. However, they are shown vulnerable against adversarial attacks: well-designed, yet imperceptible, perturbations can make the state-of-the-art deep ...

EPFL2022

Dif-MAML: Decentralized Multi-Agent Meta-Learning

Ali H. Sayed, Stefan Vlaski, Mert Kayaalp

The objective of meta-learning is to exploit knowledge obtained from observed tasks to improve adaptation to unseen tasks. Meta-learners are able to generalize better when they are trained with a larger number of observed tasks and with a larger amount of ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

U-Boost NAS: Utilization-Boosted Differentiable Neural Architecture Search

Pascal Frossard, Ahmet Caner Yüzügüler, Nikolaos Dimitriadis

Optimizing resource utilization in target platforms is key to achieving high performance during DNN inference. While optimizations have been proposed for inference latency, memory footprint, and energy consumption, prior hardware-aware neural architecture ...

SPRINGER INTERNATIONAL PUBLISHING AG2022

Neural Set Function Extensions: Learning with Discrete Functions in High Dimensions

Andreas Loukas, Nikolaos Karalias

Integrating functions on discrete domains into neural networks is key to developing their capability to reason about discrete objects. But, discrete domains are (I) not naturally amenable to gradient-based optimization, and (II) incompatible with deep lear ...

2022

Advances In Morphological Neural Networks: Training, Pruning And Enforcing Shape Constraints

Nikolaos Dimitriadis

In this paper, we study an emerging class of neural networks, the Morphological Neural networks, from some modern perspectives. Our approach utilizes ideas from tropical geometry and mathematical morphology. First, we state the training of a binary morphol ...

IEEE2021

Convex optimization in sums of Banach spaces

Michaël Unser, Shayan Aziznejad

We characterize the solution of a broad class of convex optimization problems that address the reconstruction of a function from a finite number of linear measurements. The underlying hypothesis is that the solution is decomposable as a finite sum of compo ...

2021

LENA: Communication-Efficient Distributed Learning with Self-Triggered Gradient Uploads

Martin Jaggi, Sebastian Urban Stich

In distributed optimization, parameter updates from the gradient computing node devices have to be aggregated in every iteration on the orchestrating server. When these updates are sent over an arbitrary commodity network, bandwidth and latency can be limi ...

MICROTOME PUBLISHING2021

Estimation of discrete choice models with hybrid stochastic adaptive batch size algorithms

Michel Bierlaire, Virginie Janine Camille Lurkin, Timothy Michael Hillel, Gael Lederrey

The emergence of Big Data has enabled new research perspectives in the discrete choice community. While the techniques to estimate Machine Learning models on a massive amount of data are well established, these have not yet been fully explored for the esti ...

2021

Multi-Agent Reinforcement Learning for Hyperparameter Optimization of Deep Convolutional Neural Networks

David Atienza Alonso, Marina Zapater Sancho, Arman Iranfar

Nowadays, Deep Convolutional Neural Networks (DCNNs) play a significant role in many application domains, such as, computer vision, medical imaging, and image processing. Nonetheless, designing a DCNN, able to defeat the state of the art, is a manual, chal ...

2021

Deep Generative Models and Applications

Tatjana Chavdarova

Over the past few years, there have been fundamental breakthroughs in core problems in machine learning, largely driven by advances in deep neural networks. The amount of annotated data drastically increased and supervised deep discriminative models exceed ...

EPFL2020

Machine learning methods to assist energy system optimization

Jean-Louis Scartezzini, Amarasinghage Tharindu Dasun Perera, Vahid Moussavi Nik

This study evaluates the potential of supervised and transfer learning techniques to assist energy system optimization. A surrogate model is developed with the support of a supervised learning technique (by using artificial neural network) in order to bypa ...

2019

Distributed Inference Over Networks Under Subspace Constraints

Ali H. Sayed, Stefan Vlaski, Roula Nassif

This paper considers optimization problems over networks where agents have individual objectives to meet, or individual parameter vectors to estimate, subject to subspace constraints that enforce the objectives across the network to lie in a low-dimensiona ...

IEEE2019