Publications related to Distributed Learning in Non-Convex Environments-Part II: Polynomial Escape From Saddle-Points

Related publications (38)

Understanding generalization and robustness in modern deep learning

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

EPFL2024

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Anastasiia Koloskova

Distributed learning is the key for enabling training of modern large-scale machine learning models, through parallelising the learning process. Collaborative learning is essential for learning from privacy-sensitive data that is distributed across various ...

EPFL2024

Unlabeled Principal Component Analysis and Matrix Completion

Yunzhen Yao

We introduce robust principal component analysis from a data matrix in which the entries of its columns have been corrupted by permutations, termed Unlabeled Principal Component Analysis (UPCA). Using algebraic geometry, we establish that UPCA is a well-de ...

Microtome Publishing2024

Coupling a recurrent neural network to SPAD TCSPC systems for real-time fluorescence lifetime imaging

Edoardo Charbon, Claudio Bruschini, Andrei Ardelean, Paul Mos, Yang Lin

Fluorescence lifetime imaging (FLI) has been receiving increased attention in recent years as a powerful diagnostic technique in biological and medical research. However, existing FLI systems often suffer from a tradeoff between processing speed, accuracy, ...

Nature Portfolio2024

Interpret3C: Interpretable Student Clustering Through Individualized Feature Selection

Julian Thomas Blackwell, Tanja Christina Käser Jacober, Paola Mejia Domenzain, Vinitra Swamy

Clustering in education, particularly in large-scale online environments like MOOCs, is essential for understanding and adapting to diverse student needs. However, the effectiveness of clustering depends on its interpretability, which becomes challenging w ...

2024

Social Opinion Formation and Decision Making Under Communication Trends

Ali H. Sayed, Virginia Bordignon, Mert Kayaalp

This work studies the learning process over social networks under partial and random information sharing. In traditional social learning models, agents exchange full belief information with each other while trying to infer the true state of nature. We stud ...

Ieee-Inst Electrical Electronics Engineers Inc2024

Leveraging Machine Learning for Advanced Nanoscale X-ray Analysis: Unmixing Multicomponent Signals and Enhancing Chemical Quantification

Cécile Hébert, Duncan Thomas Lindsay Alexander

Energy dispersive X-ray (EDX) spectroscopy in the transmission electron microscope is a key tool for nanomaterials analysis, providing a direct link between spatial and chemical information. However, using it for precisely determining chemical compositions ...

American Chemical Society (ACS)2024

Improving SAM Requires Rethinking its Optimization Formulation

Volkan Cevher, Kimon Antonakopoulos, Fabian Ricardo Latorre Gomez, Thomas Michaelsen Pethick, Wanyun Xie

This paper rethinks Sharpness-Aware Minimization (SAM), which is originally formulated as a zero-sum game where the weights of a network and a bounded perturbation try to minimize/maximize, respectively, the same differentiable loss. We argue that SAM shou ...

2024

Near-Minimax Optimal Estimation With Shallow ReLU Neural Networks

Rahul Parhi

We study the problem of estimating an unknown function from noisy data using shallow ReLU neural networks. The estimators we study minimize the sum of squared data-fitting errors plus a regularization term proportional to the squared Euclidean norm of the ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2023

Numerical study on the scale effect of tip vortex cavitation induced by incoming velocities and scale ratios

Mohamed Farhat

Large eddy simulations of tip vortex cavitation (TVC) around an elliptical hydrofoil is performed to study its scale effect. A satisfying agreement is obtained between the numerical and experimental data. It indicates that the scale effect of TVC is remark ...

PERGAMON-ELSEVIER SCIENCE LTD2023

The multivariate Serre conjecture ring

Luc Guyot

It is well-known that for any integral domain R, the Serre conjecture ring R(X), i.e., the localization of the univariate polynomial ring R[X] at monic polynomials, is a Bezout domain of Krull dimension

Academic Press Inc Elsevier Science2023

Simulation of flow and heat transfer in a differentially heated cubical cavity using coarse Large Eddy Simulation

Konstantin Mikityuk

Two widely used sub-grid scale models namely: the standard and the dynamic Smagorinsky models were tested in a simulation of the flow in a thermally driven 3D cavity at turbulent Rayleigh number Ra = 10(9). The cubical cavity of 0.7 m side-length is set to ...

2023

Linear Complexity Self-Attention With 3rd Order Polynomials

Grigorios Chrysos

Self-attention mechanisms and non-local blocks have become crucial building blocks for state-of-the-art neural architectures thanks to their unparalleled ability in capturing long-range dependencies in the input. However their cost is quadratic with the nu ...

Ieee Computer Soc2023

Resource aware pricing for electric vehicle charging

Nils Gustav Nilsson

Electric vehicle charging facilities offer their capacity constrained electric charge and parking to users for a fee. As electric vehicle adoption grows, so too does the potential for excessive resource utilization. In this paper, we study how prices set b ...

PERGAMON-ELSEVIER SCIENCE LTD2023

Communication-efficient distributed training of machine learning models

Thijs Vogels

In this thesis, we explore techniques for addressing the communication bottleneck in data-parallel distributed training of deep learning models. We investigate algorithms that either reduce the size of the messages that are exchanged between workers, or th ...

EPFL2023

Multi-agent Learning with Privacy Guarantees

Elsa Rizk

A multi-agent system consists of a collection of decision-making or learning agents subjected to streaming observations from some real-world phenomenon. The goal of the system is to solve some global learning or optimization problem in a distributed or dec ...

EPFL2023

Does carbon footprint reduction impair mechanical properties and service life of concrete?

Karen Scrivener, Diana Londoño Zuluaga

The aim of this study is to evaluate how much the changes in the concrete mix design, which enable carbon footprint reduction, are impacting mechanical properties and predicted service life of concrete structure. The starting point of this study was concre ...

SPRINGER2023

Universal and adaptive methods for robust stochastic optimization

Ali Kavis

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

EPFL2023

Short-term runoff forecasting in an alpine catchment with a long short-term memory neural network

Devis Tuia, Marc Conrad Russwurm

The governing hydrological processes are expected to shift under climate change in the alpine regions of Switzerland. This raises the need for more adaptive and accurate methods to estimate river flow. In high-altitude catchments influenced by snow and gla ...

2023

From Kernel Methods to Neural Networks: A Unifying Variational Formulation

Michaël Unser

The minimization of a data-fidelity term and an additive regularization functional gives rise to a powerful framework for supervised learning. In this paper, we present a unifying regularization functional that depends on an operator L\documentclass[12pt]{ ...

Springer2023