Distributed Learning in Non-Convex Environments-Part II: Polynomial Escape From Saddle-Points

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

Distributed learning is the key for enabling training of modern large-scale machine learning models, through parallelising the learning process. Collaborative learning is essential for learning from privacy-sensitive data that is distributed across various ...

We introduce robust principal component analysis from a data matrix in which the entries of its columns have been corrupted by permutations, termed Unlabeled Principal Component Analysis (UPCA). Using algebraic geometry, we establish that UPCA is a well-de ...

Fluorescence lifetime imaging (FLI) has been receiving increased attention in recent years as a powerful diagnostic technique in biological and medical research. However, existing FLI systems often suffer from a tradeoff between processing speed, accuracy, ...

Clustering in education, particularly in large-scale online environments like MOOCs, is essential for understanding and adapting to diverse student needs. However, the effectiveness of clustering depends on its interpretability, which becomes challenging w ...

This work studies the learning process over social networks under partial and random information sharing. In traditional social learning models, agents exchange full belief information with each other while trying to infer the true state of nature. We stud ...

Energy dispersive X-ray (EDX) spectroscopy in the transmission electron microscope is a key tool for nanomaterials analysis, providing a direct link between spatial and chemical information. However, using it for precisely determining chemical compositions ...

This paper rethinks Sharpness-Aware Minimization (SAM), which is originally formulated as a zero-sum game where the weights of a network and a bounded perturbation try to minimize/maximize, respectively, the same differentiable loss. We argue that SAM shou ...

We study the problem of estimating an unknown function from noisy data using shallow ReLU neural networks. The estimators we study minimize the sum of squared data-fitting errors plus a regularization term proportional to the squared Euclidean norm of the ...

Large eddy simulations of tip vortex cavitation (TVC) around an elliptical hydrofoil is performed to study its scale effect. A satisfying agreement is obtained between the numerical and experimental data. It indicates that the scale effect of TVC is remark ...

It is well-known that for any integral domain R, the Serre conjecture ring R(X), i.e., the localization of the univariate polynomial ring R[X] at monic polynomials, is a Bezout domain of Krull dimension

Two widely used sub-grid scale models namely: the standard and the dynamic Smagorinsky models were tested in a simulation of the flow in a thermally driven 3D cavity at turbulent Rayleigh number Ra = 10(9). The cubical cavity of 0.7 m side-length is set to ...

Self-attention mechanisms and non-local blocks have become crucial building blocks for state-of-the-art neural architectures thanks to their unparalleled ability in capturing long-range dependencies in the input. However their cost is quadratic with the nu ...

Electric vehicle charging facilities offer their capacity constrained electric charge and parking to users for a fee. As electric vehicle adoption grows, so too does the potential for excessive resource utilization. In this paper, we study how prices set b ...

In this thesis, we explore techniques for addressing the communication bottleneck in data-parallel distributed training of deep learning models. We investigate algorithms that either reduce the size of the messages that are exchanged between workers, or th ...

A multi-agent system consists of a collection of decision-making or learning agents subjected to streaming observations from some real-world phenomenon. The goal of the system is to solve some global learning or optimization problem in a distributed or dec ...

The aim of this study is to evaluate how much the changes in the concrete mix design, which enable carbon footprint reduction, are impacting mechanical properties and predicted service life of concrete structure. The starting point of this study was concre ...

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

The governing hydrological processes are expected to shift under climate change in the alpine regions of Switzerland. This raises the need for more adaptive and accurate methods to estimate river flow. In high-altitude catchments influenced by snow and gla ...

The minimization of a data-fidelity term and an additive regularization functional gives rise to a powerful framework for supervised learning. In this paper, we present a unifying regularization functional that depends on an operator L\documentclass[12pt]{ ...

Distributed Learning in Non-Convex Environments-Part II: Polynomial Escape From Saddle-Points

Chat with Graph Search

Understanding generalization and robustness in modern deep learning

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Unlabeled Principal Component Analysis and Matrix Completion

Coupling a recurrent neural network to SPAD TCSPC systems for real-time fluorescence lifetime imaging

Interpret3C: Interpretable Student Clustering Through Individualized Feature Selection

Social Opinion Formation and Decision Making Under Communication Trends

Leveraging Machine Learning for Advanced Nanoscale X-ray Analysis: Unmixing Multicomponent Signals and Enhancing Chemical Quantification

Improving SAM Requires Rethinking its Optimization Formulation

Near-Minimax Optimal Estimation With Shallow ReLU Neural Networks

Numerical study on the scale effect of tip vortex cavitation induced by incoming velocities and scale ratios

The multivariate Serre conjecture ring

Simulation of flow and heat transfer in a differentially heated cubical cavity using coarse Large Eddy Simulation

Linear Complexity Self-Attention With 3rd Order Polynomials

Resource aware pricing for electric vehicle charging

Communication-efficient distributed training of machine learning models

Multi-agent Learning with Privacy Guarantees

Does carbon footprint reduction impair mechanical properties and service life of concrete?

Universal and adaptive methods for robust stochastic optimization

Short-term runoff forecasting in an alpine catchment with a long short-term memory neural network

From Kernel Methods to Neural Networks: A Unifying Variational Formulation