Smooth Primal-Dual Coordinate Descent Algorithms for Nonsmooth Convex Optimization

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

This paper investigates the fundamental regression task of learning k neurons (a.k.a. teachers) from Gaussian input, using two-layer ReLU neural networks with width m (a.k.a. students) and m, k = O(1), trained via gradient descent under proper initializati ...

Contemporary AI faces important limitations that remain unresolved despite significant successes. Chief among these are the inability to acquire new knowledge without destroying old, the incomprehensible and nonengineerable nature of internal representatio ...

In April 2023, the superBIT telescope was lifted to the Earth's stratosphere by a helium-filled super-pressure balloon to acquire astronomical imaging from above (99.5% of) the Earth's atmosphere. It was launched from New Zealand and then, for 40 days, cir ...

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

Deep-learning-based digital twins (DDT) are a promising tool for data-driven system health management because they can be trained directly on operational data. A major challenge for efficient training however is that industrial datasets remain unlabeled. T ...

We address black-box convex optimization problems, where the objective and constraint functions are not explicitly known but can be sampled within the feasible set. The challenge is thus to generate a sequence of feasible points converging towards an optim ...

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

This paper considers the multi-agent linear least-squares problem in a server-agent network architecture. The system comprises multiple agents, each with a set of local data points. The agents are connected to a server, and there is no inter-agent communic ...

We propose a stochastic conditional gradient method (CGM) for minimizing convex finitesum objectives formed as a sum of smooth and non-smooth terms. Existing CGM variants for this template either suffer from slow convergence rates, or require carefully inc ...

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...

Deep neural networks have completely revolutionized the field of machine learning by achieving state-of-the-art results on various tasks ranging from computer vision to protein folding. However, their application is hindered by their large computational an ...

Modern optimization is tasked with handling applications of increasingly large scale, chiefly due to the massive amounts of widely available data and the ever-growing reach of Machine Learning. Consequently, this area of research is under steady pressure t ...

We propose an adaptive variance-reduction method, called AdaSpider, for minimization of L-smooth, non-convex functions with a finite-sum structure. In essence, AdaSpider combines an AdaGrad-inspired [Duchi et al., 2011, McMahan & Streeter, 2010], but a fai ...

Human babies have a natural desire to interact with new toys and objects, through which they learn how the world around them works, e.g., that glass shatters when dropped, but a rubber ball does not. When their predictions are proven incorrect, such as whe ...

Soft Magnetic Composites (SMCs) demonstrate excellent potentials for electromagnetic applications. The isotropy is one critical feature in designing the composite. This work presents a shape optimization algorithm for SMCs subject to isotropy and volume co ...

Singlet fission has shown potential for boosting the efficiency of solar cells, but the scarcity of suitable molecular materials hinders its implementation. We introduce an uncertainty-controlled genetic algorithm (ucGA) based on ensemble machine learning ...

In this paper, we analyze the recently proposed stochastic primal-dual hybrid gradient (SPDHG) algorithm and provide new theoretical results. In particular, we prove almost sure convergence of the iterates to a solution with convexity and linear convergenc ...

Smooth Primal-Dual Coordinate Descent Algorithms for Nonsmooth Convex Optimization

Chattez avec Graph Search

How Gradient Descent Balances Features: A Dynamical Analysis For Two-Layer Neural Networks

Foundations of a New Learning Paradigm in AI Grounded in the Principles of Evolutionary Developmental Biology

Data Downloaded via Parachute from a NASA Super-Pressure Balloon

Universal and adaptive methods for robust stochastic optimization

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

A Study on Gradient-based Meta-learning for Robust Deep Digital Twins

Safe Zeroth-Order Convex Optimization Using Quadratic Local Approximations

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Iterative pre-conditioning for expediting the distributed gradient-descent method: The case of linear least-squares problem

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

On the Generalization of Stochastic Gradient Descent with Momentum

Stop Wasting my FLOPS: Improving the Efficiency of Deep Learning Models

Scalable constrained optimization

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Seeking the new, learning from the unexpected: Computational models of surprise and novelty in the brain

Level-set-based Shape Optimization on Soft Magnetic Composites with Isotropy Constraint

Inverse Design of Singlet-Fission Materials with Uncertainty-Controlled Genetic Optimization

On The Convergence Of Stochastic Primal-Dual Hybrid Gradient