Publications associées à Sparsest Univariate Learning Models Under Lipschitz Constraint

Publications associées (18)

Eflop: a Sparsity-aware Metric for Evaluating Computational Cost in Spiking and Non-spiking Neural Networks

Deploying energy-efficient deep neural networks on energy-constrained edge devices is an important research topic in both machine learning and circuit design communities. Both artificial neural networks (ANNs) and spiking neural networks (SNNs) have been p ...

IOP Publishing Ltd2025

How Gradient Descent Balances Features: A Dynamical Analysis For Two-Layer Neural Networks

Volkan Cevher, Fanghui Liu, Zhenyu Zhu

This paper investigates the fundamental regression task of learning k neurons (a.k.a. teachers) from Gaussian input, using two-layer ReLU neural networks with width m (a.k.a. students) and m, k = O(1), trained via gradient descent under proper initializati ...

2025

No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPO

Andréa Serge Alfredo Miele, Daniil Pyatko, Caglar Gulcehre, Skander Moalla

Reinforcement learning (RL) is inherently rife with non-stationarity since the states and rewards the agent observes during training depend on its changing policy. Therefore, networks in deep RL must be capable of adapting to new observations and fitting n ...

2024

Shuffled multi-channel sparse signal recovery

Benjamin Bejar Haro

Mismatches between samples and their respective channel or target commonly arise in several real-world applications. For instance, whole-brain calcium imaging of freely moving organisms, multiple-target tracking or multi-person contactless vital sign monit ...

2024

On the number of regions of piecewise linear neural networks

Michaël Unser, Alexis Marie Frederic Goujon

Many feedforward neural networks (NNs) generate continuous and piecewise-linear (CPWL) mappings. Specifically, they partition the input domain into regions on which the mapping is affine. The number of these so-called linear regions offers a natural metric ...

2024

Diffusion Moving-Average Adaptation Over Networks

Ali H. Sayed

Recently, the diffusion moving-average (D-MA) scheme has been proposed as a way to combat noisy links over adaptive networks. However, the current theoretical results focus on networks with mean-square error costs where the optimal local solution agrees wi ...

2024

Analysis of Bootstrap and Subsampling in High-dimensional Regularized Regression

Lenka Zdeborová, Florent Gérard Krzakala, Bruno Loureiro, Adrien Vandenbroucque, Lucas Andry Clarte, Guillaume Cyprien Dalle

We investigate popular resampling methods for estimating the uncertainty of statistical models, such as subsampling, bootstrap and the jackknife, and their performance in high-dimensional supervised regression tasks. We provide a tight asymptotic descripti ...

ACM2024

Learning Barrier-Certified Polynomial Dynamical Systems for Obstacle Avoidance with Robots

Aude Billard

Established techniques that enable robots to learn from demonstrations are based on learning a stable dynamical system (DS). To increase the robots' resilience to perturbations during tasks that involve static obstacle avoidance, we propose incorporating b ...

IEEE2024

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Scott William Pesme

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

Dynamic Fourier ptychography with deep spatiotemporal priors

Michaël Unser, Pakshal Narendra Bohra

Fourier ptychography (FP) involves the acquisition of several low-resolution intensity images of a sample under varying illumination angles. They are then combined into a high-resolution complex-valued image by solving a phase-retrieval problem. The object ...

IOP Publishing Ltd2023

Beyond Spectral Gap: The Role of the Topology in Decentralized Learning

Martin Jaggi, Thijs Vogels, Hadrien Hendrikx

In data-parallel optimization of machine learning models, workers collaborate to improve their estimates of the model: more accurate gradients allow them to use larger learning rates and optimize faster. In the decentralized setting, in which workers commu ...

Microtome Publishing2023

Quantum mixed state compiling

Zoë Holmes

The task of learning a quantum circuit to prepare a given mixed state is a fundamental quantum subroutine. We present a variational quantum algorithm (VQA) to learn mixed states which is suitable for near-term hardware. Our algorithm represents a generaliz ...

IOP Publishing Ltd2023

Some first results on the consistency of spatial regression with partial differential equation regularization

Fabio Nobile

We study the consistency of the estimator in spatial regression with partial differential equa-tion (PDE) regularization. This new smoothing technique allows to accurately estimate spatial fields over complex two-dimensional domains, starting from noisy ob ...

2022

Multi-Level Monte Carlo Methods for Uncertainty Quantification and Risk-Averse Optimisation

Sundar Subramaniam Ganesh

This work aims to study the effects of wind uncertainties in civil engineering structural design. Optimising the design of a structure for safety or operability without factoring in these uncertainties can result in a design that is not robust to these per ...

EPFL2022

Regularized Diffusion Adaptation via Conjugate Smoothing

Ali H. Sayed, Stefan Vlaski

The purpose of this article is to develop and study a decentralized strategy for Pareto optimization of an aggregate cost consisting of regularized risks. Each risk is modeled as the expectation of some loss function with unknown probability distribution, ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

PDE-Constrained Optimal Control Problems with Uncertain Parameters using SAGA

Fabio Nobile, Matthieu Claude Martin

We consider an optimal control problem (OCP) for a partial differential equation (PDE) with random coefficients. The optimal control function is a deterministic, distributed forcing term that minimizes an expected quadratic regularized loss functional. For ...

2021

Efficient Proximal Mapping of the 1-path-norm of Shallow Networks

Volkan Cevher, Fabian Ricardo Latorre Gomez, Paul Thierry Yves Rolland, Shaul Nadav Hallak

We demonstrate two new important properties of the 1-path-norm of shallow neural networks. First, despite its non-smoothness and non-convexity it allows a closed form proximal operator which can be efficiently computed, allowing the use of stochastic proxi ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2020

Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions

Lenka Zdeborová

p>We study the dynamics of optimization and the generalization properties of one-hidden layer neural networks with quadratic activation function in the overparametrized regime where the layer width m is larger than the input dimension d. We conside ...

Curran Associates, Inc.2020