Sparsest Univariate Learning Models Under Lipschitz Constraint

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Deploying energy-efficient deep neural networks on energy-constrained edge devices is an important research topic in both machine learning and circuit design communities. Both artificial neural networks (ANNs) and spiking neural networks (SNNs) have been p ...

This paper investigates the fundamental regression task of learning k neurons (a.k.a. teachers) from Gaussian input, using two-layer ReLU neural networks with width m (a.k.a. students) and m, k = O(1), trained via gradient descent under proper initializati ...

Reinforcement learning (RL) is inherently rife with non-stationarity since the states and rewards the agent observes during training depend on its changing policy. Therefore, networks in deep RL must be capable of adapting to new observations and fitting n ...

Mismatches between samples and their respective channel or target commonly arise in several real-world applications. For instance, whole-brain calcium imaging of freely moving organisms, multiple-target tracking or multi-person contactless vital sign monit ...

Many feedforward neural networks (NNs) generate continuous and piecewise-linear (CPWL) mappings. Specifically, they partition the input domain into regions on which the mapping is affine. The number of these so-called linear regions offers a natural metric ...

Recently, the diffusion moving-average (D-MA) scheme has been proposed as a way to combat noisy links over adaptive networks. However, the current theoretical results focus on networks with mean-square error costs where the optimal local solution agrees wi ...

We investigate popular resampling methods for estimating the uncertainty of statistical models, such as subsampling, bootstrap and the jackknife, and their performance in high-dimensional supervised regression tasks. We provide a tight asymptotic descripti ...

Established techniques that enable robots to learn from demonstrations are based on learning a stable dynamical system (DS). To increase the robots' resilience to perturbations during tasks that involve static obstacle avoidance, we propose incorporating b ...

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

Fourier ptychography (FP) involves the acquisition of several low-resolution intensity images of a sample under varying illumination angles. They are then combined into a high-resolution complex-valued image by solving a phase-retrieval problem. The object ...

In data-parallel optimization of machine learning models, workers collaborate to improve their estimates of the model: more accurate gradients allow them to use larger learning rates and optimize faster. In the decentralized setting, in which workers commu ...

We study the consistency of the estimator in spatial regression with partial differential equa-tion (PDE) regularization. This new smoothing technique allows to accurately estimate spatial fields over complex two-dimensional domains, starting from noisy ob ...

This work aims to study the effects of wind uncertainties in civil engineering structural design. Optimising the design of a structure for safety or operability without factoring in these uncertainties can result in a design that is not robust to these per ...

The purpose of this article is to develop and study a decentralized strategy for Pareto optimization of an aggregate cost consisting of regularized risks. Each risk is modeled as the expectation of some loss function with unknown probability distribution, ...

We consider an optimal control problem (OCP) for a partial differential equation (PDE) with random coefficients. The optimal control function is a deterministic, distributed forcing term that minimizes an expected quadratic regularized loss functional. For ...

We demonstrate two new important properties of the 1-path-norm of shallow neural networks. First, despite its non-smoothness and non-convexity it allows a closed form proximal operator which can be efficiently computed, allowing the use of stochastic proxi ...

p>We study the dynamics of optimization and the generalization properties of one-hidden layer neural networks with quadratic activation function in the overparametrized regime where the layer width m is larger than the input dimension d. We conside ...

Sparsest Univariate Learning Models Under Lipschitz Constraint

Chat with Graph Search

Eflop: a Sparsity-aware Metric for Evaluating Computational Cost in Spiking and Non-spiking Neural Networks

How Gradient Descent Balances Features: A Dynamical Analysis For Two-Layer Neural Networks

No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPO

Shuffled multi-channel sparse signal recovery

On the number of regions of piecewise linear neural networks

Diffusion Moving-Average Adaptation Over Networks

Analysis of Bootstrap and Subsampling in High-dimensional Regularized Regression

Learning Barrier-Certified Polynomial Dynamical Systems for Obstacle Avoidance with Robots

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Dynamic Fourier ptychography with deep spatiotemporal priors

Beyond Spectral Gap: The Role of the Topology in Decentralized Learning

Some first results on the consistency of spatial regression with partial differential equation regularization

Multi-Level Monte Carlo Methods for Uncertainty Quantification and Risk-Averse Optimisation

Regularized Diffusion Adaptation via Conjugate Smoothing

PDE-Constrained Optimal Control Problems with Uncertain Parameters using SAGA

Efficient Proximal Mapping of the 1-path-norm of Shallow Networks

Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions