Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

The work presented in this thesis lies at the intersection of Machine Learning (ML) and control theory. The first part is dedicated to the design of Deep Neural Network (DNN) architectures with specific numerical properties, inspired by dynamical systems m ...

Turning pass-through network architectures into iterative ones, which use their own output as input, is a well-known approach for boosting performance. In this paper, we argue that such architectures offer an additional benefit: The convergence rate of the ...

This paper studies the infinite-width limit of deep linear neural networks (NNs) initialized with random parameters. We obtain that, when the number of parameters diverges, the training dynamics converge (in a precise sense) to the dynamics obtained from a ...

Translation elongation plays an important role in regulating protein concentrations in the cell, and dysregulation of this process has been linked to several human diseases. In this study, we use data from ribo-seq experiments to model ribosome densities, ...

Many feedforward neural networks (NNs) generate continuous and piecewise-linear (CPWL) mappings. Specifically, they partition the input domain into regions on which the mapping is affine. The number of these so-called linear regions offers a natural metric ...

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...

Deep digital twins (DDTs) are deep neural networks that encode the behavior of complex physical systems. DDTs are excellent system representations due to their ability to continuously adapt to operational changes and their capability to capture complex rel ...

Topographical disorientation refers to the selective inability to orient oneself in familiar surroundings. However, to date its neural correlates remain poorly understood. Here we use quantitative lesion analysis and a lesion network mapping approach in or ...

Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions

Chat with Graph Search

Navigating the Loop: From Control Systems to Neural Networks and Back

Enabling Uncertainty Estimation in Iterative Neural Networks

Infinite-width limit of deep linear neural networks

Towards improving full-length ribosome density prediction by bridging sequence and graph-based representations

On the number of regions of piecewise linear neural networks

On the Generalization of Stochastic Gradient Descent with Momentum

Digital Twin Generalization with Meta and Geometric Deep Learning

The neural correlates of topographical disorientation-a lesion analysis study