NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Here we provide the neural data, activation and predictions for the best models and result dataframes of our article "Task-driven neural network models predict neural dynamics of proprioception". It contains the behavioral and neural experimental data (cu ...

In inverse problems, the task is to reconstruct an unknown signal from its possibly noise-corrupted measurements. Penalized-likelihood-based estimation and Bayesian estimation are two powerful statistical paradigms for the resolution of such problems. They ...

Efficient sampling and approximation of Boltzmann distributions involving large sets of binary variables, or spins, are pivotal in diverse scientific fields even beyond physics. Recent advances in generative neural networks have significantly impacted this ...

Modern neuroscience research is generating increasingly large datasets, from recording thousands of neurons over long timescales to behavioral recordings of animals spanning weeks, months, or even years. Despite a great variety in recording setups and expe ...

Transformer models such as GPT generate human-like language and are predictive of human brain responses to language. Here, using functional-MRI-measured brain responses to 1,000 diverse sentences, we first show that a GPT-based encoding model can predict t ...

We present a finite elements-neural network approach for the numerical approximation of parametric partial differential equations. The algorithm generates training data from finite element simulations, and uses a data -driven (supervised) feedforward neura ...

This paper presents a comparison of machine learning (ML) methods used for three-dimensional localization of partial discharges (PD) in a power transformer tank. The study examines ML and deep learning (DL) methods, ranging from support vector machines (SV ...

The present work proposes a framework for nonlinear model order reduction based on a Graph Convolutional Autoencoder (GCA-ROM). In the reduced order modeling (ROM) context, one is interested in obtaining real -time and many-query evaluations of parametric ...

Deep learning has revolutionized the field of computer vision, a success largely attributable to the growing size of models, datasets, and computational power. Simultaneously, a critical pain point arises as several computer vision applications are deploye ...

The upcoming Square Kilometre Array Observatory will produce images of neutral hydrogen distribution during the epoch of reionization by observing the corresponding 21-cm signal. However, the 21-cm signal will be subject to instrumental limitations such as ...

Translation elongation plays an important role in regulating protein concentrations in the cell, and dysregulation of this process has been linked to several human diseases. In this study, we use data from ribo-seq experiments to model ribosome densities, ...

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

The capabilities of deep learning systems have advanced much faster than our ability to understand them. Whilst the gains from deep neural networks (DNNs) are significant, they are accompanied by a growing risk and gravity of a bad outcome. This is troubli ...

Recently, cutting-edge brain-machine interfaces (BMIs) have revealed the potential of decoders such as recurrent neural networks (RNNs) in predicting attempted handwriting [1] or speech [2], enabling rapid communication recovery after paralysis. However, c ...

Proprioception tells the brain the state of the body based on distributed sensors in the body. However, the principles that govern proprioceptive processing from those distributed sensors are poorly understood. Here, we employ a task-driven neural network ...

NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization

Chat with Graph Search

Task-driven neural network models predict neural dynamics of proprioception: Experimental data, activations and predictions of neural network models

Statistical Inference for Inverse Problems: From Sparsity-Based Methods to Neural Networks

Sparse autoregressive neural networks for classical spin systems

Robust machine learning for neuroscientific inference

Driving and suppressing the human language network using large language models

Error assessment of an adaptive finite elements-neural networks method for an elliptic parametric PDE

Partial discharge localization in power transformer tanks using machine learning methods

A graph convolutional autoencoder approach to model order reduction for parametrized PDEs

Fast and Future: Towards Efficient Forecasting in Video Semantic Segmentation

Deep learning approach for identification of H II regions during reionization in 21-cm observations - II. Foreground contamination

Towards improving full-length ribosome density prediction by bridging sequence and graph-based representations

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

On the Generalization of Stochastic Gradient Descent with Momentum

Understanding generalization and robustness in modern deep learning

Safe Deep Neural Networks

33.3 MiBMI: A 192/512-Channel 2.46mm² Miniaturized Brain-Machine Interface Chipset Enabling 31-Class Brain-to-Text Conversion Through Distinctive Neural Codes

Task-driven neural network models predict neural dynamics of proprioception: Neural network model weights