Home

Publication

Training DNNs with Hybrid Block Floating Point

About
Privacy
Disclaimer

Graph Chatbot

Related publications (38)

Page 2 of 2

Masked Training of Neural Networks with Partial Gradients

Martin Jaggi, Sebastian Urban Stich, Amirkeivan Mohtashami

State-of-the-art training algorithms for deep learning models are based on stochastic gradient descent (SGD). Recently, many variations have been explored: perturbing parameters for better accuracy (such as in Extra-gradient), limiting SGD updates to a sub ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

A Shadow Perspective on Equivariant Hochschild Homologies

Kathryn Hess Bellwald

Shadows for bicategories, defined by Ponto, provide a useful framework that generalizes classical and topological Hochschild homology. In this paper, we define Hochschild-type invariants for monoids in a symmetric monoidal, simplicial model category V, as ...

OXFORD UNIV PRESS2022

Wind farm layout and unconstrained hub height optimization using genetic algorithms applied to different power densities

Fernando Porté Agel, Nicolas Otto Kirchner Bossi

LES and wind tunnel studies have shown significant benefit when allowing turbines (T) in a wind farm to adopt different heights. This work presents two new genetic algorithms (GA) that perform wind farm layout optimization (WFLO) involving continuous and t ...

IOP Publishing2022

Visual Focus of Attention Estimation in 3D Scene with an Arbitrary Number of Targets

Jean-Marc Odobez, Rémy Alain Siegfried

Visual Focus of Attention (VFOA) estimation in conversation is challenging as it relies on difficult to estimate information (gaze) combined with scene features like target positions and other contextual information (speaking status) allowing to disambigua ...

IEEE COMPUTER SOC2021

Application of optimal spline subspaces for the removal of spurious outliers in isogeometric discretizations

Espen Sande

We show that isogeometric Galerkin discretizations of eigenvalue problems related to the Laplace operator subject to any standard type of homogeneous boundary conditions have no outliers in certain optimal spline subspaces. Roughly speaking, these optimal ...

2022

Masksembles for Uncertainty Estimation

Pascal Fua, Timur Bagautdinov, Pierre Bruno Baqué, Nikita Durasov

Deep neural networks have amply demonstrated their prowess but estimating the reliability of their predictions remains challenging. Deep Ensembles are widely considered as being one of the best methods for generating uncertainty estimates but are very expe ...

2021

Low-Power Artificial Neural Network Perceptron Based on Monolayer MoS2

Aleksandra Radenovic, Andras Kis, Zhenyu Wang, Mukesh Kumar Tripathi, Guilherme Migliato Marega

Machine learning and signal processing on the edge are poised to influence our everyday lives with devices that will learn and infer from data generated by smart sensors and other devices for the Internet of Things. The next leap toward ubiquitous electron ...

2022

Global bifurcation at isolated singular points of the Hadamard derivative

Charles Stuart

Consider F is an element of C(RxX,Y) such that F(lambda, 0) = 0 for all lambda is an element of R, where X and Y are Banach spaces. Bifurcation from the line Rx{0} of trivial solutions is investigated in cases where F(lambda, center dot ) need not be Frech ...

2021

Fourier Interpolation From Spheres

Martin Peter Stoller

In every dimension d >= 2, we give an explicit formula that expresses the values of any Schwartz function on R-d only in terms of its restrictions, and the restrictions of its Fourier transform, to all origin-centered spheres whose radius is the square roo ...

2021

Gemini: Elastic SNARKs for Diverse Environments

Alessandro Chiesa

We introduce a new class of succinct arguments, that we call elastic. Elastic SNARKs allow the prover to allocate different resources (such as memory and time) depending on the execution environment and the statement to prove. The resulting output is indep ...

SPRINGER INTERNATIONAL PUBLISHING AG2022

Spectral Coarse Spaces for the Substructured Parallel Schwarz Method

Tommaso Vanzan

The parallel Schwarz method (PSM) is an overlapping domain decomposition (DD) method to solve partial differential equations (PDEs). Similarly to classical nonoverlapping DD methods, the PSM admits a substructured formulation, that is, it can be formulated ...

2022

Invariant Integrals on Topological Groups and Applications

Vasco Schiavo

We study a fixed point property for linear actions of discrete groups on weakly complete convex proper cones in locally convex topological vector spaces. We search to understand the class of discrete groups which enjoys this property and we try to generali ...

EPFL2021

Sparsely Observed Functional Time Series: Theory and Applications

Tomas Rubin

Functional time series is a temporally ordered sequence of not necessarily independent random curves. While the statistical analysis of such data has been traditionally carried out under the assumption of completely observed functional data, it may well ha ...

EPFL2021

VWR2A: A Very-Wide-Register Reconfigurable-Array Architecture for Low-Power Embedded Devices

David Atienza Alonso, Miguel Peon Quiros, Benoît Walter Denkinger

Edge-computing requires high-performance energy-efficient embedded systems. Fixed-function or custom accelerators, such as FFT or FIR filter engines, are very efficient at implementing a particular functionality for a given set of constraints. However, the ...

ASSOC COMPUTING MACHINERY2022

Equinox: Training (for Free) on a Custom Inference Accelerator

Babak Falsafi, Martin Jaggi, Mario Paulo Drumond Lages De Oliveira, Ahmet Caner Yüzügüler, Arash Pourhabibi Zarandi, Louis Coulon

DNN inference accelerators executing online services exhibit low average loads because of service demand variability, leading to poor resource utilization. Unfortunately, reclaiming idle inference cycles is difficult as other workloads can not execute on a ...

ACM2021

Beyond Time-Average Convergence: Near-Optimal Uncoupled Online Learning via Clairvoyant Multiplicative Weights Update

Efstratios Panteleimon Skoulakis

In this paper we provide a novel and simple algorithm, Clairvoyant Multiplicative Weights Updates (CMWU), for convergence to \textit{Coarse Correlated Equilibria} (CCE) in general games. CMWU effectively corresponds to the standard MWU algorithm but where ...

2022

Semantic Perturbations with Normalizing Flows for Improved Generalization

Martin Jaggi, Sebastian Urban Stich, Tatjana Chavdarova

Data augmentation is a widely adopted technique for avoiding overfitting when training deep neural networks. However, this approach requires domain-specific knowledge and is often limited to a fixed set of hard-coded transformations. Recently, several work ...

IEEE2021

Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization)

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Zhenyu Zhu

We study the average robustness notion in deep neural networks in (selected) wide and narrow, deep and shallow, as well as lazy and non-lazy training settings. We prove that in the under-parameterized setting, width has a negative effect while it improves ...

2022