PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

This paper investigates the fundamental regression task of learning k neurons (a.k.a. teachers) from Gaussian input, using two-layer ReLU neural networks with width m (a.k.a. students) and m, k = O(1), trained via gradient descent under proper initializati ...

This work is concerned with the computation of the action of a matrix function f(A), such as the matrix exponential or the matrix square root, on a vector b. For a general matrix A, this can be done by computing the compression of A onto a suitable Krylov ...

We present a reinforcement learning framework for constructing assemblies composed of rigid parts, which are commonly seen in many historical masonry buildings and bridges. Traditional construction methods for such structures often depend on dense scaffold ...

The recent rise in interest in point clouds as an imaging modality has motivated standardization groups such as JPEG and MPEG to launch activities aiming at developing compression standards for point clouds. Lossy compression usually introduces visual arti ...

Deep neural networks (DNNs) are receiving increasing attention in wind power forecasting due to their ability to effectively capture complex patterns in wind data. However, their forecast errors are severely limited by the local optimal weight issue in opt ...

Machine learning algorithms such as Convolutional Neural Networks (CNNs) are characterized by high robustness towards quantization, supporting small-bitwidth fixed-point arithmetic at inference time with little to no degradation in accuracy. In turn, small ...

Deploying energy-efficient deep neural networks on energy-constrained edge devices is an important research topic in both machine learning and circuit design communities. Both artificial neural networks (ANNs) and spiking neural networks (SNNs) have been p ...

The desire and ability to place AI-enabled applications on the edge has grown significantly in recent years. However, the compute-, area-, and power-constrained nature of edge devices are stressed by the needs of the AI-enabled applications, due to a gener ...

Now is an exciting time for the domain of Earth observation (EO), with a multitude of diverse sensors looking at the planet from satellites, airplanes or drones. The volume of imagery acquired is massive, and hold great potential for a variety of applicati ...

The invention relates to an Imaging sensor device in a stacked arrangement comprising: - a pixel array tier comprising a plurality of pixel segments each having a plurality of pixels for photon detection each providing a digital pixel output; - a processin ...

Driven by the need for more efficient and seamless integration of physical models and data, physics -informed neural networks (PINNs) have seen a surge of interest in recent years. However, ensuring the reliability of their convergence and accuracy remains ...

As one of the major threats to women's health worldwide, breast cancer requires early diagnosis and accurate classification, since they are key to optimizing therapeutic interventions and ensuring precise prognosis. Recently, deep learning has demonstrated ...

Verification and testing of hardware heavily relies on cycle-accurate simulation of RTL. As single-processor performance is growing only slowly, conventional, single-threaded RTL simulation is becoming impractical for increasingly complex chip designs and ...

Animals have to navigate complex environments and perform intricate swimming maneuvers in the real world. To conquer these challenges, animals evolved a variety of motion control strategies. While it is known that many factors contribute to motion control, ...

The ability to train ever-larger neural networks brings artificial intelligence to the forefront of scientific and technical discoveries. However, their exponentially increasing size creates a proportionally greater demand for energy and computational hard ...

Driven by the demand for real-time processing and the need to minimize latency in AI algorithms, edge computing has experienced remarkable progress. Decision-making AI applications stand out for their heavy reliance on data-centric operations, predominantl ...

In recent years, neural quantum states have emerged as a powerful variational approach, achieving state-of-the-art accuracy when representing the ground-state wave function of a great variety of quantum many-body systems, including spin lattices, interacti ...

The use of point clouds as an imaging modality has been rapidly growing, motivating research on compression methods to enable efficient transmission and storage for many applications. While compression standards relying on conven- tional techniques such as ...

Natural language processing has experienced significant improvements with the development of Transformer-based models, which employ self-attention mechanism and pre-training strategies. However, these models still present several obstacles. A notable issue ...

Contemporary AI faces important limitations that remain unresolved despite significant successes. Chief among these are the inability to acquire new knowledge without destroying old, the incomprehensible and nonengineerable nature of internal representatio ...

PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization

Chat with Graph Search

How Gradient Descent Balances Features: A Dynamical Analysis For Two-Layer Neural Networks

SPEEDING UP KRYLOV SUBSPACE METHODS FOR COMPUTING f(A)b VIA RANDOMIZATION

Learning to Assemble with Alternative Plans

Subjective performance evaluation of bitrate allocation strategies for MPEG and JPEG Pleno point cloud compression

Improving the accuracy and interpretability of neural networks for wind power forecasting

DBFS: Dynamic Bitwidth-Frequency Scaling for Efficient Software-defined SIMD

Eflop: a Sparsity-aware Metric for Evaluating Computational Cost in Spiking and Non-spiking Neural Networks

Exploring High-Performance and Energy-Efficient Architectures for Edge AI-Enabled Applications

Interact with Earth Observation Images using AI: Transparent Methods and Evaluation for Visual Question Answering

Imaging sensor device using an array of single-photon avalanche diode photodetectors

Residual-based attention in physics-informed neural networks

Mspo: a Machine Learning Hyperparameter Optimization Method for Enhanced Breast Cancer Image Classification

Highly Parallel RTL Simulation

Using deep reinforcement learning to investigate stretch feedback during swimming of the lamprey

Training hybrid neural networks with multimode optical nonlinearities using digital twins

EdgeAI-Aware Design of In-Memory Computing Architectures

Accurate neural quantum states for interacting lattice bosons

Evaluating the effect of sparse convolutions on point cloud compression

Modeling Structured Data in Attention-based Models

Foundations of a New Learning Paradigm in AI Grounded in the Principles of Evolutionary Developmental Biology