Sparsified SGD with Memory

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

A shortcut model is a low-order model that provides a simplified representation of a complex process, thereby enabling faster and more efficient analysis, design and optimization without the computational intensity of a full-scale model. For continuous dis ...

Synthesizability in generative molecular design remains a pressing challenge. Existing methods to assess synthesizability include heuristics-based metrics or retrosynthesis models which predict a synthetic pathway. By contrast, an explicit approach anchors ...

This work aims to develop a scalable surrogate modelling framework for robust multidisciplinary design optimisation (RMDO) in complex engineering systems. Focusing on gas-bearing supported rotors, it addresses the need to maintain system performance under ...

For robots to operate in unstructured environments, they are required to interact with objects through contact. Those contacts may be used to push objects to the side, deform objects, or manipulate objects in-hand. This thesis addresses the problem of cont ...

Integer linear programming (ILP) is an elegant approach to solve linear optimization problems, naturally described using integer decision variables. Within the context of physics-inspired machine learning (ML) applied to chemistry, we demonstrate the relev ...

In the field of choice modeling, the availability of ever-larger datasets has the potential to significantly expand our understanding of human behavior, but this prospect is limited by the poor scalability of discrete choice models (DCMs): as sample sizes ...

Neural machine translation (MT) and text generation have recently reached very high levels of quality. However, both areas share a problem: in order to reach these levels, they require massive amounts of data. When this is not present, they lack generaliza ...

The monumental progress in the development of machine learning models has led to a plethora of applications with transformative effects in engineering and science. This has also turned the attention of the research community towards the pursuit of construc ...

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins. All these successful methods depend on a gradient-based learning algorithm to train a model on massive ...

We formulate computed-tomography reconstruction as a continuous-domain optimization problem with Hessian total variation (HTV) as the regularizer. HTV is a sparsity-promoting regularizer that favors continuous and piecewise-linear functions with few affine ...

The variational approach is a cornerstone of computational physics, considering both conventional and quantum computing computational platforms. The variational quantum eigensolver algorithm aims to prepare the ground state of a Hamiltonian exploiting para ...

In this article, we study the problem of Byzantine fault-tolerance in a federated optimization setting, where there is a group of agents communicating with a centralized coordinator. We allow up to

f

Byzantine-faulty agents, which may not follow a prescr ...

Distributed learning is the key for enabling training of modern large-scale machine learning models, through parallelising the learning process. Collaborative learning is essential for learning from privacy-sensitive data that is distributed across various ...

Objective. Peripheral nerve interfaces have the potential to restore sensory, motor, and visceral functions. In particular, intraneural interfaces allow targeting deep neural structures with high selectivity, even if their performance strongly depends upon ...

Modern optimization is tasked with handling applications of increasingly large scale, chiefly due to the massive amounts of widely available data and the ever-growing reach of Machine Learning. Consequently, this area of research is under steady pressure t ...

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

Designing turbocompressors is a complex and challenging task, as it involves balancing conflicting objectives such as efficiency, stability, and robustness against manufacturing deviations. This paper proposes an integrated design methodology for turbocomp ...

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...

Sparsified SGD with Memory

Chat with Graph Search

A shortcut model for batch distillation columns using pertinent effective factors

Directly optimizing for synthesizability in generative molecular design using retrosynthesis models

Dimensionless Group-Driven Ensemble Neural Networks for Robust Design Optimisation in Engineering

A Stochastic Approach to Contact-rich Manipulation

Integer Linear Programming for Unsupervised Training Set Selection in Molecular Machine Learning

Stochastic adaptive resampling for the estimation of discrete choice models

Regularization Techniques for Low-Resource Machine Translation

Probabilistic methods for neural combinatorial optimization

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Transformer Models for Vision

A Continuous-Domain Solution for Computed Tomography with Hessian Total-Variation Regularization

Phenomenological theory of variational quantum ground-state preparation

Byzantine Fault-Tolerance in Federated Local SGD Under 2f-Redundancy

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Combining biophysical models and machine learning to optimize implant geometry and stimulation protocol for intraneural electrodes

Scalable constrained optimization

Understanding generalization and robustness in modern deep learning

An Integrated Approach to Designing Robust Turbocompressors on Gas Bearings Through Surrogate Modeling and Constrained Multi-Objective Optimization

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

On the Generalization of Stochastic Gradient Descent with Momentum