Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

The present invention concerns a pivot comprising two assemblies, namely a central assembly (401) and a peripheral assembly (400). These two assemblies are mobile in rotation relative to each other around an axis of rotation (A). The pivot is characterized ...

A sample of 3984 candidates of the K+ -> pi(+)gamma gamma decay, with an estimated background of 291 +/- 14 events, was collected by the NA62 experiment at CERN during 2017-2018. In order to describe the observed di-photon mass spectrum, the next-to-leadin ...

Adversarial attacks in Natural Language Processing apply perturbations in the character or token levels. Token-level attacks, gaining prominence for their use of gradient-based methods, are susceptible to altering sentence semantics, leading to invalid adv ...

Modern optimization is tasked with handling applications of increasingly large scale, chiefly due to the massive amounts of widely available data and the ever-growing reach of Machine Learning. Consequently, this area of research is under steady pressure t ...

Collaborative learning algorithms, such as distributed SGD (or D-SGD), are prone to faulty machines that may deviate from their prescribed algorithm because of software or hardware bugs, poisoned data or malicious behaviors. While many solutions have been ...

Functional data are typically modeled as sample paths of smooth stochastic processes in order to mitigate the fact that they are often observed discretely and noisily, occasionally irregularly and sparsely. The smoothness assumption is imposed to allow for ...

Modern neuroscience research is generating increasingly large datasets, from recording thousands of neurons over long timescales to behavioral recordings of animals spanning weeks, months, or even years. Despite a great variety in recording setups and expe ...

Diagonal linear networks (DLNs) are a toy simplification of artificial neural networks; they consist in a quadratic reparametrization of linear regression inducing a sparse implicit regularization. In this paper, we describe the trajectory of the gradient ...

The shapes of galaxies, their outer regions in particular, are important guideposts to their formation and evolution. In this work, we report on the discovery of strongly box-shaped morphologies of the otherwise well-studied elliptical and lenticular galax ...

Superlattice formation afforded by metal halide perovskite nanocrystals has been a phenomenon of interest due to the high structural order induced in these self-assemblies, an order that is influenced by the surface chemistry and particle morphology of the ...

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

Selection bias may arise when data have been chosen in a way that subsequent analysis does not account for. Such bias can arise in climate event attribution studies that are performed rapidly after a devastating "trigger event'', whose occurrence correspon ...

We present a combination technique based on mixed differences of both spatial approximations and quadrature formulae for the stochastic variables to solve efficiently a class of optimal control problems (OCPs) constrained by random partial differential equ ...

Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity

Chat with Graph Search

Pivot, process for manufacturing such a pivot, oscillator comprising such a pivot, watch movement and timepiece comprising such an oscillator

Measurement of the K+ → π+γγ decay

Revisiting Character-level Adversarial Attacks for Language Models

Scalable constrained optimization

Robust Collaborative Learning with Linear Gradient Overhead

Functional data analysis with rough sample paths?

Robust machine learning for neuroscientific inference

Incremental Learning in Diagonal Linear Networks

The Halos and Environments of Nearby Galaxies (HERON) survey IV. Complexity in the boxy galaxies NGC720 and NGC2768

Long live(d) CsPbBr3 superlattices: colloidal atomic layer deposition for structural stability

Universal and adaptive methods for robust stochastic optimization

Timing and spatial selection bias in rapid extreme event attribution

A Combination Technique for Optimal Control Problems Constrained by Random PDEs