Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Concept

Generalization error

Related publications (29)

Consensus Control for Decentralized Deep Learning

Martin Jaggi, Sebastian Urban Stich, Tao Lin, Anastasiia Koloskova

Decentralized training of deep learning models enables on-device learning over networks, as well as efficient scaling to large compute clusters. Experiments in earlier works reveal that, even in a data-center setup, decentralized training often suffers fro ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data

Martin Jaggi, Sebastian Urban Stich, Tao Lin, Sai Praneeth Reddy Karimireddy

Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks. In realistic learning scenarios, the presence of heterogeneity across different clients' local datasets poses an optimization ch ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Partitioning soil organic carbon into its centennially stable and active fractions with machine-learning models based on Rock-Eval (R) thermal analysis (PARTY(SOC)v2.0 and PARTY(SOC)v2.0 (EU))

Juan Carlos Quezada Rivera

Partitioning soil organic carbon (SOC) into two kinetically different fractions that are stable or active on a century scale is key for an improved monitoring of soil health and for more accurate models of the carbon cycle. However, all existing SOC fracti ...

COPERNICUS GESELLSCHAFT MBH2021

CAFS: Cost-Aware Features Selection Method for Multimodal Stress Monitoring on Wearable Devices

David Atienza Alonso, Maria del Carmen Sandi Perez, Adriana Arza Valdes, João Pedro de Matos Rodrigues, Niloofar Momeni

Objective: Today, stress monitoring on wearable devices is challenged by the tension between high-detection accuracy and battery lifetime driven by multimodal data acquisition and processing. Limited research has addressed the classification cost on multim ...

2022

Robust Generalization via $\alpha$-Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

The aim of this work is to provide bounds connecting two probability measures of the same event using Rényi

\alpha

-Divergences and Sibson’s

\alpha

-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence and Shannon’s Mutual ...

ETHZ2020

On Vacuous and Non-Vacuous Generalization Bounds for Deep Neural Networks

Konstantinos Pitas

Deep neural networks have been empirically successful in a variety of tasks, however their theoretical understanding is still poor. In particular, modern deep neural networks have many more parameters than training data. Thus, in principle they should over ...

EPFL2020

Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimization

Lenka Zdeborová, Florent Gérard Krzakala

We consider a commonly studied supervised classification of a synthetic dataset whose labels are generated by feeding a one-layer non-linear neural network with random iid inputs. We study the generalization performances of standard classifiers in the high ...

Curran Associates, Inc.2020

Scaling description of generalization with number of parameters in deep learning

Matthieu Wyart, Clément Hongler, Mario Geiger, Franck Raymond Gabriel, Stefano Spigler, Arthur Ulysse Jacot-Guillarmod, Levent Dogus Sagun

Supervised deep learning involves the training of neural networks with a large number N of parameters. For large enough N, in the so-called over-parametrized regime, one can essentially fit the training data points. Sparsitybased arguments would suggest th ...

IOP PUBLISHING LTD2020

AL2: Progressive Activation Loss for Learning General Representations in Classification Neural Networks

Sabine Süsstrunk, Majed El Helou, Frederike Dümbgen

The large capacity of neural networks enables them to learn complex functions. To avoid overfitting, networks however require a lot of training data that can be expensive and time-consuming to collect. A common practical approach to attenuate overfitting i ...

IEEE2020