On the different regimes of stochastic gradient descent

À propos
Confidentialité
Mentions légales

Graph Chatbot

Publications associées (32)

Page 2 sur 2

Finding Actual Descent Directions For Adversarial Training

Volkan Cevher, Fabian Ricardo Latorre Gomez, Igor Krawczuk, Thomas Michaelsen Pethick, Leello Tadesse Dadi

Adversarial Training using a strong first-order adversary (PGD) is the gold standard for training Deep Neural Networks that are robust to adversarial examples. We show that, contrary to the general understanding of the method, the gradient at an optimal ad ...

2023

An exact mapping from ReLU networks to spiking neural networks

Wulfram Gerstner, Stanislaw Andrzej Wozniak, Angeliki Pantazi, Guillaume Emmanuel Fernand Bellec, Ana Stanojevic

Deep spiking neural networks (SNNs) offer the promise of low-power artificial intelligence. However, training deep SNNs from scratch or converting deep artificial neural networks to SNNs without loss of performance has been a challenge. Here we propose an ...

2023

Benefits of Max Pooling in Neural Networks: Theoretical and Experimental Evidence

François Fleuret, Kyle Michael Matoba, Nikolaos Dimitriadis

When deep neural networks became state of the art image classifiers, numerous max pooling operations were an important component of the architecture. However, modern computer vision networks typically have few, if any, max pooling operations. To understand ...

Journal of Machine Learning Research Inc.2023

Leveraging Unlabeled Data to Track Memorization

Patrick Thiran, Mahsa Forouzesh

Deep neural networks may easily memorize noisy labels present in real-world data, which degrades their ability to generalize. It is therefore important to track and evaluate the robustness of models against noisy label memorization. We propose a metric, ca ...

2023

DARE: Towards Robust Text Explanations in Biomedical and Healthcare Applications

Pascal Frossard, Ádám Dániel Ivánkay

Along with the successful deployment of deep neural networks in several application domains, the need to unravel the black-box nature of these networks has seen a significant increase recently. Several methods have been introduced to provide insight into t ...

Assoc Computational Linguistics-Acl2023

NMDA-driven dendritic modulation enables multitask representation learning in hierarchical sensory processing pathways

Willem Anna Mark Wybo, Bernd Albert Illing

While sensory representations in the brain depend on context, it remains unclear how such modulations are implemented at the biophysical level, and how processing layers further in the hierarchy can extract useful features for each possible contex-tual sta ...

National Academy of Sciences2023

Decoding cerebro-spinal signatures of human behavior: Application to motor sequence learning

Dimitri Nestor Alice Van De Ville, Nawal Noëlle Kinany

Mapping the neural patterns that drive human behavior is a key challenge in neuroscience. Even the simplest of our everyday actions stem from the dynamic and complex interplay of multiple neural structures across the central nervous system (CNS). Yet, most ...

ACADEMIC PRESS INC ELSEVIER SCIENCE2023

Monotonicity Reasoning in the Age of Neural Foundation Models

Zeming Chen

The recent advance of large language models (LLMs) demonstrates that these large-scale foundation models achieve remarkable capabilities across a wide range of language tasks and domains. The success of the statistical learning approach challenges our unde ...

Springer2023

Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Zhenyu Zhu, Yongtao Wu

Neural tangent kernel (NTK) is a powerful tool to analyze training dynamics of neural networks and their generalization bounds. The study on NTK has been devoted to typical neural network architectures, but it is incomplete for neural networks with Hadamar ...

2022

Mental Flow Estimation Through Wearable EEG

Jean-Philippe Thiran

Flow is a mental state experienced during holistic involvement in a certain task, and it is a factor that promotes motivation, development, and performance. A reliable and objective estimation of the flow is essential for moving away from the traditional s ...

IEEE2022

Autoencoders reloaded

Hervé Bourlard, Selen Hande Kabil

In Bourlard and Kamp (Biol Cybern 59(4):291-294, 1998), it was theoretically proven that autoencoders (AE) with single hidden layer (previously called "auto-associative multilayer perceptrons") were, in the best case, implementing singular value decomposit ...

SPRINGER2022

Predicting in Uncertain Environments: Methods for Robust Machine Learning

Paul Thierry Yves Rolland

One of the main goal of Artificial Intelligence is to develop models capable of providing valuable predictions in real-world environments. In particular, Machine Learning (ML) seeks to design such models by learning from examples coming from this same envi ...

EPFL2022