Concept

Fonction d'activation

Publications associées (31)

Context-Aware Learning for Generative Models

José del Rocio Millán Ruiz, Ricardo Andres Chavarriaga Lozano, Robert Leeb, Serafeim Perdikis

This work studies the class of algorithms for learning with side-information that emerges by extending generative models with embedded context-related variables. Using finite mixture models (FMMs) as the prototypical Bayesian network, we show that maximum- ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

Large-scale functional network dynamics in human callosal agenesis: Increased subcortical involvement and preserved laterality

Dimitri Nestor Alice Van De Ville, Vanessa Siffredi, Anjali Bagunu Tarun, Younes Farouj

In the human brain, the corpus callosum is the major white-matter commissural tract enabling the transmission of sensory-motor, and higher level cognitive information between homotopic regions of the two cerebral hemispheres. Despite developmental absence ...

ACADEMIC PRESS INC ELSEVIER SCIENCE2021

Product of experts for robot learning from demonstration

Emmanuel Pignat

Adaptability and ease of programming are key features necessary for a wider spread of robotics in factories and everyday assistance. Learning from demonstration (LfD) is an approach to address this problem. It aims to develop algorithms and interfaces such ...

EPFL2021

Deep Neural Networks With Trainable Activations and Controlled Lipschitz Constant

Michaël Unser, Shayan Aziznejad, Harshit Gupta, Joaquim Gonçalves Garcia Barreto Campos

We introduce a variational framework to learn the activation functions of deep neural networks. Our aim is to increase the capacity of the network while controlling an upper-bound of the actual Lipschitz constant of the input-output relation. To that end, ...

2020

Learning Activation Functions in Deep (Spline) Neural Networks

Michaël Unser, Shayan Aziznejad, Pakshal Narendra Bohra, Harshit Gupta, Joaquim Gonçalves Garcia Barreto Campos

We develop an efficient computational solution to train deep neural networks (DNN) with free-form activation functions. To make the problem well-posed, we augment the cost functional of the DNN by adding an appropriate shape regularization: the sum of the ...

2020

Disentangling feature and lazy training in deep neural networks

Matthieu Wyart, Mario Geiger, Stefano Spigler, Arthur Ulysse Jacot-Guillarmod

Two distinct limits for deep learning have been derived as the network width h -> infinity, depending on how the weights of the last layer scale with h. In the neural tangent Kernel (NTK) limit, the dynamics becomes linear in the weights and is described b ...

IOP PUBLISHING LTD2020

Efficient Proximal Mapping of the 1-path-norm of Shallow Networks

Volkan Cevher, Fabian Ricardo Latorre Gomez, Paul Thierry Yves Rolland, Shaul Nadav Hallak

We demonstrate two new important properties of the 1-path-norm of shallow neural networks. First, despite its non-smoothness and non-convexity it allows a closed form proximal operator which can be efficiently computed, allowing the use of stochastic proxi ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2020

ESSOP: Efficient and Scalable Stochastic Outer Product Architecture for Deep Learning

Irem Boybat Kara, Abu Sebastian

Deep neural networks (DNNs) have surpassed human-level accuracy in a variety of cognitive tasks but at the cost of significant memory/time requirements in DNN training. This limits their deployment in energy and memory limited applications that require rea ...

IEEE2020

On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent

Nicolas Henri Bernard Flammarion, Scott William Pesme, Aymeric Daphnis Kévin Dieuleveut

Constant step-size Stochastic Gradient Descent exhibits two phases: a transient phase during which iterates make fast progress towards the optimum, followed by a stationary phase during which iterates oscillate around the optimal point. In this paper, we s ...

2020

Scaling description of generalization with number of parameters in deep learning

Matthieu Wyart, Clément Hongler, Mario Geiger, Franck Raymond Gabriel, Stefano Spigler, Arthur Ulysse Jacot-Guillarmod, Levent Dogus Sagun

Supervised deep learning involves the training of neural networks with a large number N of parameters. For large enough N, in the so-called over-parametrized regime, one can essentially fit the training data points. Sparsitybased arguments would suggest th ...

IOP PUBLISHING LTD2020

On evaluation of thermophysical properties of transformer oil-based nanofluids: A comprehensive modeling and experimental study

Farzaneh Talebkeikhah, Mohsen Talebkeikhah

Transformer oil-based nanofluids are known to have higher thermal conductivity and heat transfer performance compared to conventional transformer oils. In this study, four different types of transformer oil-based nanofluids are synthesized using the well-k ...

ELSEVIER2020