Publications related to High-dimensional Asymptotics of Denoising Autoencoders

Related publications (15)

Deep neural networks for choice analysis: Enhancing behavioral regularity with gradient regularization

Deep neural networks (DNNs) have been increasingly applied in travel demand modeling because of their automatic feature learning, high predictive performance, and economic interpretability. Nevertheless, DNNs frequently present behaviorally irregular patte ...

PERGAMON-ELSEVIER SCIENCE LTD2024

Spatiotemporal Self-supervised Learning for Point Clouds in the Wild

Sabine Süsstrunk, Mathieu Salzmann, Tong Zhang

Self-supervised learning (SSL) has the potential to ben- efit many applications, particularly those where manually annotating data is cumbersome. One such situation is the semantic segmentation of point clouds. In this context, ex- isting methods employ co ...

IEEE2023

New insights into rodent brain microstructure and metabolism in hepatic encephalopathy

Jessie Julie Mosso

Type C hepatic encephalopathy (HE) is a severe neuropsychiatric complication of chronic liver disease, for which the prognosis is poor in the absence of liver transplantation. Cirrhosis in type C HE leads to a toxic accumulation of ammonia in the blood, wh ...

EPFL2023

MP-PCA denoising for diffusion MRS data: promises and pitfalls

Cristina Ramona Cudalbu Moldovan, Dunja Simicic, Jessie Julie Mosso

Diffusion-weighted (DW) magnetic resonance spectroscopy (MRS) suffers from a lower signal to noise ratio (SNR) compared to conventional MRS owing to the addition of diffusion attenuation. This technique can therefore strongly benefit from noise reduction s ...

ACADEMIC PRESS INC ELSEVIER SCIENCE2022

A Johnson-Lindenstrauss Framework for Randomly Initialized CNNs

Michael Christoph Gastpar, Jan Hazla, Ido Nachum

How does the geometric representation of a dataset change after the application of each randomly initialized layer of a neural network? The celebrated Johnson-Lindenstrauss lemma answers this question for linear fully-connected neural networks (FNNs), stat ...

2022

Perturbative construction of mean-field equations in extensive-rank matrix factorization and denoising

Lenka Zdeborová, Florent Gérard Krzakala

Factorization of matrices where the rank of the two factors diverges linearly with their sizes has many applications in diverse areas such as unsupervised representation learning, dictionary learning or sparse coding. We consider a setting where the two fa ...

2022

Binary Perceptron: Efficient Algorithms Can Find Solutions in a RareWell-Connected Cluster

Emmanuel Abbé

It was recently shown that almost all solutions in the symmetric binary perceptron are isolated, even at low constraint densities, suggesting that finding typical solutions is hard. In contrast, some algorithms have been shown empirically to succeed in fin ...

ASSOC COMPUTING MACHINERY2022

Contextual Semi-Supervised Learning: An Approach To Leverage Air-Surveillance and Untranscribed ATC Data in ASR Systems

Petr Motlicek, Juan Pablo Zuluaga Gomez

Air traffic management and specifically air-traffic control (ATC) rely mostly on voice communications between Air Traffic Controllers (ATCos) and pilots. In most cases, these voice communications follow a well-defined grammar that could be leveraged in Aut ...

ISCA-INT SPEECH COMMUNICATION ASSOC2021

A unified framework for Hamiltonian deep neural networks

Giancarlo Ferrari Trecate, Liang Xu, Clara Lucía Galimberti

Training deep neural networks (DNNs) can be difficult due to the occurrence of vanishing/explod- ing gradients during weight optimization. To avoid this problem, we propose a class of DNNs stemming from the time discretization of Hamiltonian systems. The t ...

2021

On some graph-based two-sample tests for high dimension, low sample size data

Soham Sarkar

Testing for equality of two high-dimensional distributions is a challenging problem, and this becomes even more challenging when the sample size is small. Over the last few decades, several graph-based two-sample tests have been proposed in the literature, ...

SPRINGER2020

Joint Segmentation and Path Classification of Curvilinear Structures

Pascal Fua, Mateusz Kozinski, Agata Justyna Mosinska

Detection of curvilinear structures in images has long been of interest. One of the most challenging aspects of this problem is inferring the graph representation of the curvilinear network. Most existing delineation approaches first perform binary segment ...

2020

Phases of two-dimensional spinless lattice fermions with first-quantized deep neural-network quantum states

Giuseppe Carleo

First-quantized deep neural network techniques are developed for analyzing strongly coupled fermionic systems on the lattice. Using a Slater-Jastrow-inspired ansatz which exploits deep residual networks with convolutional residual blocks, we approximately ...

AMER PHYSICAL SOC2020

Entropy and mutual information in models of deep neural networks

Nicolas Macris, Florent Gérard Krzakala, Lenka Zdeborová, Jean François Emmanuel Barbier, Clément Dominique Luneau

We examine a class of stochastic deep learning models with a tractable method to compute information-theoretic quantities. Our contributions are three-fold: (i) we show how entropies and mutual informations can be derived from heuristic statistical physics ...

2019

Towards End-to-End Speech Recognition

Dimitri Palaz

Standard automatic speech recognition (ASR) systems follow a divide and conquer approach to convert speech into text. Alternately, the end goal is achieved by a combination of sub-tasks, namely, feature extraction, acoustic modeling and sequence decoding, ...

EPFL2016

Parallel retrieval of correlated patterns: From Hopfield networks to Boltzmann machines

Andrea De Antoni

In this work, we first revise some extensions of the standard Hopfield model in the low storage limit, namely the correlated attractor case and the multitasking case recently introduced by the authors. The former case is based on a modification of the Hebb ...

Elsevier2013