Matthieu Wyart, Mario Geiger, Leonardo Petrini, Jonas Paccolat
We study how neural networks compress uninformative input space in models where data lie in d dimensions, but the labels of which only vary within a linear manifold of dimension d(parallel to) < d. We show that for a one-hidden-layer network initialized wi ...
IOP PUBLISHING LTD2021