Publication

Analysis of Learning a Flow-Based Generative Model From Limited Sample Complexity

2024

Abstract

We study the problem of training a flow-based generative model, parametrized by a two-layer autoencoder, to sample from a high-dimensional Gaussian mixture. We provide a sharp end-to-end analysis of the problem. First, we provide a tight closed-form characterization of the learnt velocity field, when parametrized by a shallow denoising auto-encoder trained on a finite number n of samples from the target distribution. Building on this analysis, we provide a sharp description of the corresponding generative flow, which pushes the base Gaussian density forward to an approximation of the target density. In particular, we provide closed-form formulae for the distance between the means of the generated mixture and the mean of the target mixture, which we show decays as Θ n (1 /n). Finally, this rate is shown to be in fact Bayes-optimal.

Official source

https://infoscience.epfl.ch/entities/publication/912c5bd2-d0a1-4beb-b396-3f035d71d982

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

2024

Abstract

Official source

https://infoscience.epfl.ch/entities/publication/912c5bd2-d0a1-4beb-b396-3f035d71d982

About this result

Related concepts (32)

Autoencoder

An autoencoder is a type of artificial neural network used to learn efficient codings of unlabeled data (unsupervised learning). An autoencoder learns two functions: an encoding function that transforms the input data, and a decoding function that recreates the input data from the encoded representation. The autoencoder learns an efficient representation (encoding) for a set of data, typically for dimensionality reduction. Variants exist, aiming to force the learned representations to assume useful properties.

Mixture model

In statistics, a mixture model is a probabilistic model for representing the presence of subpopulations within an overall population, without requiring that an observed data set should identify the sub-population to which an individual observation belongs. Formally a mixture model corresponds to the mixture distribution that represents the probability distribution of observations in the overall population.

Generative adversarial network

A generative adversarial network (GAN) is a class of machine learning framework and a prominent framework for approaching generative AI. The concept was initially developed by Ian Goodfellow and his colleagues in June 2014. In a GAN, two neural networks contest with each other in the form of a zero-sum game, where one agent's gain is another agent's loss. Given a training set, this technique learns to generate new data with the same statistics as the training set.

Related publications (20)

Analysis of Learning a Flow-Based Generative Model From Limited Sample Complexity

Chat with Graph Search

Observation of the Singly Cabibbo Suppressed Decay D0 →b1 (1235)-e+νe and Evidence for D+ →b1 (1235)0e+νe

Search for the Lepton Number Violating Process j/ψ = k Plus k Plus e- e- Plus c.c.*

Fundamental Limits and Solvable Models of Feature Learning in High-Dimensional Models and Attention Networks