Lecture

Kernel Density Estimation: Bandwidth Selection and Curse of Dimensionality

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture covers Kernel Density Estimation (KDE) focusing on the bandwidth selection and the curse of dimensionality. It explains the smoothed plug-in estimator, the kernel density estimator, and the asymptotic risk of KDE. The lecture discusses the bias-variance tradeoff, optimal rates for KDE, and bandwidth selection methods like pilot estimator and least squares cross-validation. It also explores the estimation of integrated mean squared error and the leave-one-out cross-validation estimator. Additionally, it delves into the generalization of KDE to higher dimensions, emphasizing the importance of bandwidth selection. The lecture concludes with a comparison between parametric and nonparametric models, highlighting the tradeoff between flexibility and efficiency.

Official source

Ontological neighbourhood

Statistics

Statistical inference: Mathematical statistics

Related concepts (32)

Nonparametric statistics

Nonparametric statistics is the type of statistics that is not restricted by assumptions concerning the nature of the population from which a sample is drawn. This is opposed to parametric statistics, for which a problem is restricted a priori by assumptions concerning the specific distribution of the population (such as the normal distribution) and parameters (such the mean or variance).

Kernel density estimation

In statistics, kernel density estimation (KDE) is the application of kernel smoothing for probability density estimation, i.e., a non-parametric method to estimate the probability density function of a random variable based on kernels as weights. KDE answers a fundamental data smoothing problem where inferences about the population are made, based on a finite data sample. In some fields such as signal processing and econometrics it is also termed the Parzen–Rosenblatt window method, after Emanuel Parzen and Murray Rosenblatt, who are usually credited with independently creating it in its current form.

Multivariate kernel density estimation

Kernel density estimation is a nonparametric technique for density estimation i.e., estimation of probability density functions, which is one of the fundamental questions in statistics. It can be viewed as a generalisation of histogram density estimation with improved statistical properties. Apart from histograms, other types of density estimators include parametric, spline, wavelet and Fourier series. Kernel density estimators were first introduced in the scientific literature for univariate data in the 1950s and 1960s and subsequently have been widely adopted.

Parametric model

In statistics, a parametric model or parametric family or finite-dimensional model is a particular class of statistical models. Specifically, a parametric model is a family of probability distributions that has a finite number of parameters. A statistical model is a collection of probability distributions on some sample space. We assume that the collection, P, is indexed by some set Θ. The set Θ is called the parameter set or, more commonly, the parameter space.

Semiparametric model

In statistics, a semiparametric model is a statistical model that has parametric and nonparametric components. A statistical model is a parameterized family of distributions: indexed by a parameter . A parametric model is a model in which the indexing parameter is a vector in -dimensional Euclidean space, for some nonnegative integer . Thus, is finite-dimensional, and . With a nonparametric model, the set of possible values of the parameter is a subset of some space , which is not necessarily finite-dimensional.

Related lectures (64)

Nonparametric Statistics: Estimation and Optimization

Explores nonparametric statistics, covering estimation methods and the bias-variance tradeoff.

Nonparametric and Bayesian Statistics

Covers nonparametric statistics, kernel density estimation, Bayesian principles, and posterior distribution summarization.

Model Selection Criteria: AIC, BIC, Cp

Explores model selection criteria like AIC, BIC, and Cp in statistics for data science.

Basic Principles of Point Estimation

Explores the Method of Moments, Bias-Variance tradeoff, Consistency, Plug-In Principle, and Likelihood Principle in point estimation.

Nonparametric Regression