Lecture

Applying the learning bound to kernel regression

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MATH-412: Statistical machine learning

A course on statistical machine learning for supervised and unsupervised learning ...

Description

This lecture applies the main theorem of the course to the case of least square regression in a Reproducing Kernel Hilbert Space (RKHS), discussing the application to LR of the Rademacher bound and the Lipschitz constant of the mapping. The slides cover various aspects of the application, including the family of functions, probability considerations, and the risk associated with the regression. The lecture delves into the complexity of the functions, the interval of values, and the rate of convergence, emphasizing the key concepts of the learning bound and kernel regression.

Instructor

Yoav Zemel

Official source

Related concepts (32)

Risk

In simple terms, risk is the possibility of something bad happening. Risk involves uncertainty about the effects/implications of an activity with respect to something that humans value (such as health, well-being, wealth, property or the environment), often focusing on negative, undesirable consequences. Many different definitions have been proposed. The international standard definition of risk for common understanding in different applications is "effect of uncertainty on objectives".

Linear least squares

Linear least squares (LLS) is the least squares approximation of linear functions to data. It is a set of formulations for solving statistical problems involved in linear regression, including variants for ordinary (unweighted), weighted, and generalized (correlated) residuals. Numerical methods for linear least squares include inverting the matrix of the normal equations and orthogonal decomposition methods. The three main linear least squares formulations are: Ordinary least squares (OLS) is the most common estimator.

Regularized least squares

Regularized least squares (RLS) is a family of methods for solving the least-squares problem while using regularization to further constrain the resulting solution. RLS is used for two main reasons. The first comes up when the number of variables in the linear system exceeds the number of observations. In such settings, the ordinary least-squares problem is ill-posed and is therefore impossible to fit because the associated optimization problem has infinitely many solutions.

Ordinary least squares

In statistics, ordinary least squares (OLS) is a type of linear least squares method for choosing the unknown parameters in a linear regression model (with fixed level-one effects of a linear function of a set of explanatory variables) by the principle of least squares: minimizing the sum of the squares of the differences between the observed dependent variable (values of the variable being observed) in the input dataset and the output of the (linear) function of the independent variable.

Least squares

The method of least squares is a standard approach in regression analysis to approximate the solution of overdetermined systems (sets of equations in which there are more equations than unknowns) by minimizing the sum of the squares of the residuals (a residual being the difference between an observed value and the fitted value provided by a model) made in the results of each individual equation. The most important application is in data fitting.

Related lectures (64)

Confidence Intervals: Definition and Estimation

Explains confidence intervals, parameter estimation methods, and the central limit theorem in statistical inference.

Linear Regression: Estimation and Inference

Explores linear regression estimation, linearity assumptions, and statistical tests in the context of model comparison.

Bayesian Inference: Estimation & Demystification

Covers the concepts of Demystification, Estimation, and Bayesian Inference in the context of Bayesian statistics.

Risk Analysis in Project Management

Explores risk analysis in project management, covering identification, prevention, quantification, and lean management.

Elements of Statistics: Probability and Random Variables