Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Projection Pursuit Regression: Nonlinear Modeling and Interpretability

Description

This lecture introduces Projection Pursuit Regression, a method inspired by tomography, for modeling nonlinear relationships in data. It discusses the decomposition of functions into smooth components and the challenges of interpretability when embracing nonlinearity. The instructor explains the fitting process, the trade-offs between interpretability and complexity, and the use of neural networks for regression. The lecture concludes with a discussion on the computational aspects of fitting complex models and the implications for data interpretation.

Official source

Related concepts (32)

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

Computational complexity

In computer science, the computational complexity or simply complexity of an algorithm is the amount of resources required to run it. Particular focus is given to computation time (generally measured by the number of needed elementary operations) and memory storage requirements. The complexity of a problem is the complexity of the best algorithms that allow solving the problem. The study of the complexity of explicitly given algorithms is called analysis of algorithms, while the study of the complexity of problems is called computational complexity theory.

Computational complexity theory

In theoretical computer science and mathematics, computational complexity theory focuses on classifying computational problems according to their resource usage, and relating these classes to each other. A computational problem is a task solved by a computer. A computation problem is solvable by mechanical application of mathematical steps, such as an algorithm. A problem is regarded as inherently difficult if its solution requires significant resources, whatever the algorithm used.

Neural network

A neural network can refer to a neural circuit of biological neurons (sometimes also called a biological neural network), a network of artificial neurons or nodes in the case of an artificial neural network. Artificial neural networks are used for solving artificial intelligence (AI) problems; they model connections of biological neurons as weights between nodes. A positive weight reflects an excitatory connection, while negative values mean inhibitory connections. All inputs are modified by a weight and summed.

Nonlinear regression

In statistics, nonlinear regression is a form of regression analysis in which observational data are modeled by a function which is a nonlinear combination of the model parameters and depends on one or more independent variables. The data are fitted by a method of successive approximations. In nonlinear regression, a statistical model of the form, relates a vector of independent variables, , and its associated observed dependent variables, . The function is nonlinear in the components of the vector of parameters , but otherwise arbitrary.

Related lectures (40)

Nonlinear ML Algorithms

Introduces nonlinear ML algorithms, covering nearest neighbor, k-NN, polynomial curve fitting, model complexity, overfitting, and regularization.

Generalization in Deep Learning

Explores generalization in deep learning, covering model complexity, implicit bias, and the double descent phenomenon.

Overfitting: Symptoms and Characteristics

Explores overfitting in polynomial regression, emphasizing the importance of generalization in machine learning and statistics.