Lecture

Optimization Problems: Gradient Descent Algorithm

Description

This lecture introduces a general formulation for optimization problems, which can be applied to linear regression, logistic regression, and support vector machines. By mapping these machine learning problems to this formulation, one can utilize the gradient descent algorithm to iteratively move in the opposite direction of the gradient until reaching the function's minimum.

In MOOC

IoT Systems and Industrial Applications with Design Thinking

Official source

Ontological neighbourhood

Statistics

Data analysis: Regression analysis

Related concepts (24)

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

Logistic regression

In statistics, the logistic model (or logit model) is a statistical model that models the probability of an event taking place by having the log-odds for the event be a linear combination of one or more independent variables. In regression analysis, logistic regression (or logit regression) is estimating the parameters of a logistic model (the coefficients in the linear combination).

Nonlinear regression

In statistics, nonlinear regression is a form of regression analysis in which observational data are modeled by a function which is a nonlinear combination of the model parameters and depends on one or more independent variables. The data are fitted by a method of successive approximations. In nonlinear regression, a statistical model of the form, relates a vector of independent variables, , and its associated observed dependent variables, . The function is nonlinear in the components of the vector of parameters , but otherwise arbitrary.

Bayesian linear regression

Bayesian linear regression is a type of conditional modeling in which the mean of one variable is described by a linear combination of other variables, with the goal of obtaining the posterior probability of the regression coefficients (as well as other parameters describing the distribution of the regressand) and ultimately allowing the out-of-sample prediction of the regressand (often labelled ) conditional on observed values of the regressors (usually ).

Multinomial logistic regression

In statistics, multinomial logistic regression is a classification method that generalizes logistic regression to multiclass problems, i.e. with more than two possible discrete outcomes. That is, it is a model that is used to predict the probabilities of the different possible outcomes of a categorically distributed dependent variable, given a set of independent variables (which may be real-valued, binary-valued, categorical-valued, etc.).

Related lectures (35)

Optimization in Machine Learning: Gradient Descent

Covers optimization in machine learning, focusing on gradient descent for linear and logistic regression, stochastic gradient descent, and practical considerations.

Linear and Logistic Regression

Covers linear and logistic regression, including underfitting, overfitting, and performance metrics.

Linear Regression: Statistical Inference and Regularization

Covers the probabilistic model for linear regression and the importance of regularization techniques.

Overfitting, Cross-validation, Regularization

Explores overfitting, cross-validation, and regularization in machine learning, emphasizing model complexity and the importance of regularization strength.

Linear Regression: Basics and Applications