Lecture

Optimization Problems: Gradient Descent Algorithm

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture introduces a general formulation for optimization problems, which can be applied to linear regression, logistic regression, and support vector machines. By mapping these machine learning problems to this formulation, one can utilize the gradient descent algorithm to iteratively move in the opposite direction of the gradient until reaching the function's minimum.

In MOOC

IoT Systems and Industrial Applications with Design Thinking

Internet of Things (IoT) and smart connected devices have radically changed the way our world works and how companies operate and create new businesses. Do you want to learn about IoT fundamentals? This is a course for you! In this course we provide an overview of the underlaying IoT technologies and its business applications. This course is an introduction to IoT and explains why it is a continuously evolving concept with many industrial applications. You will learn the fundamental building blocks of IoT

Official source

Ontological neighbourhood

Statistics

Data analysis: Regression analysis

Related concepts (24)

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

Logistic regression

In statistics, the logistic model (or logit model) is a statistical model that models the probability of an event taking place by having the log-odds for the event be a linear combination of one or more independent variables. In regression analysis, logistic regression (or logit regression) is estimating the parameters of a logistic model (the coefficients in the linear combination).

Nonlinear regression

In statistics, nonlinear regression is a form of regression analysis in which observational data are modeled by a function which is a nonlinear combination of the model parameters and depends on one or more independent variables. The data are fitted by a method of successive approximations. In nonlinear regression, a statistical model of the form, relates a vector of independent variables, , and its associated observed dependent variables, . The function is nonlinear in the components of the vector of parameters , but otherwise arbitrary.

Bayesian linear regression

Bayesian linear regression is a type of conditional modeling in which the mean of one variable is described by a linear combination of other variables, with the goal of obtaining the posterior probability of the regression coefficients (as well as other parameters describing the distribution of the regressand) and ultimately allowing the out-of-sample prediction of the regressand (often labelled ) conditional on observed values of the regressors (usually ).

Multinomial logistic regression

In statistics, multinomial logistic regression is a classification method that generalizes logistic regression to multiclass problems, i.e. with more than two possible discrete outcomes. That is, it is a model that is used to predict the probabilities of the different possible outcomes of a categorically distributed dependent variable, given a set of independent variables (which may be real-valued, binary-valued, categorical-valued, etc.).

Related lectures (64)

Optimization in Machine Learning: Gradient Descent

Covers optimization in machine learning, focusing on gradient descent for linear and logistic regression, stochastic gradient descent, and practical considerations.

Classification Algorithms: Generative and Discriminative Approaches

Explores generative and discriminative classification algorithms, emphasizing their applications and differences in machine learning tasks.

Linear and Logistic Regression

Covers linear and logistic regression, including underfitting, overfitting, and performance metrics.

Cross-Validation: Techniques and Applications

Explores cross-validation, overfitting, regularization, and regression techniques in machine learning.

Linear Regression and Logistic Regression