Lecture

Boosting: Adaboost Algorithm

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MATH-412: Statistical machine learning

A course on statistical machine learning for supervised and unsupervised learning ...

Description

This lecture covers the concept of boosting, focusing on the Adaboost algorithm and forward stagewise additive modeling. It explains the derivation of Adaboost as FSAM on the exponential loss, boosted trees, and interpretations of Adaboost. The lecture also discusses the solutions for B and G, comparing with Adaboost, boosting of trees, and general gradient tree boosting for classification.

Instructor

Yoav Zemel

Official source

Ontological neighbourhood

Information engineering

Machine learning: Decision tree learning

Related concepts (27)

Boosting (machine learning)

In machine learning, boosting is an ensemble meta-algorithm for primarily reducing bias, and also variance in supervised learning, and a family of machine learning algorithms that convert weak learners to strong ones. Boosting is based on the question posed by Kearns and Valiant (1988, 1989): "Can a set of weak learners create a single strong learner?" A weak learner is defined to be a classifier that is only slightly correlated with the true classification (it can label examples better than random guessing).

Decision tree

A decision tree is a decision support hierarchical model that uses a tree-like model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm that only contains conditional control statements. Decision trees are commonly used in operations research, specifically in decision analysis, to help identify a strategy most likely to reach a goal, but are also a popular tool in machine learning.

XGBoost

XGBoost (eXtreme Gradient Boosting) is an open-source software library which provides a regularizing gradient boosting framework for C++, Java, Python, R, Julia, Perl, and Scala. It works on Linux, Windows, and macOS. From the project description, it aims to provide a "Scalable, Portable and Distributed Gradient Boosting (GBM, GBRT, GBDT) Library". It runs on a single machine, as well as the distributed processing frameworks Apache Hadoop, Apache Spark, Apache Flink, and Dask.

Logistic regression

In statistics, the logistic model (or logit model) is a statistical model that models the probability of an event taking place by having the log-odds for the event be a linear combination of one or more independent variables. In regression analysis, logistic regression (or logit regression) is estimating the parameters of a logistic model (the coefficients in the linear combination).

Discrete choice

In economics, discrete choice models, or qualitative choice models, describe, explain, and predict choices between two or more discrete alternatives, such as entering or not entering the labor market, or choosing between modes of transport. Such choices contrast with standard consumption models in which the quantity of each good consumed is assumed to be a continuous variable. In the continuous case, calculus methods (e.g. first-order conditions) can be used to determine the optimum amount chosen, and demand can be modeled empirically using regression analysis.

Related lectures (64)

Regression Trees and Ensemble Methods in Machine Learning

Discusses regression trees, ensemble methods, and their applications in predicting used car prices and stock returns.

Structured Classifications: Decision Trees and Boosting

Explores decision trees, overfitting elimination, boosting techniques, and their practical applications in predictive modeling.

Decision Trees and Random Forests: Concepts and Applications

Discusses decision trees and random forests, focusing on their structure, optimization, and application in regression and classification tasks.

Decision Trees and Boosting

Explores decision trees in machine learning, their flexibility, impurity criteria, and introduces boosting methods like Adaboost.

Decision Trees: Induction & Attributes