Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Evaluation of Binary Classifiers

In course

MATH-412: Statistical machine learning

A course on statistical machine learning for supervised and unsupervised learning ...

Description

This lecture covers the evaluation of binary classifiers, focusing on metrics such as recall, sensitivity, specificity, false positive and false negative rates, precision, ROC curves, convex combinations, and the relation between metrics measures and misclassification error. It also delves into the precision-recall curve and performance measures derived from the ROC curve.

Instructor

Yoav Zemel

Official source

Ontological neighbourhood

Statistics

Data analysis: Statistical classification

Related concepts (23)

Sensitivity and specificity

Sensitivity and specificity mathematically describe the accuracy of a test that reports the presence or absence of a condition. If individuals who have the condition are considered "positive" and those who do not are considered "negative", then sensitivity is a measure of how well a test can identify true positives and specificity is a measure of how well a test can identify true negatives: Sensitivity (true positive rate) is the probability of a positive test result, conditioned on the individual truly being positive.

Positive and negative predictive values

The positive and negative predictive values (PPV and NPV respectively) are the proportions of positive and negative results in statistics and diagnostic tests that are true positive and true negative results, respectively. The PPV and NPV describe the performance of a diagnostic test or other statistical measure. A high result can be interpreted as indicating the accuracy of such a statistic. The PPV and NPV are not intrinsic to the test (as true positive rate and true negative rate are); they depend also on the prevalence.

False positives and false negatives

A false positive is an error in binary classification in which a test result incorrectly indicates the presence of a condition (such as a disease when the disease is not present), while a false negative is the opposite error, where the test result incorrectly indicates the absence of a condition when it is actually present. These are the two kinds of errors in a binary test, in contrast to the two kinds of correct result (a and a ).

Binary classification

Binary classification is the task of classifying the elements of a set into two groups (each called class) on the basis of a classification rule. Typical binary classification problems include: Medical testing to determine if a patient has certain disease or not; Quality control in industry, deciding whether a specification has been met; In information retrieval, deciding whether a page should be in the result set of a search or not. Binary classification is dichotomization applied to a practical situation.

Precision and recall

In pattern recognition, information retrieval, object detection and classification (machine learning), precision and recall are performance metrics that apply to data retrieved from a collection, corpus or sample space. Precision (also called positive predictive value) is the fraction of relevant instances among the retrieved instances. Written as a formula:. Recall (also known as sensitivity) is the fraction of relevant instances that were retrieved. Written as a formula: . Both precision and recall are therefore based on relevance.

Related lectures (34)

Generalized Linear Regression: Classification

Explores Generalized Linear Regression, Classification, confusion matrices, ROC curves, and noise in data.

Fairness in Machine Learning

Explores sources of unfairness in machine learning, the importance of fairness metrics, and evaluating model predictions using various fairness metrics.

Quantifying Performance: Misclassification and F-Measure

Covers quantifying performance through true positives, false negatives, and false positives in machine learning.

Classification: Decision Trees and kNN

Introduces decision trees and k-nearest neighbors for classification tasks, exploring metrics like accuracy and AUC.