Lecture

Evaluation of Binary Classifiers

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MATH-412: Statistical machine learning

A course on statistical machine learning for supervised and unsupervised learning ...

Description

This lecture covers the evaluation of binary classifiers, focusing on metrics such as recall, sensitivity, specificity, false positive and false negative rates, precision, ROC curves, convex combinations, and the relation between metrics measures and misclassification error. It also delves into the precision-recall curve and performance measures derived from the ROC curve.

Instructor

Yoav Zemel

Official source

Ontological neighbourhood

Statistics

Data analysis: Statistical classification

Related concepts (23)

Positive and negative predictive values

The positive and negative predictive values (PPV and NPV respectively) are the proportions of positive and negative results in statistics and diagnostic tests that are true positive and true negative results, respectively. The PPV and NPV describe the performance of a diagnostic test or other statistical measure. A high result can be interpreted as indicating the accuracy of such a statistic. The PPV and NPV are not intrinsic to the test (as true positive rate and true negative rate are); they depend also on the prevalence.

False positives and false negatives

A false positive is an error in binary classification in which a test result incorrectly indicates the presence of a condition (such as a disease when the disease is not present), while a false negative is the opposite error, where the test result incorrectly indicates the absence of a condition when it is actually present. These are the two kinds of errors in a binary test, in contrast to the two kinds of correct result (a and a ).

Sensitivity and specificity

Sensitivity and specificity mathematically describe the accuracy of a test that reports the presence or absence of a condition. If individuals who have the condition are considered "positive" and those who do not are considered "negative", then sensitivity is a measure of how well a test can identify true positives and specificity is a measure of how well a test can identify true negatives: Sensitivity (true positive rate) is the probability of a positive test result, conditioned on the individual truly being positive.

False positive rate

In statistics, when performing multiple comparisons, a false positive ratio (also known as fall-out or false alarm ratio) is the probability of falsely rejecting the null hypothesis for a particular test. The false positive rate is calculated as the ratio between the number of negative events wrongly categorized as positive (false positives) and the total number of actual negative events (regardless of classification). The false positive rate (or "false alarm rate") usually refers to the expectancy of the false positive ratio.

False discovery rate

In statistics, the false discovery rate (FDR) is a method of conceptualizing the rate of type I errors in null hypothesis testing when conducting multiple comparisons. FDR-controlling procedures are designed to control the FDR, which is the expected proportion of "discoveries" (rejected null hypotheses) that are false (incorrect rejections of the null). Equivalently, the FDR is the expected ratio of the number of false positive classifications (false discoveries) to the total number of positive classifications (rejections of the null).

Related lectures (64)

Linear Regression: Basics

Covers the basics of linear regression, binary and multi-class classification, and evaluation metrics.

Performance Criteria: Confusion Matrix, Recall, Precision, Accuracy

Explores performance criteria in supervised learning, emphasizing precision, recall, and specificity in model evaluation.

Generalized Linear Regression: Classification

Explores Generalized Linear Regression, Classification, confusion matrices, ROC curves, and noise in data.

Performance Evaluation: Bootstrap and Performance Metrics

Explores machine learning model evaluation using leave-one-out, bootstrap, and performance metrics like recall and precision.

Linear Models for Classification: Multi-Class Extensions