Lecture

Evaluating Information Retrieval

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

CS-423: Distributed information systems

This course introduces the foundations of information retrieval, data mining and knowledge bases, which constitute the foundations of today's Web-based distributed information systems. ...

Description

This lecture covers the evaluation of information retrieval models, including the importance of matching user needs, test collections with test queries, recall, precision, F-Measure, precision/recall tradeoff, and measures like MAP. It also explains how to evaluate ranked retrieval systems using precision-recall plots and interpolated precision.

Instructor

Karl Aberer

Karl Aberer received his PhD in mathematics in 1991 from the ETH Zürich. From 1991 to 1992 he was postdoctoral fellow at the International Computer Science Institute (ICSI) at the University of California, Berkeley. In 1992, he joined the Integrated Publication and Information Systems institute (IPSI) of GMD in Germany, where he was leading the research division Open Adaptive Information Management Systems. In 2000 he joined EPFL as full professor. Since 2005 he is the director of the Swiss National Research Center for Mobile Information and Communication Systems ( NCCR-MICS, www.mics.ch ). He is member of the editorial boards of VLDB Journal, ACM Transaction on Autonomous and Adaptive Systems and World Wide Web Journal. He has been consulting for the Swiss government in research and science policy as a member of the Swiss Research and Technology Council ( SWTR ) from 2003 - 2011.

Official source

Related concepts (22)

Information retrieval

Information retrieval (IR) in computing and information science is the process of obtaining information system resources that are relevant to an information need from a collection of those resources. Searches can be based on full-text or other content-based indexing. Information retrieval is the science of searching for information in a document, searching for documents themselves, and also searching for the metadata that describes data, and for databases of texts, images or sounds.

Outer measure

In the mathematical field of measure theory, an outer measure or exterior measure is a function defined on all subsets of a given set with values in the extended real numbers satisfying some additional technical conditions. The theory of outer measures was first introduced by Constantin Carathéodory to provide an abstract basis for the theory of measurable sets and countably additive measures.

Radon measure

In mathematics (specifically in measure theory), a Radon measure, named after Johann Radon, is a measure on the σ-algebra of Borel sets of a Hausdorff topological space X that is finite on all compact sets, outer regular on all Borel sets, and inner regular on open sets. These conditions guarantee that the measure is "compatible" with the topology of the space, and most measures used in mathematical analysis and in number theory are indeed Radon measures.

Lebesgue measure

In measure theory, a branch of mathematics, the Lebesgue measure, named after French mathematician Henri Lebesgue, is the standard way of assigning a measure to subsets of n-dimensional Euclidean space. For n = 1, 2, or 3, it coincides with the standard measure of length, area, or volume. In general, it is also called n-dimensional volume, n''-volume, or simply volume. It is used throughout real analysis, in particular to define Lebesgue integration.

Signed measure

In mathematics, signed measure is a generalization of the concept of (positive) measure by allowing the set function to take negative values. There are two slightly different concepts of a signed measure, depending on whether or not one allows it to take infinite values. Signed measures are usually only allowed to take finite real values, while some textbooks allow them to take infinite values. To avoid confusion, this article will call these two cases "finite signed measures" and "extended signed measures".

Related lectures (56)

Performance Evaluation: Bootstrap and Performance Metrics

Explores machine learning model evaluation using leave-one-out, bootstrap, and performance metrics like recall and precision.

Information Retrieval Basics: Document Frequency and Precision

Introduces information retrieval basics, emphasizing document frequency and precision in evaluating retrieval quality.

Performance Criteria: Confusion Matrix, Recall, Precision, Accuracy

Explores performance criteria in supervised learning, emphasizing precision, recall, and specificity in model evaluation.

Information Retrieval Basics: Document Length and Normalization

Explores document length, normalization, bias compensation, and retrieval model evaluation in information retrieval.

Information Retrieval Basics: Overview and Text-Based Retrieval