Lecture

Temporality and Entity Resolution

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

CS-422: Database systems

This course is intended for students who want to understand modern large-scale data analysis systems and database systems. It covers a wide range of topics and technologies, and will prepare students to be able to build such systems as well as read and und ...

Description

This lecture covers the challenges of dealing with temporality in data, including the time of data entry and the time when a recorded phenomenon is considered true. It also delves into entity resolution, which involves identifying and merging duplicate entity profiles across datasets. Various techniques such as fuzzy matching, deduplication, and similarity metrics like Jaccard similarity are discussed. The lecture further explores the complexities of duplicate entity detection, data deduplication, and the computational costs involved. Strategies for reducing the computational cost of duplicate detection, such as blocking for candidate selection and q-gram set join, are explained. The session concludes with a summary of entity resolution and data wrangling tasks, emphasizing the importance of optimizations to make clustering more efficient.

Instructor

Anastasia Ailamaki

Expertise Database management systems, scientific applications, computer architecture Teaching & PhD PhD Students Hamish Nicholson, Anna Herlihy, Yi Jiang, Eleni Zapridou, Georgiy Konstantinovich Lebedev, Ioanna Tsakalidou Past EPFL PhD Students Farhan Tauheed, Radu Ioan Stoica, Manos Athanassoulis, Pinar Tözün, Adrian Daniel Popescu, Ioannis Alagiannis, Renata Borovica-Gajic, Danica Porobic, Iraklis Psaroudakis, Manos Karpathiotakis, Matthaios Alexandros Olma, Mirjana Pavlovic, Eleni Tzirita Zacharatou,

Official source

Related concepts (30)

String metric

In mathematics and computer science, a string metric (also known as a string similarity metric or string distance function) is a metric that measures distance ("inverse similarity") between two text strings for approximate string matching or comparison and in fuzzy string searching. A requirement for a string metric (e.g. in contrast to string matching) is fulfillment of the triangle inequality. For example, the strings "Sam" and "Samuel" can be considered to be close.

Data wrangling

Data wrangling, sometimes referred to as data munging, is the process of transforming and mapping data from one "raw" data form into another format with the intent of making it more appropriate and valuable for a variety of downstream purposes such as analytics. The goal of data wrangling is to assure quality and useful data. Data analysts typically spend the majority of their time in the process of data wrangling compared to the actual analysis of the data.

Hierarchical clustering

In data mining and statistics, hierarchical clustering (also called hierarchical cluster analysis or HCA) is a method of cluster analysis that seeks to build a hierarchy of clusters. Strategies for hierarchical clustering generally fall into two categories: Agglomerative: This is a "bottom-up" approach: Each observation starts in its own cluster, and pairs of clusters are merged as one moves up the hierarchy. Divisive: This is a "top-down" approach: All observations start in one cluster, and splits are performed recursively as one moves down the hierarchy.

Cluster analysis

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense) to each other than to those in other groups (clusters). It is a main task of exploratory data analysis, and a common technique for statistical data analysis, used in many fields, including pattern recognition, , information retrieval, bioinformatics, data compression, computer graphics and machine learning.

Principle of relativity

In physics, the principle of relativity is the requirement that the equations describing the laws of physics have the same form in all admissible frames of reference. For example, in the framework of special relativity the Maxwell equations have the same form in all inertial frames of reference. In the framework of general relativity the Maxwell equations or the Einstein field equations have the same form in arbitrary frames of reference.

Related lectures (44)

Graph Coloring II

Explores advanced graph coloring concepts, including planted coloring, rigidity threshold, and frozen variables in BP fixed points.

Advanced Structure Discovery: Distance Metrics and Time Series Data

Explores clustering algorithms, distance metrics, and time series data analysis techniques.

Data Wrangling and Analysis

Covers a homework assignment on data wrangling and analysis using Python's pandas library for real-world datasets.

Data Modeling: Concepts and Applications

Explores data modeling concepts, SQL implementations, and practical applications in handling missing data.

Clustering: K-Means