RecordLinkage is a powerful and modular Python toolkit for record linkage and duplicate detection. It provides methods for indexing, comparing records with various similarity measures, and classifying matches, leveraging pandas and numpy for efficient data handling. The library is actively maintained (version 0.16) and suitable for research and linking small to medium-sized datasets.
pip install recordlinkageVerified import paths — ran on the pinned version, not inferred.
This quickstart demonstrates the core steps of record linkage: generating candidate pairs using blocking on a common attribute (city/town), comparing these pairs using string similarity and exact matches, and then classifying likely matches based on the comparison scores.
Review the release notes for version 0.15 to update deprecated class usages and ensure your environment meets the new minimum Python (3.8+) and pandas (1.0+) requirements.
Carefully select blocking keys, considering multiple keys (union of blocks), using more error-tolerant indexing methods (e.g., sorted neighbourhood, phonetic blocking), or applying pre-processing to standardize blocking fields.
Benchmark different string comparison methods for your specific data and choose a balance between accuracy and performance. Prioritize 'jellyfish' for string comparisons and ensure the Rust-backed version is active (check `import jellyfish.rustyfish` does not raise an exception).
Implement robust pre-processing steps using `recordlinkage.standardise` or other data cleaning libraries to normalize names, addresses, dates, and other attributes before attempting to link records.
pip install recordlinkage
Import the class from its correct submodule, for example: `from recordlinkage.index import Block` or `from recordlinkage.compare import String`.
Use a valid string comparison method name, such as 'jarowinkler', 'levenshtein', 'damerau_levenshtein', 'jaro', 'exact', 'qgram', 'cosine', or 'smith_waterman'.