The `simhash` library provides a Python implementation of the Simhash Algorithm, a technique for quickly finding near-duplicate documents or comparing the similarity of two texts or data objects. It's highly useful for tasks like large-scale content deduplication, spam detection, and content recommendation, offering a fast way to identify perceptually similar items. The current version is 2.1.2, and it follows an irregular release cadence based on contributions and bug fixes.
pip install simhashVerified import paths — ran on the pinned version, not inferred.
This quickstart demonstrates how to create `Simhash` objects from strings and calculate the Hamming distance between them. A smaller distance indicates greater similarity. The choice of similarity threshold depends on your specific application.
Implement a custom callable function for feature extraction that takes a string and returns an iterable of features (strings or numbers), then pass it as `f=your_function` to the `Simhash` constructor.
To compare the actual hash values, use `simhash_obj_a.value == simhash_obj_b.value`. To measure similarity, use `simhash_obj_a.distance(simhash_obj_b)` which returns the Hamming distance.
Interpret `distance(other)` results carefully: 0 means identical hashes, while higher numbers mean less similar. Define an appropriate distance threshold for your application's 'similar' definition (e.g., distance < 3).
Run `pip install simhash` in your terminal to install the library.
Ensure the `f` parameter is assigned a function that takes a string and returns an iterable of features, e.g., `Simhash(text, f=my_feature_extractor_function)`.
Check the input string for content. If using a custom `f` function, ensure it correctly processes the input and returns at least one feature for non-empty or meaningful inputs. For example, your tokenizer might be too aggressive, removing all tokens.
No dependency data recorded yet.