marin-dupekit

library0.1.1pypypi✓ verified 80d ago

Marin Dupekit is a Rust-accelerated Python library for text de-duplication, optimized for performance with large datasets. Current version 0.1.1, active development with monthly releases.

devops

pip install marin-dupekit

github ↗homepage ↗

INSTALL

IMPORT

SIG · MARIN-DUPEKIT

marin-dupekit

devopspythonv0.1.1

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

Bloom

✓ from dupekit import Bloom

✗ from marin_dupekit import deduplicate

Document

✓ from dupekit import Document

process_native

✓ from dupekit import process_native

Basic deduplication with default settings.

from marin_dupekit import deduplicate texts = [ "Hello world", "Hello world!", "Goodbye world", ] results = deduplicate(texts, threshold=0.8) print(results)

Debug

Known issues

breakingThe function signature changed in 0.1.0: 'threshold' is now required (no default).

fix

Update calls: deduplicate(texts, threshold=0.8)

affects: <0.1.0

gotchaInput texts must be strings, other types cause TypeError.

fix

Convert inputs to str before calling deduplicate.

affects: all

Upgrade

Version history

0.1.1latest on PyPI · released May 7, 2026

Audit

Dependencies

No dependency data recorded yet.

Agent activity

2 hits · last 30 days

node

Resources

githubhttps://github.com/marin-community/marin ↗

homepagehttps://github.com/marin-community/marin ↗