rank-bm25

library0.2.2pypypi✓ verified 27d ago

Provides various BM25 algorithms (BM25Okapi, BM25L, BM25Plus) for document ranking based on a corpus of tokenized documents. It's currently at version 0.2.2 and appears to have an infrequent release cadence, with the latest update adding support for non-iterable corpuses.

ai-ml data

pip install rank-bm25

github ↗package ↗

INSTALL

IMPORT

SIG · RANK-BM25

rank-bm25

ai-mlpythonv0.2.2

Install

3.6s avg

Import

280ms

Disk

89MB

Pass rate

10/ 10

Env Coverage10 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v0.2.2 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.95 runs

installs and imports cleanly · install 0.0s · import 0.278s · 89.4MB

glibc

py 3.10–3.95 runs

installs and imports cleanly · install 3.6s · import 0.282s · 86MB

89MB installed

● package 89MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

BM25Okapi

✓ from rank_bm25 import BM25Okapi

✗ from rankbm25 import BM25Okapi

The PyPI package is 'rank-bm25' but the module name for import is 'rank_bm25' (with an underscore).

BM25L

✓ from rank_bm25 import BM25L

✗ from rankbm25 import BM25L

The PyPI package is 'rank-bm25' but the module name for import is 'rank_bm25' (with an underscore).

BM25Plus

✓ from rank_bm25 import BM25Plus

✗ from rankbm25 import BM25Plus

The PyPI package is 'rank-bm25' but the module name for import is 'rank_bm25' (with an underscore).

This example demonstrates how to initialize BM25Okapi with a tokenized corpus and then retrieve scores and top-N documents for a given tokenized query.

from rank_bm25 import BM25Okapi corpus = [ "Hello there, this is a document.", "This document is about BM25.", "Hello, how are you today?", "BM25 is a ranking algorithm.", ] # Tokenize the corpus (essential step) tokenized_corpus = [doc.lower().split(" ") for doc in corpus] bm25 = BM25Okapi(tokenized_corpus) query = "BM25 ranking algorithm" tokenized_query = query.lower().split(" ") doc_scores = bm25.get_scores(tokenized_query) print(f"Document scores: {doc_scores}") top_n = bm25.get_top_n(tokenized_query, corpus, n=2) print(f"Top 2 documents: {top_n}")

Debug

Known issues

gotchaThe BM25 algorithms expect a pre-tokenized corpus (a list of lists of strings) and tokenized queries, not raw strings. Each sub-list represents a document's tokens.

fix

Ensure your corpus (list of documents) and queries are tokenized into lists of words (e.g., using `doc.split(' ')` or an NLP tokenizer) before passing them to the BM25 constructor or `get_scores`/`get_top_n` methods.

affects: All versions

gotchaThe PyPI package name for installation is `rank-bm25` (with a hyphen), but the Python module you import into your code is `rank_bm25` (with an underscore).

fix

Use `pip install rank-bm25` for installation and `from rank_bm25 import ...` for importing classes like `BM25Okapi`.

affects: All versions

gotchaPrior to version 0.2.2, passing non-iterable corpuses (e.g., generators) to the BM25 constructor was not officially supported and could lead to unexpected behavior or errors. While 0.2.2 added support, be mindful of generator behavior.

fix

For versions prior to 0.2.2, ensure your corpus is a concrete list of tokenized documents. For 0.2.2+, generators are supported, but remember they are single-pass and may need to be re-initialized if reused.

affects: < 0.2.2

Upgrade

Version history

0.2.2latest on PyPI · released Feb 16, 2022

Audit

Dependencies

No dependency data recorded yet.

Agent activity

18 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/dorianbrown/rank_bm25 ↗

packagehttps://pypi.org/project/rank-bm25/ ↗