Registry / ai-ml / keybert

keybert

JSON →
library0.9.0pypypi✓ verified 22d ago

KeyBERT is a minimal and easy-to-use Python library for keyword extraction that leverages state-of-the-art BERT embeddings to identify keywords and keyphrases most similar to a given document. Currently at version 0.9.0, it maintains an active release cadence with frequent updates improving performance, adding new features like LLM integration, and extending model backend support.

pip install keybert
INSTALL
IMPORT
SIG · KEYBERT
K
keybert
ai-mlpythonv0.9.0
Install
Import
Disk
Pass rate
0/ 10
Env Coverage0 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v0.9.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
glibc
py 3.10
✕ build_error
1/3 runs
py 3.11
✕ build_error
1/3 runs
py 3.12
✕ build_error
1/3 runs
py 3.13
✕ build_error
2/3 runs
py 3.9
✕ build_error
1/3 runs
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

KeyBERT
from keybert import KeyBERT
KeyLLM
from keybert import KeyLLM
from keybert.llm import KeyLLM
KeyLLM is typically imported directly from keybert for convenience, though its components like OpenAI backend are in keybert.llm.
OpenAI
from keybert.llm import OpenAI
from keybert import OpenAI
OpenAI LLM integration class is specifically located in the keybert.llm submodule.

Initialize the KeyBERT model and use the `extract_keywords` method to retrieve relevant keywords from a document. The `top_n` parameter controls the number of keywords returned. Further parameters like `keyphrase_ngram_range`, `stop_words`, `use_mmr`, and `diversity` can be used to customize and diversify the extraction results.

from keybert import KeyBERT doc = """ Supervised learning is the machine learning task of learning a function that maps an input to an output based on example input-output pairs. It infers a function from labeled training data consisting of a set of training examples. In supervised learning, each example is a pair consisting of an input object (typically a vector) and a desired output value (also called the supervisory signal). """ kw_model = KeyBERT() keywords = kw_model.extract_keywords(doc, top_n=5) print(keywords) # Example with diversification (Maximal Marginal Relevance) keywords_mmr = kw_model.extract_keywords(doc, keyphrase_ngram_range=(1, 3), stop_words='english', use_mmr=True, diversity=0.7, top_n=5) print(keywords_mmr)
Debug
Known issues
breakingSupport for Python versions 3.6 and 3.7 was dropped in KeyBERT version 0.8.5. Users on older Python versions must upgrade to Python 3.8 or newer.
fix
Upgrade your Python environment to version 3.8 or higher.
affects: >=0.8.5
breakingKeyBERT's `KeyLLM` integration with the OpenAI API required updates for `openai>=1`. Older `openai` library versions (e.g., pre-1.0) are incompatible.
fix
Ensure your `openai` package is version 1.0 or higher when using `KeyLLM` with OpenAI models. (e.g., `pip install openai>=1`).
affects: >=0.8.3
gotchaFor large datasets or improved performance, using a GPU is highly recommended. Processing multiple documents in a single `extract_keywords` call significantly speeds up inference by embedding words only once.
fix
Run KeyBERT on a system with a GPU. When processing many documents, pass a list of documents to `kw_model.extract_keywords()` instead of iterating over them individually.
affects: All
gotchaBy default, KeyBERT uses cosine similarity which may result in very similar keywords. To get more diverse keywords, leverage diversification techniques like 'Max Sum Distance' or 'Maximal Marginal Relevance' (MMR).
fix
Pass `use_mmr=True` (and optionally `diversity` parameter) or `use_maxsum=True` (and `nr_candidates`) to the `extract_keywords` method to enable diversification.
affects: All
gotchaKeyBERT generally doesn't require extensive text preprocessing due to BERT's contextual understanding. However, noisy data (e.g., HTML tags) can negatively impact results.
fix
While not always necessary, clean documents of irrelevant noise like HTML tags or other structural elements that do not contribute to the semantic meaning for better keyword extraction.
affects: All
Errors
Common errors & fixes
OSError: Model name 'distilbert-base-nli-mean-token' was not found in model name list
This error typically occurs when the specified Sentence-Transformers model name is incorrect or misspelled, preventing KeyBERT from downloading or loading the pre-trained embedding model. A common mistake is missing the 's' at the end of 'mean-tokens'.
fix
Correct the model name to the official version, often by adding an 's' to 'mean-token', e.g., `model = KeyBERT('distilbert-base-nli-mean-tokens')`. Refer to the Sentence-Transformers documentation for correct model identifiers.
cannot import name 'KeyBERT' from 'keybert'
This import error usually happens when your Python script is named `keybert.py`. Python tries to import `KeyBERT` from your local file instead of the installed library, leading to a circular import or an inability to find the `KeyBERT` class within your script.
fix
Rename your Python file to something other than `keybert.py` (e.g., `my_keybert_script.py`) to avoid conflicts with the installed `keybert` package.
TypeError: 'DistilBertTokenizer' object is not callable
This error can occur if there's a version mismatch or an incorrect usage pattern when KeyBERT internally tries to use components from the `transformers` library, specifically the tokenizer. It implies that a tokenizer object is being treated as a function.
fix
Ensure that your `transformers` and `sentence-transformers` libraries are up-to-date and compatible with your `keybert` version. Often, upgrading these dependencies (`pip install --upgrade transformers sentence-transformers keybert`) resolves such internal compatibility issues.
AttributeError: 'tuple' object has no attribute 'page_content'
This error typically arises when using KeyLLM (KeyBERT's LLM integration) with external libraries like LangChain, where the output format or expected object type for documents or messages does not match what KeyLLM anticipates. It indicates that a tuple was received when an object with a `page_content` attribute (like a LangChain `Document`) was expected.
fix
Ensure that the data being passed to KeyLLM, especially when integrated with LangChain, is correctly formatted as `langchain.schema.Document` objects or compatible structures, rather than raw tuples or strings, to align with the expected input type for keyword extraction.
Upgrade
Version history
0.9.0latest on PyPI · released Feb 7, 2025
Audit
Dependencies
sentence-transformersrequiredDefault and recommended backend for embedding models.
torchrequiredRequired by sentence-transformers for BERT models.
scikit-learnrequiredUsed for CountVectorizer for candidate keyword generation.
openaioptionalRequired for using KeyLLM with OpenAI models.
flairoptionalOptional embedding model backend.
gensimoptionalOptional embedding model backend.
spacyoptionalOptional embedding model backend.
tensorflow_textoptionalRequired for 'use' (Universal Sentence Encoder) optional backend.
model2vecoptionalUsed for lightweight installation, provides alternative embeddings.
Agent activity
18 hits · last 30 days
node
16
OpenAI (training)
1
Resources
keybert — pip install keybert · libregistry