Install & Compatibility
Where this runs
tested against v0.3.4 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.95 runs
installs and imports cleanly · install 0.0s · import 0.000s · 18.4MB
glibcpy 3.10–3.95 runs
installs and imports cleanly · install 1.8s · import 0.000s · 19MB
16MB installed
● package 16MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
Segmenter
✓ import pysbd; segmenter = pysbd.Segmenter(...)
Primary class for sentence segmentation
PySBDFactory
✓ from pysbd.utils import PySBDFactory
✗ from pysbd.segmenter import PySBDFactory
PySBDFactory is specifically for spaCy integration and resides in pysbd.utils, not directly under pysbd or pysbd.segmenter.
This example demonstrates basic sentence segmentation using the `Segmenter` class for English text. The `clean=False` parameter is used to prevent aggressive text cleaning.
import pysbd
text = "Dr. Smith went to the U.S. last week. He said, 'Hello!' How are you?"
# Initialize segmenter for English
segmenter = pysbd.Segmenter(language="en", clean=False)
sentences = segmenter.segment(text)
for i, sent in enumerate(sentences):
print(f"Sentence {i+1}: {sent}")
Debug
Known issues
breakingWhen integrating with spaCy, examples for spaCy v2.x using `nlp.add_pipe(PySBDFactory(nlp))` are not compatible with spaCy v3.x. The `add_pipe` API changed to expect a string name of the registered component factory.fixFor spaCy v3.x, use `@Language.factory('pysbd_segmenter')` decorator on a custom factory function, then `nlp.add_pipe('pysbd_segmenter')`. Refer to spaCy's updated documentation for custom pipeline components. affects: spaCy v3.x and later
gotchaSegmentation results might differ slightly when using `pysbd.Segmenter` directly compared to using `pysbd` as a spaCy pipeline component, especially with quoted text or complex punctuation.fixIf precise, identical segmentation is critical across different integration methods, thoroughly test both approaches with your specific text data. Direct usage (`pysbd.Segmenter().segment(text)`) generally aligns with the expected rule-based output.
affects: All versions
gotchaThe `clean=True` parameter in the `Segmenter` constructor performs aggressive pre-filtering of the input text, removing repeated punctuation, line breaks, URLs, and HTML tags. This might alter the original text more than desired for certain NLP tasks.fixIf you need to preserve the original text as much as possible, set `clean=False` (which is the default in some examples but not universally). Implement custom cleaning steps if specific pre-processing is required.
affects: All versions
gotchaBy default, `pysbd.Segmenter` returns a list of strings. If you require character offsets into the original text for non-destructive tokenization, you must initialize `Segmenter` with `char_span=True`.fixInitialize the segmenter as `segmenter = pysbd.Segmenter(language='en', char_span=True)`. The output will then be a list of `TextSpan` objects, each containing the segmented sentence, its start, and end character indices.
affects: All versions
gotchaWhile highly accurate, `pysbd` is a rule-based system implemented in Python. It may be slower compared to some sentence boundary detection alternatives that are implemented in lower-level languages like C++ or optimized using Cython.fixConsider performance implications for very large text corpora or real-time applications. Benchmark `pysbd` against other tools if speed is a primary concern. For high accuracy where speed is not the absolute bottleneck, `pysbd` remains a strong choice.
affects: All versions
Upgrade
Version history
0.3.4latest on PyPI · released Feb 11, 2021
Audit
Dependencies
spacyoptionalOptional, for integration as a spaCy pipeline component.