Install & Compatibility
Where this runs
tested against v2.6.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.95 runs
installs and imports cleanly · install 0.0s · import 0.308s · 106.1MB
glibcpy 3.10–3.95 runs
installs and imports cleanly · install 5.1s · import 0.306s · 103MB
106MB installed
● package 106MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
corpus_bleu
✓ from sacrebleu import corpus_bleu
✗ import sacrebleu
BLEU
✓ from sacrebleu import BLEU
✗ import sacrebleu
sentence_bleu
✓ from sacrebleu import sentence_bleu
✗ import sacrebleu
Calculates the corpus BLEU score for a given hypothesis against one or more reference translations. Outputs the raw score and the formatted string.
import sacrebleu
# Example hypothesis and reference sentences
hypothesis = "The cat sat on the mat."
references = [
"The cat is on the mat.",
"A cat sat on the mat."
]
# Calculate corpus BLEU score
# Note: sacrebleu expects lists of sentences, even for a single hypothesis/reference
bleu_score = sacrebleu.corpus_bleu([hypothesis], [references])
print(f"BLEU score: {bleu_score.score:.2f}")
print(f"BLEU string: {bleu_score.format()}")
sacrebleu --version
Debug
Known issues
breakingThe default output format for the CLI utility changed from single-line to JSON. Tools or scripts parsing sacrebleu's standard output will need to adapt.fixUpdate scripts to parse JSON output. Use the `--format text` option to revert to the old single-line format for compatibility, e.g., `sacrebleu -t wmt14 -l en-de --format text`.
affects: >=2.0.0
breakingPython 3.5, 3.6, and 3.8 support have been dropped in recent major versions. The library now requires Python >=3.9.fixUpgrade your Python environment to 3.9 or higher. For older Python versions, use an earlier sacrebleu version (e.g., sacrebleu<2.0 for Python <3.6, sacrebleu<2.6 for Python 3.8).
affects: >=2.0.0, >=2.6.0
breakingThe default smoothing method and floor value for `corpus_bleu()` and `sentence_bleu()` changed, potentially yielding different scores compared to earlier versions.fixReview your code to ensure the desired smoothing method (`floor`, `add-k`, `exp`) and `smooth_value` are explicitly passed to `corpus_bleu()` or `sentence_bleu()` to ensure consistent scoring behavior.
affects: >=1.5.0
gotchaSome language-specific tokenizers require extra dependencies that are not installed by default. For example, Japanese (`-tok ja`) and Korean (`-tok ko-mecab`) tokenizers need additional packages.fixInstall with optional extras, e.g., `pip install sacrebleu[ja]` for Japanese. For Korean tokenizers like `ko-mecab`, you might need to manually install `mecab-ko`.
affects: All versions
Upgrade
Version history
2.6.0latest on PyPI · released Jan 12, 2026
Audit
Dependencies
mecab-kooptionalRequired for Korean tokenization (e.g., '--tok ko-mecab').
sacrebleu[ja]optionalExtra for Japanese tokenization, installing `mecab3-python`.