Install & Compatibility
Where this runs
tested against v4.1 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.95 runs
build_error
glibcpy 3.10–3.95 runs
installs and imports cleanly · install 16.8s · import 2.986s · 346MB
389MB installed
● package 389MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
DiarizationErrorRate
✓ from pyannote.metrics.diarization import DiarizationErrorRate
Primary import for computing the Diarization Error Rate.
Annotation
✓ from pyannote.core import Annotation
Required from 'pyannote.core' to define ground truth and hypothesis segments for evaluation.
Segment
✓ from pyannote.core import Segment
Required from 'pyannote.core' to define temporal segments within annotations.
This quickstart demonstrates how to compute the Diarization Error Rate (DER) using `pyannote.metrics`. It involves creating `Annotation` objects for both the reference and hypothesis, defining temporal `Segment`s with speaker labels, and then instantiating and calling the `DiarizationErrorRate` class.
from pyannote.core import Segment, Annotation
from pyannote.metrics.diarization import DiarizationErrorRate
# Define a reference (ground truth) annotation
reference = Annotation(uri='file1')
reference[Segment(0, 10)] = 'A'
reference[Segment(12, 20)] = 'B'
reference[Segment(24, 27)] = 'A'
reference[Segment(30, 40)] = 'C'
# Define a hypothesis (system output) annotation
hypothesis = Annotation(uri='file1')
hypothesis[Segment(2, 13)] = 'a'
hypothesis[Segment(13, 14)] = 'd'
hypothesis[Segment(14, 20)] = 'b'
hypothesis[Segment(22, 38)] = 'c'
hypothesis[Segment(38, 40)] = 'd'
# Instantiate the Diarization Error Rate metric
metric = DiarizationErrorRate()
# Compute the DER
der_value = metric(reference, hypothesis)
print(f"Diarization Error Rate: {der_value:.3f}")
pyannote-metrics --version
Debug
Known issues
breakingVersion 3.3.0 introduced a breaking change by improving diarization purity and coverage to explicitly account for overlapping regions, which might alter previously obtained metric values for systems that handle overlap differently.fixRe-evaluate existing systems with the new metric behavior. Understand how overlapping speech is handled by your diarization system and pyannote.metrics to correctly interpret results.
affects: >=3.3.0
gotchaComparison of evaluation scores across different diarization evaluation tools (e.g., `pyannote.metrics` vs. `md-eval`) is not recommended due to varying design choices, default parameters (like collar size), and handling of speaker mapping and overlapping speech.fixAlways use the evaluation tool and parameters (e.g., `collar`, `skip_overlap`) specified by the benchmark you are targeting, and explicitly report all settings. Avoid direct comparisons of scores obtained from different tools.
affects: All versions
gotchaThe `collar` parameter, typically set to 0.25 (250 ms exclusion around boundaries), significantly impacts DER. Manual annotations often lack audio sample-level precision, making a collar common practice. However, strict benchmarks may use `collar=0.0`.fixExplicitly define and report the `collar` setting in your experiments. Be aware that different collar values can change DER by several percentage points, making results incomparable if not standardized.
affects: All versions
breakingOlder versions (2.0.1) dropped support for Python 2.7 and all file formats except RTTM for evaluation. Ensure your environment uses Python 3.10+ and RTTM for input annotations.fixMigrate to Python 3.10 or newer and convert non-RTTM annotation files to RTTM format for compatibility.
affects: <2.0.1 (upgrade paths)
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'pyannote.metrics'
The `pyannote.metrics` library or one of its core dependencies like `pyannote.core` is not installed, or the Python environment where the code is run does not have access to the installed package.
fixEnsure the library is correctly installed using pip: `pip install pyannote.metrics`. If using a virtual environment, activate it before installation. Also, ensure `pyannote.core` is installed, as it's a dependency: `pip install pyannote.core`.
ZeroDivisionError: float division by zero
This error often occurs when calculating metrics like `SegmentationPurity` or `SegmentationCoverage` on empty reference or hypothesis annotations, leading to division by zero in the metric calculation.
fixBefore computing metrics, ensure that both the reference and hypothesis `Annotation` objects contain actual segments. You may need to add checks for empty annotations (e.g., `if len(reference) > 0 and len(hypothesis) > 0:`) and handle such cases gracefully, perhaps by assigning a default value or skipping the metric computation.
ValueError: zero-size array to reduction operation maximum which has no identity
Similar to `ZeroDivisionError`, this `ValueError` arises when metrics (like `SegmentationCoverage` or `SegmentationPurityCoverageFMeasure`) attempt to perform reduction operations (like finding a maximum) on arrays that are empty, which can happen with empty reference or hypothesis annotations.
fixVerify that the `Annotation` objects (reference and hypothesis) used for metric computation are not empty. Implement checks to ensure annotations contain segments before calling metric functions to avoid operating on zero-size arrays.
TypeError: get_protocol() got an unexpected keyword argument 'progress'
This error typically indicates an incompatibility between versions of `pyannote.database` and `pyannote.pipeline` (or other components of the pyannote ecosystem). An older version of `pyannote.database`'s `get_protocol` function might not accept the `progress` argument, which was introduced or removed in different versions.
fixUpgrade all `pyannote` libraries to compatible versions, ideally the latest stable releases, to ensure consistent API usage across the ecosystem. For example, `pip install --upgrade pyannote.database pyannote.pipeline`.
Upgrade
Version history
4.1latest on PyPI · released May 6, 2026
Audit
Dependencies
pyannote.corerequiredCore data structures for handling annotations and segments, fundamental for defining reference and hypothesis inputs to metrics.
pyannote.databaseoptionalProvides reproducible experimental protocols for multimedia databases, often used in conjunction with metrics for standardized evaluation.