seqeval

library1.2.2pypypi✓ verified 35d ago

seqeval is a Python framework for sequence labeling evaluation. It provides metrics like F1 score, precision, recall, and a detailed classification report for tasks such as named-entity recognition and part-of-speech tagging. It is currently at version 1.2.2 and maintains an active release cadence, with updates often focusing on performance improvements and additional evaluation schemes.

ai-ml testing

github ↗package ↗

Install & Compatibility

Where this runs

tested against v1.2.2 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.920 runs

build_error

glibc

py 3.10–3.920 runs

installs and imports cleanly · install 11.5s · import 3.577s · 272MB

281MB installed

● package 281MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

from seqeval.metrics import accuracy_score

from seqeval.metrics import precision_score

from seqeval.metrics import recall_score

from seqeval.metrics import f1_score

from seqeval.metrics import classification_report

Used to specify a tagging scheme for metrics when needed.

from seqeval.scheme import IOB2

Calculate the F1-score and generate a detailed classification report for sequence labeling predictions.

from seqeval.metrics import f1_score from seqeval.metrics import classification_report y_true = [['O', 'O', 'B-MISC', 'I-MISC', 'B-MISC', 'O', 'O'], ['B-PER', 'I-PER', 'O']] y_pred = [['O', 'O', 'B-MISC', 'I-MISC', 'B-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']] # Compute F1-score (micro average by default) print(f"F1 Score (micro): {f1_score(y_true, y_pred):.2f}") # Compute F1-score with different averaging methods print(f"F1 Score (average=None): {f1_score(y_true, y_pred, average=None)}") # Generate a full classification report report = classification_report(y_true, y_pred, digits=2) print("\nClassification Report:\n", report)

Debug

Known footguns

breakingThe `classification_report` behavior changed significantly in v1.0.0. It now allows explicit specification of the evaluation scheme (e.g., IOB1, IOB2, BILOU) which can alter how entities are counted. Older code relying on implicit scheme assumptions might produce different results.

gotchaseqeval's metrics (especially F1, precision, recall) are calculated differently from `scikit-learn`'s for sequence labeling tasks. `seqeval` specifically evaluates *entities*, primarily focusing on non-'O' (Outside) tags, and does not count correctly predicted 'O' tags as true positives, only misclassified ones. `scikit-learn`'s metrics, when applied naively to token-level tags, will include all 'O' tags in its calculations, potentially leading to inflated scores if 'O' tags are abundant and correctly predicted.

gotchaWhen using `mode='strict'` (e.g., in `f1_score` or `classification_report`), only exact matches for entity spans (including both boundaries and type) are considered correct. This is often the desired behavior for robust evaluation but can result in lower scores compared to the default mode which might be more lenient.

gotchaInput `y_true` and `y_pred` for all `seqeval.metrics` functions must be lists of lists of *strings* (e.g., `[['B-PER', 'I-PER', 'O']]`), representing the sequence tags. Providing numerical label IDs instead of string tags will result in errors.

gotchaPerformance of evaluation, particularly in `strict` mode, was significantly improved in version 1.2.1. Older versions might exhibit slower computation times for large datasets or extensive evaluations.

Upgrade

Version history

Breaking-change detection hasn't run for this library yet.

Audit

Security & dependencies

CVE tracking and dependency tree are planned for a later release.

Agent activity

31 hits · last 30 days

chatgpt-user

petalbot

ahrefsbot

amazonbot

bytedance

script

googlebot

oai-searchbot

Resources

githubhttps://github.com/chakki-works/seqeval ↗

packagehttps://pypi.org/project/seqeval/ ↗