pyctcdecode

library0.5.0pypypi✓ verified 88d ago

pyctcdecode is a Python library that provides a standalone beam search decoder for CTC (Connectionist Temporal Classification) models. It allows for efficient decoding of CTC outputs and seamlessly integrates with KenLM language models to improve speech recognition accuracy. The current version is 0.5.0, and it follows an active release cadence, with updates addressing features, performance, and bug fixes.

ai-ml

pip install pyctcdecode

github ↗package ↗

INSTALL

IMPORT

SIG · PYCTCDECODE

pyctcdecode

ai-mlpythonv0.5.0

Install

4.5s avg

Import

403ms

Disk

96MB

Pass rate

8/ 10

Env Coverage8 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v0.5.0 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

glibc

py 3.10

✓ —

✓ 4.55s

py 3.11

✓ —

✓ 4.25s

py 3.12

✓ —

✓ 3.85s

py 3.13

✕ build_error

py 3.9

✓ —

✓ 5.43s

96MB installed

● package 96MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

BeamSearchDecoderCTC

✓ from pyctcdecode import BeamSearchDecoderCTC

LanguageModel

✓ from pyctcdecode import LanguageModel

Alphabet

✓ from pyctcdecode.alphabet import Alphabet

BLANK_TOKEN

✓ from pyctcdecode.alphabet import BLANK_TOKEN

get_alphabet

✓ from pyctcdecode.alphabet import get_alphabet

This quickstart demonstrates how to initialize `BeamSearchDecoderCTC` with a custom alphabet and decode dummy CTC logits. It shows the basic usage without a language model. For real-world applications, integrating with a KenLM language model is highly recommended for improved accuracy.

from pyctcdecode import BeamSearchDecoderCTC from pyctcdecode.alphabet import BLANK_TOKEN, get_alphabet import numpy as np # Define your model's alphabet. The BLANK_TOKEN must be the first element. # This example uses a common alphabet for English speech recognition. labels = [BLANK_TOKEN] + list("abcdefghijklmnopqrstuvwxyz '") alphabet = get_alphabet(labels) # Create dummy CTC output (logits) for demonstration. # In a real scenario, these would come from your deep learning model. # Shape: (time_steps, alphabet_size) time_steps = 50 logits = np.random.rand(time_steps, len(labels)).astype(np.float32) # Initialize the decoder without a language model. # For better accuracy, integrate with a KenLM language model (see warnings). decoder = BeamSearchDecoderCTC(alphabet) # Decode the logits. The decode method returns a list of hypotheses. # We take the first (most probable) one. hypotheses = decoder.decode(logits) decoded_text = hypotheses[0] print(f"Decoded text (example): {decoded_text}")

Debug

Known issues

gotchaThe KenLM C++ library, an optional but highly recommended dependency for language model integration, can be challenging to install due to its native compilation requirements (e.g., Boost, Zlib, Bzip2, CMake).

fix

For common Linux distributions, ensure development headers for `boost`, `zlib`, `bzip2`, and `cmake` are installed. For example: `sudo apt-get install libboost-all-dev liblzma-dev libbz2-dev cmake`. Then retry `pip install pyctcdecode[kenlm]` or consult KenLM's specific build instructions if issues persist.

affects: All versions requiring KenLM (0.1.0+)

gotchaThe CTC blank token MUST be the first element in the alphabet list provided to `get_alphabet` or `Alphabet` constructor. Incorrect positioning will lead to incorrect decoding results or runtime errors.

fix

Always construct your alphabet list by placing `BLANK_TOKEN` (or an empty string) as the first element. Example: `labels = [BLANK_TOKEN] + list('abc')`. Ensure the total length of the alphabet matches your model's CTC output dimension.

affects: All versions

gotchaLoading a KenLM language model requires a valid `.arpa` file. Incorrect file paths, malformed `.arpa` files, or extremely large language models can cause `FileNotFoundError`, `MemoryError`, or excessively slow initialization.

fix

Ensure the `.arpa` file exists and is accessible. Verify its format using KenLM's tools if possible. For very large models, consider reducing the beam width or using a smaller language model for initial testing to mitigate performance issues.

affects: All versions using KenLM

Errors

Common errors & fixes

ModuleNotFoundError: No module named 'kenlm'

The `pyctcdecode[kenlm]` extra was not installed, or its installation failed during compilation, preventing the import of the `kenlm` module required for language model functionality.

fix

Install with KenLM support using `pip install pyctcdecode[kenlm]`. If this fails, review system dependencies for KenLM (Boost, Zlib, Bzip2, CMake) as described in the warnings.

error: Boost library not found or not configured. Set BOOST_ROOT or BOOST_INCLUDEDIR to point to the Boost install.

During `pyctcdecode[kenlm]` installation, the underlying KenLM C++ compilation process failed to locate the necessary Boost library, which is a critical dependency.

fix

Install Boost development headers on your system (e.g., `sudo apt-get install libboost-all-dev` on Debian/Ubuntu, or `brew install boost` on macOS). Ensure `cmake` is also installed. If Boost is in a non-standard location, you might need to manually set environment variables like `BOOST_ROOT` or `BOOST_INCLUDEDIR`.

IndexError: list index out of range

This often occurs when the `alphabet` passed to the decoder does not correctly map to the CTC model's output dimension, or more commonly, if the blank token is not positioned as the very first element in the alphabet list.

fix

Verify that your `labels` list used to create the `alphabet` has the `BLANK_TOKEN` (or an empty string) as its first element, and that the total length of your alphabet exactly matches the last dimension of your CTC model's output logits.

Upgrade

Version history

0.5.0latest on PyPI · released Jan 20, 2023

Audit

Dependencies

kenlmoptionalOptional, required for language model integration, which significantly improves decoding accuracy for speech recognition tasks. Its installation often requires system-level C++ build tools.

numpyrequiredCore dependency for numerical operations.

scipyrequiredCore dependency for scientific computing functions.

tqdmrequiredUsed for progress bars.

pygtrierequiredUsed for efficient prefix tree operations in beam search.

Agent activity

9 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/kensho-technologies/pyctcdecode ↗

packagehttps://pypi.org/project/pyctcdecode/ ↗