Registry / ai-ml / sudachipy

sudachipy

JSON →
library0.6.11pypypi✓ verified 22d ago

SudachiPy is a Python binding for Sudachi.rs, a Japanese morphological analyzer implemented in Rust. It provides multi-granular tokenization for Japanese text, handling various linguistic nuances. The current version is 0.6.10, with releases typically occurring every few months to incorporate updates, bug fixes, and Python version support.

pip install sudachipy sudachidict_core
INSTALL
IMPORT
SIG · SUDACHIPY
S
sudachipy
ai-mlpythonv0.6.11
Install
3.3s avg
Import
58ms
Disk
228MB
Pass rate
5/ 10
Env Coverage5 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v0.6.11 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.95 runs
build_error
glibc
py 3.103.95 runs
installs and imports cleanly · install 3.3s · import 0.058s · 230MB
228MB installed
● package 228MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

Dictionary
from sudachipy import Dictionary
from sudachipy.dictionary import Dictionary
Importing from sudachipy.dictionary is deprecated as of recent versions.
Tokenizer
from sudachipy import Tokenizer
from sudachipy.tokenizer import Tokenizer
Importing from sudachipy.tokenizer is deprecated as of recent versions.
SplitMode
from sudachipy import SplitMode

This quickstart demonstrates how to initialize the SudachiPy tokenizer with a default dictionary and perform multi-granular tokenization on Japanese text. It also shows how to access basic information for individual morphemes.

from sudachipy import Dictionary, SplitMode # Initialize the tokenizer with the default (core) dictionary tokenizer = Dictionary().create() text = "すもももももももものうち" # Tokenize in SplitMode.C (shortest segmentation) morphemes_c = tokenizer.tokenize(text, SplitMode.C) print("SplitMode.C:", [m.surface() for m in morphemes_c]) # Tokenize in SplitMode.A (medium segmentation) morphemes_a = tokenizer.tokenize(text, SplitMode.A) print("SplitMode.A:", [m.surface() for m in morphemes_a]) # Access morpheme details if morphemes_c: first_morpheme = morphemes_c[0] print(f"\nFirst morpheme (C): {first_morpheme.surface()}") print(f" Reading form: {first_morpheme.reading_form()}") print(f" Part of Speech: {first_morpheme.part_of_speech()}")
Debug
Known issues
breakingSupport for Python 3.6, 3.7, and 3.8 has been removed in versions 0.6.4 and 0.6.9, respectively. Ensure you are using Python 3.9 or newer.
fix
Upgrade your Python environment to 3.9 or a later supported version.
affects: >=0.6.4, >=0.6.9
breakingThe `sudachipy link` command, used for managing dictionary paths, was removed in v0.5.2 and later. Dictionary specification methods now rely on `config_path` or `dict_type` arguments to `Dictionary()` or CLI options.
fix
Refer to the documentation for updated methods of specifying dictionary paths or types (e.g., `Dictionary(dict_type='full')`).
affects: >=0.5.2
deprecatedDirect imports like `from sudachipy.dictionary import Dictionary` and `from sudachipy.tokenizer import Tokenizer` are deprecated. Import `Dictionary`, `Tokenizer`, and `SplitMode` directly from the top-level `sudachipy` package.
fix
Change import statements to `from sudachipy import Dictionary, Tokenizer, SplitMode`.
affects: >=0.6.x
gotchaSudachiPy requires a dictionary package (e.g., `sudachidict_core`) to be installed separately. It is not included in the main `sudachipy` package.
fix
Always install a dictionary package alongside `sudachipy`, e.g., `pip install sudachipy sudachidict_core`.
affects: All versions
gotchaBuilding SudachiPy from source (common on ARM64 Linux/macOS if no pre-built wheel is available) requires the Rust compiler toolchain and `setuptools-rust` to be installed in your environment.
fix
Install Rust and `setuptools-rust` (e.g., `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && pip install setuptools-rust`).
affects: All versions (when building from source)
gotchaThe `mode` parameter in the `Tokenizer.tokenize()` method is deprecated. Pass the analysis mode when creating the `Tokenizer` instance, or use `Morpheme.split()` for multi-level splitting.
fix
Create different `Tokenizer` instances for different modes, e.g., `tokenizer_a = Dictionary().create(mode=SplitMode.A)`.
affects: >=0.6.x
gotchaDictionary resource path resolution logic changed in v0.6.3. Paths are now resolved in a specific order: absolute paths, relative to config `path`, relative to `resource_dir` param, relative to config file, relative to current directory.
fix
If custom dictionary paths are used, verify they align with the new resolution order. Consider using the `resource_dir` parameter in `Dictionary()` constructor or `config_path` for explicit control.
affects: >=0.6.3
Upgrade
Version history
0.6.11latest on PyPI · released Apr 13, 2026
Audit
Dependencies
sudachidict_corerequiredRequired for dictionary functionality; not bundled with sudachipy.
setuptools-rustoptionalRequired for building from source, particularly on ARM architectures.
rustoptionalRust compiler toolchain required for building from source (e.g., on ARM64 Linux/macOS).
Agent activity
18 hits · last 30 days
node
14
OpenAI (training)
1
Resources
sudachipy — pip install sudachipy · libregistry