PyThaiNLP is a Python library for natural language processing (NLP) of the Thai language. It provides standard NLP functions like word and sentence segmentation, part-of-speech tagging, transliteration, and various utilities. The library is actively maintained, with version 5.3.4 as the current stable release, and new minor updates for the 5.x series are still being released, with a major 6.0 release expected to introduce breaking changes.
pip install pythainlpVerified import paths — ran on the pinned version, not inferred.
This quickstart demonstrates basic word and sentence tokenization using PyThaiNLP's default engine. Many other tokenization engines are available and can be specified with the `engine` parameter (e.g., `engine="icu"`).
Review the migration guide for PyThaiNLP 6.0 when it's released and ensure your environment uses Python 3.9 or newer.
Update environment variable usage: `PYTHAINLP_DATA` instead of `PYTHAINLP_DATA_DIR`, and `PYTHAINLP_READ_ONLY` instead of `PYTHAINLP_READ_MODE`.
Be aware of this initial delay in performance-sensitive applications. Consider pre-loading necessary models or data if consistent, immediate response times are critical.
For PyICU on Windows, check `https://www.lfd.uci.edu/~gohlke/pythonlibs/` for pre-built wheels. For `python-crfsuite` on Python 3.10+, refer to PyThaiNLP's FAQ for workarounds or ensure you're using a compatible Python version or installation method for that dependency.
Set `PYTHAINLP_DATA` to a writable local directory (e.g., `./pythainlp-data`) within the distributed function on each worker node before any data access.
pip install pythainlp
Download the necessary model(s) using `pythainlp.tools.download("model_name")`, replacing `model_name` with the specific model needed (e.g., "attacut", "perceptron_tagger").Install the `tltk` dependency separately: `pip install tltk`.
Import `word_tokenize` from the correct submodule: `from pythainlp.tokenize import word_tokenize`.