nltk

library3.9.4pypypi✓ verified 52d ago

NLTK (Natural Language Toolkit) is a leading open-source Python library for Natural Language Processing (NLP). It provides easy-to-use interfaces to over 50 corpora and lexical resources, along with a suite of text processing libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning. Currently at version 3.9.4, NLTK generally follows a release cadence of a few minor versions per year, with more significant updates addressing security and Python compatibility as needed.

ai-ml

pip install nltk

github ↗package ↗homepage ↗docs ↗

Install & Compatibility

Where this runs

tested against v3.9.4 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.925 runs

installs and imports cleanly · install 0.0s · import 0.876s · 35.4MB

glibc

py 3.10–3.925 runs

installs and imports cleanly · install 3.3s · import 0.821s · 36MB

35MB installed

● package 35MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

nltk

✓ import nltk

Most common and provides access to core functionalities and submodules like nltk.word_tokenize, nltk.pos_tag.

word_tokenize

✓ from nltk.tokenize import word_tokenize

Direct import for specific tokenization functions.

PorterStemmer

✓ from nltk.stem import PorterStemmer

Direct import for a specific stemmer.

This quickstart demonstrates basic text tokenization and Part-of-Speech (POS) tagging using NLTK. It includes checks to download the 'punkt' tokenizer and 'averaged_perceptron_tagger' if they are not already present, which are common requirements for many NLTK operations. This ensures the example is runnable out-of-the-box.

import nltk from nltk.tokenize import word_tokenize from nltk.tag import pos_tag # Download necessary NLTK data (run once) try: nltk.data.find('tokenizers/punkt') except nltk.downloader.DownloadError: nltk.download('punkt') try: nltk.data.find('taggers/averaged_perceptron_tagger') except nltk.downloader.DownloadError: nltk.download('averaged_perceptron_tagger') text = "NLTK is a powerful library for natural language processing." # Tokenization tokens = word_tokenize(text) print(f"Tokens: {tokens}") # Part-of-Speech Tagging tagged_tokens = pos_tag(tokens) print(f"POS Tagged: {tagged_tokens}")

Debug

Known issues

breakingNLTK 3.9 introduced a breaking change by replacing pickled models (e.g., for `punkt`, chunkers, taggers) with new pickle-free `_tab` packages to fix security vulnerability CVE-2024-39705. Older versions using pickled models may be insecure or incompatible with newer NLTK versions.

fix

Upgrade NLTK to version 3.9 or higher. Ensure your application is updated to use the new `_tab` packages or re-download corpora with `nltk.download()` after upgrading. Specifically, NLTK 3.9.3 fixed CVE-2025-14009 related to secure ZIP extraction.

affects: <3.9

gotchaMany NLTK functionalities (e.g., tokenizers, taggers, corpora) require downloading specific datasets. Failing to download them will result in `Resource Not Found` errors. Running `nltk.download('all')` can be resource-intensive and unsuitable for production environments.

fix

Before using a specific NLTK module that relies on external data, ensure the necessary data is downloaded. For production, explicitly download only the required packages using `nltk.download('package_name')` once during setup, or use `nltk.data.path.append('/path/to/nltk_data')` to point to pre-downloaded data. For example, `nltk.download('punkt')` for the Punkt tokenizer.

affects: All versions

breakingThe `nltk.downloader.DownloadError` exception class was deprecated and removed in NLTK versions 3.8.1 and higher. Code attempting to catch `nltk.downloader.DownloadError` will raise an `AttributeError`. The replacement is `nltk.downloader.NLTKDownloadError` or its base class `nltk.downloader.NLTKDownloaderException`.

fix

Update exception handling in your code to catch `nltk.downloader.NLTKDownloadError` or `nltk.downloader.NLTKDownloaderException` instead of `nltk.downloader.DownloadError`.

affects: >=3.8.1

Errors

Common errors & fixes

ModuleNotFoundError: No module named 'nltk'

The NLTK library is not installed in the Python environment you are currently using, or there's an issue with your Python PATH.

fix

Install NLTK using pip: `pip install nltk` or `pip3 install nltk`.

LookupError: Resource 'punkt' not found. Please use the NLTK Downloader to obtain the resource:

NLTK requires additional data packages (like 'punkt' for tokenization, 'stopwords' for stop word lists, 'wordnet' for lexical resources, etc.) that are not included in the initial library installation and must be downloaded separately.

fix

Open a Python interpreter and run `import nltk; nltk.download('punkt')` to download the specific 'punkt' tokenizer. For other resources, replace 'punkt' with the name of the missing resource (e.g., 'stopwords', 'wordnet', 'averaged_perceptron_tagger'), or run `nltk.download('all')` to download all popular NLTK data collections.

Error loading [resource name]: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed:

The NLTK data downloader is encountering an SSL certificate verification issue, often due to corporate network proxies, firewalls, or an outdated Python installation's certificate store.

fix

Bypass SSL verification for the NLTK download. In your Python script or interpreter, add the following before `nltk.download()`: `import ssl; try: _create_unverified_https_context = ssl._create_unverified_https_context except AttributeError: pass else: ssl._create_default_https_context = _create_unverified_https_context; nltk.download('popular')` (or the specific resource you need).

AttributeError: module 'nltk' has no attribute 'download'

This usually happens if you've inadvertently named one of your Python files 'nltk.py', which causes Python to import your local file instead of the actual NLTK library, or if you're using a very old or corrupted NLTK installation.

fix

Rename your Python script if it's named `nltk.py` (or any other name that conflicts with an NLTK module). If that's not the case, ensure NLTK is properly installed and updated by running `pip install --upgrade nltk`.

Upgrade

Version history

3.9.4latest on PyPI

Audit

Dependencies

No dependency data recorded yet.

Agent activity

12 hits · last 30 days

seranking-bot

node

ahrefsbot

Amazon

Resources

githubhttps://github.com/nltk/nltk ↗

packagehttps://pypi.org/project/nltk/ ↗

homepagehttps://www.nltk.org/ ↗

docshttps://www.nltk.org/ ↗