cleantext

library1.1.4pypypi✓ verified 85d ago

An open-source Python package to clean raw text data. Version 1.1.4 provides simple functions to normalize whitespace, remove URLs, emojis, numbers, punctuation, and more. The package is relatively stable with low release cadence. It is widely used for quick text preprocessing in NLP pipelines.

ai-ml

pip install cleantext

github ↗package ↗

INSTALL

IMPORT

SIG · CLEANTEXT

cleantext

ai-mlpythonv1.1.4

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

clean

✓ from cleantext import clean

Direct import is straightforward

Basic example: removes URLs, numbers, punctuation, and extra spaces, leaving text intact.

from cleantext import clean text = "Hello! Check out https://example.com 😊 I have 5 apples..." cleaned = clean( text, extra_spaces=True, lowercase=False, numbers=False, punct=False, replace_with_url='<URL>', replace_with_number='<NUM>', replace_with_punct='', lang='en' ) print(cleaned)

Debug

Known issues

gotchaThe 'clean' function returns a list of tokens by default (not a string). If you expect a single string, set 'lowercase=False' and then join the list, or check output type.

fix

Use 'cleaned_text = ' '.join(clean(...))' if you need a single string.

affects: all versions

deprecatedThe 'fix_unicode' parameter was removed in version 1.0.0. Using it raises an error.

fix

Remove 'fix_unicode' from the call. Unicode normalization is now handled automatically.

affects: >=1.0.0

breakingIn version 1.0.0, the 'clean' function changed its default parameter values and removed some arguments like 'fix_unicode'. Code written for <1.0.0 may break.

fix

Review your function calls and adjust parameter names. For example, 'no_urls' is now 'replace_with_url'.

affects: >=1.0.0

gotchaThe 'clean' function can remove too much. For instance, setting 'punct=False' removes all punctuation including periods in abbreviations.

fix

Use the 'replace_with_punct' parameter with a placeholder instead of removing punctuation entirely.

affects: all versions

Errors

Common errors & fixes

ModuleNotFoundError: No module named 'cleantext'

Package not installed.

fix

Run 'pip install cleantext' to install the package.

TypeError: clean() got an unexpected keyword argument 'fix_unicode'

The 'fix_unicode' parameter was removed in version 1.0.0.

fix

Remove 'fix_unicode' from your function call. Unicode normalization is handled automatically.

clean() returns a list, not string

The 'clean' function returns a list of tokens by default.

fix

Use 'cleaned_text = ' '.join(clean(...))' to get a single string.

Upgrade

Version history

1.1.4latest on PyPI · released Dec 29, 2021

Audit

Dependencies

No dependency data recorded yet.

Agent activity

14 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/prasanthg3/cleantext ↗

packagehttps://pypi.org/project/cleantext/ ↗