Registry / ai-ml / gibberish-detector

gibberish-detector

JSON →
library0.1.1pypypi✓ verified 86d ago

The `gibberish-detector` Python library, currently at version 0.1.1, identifies nonsensical strings using a Markov Chain-based model. It's an adaptation of an earlier project, updated for Python 3. Users first train a model on a corpus of 'good' text to understand character transition probabilities, and then use this model to determine if new input strings are gibberish. The library is primarily maintained as a utility for text validation and spam filtering, with updates occurring on an infrequent basis.

pip install gibberish-detector
INSTALL
IMPORT
SIG · GIBBERISH-DETECTOR
G
gibberish-detector
ai-mlpythonv0.1.1
Install
1.5s avg
Import
17ms
Disk
16MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v0.1.1 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.910 runs
installs and imports cleanly · install 0.0s · import 0.017s · 17.9MB
glibc
py 3.103.910 runs
installs and imports cleanly · install 1.5s · import 0.016s · 18MB
16MB installed
● package 16MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

detector
from gibberish_detector import detector

This quickstart demonstrates how to import the `gibberish_detector` and use a trained model to detect gibberish. Note that a valid trained model file is essential for the `create_from_model` function to work correctly. The provided code creates a dummy model file, which will likely cause an error upon loading but illustrates the API usage. For actual detection, you must first train a model using the `gibberish-detector train` command-line tool, providing a large text file of 'good' (non-gibberish) text, and then point `create_from_model` to your generated model file.

import os import tempfile # NOTE: In a real scenario, you would train a model on a large text file. # For this quickstart, we'll create a dummy model file for demonstration. # A proper model training example would be: # gibberish-detector train examples/big.txt > big.model # Create a dummy model file for demonstration purposes # This content is NOT a valid gibberish-detector model and will likely fail. # It's purely to show the API usage. A real model is a JSON file. model_content = "{}" with tempfile.NamedTemporaryFile(mode='w', delete=False, suffix='.model', encoding='utf-8') as tmp_model_file: tmp_model_file.write(model_content) model_path = tmp_model_file.name try: from gibberish_detector import detector # Attempt to load from the dummy model file # This will likely fail with a JSONDecodeError or similar since it's an empty dict string. # In a real application, ensure your model_path points to a valid, trained model file. print(f"Attempting to load model from: {model_path}") my_detector = detector.create_from_model(model_path) # Example usage with a loaded detector print(f"'superman' is gibberish: {my_detector.is_gibberish('superman')}") print(f"'ertrjiloifdfyyoiu' is gibberish: {my_detector.is_gibberish('ertrjiloifdfyyoiu')}") except Exception as e: print(f"Could not run quickstart due to an error. This is expected if the model_path is not a valid trained model. Error: {e}") print("To run properly, first train a model using the command line tool:") print(" gibberish-detector train <path_to_good_text_file> > your_model.model") print("Then, replace 'model_path' above with 'your_model.model'.") finally: # Clean up the dummy model file if os.path.exists(model_path): os.remove(model_path)
Debug
Known issues
gotchaThe library requires a pre-trained model file to detect gibberish. Simply installing the package does not provide a functional model out-of-the-box. Attempting to use `create_from_model` without a valid model file will result in errors.
fix
First, train a model using the command-line interface: `gibberish-detector train <path_to_good_text_file> > your_model.model`. Then, load this generated model file using `detector.create_from_model('your_model.model')`.
affects: 0.1.1
gotchaThe effectiveness of gibberish detection heavily depends on the quality and size of the training data. A model trained on a small or unrepresentative dataset may produce inaccurate results.
fix
Use a large corpus of relevant, non-gibberish text (e.g., several megabytes of English text) to train your model for optimal performance. The GitHub README suggests `examples/big.txt` for training.
affects: 0.1.1
Errors
Common errors & fixes
FileNotFoundError: [Errno 2] No such file or directory: 'big.model'
The `detector.create_from_model()` function was called with a model file path that does not exist or is incorrect.
fix
Ensure you have trained a model and provided the correct path to the `.model` file. Example training: `gibberish-detector train examples/big.txt > big.model`.
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The model file specified is either empty, corrupted, or not a valid JSON format that the `gibberish-detector` expects.
fix
Verify the integrity of your `.model` file. Retrain the model if necessary using `gibberish-detector train <path_to_good_text_file> > your_model.model` to ensure a correctly formatted model file is generated.
Upgrade
Version history
0.1.1latest on PyPI · released Mar 4, 2021
Audit
Dependencies

No dependency data recorded yet.

Agent activity
12 hits · last 30 days
node
10
Amazon
1
OpenAI (training)
1
Resources
gibberish-detector — pip install gibberish-detector · libregistry