readabilipy

library0.3.0pypypi✓ verified 25d ago

ReadabiliPy is a Python library that provides a wrapper for Mozilla's Readability.js, a powerful tool for extracting the main content from HTML pages. It also includes pure Python article extraction routines. The library augments the Readability.js output to include plain text representations of article paragraphs. The current version is 0.3.0, and it has an active development status, with updates released periodically.

web-framework

pip install readabilipy

github ↗package ↗

INSTALL

IMPORT

SIG · READABILIPY

readabilipy

web-frameworkpythonv0.3.0

Install

3.3s avg

Import

561ms

Disk

34MB

Pass rate

10/ 10

Env Coverage10 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v0.3.0 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.95 runs

installs and imports cleanly · install 0.0s · import 0.582s · 35.3MB

glibc

py 3.10–3.95 runs

installs and imports cleanly · install 3.3s · import 0.540s · 36MB

34MB installed

● package 34MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

simple_json_from_html_string

✓ from readabilipy import simple_json_from_html_string

This is the primary function for extracting article content from an HTML string.

This quickstart demonstrates how to use `readabilipy` to extract article content from an HTML string. It shows both the `use_readability=True` option (which leverages Mozilla's Readability.js via Node.js if available) and the `use_readability=False` option (for the pure Python implementation). Note that the results may differ between the two methods.

import requests from readabilipy import simple_json_from_html_string # Example HTML content (or fetch from a URL) url = "https://en.wikipedia.org/wiki/Python_(programming_language)" try: req = requests.get(url, timeout=10) req.raise_for_status() # Raise an exception for HTTP errors html_content = req.text except requests.exceptions.RequestException as e: print(f"Error fetching URL: {e}") html_content = "<html><body><h1>Example Article</h1><p>This is a paragraph.</p></body></html>" # Extract article using Readability.js (requires Node.js installed) # Set use_readability=True to enable the Node.js wrapper # If Node.js is not found, it will fall back to the Python-only parser article_js = simple_json_from_html_string(html_content, use_readability=True) print("--- Extracted with Readability.js (or Python fallback) ---") print(f"Title: {article_js.get('title')}") print(f"Content snippet: {article_js.get('plain_text', [''])[0][:100]}...") # Extract article using the pure Python implementation article_py = simple_json_from_html_string(html_content, use_readability=False) print("\n--- Extracted with Pure Python ---") print(f"Title: {article_py.get('title')}") print(f"Content snippet: {article_py.get('plain_text', [''])[0][:100]}...")

Debug

Known issues

gotchaTo utilize Mozilla's Readability.js functionality, you must have Node.js (version 14 or higher) installed and accessible in your system's PATH. Without Node.js, `readabilipy` will silently fall back to its pure Python extraction routines when `use_readability=True` is specified.

fix

Install Node.js (v14+) from nodejs.org or ensure it's in your system's PATH. If Node.js is not an option, set `use_readability=False` to explicitly use the Python-only extractor.

affects: <=0.3.0

gotchaThe `use_readability` flag (defaulting to `True` in `simple_json_from_html_string`) controls whether the Node.js-based Readability.js wrapper or the pure Python extractor is used. The results from these two methods can differ significantly for certain articles.

fix

Always explicitly set `use_readability=True` or `use_readability=False` based on your desired behavior and ensure Node.js is correctly installed if you intend to use the Readability.js wrapper.

affects: <=0.3.0

breakingPrior to v0.3.0, users frequently encountered `UnicodeEncodeError` and `UnicodeDecodeError` when processing certain HTML content due to encoding issues with external Node.js subprocess calls and file handling.

fix

Upgrade to `readabilipy` v0.3.0 or newer. Ensure your input HTML is correctly encoded, preferably UTF-8.

affects: <0.3.0

gotchaVersions prior to v0.3.0 had a bug related to changes in the working directory during article extraction, potentially leading to incorrect file paths or failures when using the Readability.js wrapper.

fix

Upgrade to `readabilipy` v0.3.0 or newer, which includes a fix for this working directory bug. If upgrading is not possible, ensure your application does not change the current working directory while `readabilipy` is processing HTML.

affects: <0.3.0

Errors

Common errors & fixes

ModuleNotFoundError: No module named 'readabilipy'

The `readabilipy` package is not installed in the current Python environment.

fix

pip install readabilipy

playwright.sync_api._generated.Error: browserType.launch: Playwright was not able to find a suitable browser executable.

Playwright browser executables (e.g., Chromium, Firefox, WebKit) are not installed, which are required for `readabilipy` to render JavaScript content.

fix

Run `playwright install` in your terminal to download the necessary browser binaries.

ValueError: Invalid URL 'example.com'

The URL provided to `readabilipy.extract_from_url` is malformed or does not include a valid scheme (e.g., `http://` or `https://`).

fix

Ensure the URL string is a complete and valid URL, including the protocol (e.g., `https://www.example.com/article`).

TypeError: expected string or bytes-like object

The `readabilipy.extract_from_html` function received an argument that was not a string or bytes-like object (e.g., an integer or None).

fix

Provide the raw HTML content as a string (or bytes) to the `html` parameter of `readabilipy.extract_from_html`.

Upgrade

Version history

0.3.0latest on PyPI · released Dec 2, 2024

Audit

Dependencies

Node.jsoptionalRequired for using Mozilla's Readability.js wrapper functionality (version 14 or higher). Not needed for the pure Python extractor.

beautifulsoup4requiredRuntime dependency for HTML parsing.

html5librequiredRuntime dependency for HTML parsing.

lxmlrequiredRuntime dependency for HTML parsing.

regexrequiredRuntime dependency for text processing.

Agent activity

3 hits · last 30 days

node

Resources

githubhttps://github.com/alan-turing-institute/ReadabiliPy ↗

packagehttps://pypi.org/project/readabilipy/ ↗