Registry / data / htmldate

htmldate

JSON →
library1.9.4pypypi✓ verified 49d ago

htmldate is a Python library designed for fast and robust extraction of original and updated publication dates from URLs and web pages. It is actively maintained with frequent minor releases, often addressing bug fixes, dependency updates, and improvements to extraction heuristics.

dataweb-framework
pip install htmldate
Install & Compatibility
Where this runs
tested against v1.9.4 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.930 runs
installs and imports cleanly · install 0.0s · import 1.696s · 41.2MB
glibc
py 3.103.930 runs
installs and imports cleanly · install 3.7s · import 1.561s · 42MB
40MB installed
● package 40MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

find_date
from htmldate import find_date

To use `htmldate`, import the `find_date` function. It can extract dates directly from a URL or from an HTML string. When providing an HTML string, it's often useful to also provide the `url` parameter for better relative path resolution and more accurate heuristics. The `originaldate` parameter allows you to prioritize the earliest found date (original publication) over potentially updated dates.

import requests from htmldate import find_date # Example 1: Extract date from a URL url = 'https://www.example.com/news/article' # For real-world usage, consider handling network errors # Example using a placeholder URL, replace with a real one for testing # html_content = requests.get(url, timeout=10).text # Using a mock HTML content for reproducibility html_content = """ <html><head><meta property="article:published_time" content="2023-10-26T10:00:00Z"></head> <body><h1>Latest News</h1><p>Published: October 26, 2023</p></body></html> """ date_from_url = find_date(url=url, html=html_content) print(f"Date extracted from URL: {date_from_url}") # Example 2: Extract original publication date (if available and different from updated) # The 'originaldate' parameter hints the extractor to prioritize the earliest date. html_content_updated = """ <html><head><meta property="article:published_time" content="2023-10-26T10:00:00Z"> <meta property="article:modified_time" content="2024-03-15T14:30:00Z"></head> <body><h1>Latest News</h1><p>Published: October 26, 2023</p><p>Last Updated: March 15, 2024</p></body></html> """ original_date = find_date(html=html_content_updated, originaldate=True) updated_date = find_date(html=html_content_updated, originaldate=False) print(f"Original Date: {original_date}") print(f"Updated Date: {updated_date}")
htmldate --version
Debug
Known issues
breakingAs of `v1.9.0`, htmldate officially focuses on and requires Python 3.8 or newer. Older Python versions are no longer supported and may encounter compatibility issues or installation failures.
fix
Ensure your project runs on Python 3.8 or a newer version.
affects: >=1.9.0
gotchaThe `originaldate` parameter behavior was fixed in `v1.7.0` to more accurately distinguish between original publication dates and updated dates from meta properties. If you relied on the previous behavior (pre-1.7.0) for this distinction, your results might change.
fix
Review your code if you used `originaldate=True` in versions prior to `1.7.0` and verify the extracted dates are as expected after upgrading.
affects: <1.7.0
gotchaIn `v1.6.0`, the library introduced stricter extraction patterns and replaced `lxml.html.Cleaner` for a focus on precision. This might result in `htmldate` no longer finding a date on some pages where it previously did, or extracting a different (and potentially more accurate) date.
fix
Be aware that date extraction results might vary for some URLs when upgrading from versions older than `1.6.0` due to refined heuristics. Evaluate critical extractions after upgrade.
affects: <1.6.0
breakingThe `requests` library, a mandatory dependency for `htmldate`, was not found in the environment, leading to a `ModuleNotFoundError` and preventing `htmldate` from functioning. This typically indicates an incomplete installation or an improperly configured environment where dependencies are not automatically resolved.
fix
Ensure `requests` is correctly installed in your environment. Installing `htmldate` via `pip install htmldate` within a dedicated virtual environment is the recommended approach to automatically resolve and install its dependencies.
affects: >=0.1.0
breakingThe 'htmldate' library critically depends on the 'requests' package. Failure to install 'requests' will result in a 'ModuleNotFoundError', preventing 'htmldate' from functioning.
fix
Ensure the 'requests' library is installed in your Python environment. When installing 'htmldate' via pip, 'requests' should be automatically installed as a dependency. If running custom scripts, explicitly install 'requests' (e.g., `pip install requests`).
affects: all
Errors
Common errors & fixes
ImportError: cannot import name etree
This error often occurs when the `lxml` library, a core dependency of `htmldate` for HTML parsing, is either not properly installed, has a corrupted installation, or an incompatible version, preventing the `etree` module from being imported correctly.
fix
Reinstall `lxml` to ensure a clean and compatible installation: `pip uninstall lxml` followed by `pip install lxml`. If using a specific Python 3 environment, use `pip3`.
htmldate.find_date() returns None
The `find_date()` function returns `None` when it cannot successfully extract a publication or modification date from the provided URL or HTML content. This can be due to the absence of clear date patterns on the webpage, an unsupported website structure, or a bug in an older version of the `htmldate` library.
fix
First, update `htmldate` to the latest version (`pip install -U htmldate`) as bug fixes related to date extraction heuristics are frequent. If the issue persists, the target webpage may genuinely lack discernible date metadata for the library to find.
ModuleNotFoundError: No module named 'htmldate'
This error indicates that the `htmldate` package is not installed in the Python environment currently being used, or the Python interpreter cannot locate the installed package.
fix
Install the `htmldate` package using pip: `pip install htmldate`. If already installed, verify that you are running your script with the Python interpreter where `htmldate` was installed (e.g., by activating the correct virtual environment).
pip install htmldate[speed] fails or cchardet issues
The optional `[speed]` installation attempts to install `cchardet` for faster character encoding detection, but `cchardet` can have compilation or compatibility issues on certain platforms (e.g., some macOS versions), leading to installation failures or runtime problems.
fix
If encountering issues with the `[speed]` extra, install `htmldate` without it: `pip install htmldate`. `htmldate` will then fall back to `charset_normalizer` for character detection, which is typically more broadly compatible. If `cchardet` was partially installed and causing issues, uninstall it explicitly: `pip uninstall cchardet`.
Upgrade
Version history
1.10.0latest on PyPI
Audit
Dependencies
lxmlrequiredCore dependency for parsing HTML content efficiently.
requestsrequiredUsed internally by `find_date` when fetching content from URLs.
Agent activity
11 hits · last 30 days
seranking-bot
4
ahrefsbot
3
node
2
mj12bot
1
Resources