Install & Compatibility
Where this runs
tested against v0.10.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.910 runs
installs and imports cleanly · install 0.0s · import 0.000s · 40.4MB
glibcpy 3.10–3.910 runs
installs and imports cleanly · install 4.7s · import 0.000s · 41MB
39MB installed
● package 39MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
HTMLSession
✓ from requests_html import HTMLSession
✗ from requests_html import HTMLSession
Demonstrates basic synchronous HTML retrieval and parsing using CSS selectors and extracting absolute links. An commented-out example shows how to use JavaScript rendering, which requires Pyppeteer and will download Chromium on its first invocation.
from requests_html import HTMLSession
session = HTMLSession()
r = session.get('https://www.python.org/')
# Extract title using CSS selector
title = r.html.find('title', first=True).text
print(f"Page title: {title}")
# Extract all absolute links
print("Absolute links:")
for link in r.html.absolute_links:
if 'docs' in link:
print(link)
# Example for JavaScript rendering (requires pyppeteer and Chromium)
# To run this, ensure pyppeteer is installed and Chromium is downloaded.
# r_js = session.get('https://pyppeteer.github.io/')
# r_js.html.render(sleep=1)
# js_content = r_js.html.find('#example-id', first=True).text
# print(f"JS rendered content: {js_content}")
session.close()
Debug
Known issues
gotchaJavaScript rendering (using `r.html.render()`) requires `pyppeteer` and will automatically download a Chromium browser into your home directory the first time it's invoked. This can take some time and consume disk space.fixBe aware of the initial setup time and disk usage. Ensure sufficient network access for the Chromium download.
affects: 0.10.0 and earlier
gotchaThe `requests-html` library's latest release on PyPI is from February 2019. While functional, it might not receive frequent updates compared to its core dependency `requests`. Community contributions via GitHub are ongoing, but new features or critical bug fixes may not be immediately released to PyPI.fixConsider checking the GitHub repository for the latest development if encountering issues not resolved in the PyPI version. For very cutting-edge web scraping needs, alternatives might offer more active development.
affects: 0.10.0 and earlier
gotchaAsynchronous support (`AsyncHTMLSession`) requires Python 3.6+ and the `requests-html[async]` installation. The `.run()` method for `AsyncHTMLSession` executes coroutines and its results list order reflects the completion order, not the order coroutines were passed.fixInstall with `pip install requests-html[async]`. When using `asession.run()`, do not assume the order of results matches the input order of coroutines.
affects: 0.10.0 and earlier
breakingOlder documentation and some historical context indicated stricter Python 3.6 support. While `requests-html` generally functions with newer Python 3 versions (e.g., 3.7+), direct compatibility guarantees were historically tied to 3.6. Always test thoroughly with your specific Python version.fixEnsure `requests-html` is installed in a Python 3.6+ environment. Report any compatibility issues with newer Python versions to the project's GitHub.
affects: Potentially older sub-versions (pre-0.10.0) or specific environments
Upgrade
Version history
0.10.0latest on PyPI · released Feb 17, 2019
Audit
Dependencies
requestsrequiredCore HTTP request functionality.
pyqueryrequiredCSS selector support (jQuery-style).
lxmlrequiredEfficient HTML/XML parsing backend.
pyppeteeroptionalRequired for JavaScript rendering (downloads Chromium on first use).
w3librequiredURL parsing and encoding detection (integrated for robustness).