Registry / http-networking / cloudscraper

cloudscraper

library1.2.71pypypi✓ verified 23d ago

Cloudscraper is a Python library built on top of the `requests` library, designed to bypass Cloudflare's anti-bot page (also known as "I'm Under Attack Mode" or IUAM). It achieves this by mimicking a real web browser, handling JavaScript challenges, and managing cookies automatically, allowing users to scrape websites protected by Cloudflare. The library is actively maintained, with frequent updates to adapt to Cloudflare's evolving security measures. The current PyPI version is 1.2.71, though a major version 3.0.0 has been released on GitHub with significant changes.

http-networking web-framework

pip install cloudscraper

github ↗package ↗

INSTALL

IMPORT

SIG · CLOUDSCRAPER

cloudscraper

http-networkingpythonv1.2.71

Install

2.3s avg

Import

395ms

Disk

22MB

Pass rate

10/ 10

Env Coverage10 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v1.2.71 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.95 runs

installs and imports cleanly · install 0.0s · import 0.416s · 24MB

glibc

py 3.10–3.95 runs

installs and imports cleanly · install 2.3s · import 0.374s · 24MB

22MB installed

● package 22MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

cloudscraper

✓ import cloudscraper

✗ from cloudscraper import CloudScraper; scraper = CloudScraper()

While `CloudScraper()` directly works, `create_scraper()` is the recommended factory function as it handles initial setup and configuration best, especially in newer versions.

The simplest way to use `cloudscraper` is by calling `cloudscraper.create_scraper()`, which returns a `CloudScraper` instance. This instance behaves similarly to a `requests.Session` object. Any requests made through this session to Cloudflare-protected websites will automatically attempt to bypass the anti-bot measures. The script includes a placeholder URL and demonstrates basic error handling for a 403 status code, which is common if Cloudflare still blocks the request. It's recommended to set a realistic User-Agent for better stealth.

import cloudscraper import os # Instantiate a CloudScraper session. This object works like a requests.Session scraper = cloudscraper.create_scraper( # Optionally, provide a `requests` Session object to base it on # sess=requests.Session(), # Or configure an interpreter, e.g., 'nodejs' if installed for better performance # interpreter='nodejs' ) # Make a GET request to a Cloudflare-protected site # Replace 'http://somesite.com' with your target URL # For demonstration, we'll use a placeholder URL or a test site # You might need to set headers, e.g., a User-Agent, for more realistic requests headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36'} url = os.environ.get('TARGET_URL', 'https://nowsecure.nl') # A common test site for bot detection try: response = scraper.get(url, headers=headers) response.raise_for_status() # Raise an exception for bad status codes print(f"Successfully accessed {url} (Status: {response.status_code})") # print(response.text[:500]) # Print first 500 characters of content except Exception as e: print(f"Failed to access {url}: {e}") if response.status_code == 403: print("Access denied (403 Forbidden). Cloudflare protection might be too strong or settings need adjustment.")

Debug

Known issues

breakingCloudscraper v3.0.0 (released on GitHub, potentially available via `pip install cloudscraper>=3.0.0` or direct GitHub install) introduces breaking changes, including a minimum Python version requirement of 3.8+ (dropping Python 3.6 and 3.7 support) and significant dependency upgrades. Python 2 compatibility code has also been removed. If upgrading from an older 1.x or 2.x version, ensure your environment meets these new requirements.

fix

Upgrade Python to 3.8 or newer. Update all project dependencies to their latest stable versions. Review `cloudscraper`'s GitHub `UPGRADE_SUMMARY.md` for specific changes if migrating an existing codebase. Use `pip install --upgrade cloudscraper` or `pip install cloudscraper>=3.0.0`.

affects: < 3.0.0

gotchaCloudflare continuously updates its anti-bot techniques, leading to an 'arms race' where `cloudscraper` versions can become outdated and cease to work against the latest Cloudflare protections. This often results in `403 Forbidden` errors or being stuck on a Cloudflare challenge page.

fix

Keep `cloudscraper` updated to the latest version. If issues persist, enable debug mode (`cloudscraper.create_scraper(debug=True)`) to diagnose. Consider adjusting scraper settings like `delay`, `browser` profile, or trying different JavaScript interpreters (e.g., Node.js if installed).

affects: All versions

gotchaWhile `cloudscraper` can handle JavaScript challenges, it does not natively solve CAPTCHA challenges (like reCAPTCHA or hCaptcha) that Cloudflare might present. Users will encounter blocks if CAPTCHAs are triggered.

fix

Integrate with a third-party CAPTCHA solving service (e.g., 2Captcha, Anti-Captcha) via `cloudscraper`'s built-in support. This typically requires an API key and an active subscription to the service.

affects: All versions

gotchaCloudscraper is not a full-fledged browser and may not suffice for complex web scraping scenarios involving heavy client-side JavaScript rendering, dynamic content loaded via XHR, or highly sophisticated Cloudflare protection layers. It primarily tackles JavaScript challenges and cookie management.

fix

For highly dynamic or heavily protected websites, consider using headless browser automation tools like Selenium or Playwright, or utilizing specialized managed scraping APIs that handle these complexities.

affects: All versions

Errors

Common errors & fixes

ModuleNotFoundError: No module named 'cloudscraper'

The `cloudscraper` package is not installed in the Python environment being used, or the wrong Python environment is active. This can also occur if Cloudscraper was installed for a different Python version than the one currently executing the script.

fix

Install `cloudscraper` using pip: `pip install cloudscraper`. If using a virtual environment, ensure it is activated before installation. Verify the correct Python interpreter is being used.

cloudscraper not bypassing Cloudflare (e.g., receiving 403 Forbidden, Cloudflare CAPTCHA page in response content)

Cloudflare has updated its anti-bot mechanisms, rendering the current `cloudscraper` version or its configuration ineffective against the target website's protection. This can also be due to aggressive scraping patterns or Cloudflare detecting the automated nature of requests.

fix

Update `cloudscraper` to the latest version: `pip install --upgrade cloudscraper`. Consider implementing delays between requests, rotating IP addresses (proxies), or adjusting browser emulation parameters. If the issue persists, Cloudflare's protection might be too advanced for `cloudscraper` and may require alternative tools like Selenium/Playwright or specialized scraping APIs.

cloudscraper.exceptions.CloudflareChallengeError: Detected a Cloudflare version 2 challenge

The target website is protected by a newer version of Cloudflare's challenge (version 2 or later) that `cloudscraper`'s open-source version cannot automatically bypass.

fix

For Cloudflare version 2 challenges, `cloudscraper`'s free version often cannot provide an automatic bypass. One suggested solution is to integrate a third-party CAPTCHA solving service (e.g., 2Captcha, Anti-Captcha) by providing the API key to `cloudscraper.create_scraper()`. Alternatively, more robust browser automation tools (like Selenium or Playwright) might be necessary.

AttributeError: 'SSLContext' object has no attribute 'orig_wrap_socket'

This error typically arises from an incompatibility between `cloudscraper` (specifically newer versions) and older Python versions, such as Python 3.6, due to changes in the `ssl` module.

fix

Upgrade your Python version to 3.7 or higher, as `cloudscraper` works best with newer Python versions. If upgrading Python is not immediately feasible, try downgrading `cloudscraper` to an older, compatible version (e.g., `pip install cloudscraper==1.2.58`).

requests.exceptions.HTTPError: 403 Client Error: Forbidden for url:

`cloudscraper` failed to bypass the Cloudflare challenge, resulting in the server returning a 403 Forbidden status code, indicating the request was blocked.

fix

Ensure `cloudscraper` and `js2py` are updated (`pip install --upgrade cloudscraper js2py`), and consider passing a custom `user_agent` or specific `headers` to the scraper.

Upgrade

Version history

1.2.71latest on PyPI · released Apr 25, 2023

Audit

Dependencies

requestsrequiredCore HTTP client functionality, Cloudscraper extends its Session object.

requests_toolbeltrequiredRequired for certain HTTP utilities.

pyparsingrequiredUsed for parsing JavaScript challenges.

pyOpenSSLrequiredFor advanced TLS/SSL capabilities.

pycryptodomerequiredFor cryptographic operations if required by challenges.

websocket-clientrequiredFor WebSocket handling if Cloudflare uses it.

js2pyrequiredDefault pure Python JavaScript interpreter for solving challenges.

brotlirequiredFor Brotli compression support in HTTP requests.

certifirequiredProvides a curated list of trusted root certificates.

nodejsoptionalRecommended for faster and more robust JavaScript challenge solving, requires separate installation.

Agent activity

42 hits · last 30 days

node

Amazon

OpenAI (training)

Resources

githubhttps://github.com/venomous/cloudscraper ↗

packagehttps://pypi.org/project/cloudscraper/ ↗