Registry / web-framework / scrapy-splash

scrapy-splash

library0.11.1pypypiunverified

scrapy-splash provides JavaScript support for Scrapy using Splash, a headless browser. Version 0.11.1 is the latest release. The library allows rendering JavaScript-heavy pages by delegating requests to a Splash instance. Releases are sporadic; the last major update (0.10.0) added support for Python 3.12/3.13 and Scrapy 2.12+, and deprecated old dupefilter/cache storage components.

web-framework http-networking

pip install scrapy-splash

github ↗package ↗

INSTALL

IMPORT

SIG · SCRAPY-SPLASH

scrapy-splash

web-frameworkpythonv0.11.1

Install

7.6s avg

Import

1285ms

Disk

92MB

Pass rate

10/ 10

Env Coverage10 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v0.11.1 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.95 runs

installs and imports cleanly · install 0.0s · import 0.258s · 89MB

glibc

py 3.10–3.95 runs

installs and imports cleanly · install 7.6s · import 0.256s · 90MB

92MB installed

● package 92MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

SplashRequest

✓ from scrapy_splash import SplashRequest

✗ from scrapy_splash import SplashRequest

Basic spider using SplashRequest to render JavaScript.

import scrapy from scrapy_splash import SplashRequest class MySpider(scrapy.Spider): name = 'myspider' start_urls = ['http://example.com'] def start_requests(self): for url in self.start_urls: yield SplashRequest(url, self.parse, args={'wait': 0.5}) def parse(self, response): # response is a SplashJsonResponse yield {'title': response.css('title::text').get()}

Debug

Known issues

breakingIn scrapy-splash 0.10.0, SplashAwareDupeFilter and SplashAwareFSCacheStorage are deprecated. You must remove them from your settings and use the default Scrapy components (DUPEFILTER_CLASS and FILES_STORE_S3_ACL etc.) instead. A new SplashRequestFingerprinter component is provided to maintain request fingerprinting for Splash requests.

fix

Remove DUPEFILTER_CLASS = 'scrapy_splash.SplashAwareDupeFilter' from settings. Instead, set SPLASH_REQUEST_FINGERPRINTER_CLASS = 'scrapy_splash.SplashRequestFingerprinter' and let Scrapy use its default dupefilter.

affects: >=0.10.0

gotchaHttpAuthMiddleware credentials leak: If you use http_user and http_pass spider attributes for Splash authentication, those credentials are sent to every non-Splash request (including robots.txt). Use SPLASH_USER and SPLASH_PASS settings instead.

fix

Set SPLASH_USER and SPLASH_PASS in settings.py and remove http_user/http_pass from spider attributes. Upgrade to >=0.8.0.

affects: <0.8.0

deprecatedSplashJsonResponse.body_as_unicode() is deprecated since 0.9.0; use .text instead.

fix

Replace calls to response.body_as_unicode() with response.text.

affects: >=0.9.0

Upgrade

Version history

0.11.1latest on PyPI · released Feb 11, 2025

Audit

Dependencies

Scrapyrequiredscrapy-splash is a Scrapy plugin; requires Scrapy >= 1.0

SplashrequiredSplash HTTP API service must be running; not a Python dependency but required at runtime

Agent activity

29 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/scrapy-plugins/scrapy-splash ↗

packagehttps://pypi.org/project/scrapy-splash/ ↗