CacheControl provides an HTTP caching layer for the popular `requests` library, mimicking the caching algorithms found in `httplib2`. It aims to make `requests` sessions thread-safe and efficient by persisting HTTP responses according to cache-control headers. The library is actively maintained, with frequent updates addressing Python version compatibility, bug fixes, and serialization improvements.
Install & Compatibility
Where this runs
tested against v0.14.4 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.950 runs
installs and imports cleanly · install 0.0s · import 0.638s · 22.7MB
glibcpy 3.10–3.950 runs
installs and imports cleanly · install 2.3s · import 0.568s · 24MB
21MB installed
● package 21MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
CacheControl
✓ from cachecontrol import CacheControl
The primary class to wrap a requests.Session for caching.
FileCache
✓ from cachecontrol.caches.file_cache import FileCache
Used for persisting cache entries to the filesystem, requires `cachecontrol[filecache]`.
CacheControlAdapter
✓ from cachecontrol.adapter import CacheControlAdapter
Alternative way to apply caching via requests' Transport Adapters, often used with `session.mount()`.
This quickstart demonstrates how to set up `CacheControl` with `requests` using a persistent `FileCache`. The first `GET` request will fetch data from the network and cache it. Subsequent requests to the same URL, if cacheable, will be served from the local cache.
import requests
from cachecontrol import CacheControl
from cachecontrol.caches.file_cache import FileCache
# Create a standard requests session
sess = requests.Session()
# Wrap the session with CacheControl using a FileCache for persistent storage
# Replace '.web_cache' with your desired cache directory
cached_sess = CacheControl(sess, cache=FileCache('.web_cache'))
# Make a request - the response will be cached if HTTP headers allow
response = cached_sess.get('https://httpbin.org/cache/60')
print(f"First request status: {response.status_code}")
print(f"From cache (should be False): {getattr(response, 'from_cache', False)}")
# Make the same request again - it should now be served from cache
response = cached_sess.get('https://httpbin.org/cache/60')
print(f"Second request status: {response.status_code}")
print(f"From cache (should be True): {getattr(response, 'from_cache', False)}")
# Clean up the cache directory (optional for a real app)
# import shutil
# shutil.rmtree('.web_cache', ignore_errors=True)
Debug
Known issues
breakingPython 3.8 support was dropped in v0.14.3. Python versions older than 3.10 are no longer officially supported as of v0.14.4. Ensure your environment meets the `>=3.10` requirement.fixUpgrade your Python interpreter to version 3.10 or newer, or pin `cachecontrol` to an older compatible version.
affects: >=0.14.3
breakingSerialization format changes: Version `0.13.1` removed support for older serialization formats (v1 and v2). Caches created with very old versions of `cachecontrol` (before `msgpack` was introduced around v0.12.0) will be unreadable after upgrading.fixClear any existing cache directories or stores before upgrading to v0.13.1 or newer if you suspect they were created with very old versions of the library.
affects: >=0.13.1
gotchaThe `msgpack` dependency has a version constraint (`<2.0.0`) since `v0.14.0`. If other libraries in your project require `msgpack >= 2.0.0`, you might encounter dependency conflicts.fixCheck your project's dependency tree for `msgpack` conflicts. You may need to use a virtual environment or dependency resolver like `pip-tools` or `Poetry` to manage versions. Consider using a different cache backend if `msgpack` conflicts are unavoidable.
affects: >=0.14.0
gotchaOlder versions of `cachecontrol` (pre-`v0.12.13`/`v0.13.0`) might have compatibility issues with `requests` sessions using `urllib3 2.0+`, leading to `IncompleteRead` errors.fixUpgrade `cachecontrol` to at least `0.13.0` (or `0.12.13` if staying on the `0.12.x` line) to ensure compatibility with `urllib3 2.0` and newer `requests` versions.
affects: <0.12.13 || <0.13.0
gotchaA race condition when overwriting cache entries was fixed in `v0.14.2`. Concurrent writes to the same cache file could lead to corruption in earlier versions.fixUpgrade to `cachecontrol` version `0.14.2` or newer, especially if your application involves high concurrency or multiple processes accessing the same `FileCache` directory.
affects: <0.14.2
gotchaMemory usage with `DictCache` or older `FileCache` implementations can be excessive for large binary responses. `SeparateBodyFileCache` was introduced for better memory efficiency by streaming large bodies.fixConsider using `SeparateBodyFileCache` (available from `v0.12.11`) instead of `FileCache` or `DictCache` for memory-intensive caching scenarios, and ensure you have `filelock` installed.
affects: <0.12.11
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'pip._vendor.cachecontrol'
This error typically occurs when your `pip` installation is corrupted or an internal dependency (like `cachecontrol`, which `pip` uses) is missing or cannot be found within `pip`'s vendorized packages. It's not usually a direct issue with the `cachecontrol` library itself but rather with `pip`.
fixReinstalling or upgrading `pip` often resolves this. You can try `python -m ensurepip` or `python -m pip install --upgrade pip --force-reinstall`.
AttributeError: 'dict' object has no attribute 'cache_control'
This error arises when you attempt to access a `.cache_control` attribute on a dictionary object, usually when trying to get caching information directly from a `requests` response object or a generic dictionary instead of the `CacheControl` wrapped session or its response object, which exposes such attributes.
fixEnsure you are interacting with a `CacheControl` wrapped session and accessing the cache control headers appropriately, typically through HTTP response headers or methods provided by `CacheControl`, not directly as an attribute on a raw response dictionary. If attempting to check `Cache-Control` HTTP headers, access `response.headers['Cache-Control']` on a standard `requests` response.
CacheControl not caching (requests are not being cached as expected)
By default, `CacheControl` uses an in-memory cache, meaning the cache is cleared when the program exits. If you expect caching to persist across different runs of your application, you need to explicitly configure a persistent cache backend (e.g., `FileCache`).
fixInitialize `CacheControl` with a persistent cache backend, such as `FileCache`.
```python
import requests
from cachecontrol import CacheControl
from cachecontrol.caches import FileCache
sess = requests.session()
cached_sess = CacheControl(sess, cache=FileCache('.web_cache'))
response = cached_sess.get('https://example.com')
``` Audit
Dependencies
requestsrequiredCore integration; CacheControl extends requests.Session objects.
msgpackoptionalOptional dependency for FileCache and other disk-based caches for efficient serialization, specifically msgpack < 2.0.0.
filelockoptionalOptional dependency for FileCache and SeparateBodyFileCache to prevent race conditions during file access.