Registry / data / fastdownload

fastdownload

JSON →
library0.0.7pypypi✓ verified 24d ago

Fastdownload is a Python library designed for easily downloading, verifying, and extracting data archives. It ensures users always have the latest versions of datasets or other files by checking against file size and hash information. The library is a core component of the fastai ecosystem for managing datasets. The current version is 0.0.7, and releases occur as needed for bug fixes and new features.

pip install fastdownload
INSTALL
IMPORT
SIG · FASTDOWNLOAD
F
fastdownload
datapythonv0.0.7
Install
3.8s avg
Import
664ms
Disk
43MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v0.0.7 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.95 runs
installs and imports cleanly · install 0.0s · import 0.718s · 46.3MB
glibc
py 3.103.95 runs
installs and imports cleanly · install 3.8s · import 0.610s · 51MB
43MB installed
● package 43MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

FastDownload
from fastdownload import FastDownload
The primary class for managing downloads and extractions.
download_url
from fastdownload.core import download_url
from fastdownload import download_url
While 'from fastdownload import download_url' often works due to internal imports, the canonical path for direct access to the helper function is within `fastdownload.core`.

This quickstart demonstrates how to use `FastDownload` to download and extract a remote archive to a temporary directory. It also shows how to download a file without immediate extraction. The `base` parameter is used to direct all downloaded and extracted content to a specified location, overriding the default `~/.fastdownload`.

import os import tempfile import shutil from pathlib import Path from fastdownload import FastDownload # Create a temporary directory for demonstration tmp_dir = Path(tempfile.mkdtemp()) try: fd = FastDownload(base=tmp_dir) # URL for a small, publicly available archive (e.g., MNIST tiny dataset from fast.ai) test_url = 'https://s3.amazonaws.com/fast-ai-sample/mnist_tiny.tgz' print(f"Downloading and extracting from {test_url} to {tmp_dir}...") extracted_path = fd.get(test_url) print(f"Successfully downloaded and extracted to: {extracted_path}") print(f"Contents of the extracted directory: {list(extracted_path.iterdir())}") # Example of downloading only without extraction print(f"\nDownloading only (no extraction) from {test_url} to {tmp_dir / 'archive'}...") downloaded_file_path = fd.download(test_url) print(f"Successfully downloaded to: {downloaded_file_path}") finally: # Clean up the temporary directory if tmp_dir.exists(): print(f"\nCleaning up temporary directory: {tmp_dir}") shutil.rmtree(tmp_dir)
Debug
Known issues
gotchaBy default, fastdownload stores archives in `~/.fastdownload/archive` and extracts data to `~/.fastdownload/data`. This can fill up disk space if not managed.
fix
Initialize `FastDownload` with a `base` parameter pointing to a preferred storage location, e.g., `FastDownload(base=Path('/path/to/my/data'))`.
affects: All versions
gotchaFastdownload will automatically verify file sizes and hashes if a `download_checks.py` file is present (typically generated when publishing datasets with fastdownload). If local files don't match, it may re-download.
fix
To force a re-download regardless of local state or checks, use `fd.get(url, force=True)` or `fd.download(url, force=True)`.
affects: All versions
gotchaThere is unrelated adware also named 'Fastdownload' that can cause browser issues. Ensure you are installing the Python library `fastdownload` (by fastai) and not any other software.
fix
Always verify the package source and use `pip install fastdownload` to ensure you're getting the correct Python library.
affects: N/A (name collision)
gotchaThe `FastDownload` class provides `get`, `download`, and `extract` methods for different behaviors. `get` downloads and extracts, `download` only downloads, and `extract` only extracts (assuming the file is already downloaded). Confusing these can lead to unexpected behavior.
fix
Choose the method that matches your intended operation: `fd.get(url)` for full download and extraction, `fd.download(url)` for just the archive, or `fd.extract(url)` if the archive is already present.
affects: All versions
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'fastdownload'
The 'fastdownload' package is not installed in the Python environment.
fix
Install the package using pip: 'pip install fastdownload'.
ImportError: cannot import name 'FastDownload' from 'fastdownload'
The 'FastDownload' class is not found in the 'fastdownload' module, possibly due to an incorrect import statement.
fix
Ensure the correct import statement: 'from fastdownload import FastDownload'.
AttributeError: module 'fastdownload' has no attribute 'get'
Attempting to call a method 'get' directly on the 'fastdownload' module instead of on an instance of 'FastDownload'.
fix
Create an instance of 'FastDownload' and call 'get' on it: 'd = FastDownload(); d.get(url)'.
from fastdownload import fastdownload
The main class in the fastdownload library is named `FastDownload` (with a capital 'F' and 'D'), not `fastdownload`.
fix
from fastdownload import FastDownload
ImportError: cannot import name 'download_url' from 'fastdownload'
While `download_url` might be re-exported at the top level of `fastdownload` in some versions, its canonical and most reliable import path is from the `fastdownload.core` submodule.
fix
from fastdownload.core import download_url
Install Issues
1 verified issue
ImportError: cannot import name 'fastdownload' from 'fastdownload'All platforms · Python 3.9 · 3.10 · 3.11 · 3.12
Upgrade
Version history
0.0.7latest on PyPI · released Jul 7, 2022
Audit
Dependencies
fastcorerequiredProvides core utilities and extensions for fastai projects, which fastdownload relies on.
fastprogressrequiredUsed for displaying progress bars during downloads.
Agent activity
16 hits · last 30 days
node
14
Resources
fastdownload — pip install fastdownload · libregistry