img2dataset

library1.47.0pypypi✓ verified 84d ago

img2dataset is a high-performance library to download and convert a set of image URLs into an image dataset, supporting various output formats such as parquet, webdataset, and local folders. Current version is 1.47.0, with frequent releases every few weeks.

data ai-ml

pip install img2dataset

github ↗package ↗

INSTALL

IMPORT

SIG · IMG2DATASET

img2dataset

datapythonv1.47.0

Install

22.7s avg

Import

6197ms

Disk

735MB

Pass rate

4/ 10

Env Coverage4 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v1.47.0 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

glibc

py 3.10

✕ build_error

✓ 23.5s

py 3.11

✕ build_error

py 3.12

✕ build_error

✓ 20.2s

py 3.13

✕ build_error

✓ 20.2s

py 3.9

✕ build_error

✓ 26.8s

735MB installed

● package 735MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

download

✓ from img2dataset import download

Main entry point function.

Download images from a CSV file with columns 'url' (and optional 'caption'). Saves as individual files in output_folder.

from img2dataset import download download( url_list="https://example.com/image_list.csv", output_folder="/path/to/output", thread_count=4, output_format="files", input_format="csv", url_col="url", caption_col=None, process_count=1, )

img2dataset --version

Debug

Known issues

gotchaIf you use `output_format='parquet'` and have many urls, ensure you have enough disk space for intermediate files. The library may write temporary files to the system temp directory, which could fill up /tmp.

fix

Set the environment variable `TMPDIR` to a directory with sufficient space.

affects: all

breakingIn version 1.40.0, the `image_size` parameter was replaced by `resize_mode` and `resize_size`. Existing code using `image_size` will break.

fix

Replace `image_size=256` with `resize_mode='center_crop'` and `resize_size=256`.

affects: >=1.40.0

gotchaWhen using `output_format='webdataset'`, the output consists of tar files with a fixed number of samples per tar (default 10000). If you have a small dataset, you might get only one tar file. This can cause issues with some PyTorch DataLoaders expecting shards.

fix

Use `output_format='files'` for small datasets or adjust `webdataset_num_samples_per_shard`.

affects: all

deprecatedThe `input_format` parameter with value 'list' is deprecated since version 1.35.0. Use 'csv' or 'text' instead.

fix

Change `input_format='list'` to `input_format='csv'` and adjust input file format accordingly.

affects: >=1.35.0

Errors

Common errors & fixes

img2dataset.errors.DatasetDownloaderError: Failed to download ...

The URL is malformed, the server returns a non-200 status, or the connection times out.

fix

Check that the URL is valid and accessible. Increase `timeout` parameter or set `retry_delay` and `max_retries`.

FileNotFoundError: [Errno 2] No such file or directory: 'urls.csv'

The path to the URL list file is incorrect or the file does not exist.

fix

Provide an absolute path or ensure the file exists relative to the current working directory.

Upgrade

Version history

1.47.0latest on PyPI · released Aug 9, 2025

Audit

Dependencies

No dependency data recorded yet.

Agent activity

6 hits · last 30 days

node

Resources

githubhttps://github.com/rom1504/img2dataset ↗

packagehttps://pypi.org/project/img2dataset/ ↗