Install & Compatibility
Where this runs
tested against v0.1.6 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
2227MB installed
● package 2227MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
fid
✓ from cleanfid import fid
The primary interface for `clean-fid` is the `fid` module, which exposes functions like `compute_fid` and `compute_kid`.
This quickstart demonstrates how to compute the FID score between two folders of images using `clean-fid`. Replace `path_to_real_images` and `path_to_generated_images` with actual directories containing your image files. The library also supports computing FID against pre-computed statistics for standard datasets or using a generative model directly.
import os
from cleanfid import fid
# Create dummy image folders for demonstration
if not os.path.exists('path_to_real_images'):
os.makedirs('path_to_real_images', exist_ok=True)
# In a real scenario, populate this folder with real images
# For this example, we'll just create a dummy file
with open('path_to_real_images/dummy_real.txt', 'w') as f: pass
if not os.path.exists('path_to_generated_images'):
os.makedirs('path_to_generated_images', exist_ok=True)
# In a real scenario, populate this folder with generated images
# For this example, we'll just create a dummy file
with open('path_to_generated_images/dummy_gen.txt', 'w') as f: pass
# Compute FID between two image folders
fdir1 = "path_to_real_images"
fdir2 = "path_to_generated_images"
# Note: For actual FID computation, these folders need to contain actual images.
# The dummy files above will cause an error when clean-fid tries to load images.
# This is a placeholder for a runnable example structure.
# Example of computing FID (will likely fail with dummy folders but shows API)
try:
score = fid.compute_fid(fdir1, fdir2)
print(f"Computed FID: {score}")
except Exception as e:
print(f"Error computing FID (expected with dummy data): {e}")
# Example with pre-computed dataset statistics (requires actual dataset name like 'FFHQ')
# score_with_stats = fid.compute_fid(fdir2, dataset_name="FFHQ", dataset_res=1024, dataset_split="trainval70k")
# print(f"Computed FID with FFHQ stats: {score_with_stats}")
Debug
Known issues
gotchaFID scores from other libraries might be inconsistent. `clean-fid` aims to provide a 'clean' and reproducible FID by correctly implementing image resizing and quantization steps, which are often sources of discrepancy in other implementations (e.g., PyTorch-FID, TensorFlow-FID).fixAlways use `clean-fid` for new evaluations or for comparing with 'clean' results. Be aware that scores from other libraries might not be directly comparable without using `clean-fid`'s legacy modes.
affects: All versions
gotchaUsing `mode='legacy_pytorch'` or `mode='legacy_tensorflow'` will reproduce potentially inconsistent FID scores from older implementations. While useful for backward compatibility and comparison with existing literature, these modes do not reflect the 'clean' FID standard.fixFor new, robust, and comparable evaluations, prefer the default `mode='clean'` (or omit the `mode` argument as 'clean' is default). Only use `legacy` modes when explicitly needing to match a specific prior work's (potentially flawed) FID calculation.
affects: All versions
gotchaJPEG compression, even if perceptually subtle, can significantly impact FID scores. If your generated or real images are compressed (e.g., for storage efficiency), it can lead to different FID values compared to uncompressed images.fixEnsure consistent image compression (or lack thereof) across all image sets (real, generated, and any pre-computed statistics) for meaningful FID comparisons. When comparing with other works, verify their image compression practices.
affects: All versions
breakingThe `clip` dependency for CLIP-FID is installed directly from GitHub (`clip @ git+https://github.com/openai/CLIP.git`). This direct Git installation can sometimes cause issues with dependency resolvers or break if the upstream repository changes in an incompatible way.fixIf encountering installation issues with `clip`, try installing it manually before `clean-fid` or ensure your environment supports direct Git installs. Monitor the `openai/CLIP` repository for breaking changes if you rely on CLIP-FID functionality.
affects: All versions requiring CLIP-FID
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'cleanfid'
The clean-fid library is not installed in the Python environment or is not accessible from where the script is being run.
fixEnsure the library is installed using pip: `pip install clean-fid` or `pip install -e .` if installing from source. Verify the correct Python environment is activated.
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory.
This error typically occurs when the pre-trained InceptionV3 model checkpoint, downloaded by clean-fid, is corrupted, incomplete, or cannot be accessed due to file system issues or permissions.
fixDelete the corrupted model file(s) from the cache directory (usually in `~/.cache/torch/hub/checkpoints/` or the specified `model_path` in `FID` constructor) and rerun the code to trigger a fresh download. Ensure sufficient disk space and network connectivity for the download.
ValueError: Path does not exist
The directory path provided to `clean-fid.fid.compute_fid` or `compute_kid` for input images does not exist on the file system or is misspelled.
fixVerify that the `fdir1` and `fdir2` (or `dataset_path` for custom stats) arguments point to existing and correctly spelled directories. Use `os.path.exists()` to debug the paths. Ensure the paths are absolute or relative to the script's execution directory.
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!
This error occurs when some tensors (e.g., input images) are on the CPU while the model or other tensors are on the GPU (CUDA), or vice-versa, leading to a device mismatch during computation.
fixExplicitly move all relevant tensors and the model to the same device (either 'cpu' or 'cuda') using `.to(device)` before performing operations. The `cleanfid.fid.FID` constructor also accepts a `device` argument.
ValueError: The input must have 3 channels; got `input_shape=(H, W, 1)`
The feature extraction models (like InceptionV3) used by clean-fid expect 3-channel RGB images, but the provided input images are grayscale (1 channel) or have an unexpected number of channels (e.g., 4 channels for RGBA).
fixConvert all input images to a 3-channel RGB format before passing them to clean-fid. For example, using PIL: `image.convert('RGB')` for grayscale or `image.convert('RGB')` after dropping the alpha channel for RGBA. Upgrade
Version history
0.1.35latest on PyPI · released Dec 18, 2022
Audit
Dependencies
torchrequiredCore deep learning framework.
torchvisionrequiredImage datasets and models for PyTorch.
numpyrequiredNumerical computing.
PillowrequiredImage processing.
scipyrequiredScientific computing for statistical operations.
tqdmrequiredProgress bars.
requestsrequiredFetching models and data.
lmdboptionalUsed for efficient data storage/retrieval (e.g., cached features).
lpipsoptionalUsed for LPIPS metric, if enabled.
ninjaoptionalBuild system, often used by PyTorch extensions.
cythonoptionalFor C extensions, if any.
ffmpegoptionalMultimedia processing, if needed for video datasets/inputs.
clip @ git+https://github.com/openai/CLIP.gitoptionalRequired for CLIP-FID calculations.
tensorflow-cpuoptionalOnly required for `mode="legacy_tensorflow"` to reproduce TensorFlow-based legacy FID scores. May conflict with PyTorch CUDA installations.