Install & Compatibility
Where this runs
tested against v1.0.1 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
clip
✓ import clip
✗ import clip
This quickstart loads a pre-trained CLIP model (ViT-B/32) and its associated preprocessing function. It then processes a dummy image and a list of text labels, computes the image and text features, and calculates the similarity scores to predict the most relevant text snippet for the image. It runs on CUDA if available, otherwise falls back to CPU.
import torch
import clip
from PIL import Image
import os
# Ensure 'CLIP.png' exists or replace with a valid image path
# For demonstration, let's create a dummy image file if it doesn't exist
if not os.path.exists('CLIP.png'):
try:
from io import BytesIO
import base64
dummy_image_b64 = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII="
img_data = base64.b64decode(dummy_image_b64)
with open('CLIP.png', 'wb') as f:
f.write(img_data)
except ImportError:
print("Pillow not installed or cannot create dummy image. Please provide a real image path.")
exit()
device = "cuda" if torch.cuda.is_available() else "cpu"
model, preprocess = clip.load("ViT-B/32", device=device)
image = preprocess(Image.open("CLIP.png")).unsqueeze(0).to(device)
text = clip.tokenize(["a diagram", "a dog", "a cat"]).to(device)
with torch.no_grad():
image_features = model.encode_image(image)
text_features = model.encode_text(text)
logits_per_image, logits_per_text = model(image, text)
probs = logits_per_image.softmax(dim=-1).cpu().numpy()
print("Label probs:", probs)
# Expected output for a blank image and these labels might be somewhat uniform or biased,
# but demonstrates the process. For a real image, probabilities would be skewed.
Debug
Known issues
gotchaThe `openai-clip` package on PyPI is an unofficial wrapper around OpenAI's official CLIP GitHub repository. For the most up-to-date and officially supported version, it is recommended to install directly from the OpenAI CLIP GitHub repository.fixUse `pip install git+https://github.com/openai/CLIP.git` instead of `pip install openai-clip`.
affects: All versions of `openai-clip` on PyPI (1.0.1+)
breakingNewer versions of `setuptools` (81+) cause build failures due to the removal of `pkg_resources`, which `clip` (from OpenAI's GitHub) might still implicitly use.fixPin `setuptools` to a version below 81 (`pip install 'setuptools<81'`) or ensure your Python environment uses compatible versions if encountering build errors during installation.
affects: Potentially `openai-clip` 1.0.1 when used with `setuptools>=81` and Python versions where `pkg_resources` is removed (e.g., Python 3.12+).
gotchaCLIP's performance can be sensitive to the phrasing of text prompts ('prompt engineering'). Slight variations in wording can significantly impact classification accuracy.fixExperiment with different prompt templates (e.g., 'a photo of {class}', 'this is a {class}') and systematically test prompt variations for your specific use case. Consider ensembling predictions from multiple prompts. affects: All versions
gotchaCLIP may struggle with tasks requiring precise spatial reasoning, counting, or very fine-grained classification (e.g., distinguishing between similar car models or flower species). It also exhibits poor generalization to images not well-represented in its pre-training data.fixBe aware of these limitations. For highly specific or fine-grained tasks, fine-tuning CLIP on domain-specific data or using task-specific models might be necessary. Augmenting prompts or using retrieval-augmented approaches can sometimes help.
affects: All versions
gotchaModels trained on internet-scale data like CLIP can inherit and exhibit social biases present in the training datasets.fixExercise caution when deploying CLIP in sensitive applications. Conduct thorough bias evaluations for your specific use cases and consider ethical implications. Techniques like debiasing embeddings or careful prompt engineering can mitigate some issues.
affects: All versions
Upgrade
Version history
1.0.1latest on PyPI · released Jul 19, 2022
Audit
Dependencies
torchrequiredCore deep learning framework for model execution.
torchvisionrequiredProvides image datasets, models, and transformations for PyTorch.
ftfyrequiredHandles text encoding issues in tokenizer.
regexrequiredUsed for tokenizer operations.
tqdmrequiredFor progress bars during model loading/processing.
PillowrequiredFor image manipulation (PIL.Image).