Registry / ai-ml / inference-gpu

inference-gpu

JSON →
library1.5.1pypypiunverified

Roboflow Inference provides a robust framework for deploying computer vision models across various devices and environments without requiring deep machine learning expertise. This GPU-specific variant leverages CUDA for accelerated inference. Currently at version 1.2.2, the library maintains an active release cadence with frequent patches and minor updates.

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 pip install inference-gpu
INSTALL
IMPORT
SIG · INFERENCE-GPU
I
inference-gpu
ai-mlpythonv1.5.1
Install
Import
Disk
Pass rate
0/ 10
Env Coverage0 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v? · pip install
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.910 runs
timeout
glibc
py 3.103.910 runs
timeout
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

get_roboflow_model
from inference import get_roboflow_model
Stream
from inference.core.interfaces.stream.Stream
Used for real-time video stream processing.
InferenceHTTPClient
from inference.core.roboflow_inference import InferenceHTTPClient
from inference import InferenceHTTPClient
Path changed in recent 1.x versions. The fully qualified path is now required.

This quickstart demonstrates how to load a Roboflow model using `get_roboflow_model` and perform inference on an image. It highlights the use of environment variables for API keys and model identification, which is recommended for security and flexibility. Replace 'YOUR_API_KEY' and 'YOUR_PROJECT/YOUR_VERSION' with your actual credentials and model details.

import os from inference import get_roboflow_model # Set your Roboflow API key, project ID, and version ROBOFLOW_API_KEY = os.environ.get('ROBOFLOW_API_KEY', 'YOUR_API_KEY') ROBOFLOW_MODEL_ID = os.environ.get('ROBOFLOW_MODEL_ID', 'YOUR_PROJECT/YOUR_VERSION') # Ensure API key is set if ROBOFLOW_API_KEY == 'YOUR_API_KEY': print("Warning: ROBOFLOW_API_KEY not set. Using placeholder.") # Load the model print(f"Loading model: {ROBOFLOW_MODEL_ID}") model = get_roboflow_model(model_id=ROBOFLOW_MODEL_ID, api_key=ROBOFLOW_API_KEY) # Example inference (using a dummy image path) # Replace 'your_image.jpg' with a real image path or PIL Image object # For local testing, you might need a dummy image or to adjust the source. try: # This part assumes you have an image file named 'test_image.jpg' # For a real example, replace with a valid image path. # Download a sample image for testing, e.g., from Roboflow Universe. # Example: 'https://i.imgur.com/your_image.jpg' results = model.infer(image_path="test_image.jpg") print("Inference successful!") # print(results) except FileNotFoundError: print("Error: test_image.jpg not found. Please provide a valid image path for inference.") except Exception as e: print(f"An error occurred during inference: {e}") # You can also infer on a PIL Image or numpy array # from PIL import Image # import numpy as np # dummy_image = Image.new('RGB', (640, 480), color = 'red') # results = model.infer(image=np.array(dummy_image)) # print("Inference successful with dummy image!")
inference --version
Debug
Known issues
breakingStarting with v1.2.0, `inference-models` became the default backend engine. While the old backend is available in opt-out mode, this change might affect performance or behavior for existing codebases not explicitly configured.
fix
Review model loading and inference logic. If you encounter issues, consult the official documentation for the `inference-models` engine or explicitly configure the legacy backend if necessary (though migrating to the new backend is recommended).
affects: >=1.2.0
gotchaFor GPU acceleration, `torch` and `torchvision` must be installed *separately* and *prior* to `inference-gpu`, ensuring the correct CUDA version is specified (e.g., `cu118` or `cu121`). Failing to do so will result in CPU-only inference or installation failures.
fix
Always follow the recommended installation command: `pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cuXXX` before `pip install inference-gpu`, replacing `XXX` with your CUDA version.
affects: >=1.0.0
deprecatedPython 3.9 reached End of Life and support for it has been deprecated with `inference` v1.1.0.
fix
Upgrade your Python environment to 3.10, 3.11, or 3.12 (as specified by `requires_python`).
affects: >=1.1.0
gotchaAPI Keys, project IDs, and model versions are mandatory for `get_roboflow_model` and `InferenceHTTPClient`. Incorrect or missing credentials will prevent model loading or inference.
fix
Ensure `ROBOFLOW_API_KEY` and `ROBOFLOW_MODEL_ID` (project/version) are correctly set, preferably via environment variables, or passed directly to the model loading function.
affects: All versions
Upgrade
Version history
1.5.1latest on PyPI · released Aug 28, 2026
Audit
Dependencies
torchrequiredEssential for GPU acceleration; must be installed separately with correct CUDA version prior to inference-gpu for optimal performance.
torchvisionrequiredEssential for GPU acceleration; must be installed separately with correct CUDA version prior to inference-gpu for optimal performance.
Agent activity
9 hits · last 30 days
node
8
Resources