Install & Compatibility
Where this runs
tested against v0.1.170 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.910 runs
installs and imports cleanly · install 0.0s · import 0.000s · 19.8MB
glibcpy 3.10–3.910 runs
installs and imports cleanly · install 1.7s · import 0.000s · 20MB
18MB installed
● package 18MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
BaseInferenceService
✓ from matrice_inference.base_inference import BaseInferenceService
✗ from matrice_inference.base_inference import BaseInferenceService
This quickstart demonstrates how to create a basic inference service using `matrice-inference`. It involves defining custom request/response models, implementing `BaseInferenceService` with `warmup` and `predict` methods, configuring the service, and finally using `create_app` to generate a runnable FastAPI application. Save the code as `main.py` and run it with `uvicorn main:app --host 0.0.0.0 --port 8000`. You can then interact with the generated API at `http://localhost:8000/docs`.
import uvicorn
from fastapi import FastAPI
from pydantic import BaseModel
from typing import Any, TypeVar, Generic, Awaitable
import asyncio # Required for async in warmup/predict
from matrice_inference.api.app import create_app
from matrice_inference.base_inference import BaseInferenceService
from matrice_inference.config import InferenceConfig
# Define your custom request and response models
class MyInferenceRequest(BaseModel):
text: str
upper_case: bool = False
class MyInferenceResponse(BaseModel):
processed_text: str
original_length: int
# Implement your inference service
class MyService(BaseInferenceService[MyInferenceRequest, MyInferenceResponse]):
def __init__(self, config: InferenceConfig):
super().__init__(config)
self.is_ready = False
print(f"Service '{config.service_name}' initialized.")
async def warmup(self):
"""Simulate loading a model."""
print("Warming up MyService...")
await asyncio.sleep(0.01) # Simulate async I/O
self.is_ready = True
print("MyService is ready.")
async def predict(self, request: MyInferenceRequest) -> MyInferenceResponse:
"""Perform actual inference."""
if not self.is_ready:
raise RuntimeError("Service not warmed up.")
processed_text = request.text
if request.upper_case:
processed_text = request.text.upper()
return MyInferenceResponse(
processed_text=processed_text,
original_length=len(request.text)
)
# Create a minimal configuration
inference_config = InferenceConfig(
service_name="MyUpperCaseService",
model_name="text_processor",
model_version="1.0.0"
)
# Instantiate your service
my_service = MyService(inference_config)
# Create the FastAPI application
app: FastAPI = create_app(
inference_service=my_service,
request_model=MyInferenceRequest,
response_model=MyInferenceResponse
)
# To run this, save as `main.py` and execute in your terminal:
# uvicorn main:app --host 0.0.0.0 --port 8000
# Then open http://localhost:8000/docs in your browser to test the API.
Debug
Known issues
breakingAs a pre-1.0 library (version 0.1.x), `matrice-inference` does not guarantee API stability between minor releases. Expect frequent breaking changes to class signatures, function parameters, or module structures without explicit warnings in a changelog.fixRefer to the GitHub repository's latest source code for up-to-date API usage. Pin dependencies to exact versions (e.g., `matrice-inference==0.1.166`) and test thoroughly before upgrading.
affects: All versions < 1.0.0
gotchaThe `BaseInferenceService` is an abstract base class requiring all abstract methods (`warmup` and `predict`) to be implemented as `async` functions in your concrete service class. Forgetting `async` or not implementing a method will result in a `TypeError`.fixEnsure your service class properly subclasses `BaseInferenceService` and implements both `async def warmup(self):` and `async def predict(self, request: RequestType) -> ResponseType:` methods.
affects: All versions
gotcha`matrice-inference` has strict Pydantic v2 dependencies (e.g., `pydantic>=2.7.0,<2.8.0`). If your project uses Pydantic v1 or an incompatible Pydantic v2 range due to other dependencies, you will encounter `PydanticImportError` or other runtime issues.fixUse a dedicated virtual environment. Ensure all project dependencies are compatible with the required Pydantic v2 range. If conflicts arise, consider using tools like `pipdeptree` or `poetry` to resolve dependency graph issues, or temporarily isolate `matrice-inference` in a microservice.
affects: All versions
gotchaThis library is primarily an internal utility for Matrice.ai. Its documentation for external users is minimal, and some assumptions about the operational environment or pre-existing infrastructure might not be explicitly stated, leading to unexpected behavior in different setups.fixCarefully review the source code for implicit configurations or dependencies. Be prepared to debug unexpected startup or runtime issues that might stem from environment mismatches. If possible, consult the maintainers for clarification on external use.
affects: All versions
Upgrade
Version history
0.1.170latest on PyPI · released Jun 15, 2026
Audit
Dependencies
fastapirequiredCore web framework for building inference APIs.
uvicornrequiredASGI server for running the FastAPI application.
pydanticrequiredData validation and settings management, heavily used for request/response models and configuration.
numpyrequiredNumerical computing, common in ML workflows.
typing_extensionsrequiredBackports and enhancements for Python's typing module.
opencv-python-headlessrequiredOpenCV library without GUI dependencies, often used for image processing in ML.
logururequiredFlexible logging library used internally.
python-multipartrequiredSupport for parsing multipart/form-data, used by FastAPI for file uploads.
psutilrequiredCross-platform library for retrieving process and system utilization.