Install & Compatibility
Where this runs
tested against v1.7.1 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
py 3.10
✕ build_error
✓ 22.19s
py 3.11
✕ build_error
✓ 19.25s
py 3.12
✕ build_error
✓ 16.9s
py 3.13
✕ build_error
✓ 15.09s
py 3.9
✕ build_error
✕ build_error
268MB installed
● package 268MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
MLModel
✓ from mlserver import MLModel
✗ from mlserver.model import MLModel
The primary base class for implementing custom ML models. The import path changed from `mlserver.model` to `mlserver` in 1.x.
InferenceRequest
✓ from mlserver.types import InferenceRequest
Required for defining input payloads according to the V2 inference protocol.
InferenceResponse
✓ from mlserver.types import InferenceResponse
Required for defining output payloads according to the V2 inference protocol.
cli.main
✓ from mlserver.cli import main
The entry point for MLServer's command-line interface, used to start the server.
This quickstart defines a simple `MyModel` that doubles its input. Save the code as `model.py`, then use the `mlserver start .` command to run the server. An example `curl` command is provided to demonstrate how to send an inference request to the running server.
from mlserver import MLModel
from mlserver.types import InferenceRequest, InferenceResponse, ResponseOutput
import numpy as np
class MyModel(MLModel):
async def load(self):
# In a real scenario, load your model artifacts here
self.model = lambda x: x * 2 # A simple dummy function
self.ready = True
async def predict(self, request: InferenceRequest) -> InferenceResponse:
input_data = request.inputs[0].data.__root__
input_array = np.array(input_data).astype(np.float32)
output_array = self.model(input_array)
return InferenceResponse(
outputs=[
ResponseOutput(
name="output-0",
shape=output_array.shape,
datatype="FP32",
data=output_array.tolist(),
)
]
)
# To run this model:
# 1. Save the above code as `model.py` in an empty directory.
# 2. Open your terminal in that directory.
# 3. Ensure mlserver and numpy are installed: `pip install mlserver numpy`
# 4. Run the MLServer: `mlserver start .`
#
# You can then send an inference request (e.g., using curl in a new terminal):
# curl -X POST 'http://localhost:8080/v2/models/MyModel/infer' \
# -H 'Content-Type: application/json' \
# -d '{
# "inputs": [
# {
# "name": "input-0",
# "shape": [1, 2],
# "datatype": "FP32",
# "data": [10.0, 20.0]
# }
# ]
# }'
mlserver --version
Errors
Common errors & fixes
No MLModel class found in module 'model'
The `model.py` file does not define a class that inherits from `mlserver.MLModel`, or the class is not discoverable (e.g., typo in class name, wrong file path).
fixEnsure your model class is named `MyModel` (or matches `MLSERVER_MODEL_NAME` env var) and correctly subclasses `mlserver.MLModel`. Verify `model.py` is in the directory you're running `mlserver start` from, or specify its path.
RequestValidationError: 1 validation error for InferenceRequest
The incoming `InferenceRequest` JSON payload does not conform to the V2 Inference Protocol specification (e.g., missing required fields like `name`, `shape`, `datatype` for inputs, or incorrect data types/shapes).
fixReview the structure of your `InferenceRequest` to ensure it precisely matches the V2 protocol. Pay close attention to the `inputs` array's contents and their types.
ModuleNotFoundError: No module named 'mlserver_tensorflow'
You are attempting to serve a TensorFlow model, but the `mlserver-tensorflow` runtime library is not installed.
fixInstall the specific runtime package for TensorFlow: `pip install mlserver-tensorflow`.
Input '...' missing 'name' field
The V2 Inference Protocol mandates that all inputs and outputs have a 'name' field, which is missing in your request or model's response.
fixAdd a unique `name` string to each `ResponseOutput` object within your `InferenceRequest` and `InferenceResponse` (e.g., `name='input-0'`, `name='output-0'`).
Upgrade
Version history
1.7.1latest on PyPI · released Jun 6, 2025
Audit
Dependencies
No dependency data recorded yet.