Registry / ai-ml / mlserver-mlflow

mlserver-mlflow

library1.7.1pypypi✓ verified 85d ago

mlserver-mlflow provides an MLflow runtime for MLServer, allowing users to serve models logged with MLflow using the MLServer inference server. It's currently at version 1.7.1 and maintains a release cadence aligned with MLServer's development, receiving updates for bug fixes and compatibility with new MLflow/MLServer versions.

ai-ml data

pip install mlserver-mlflow

package ↗

INSTALL

IMPORT

SIG · MLSERVER-MLFLOW

mlserver-mlflow

ai-mlpythonv1.7.1

Install

44.2s avg

Import

6872ms

Disk

858MB

Pass rate

4/ 10

Env Coverage4 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v1.7.1 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

glibc

py 3.10

✕ build_error

✓ 45s

py 3.11

✕ build_error

✓ 43.55s

py 3.12

✕ build_error

✓ 38.55s

py 3.13

✕ build_error

✓ 49.7s

py 3.9

✕ build_error

858MB installed

● package 858MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

MLflowRuntime

✓ from mlserver_mlflow import MLflowRuntime

This quickstart demonstrates how to programmatically use `MLflowRuntime` to load an MLflow model and perform an inference. It first creates a dummy MLflow model and logs it locally, then uses its URI to instantiate `MLflowRuntime` within MLServer's `ModelSettings`, loads the model, and makes a prediction. The `asyncio.run(main())` block executes the asynchronous model loading and inference.

import os import tempfile import mlflow import mlflow.sklearn from sklearn.linear_model import LogisticRegression import numpy as np import asyncio from mlserver_mlflow import MLflowRuntime from mlserver.settings import ModelSettings from mlserver.types import InferenceRequest, RequestInput # 1. Create a dummy MLflow model and log it locally # (In a real scenario, this model would already be logged) temp_dir = tempfile.TemporaryDirectory() model_base_path = os.path.join(temp_dir.name, "mlflow_models") mlflow.set_tracking_uri(f"file://{model_base_path}/mlruns") with mlflow.start_run(): model = LogisticRegression() model.fit(np.array([[0,0],[1,1]]), np.array([0,1])) mlflow.sklearn.log_model(model, "model_artifact") model_uri = f"file://{mlflow.active_run().info.artifact_uri}/model_artifact" # 2. Instantiate and load MLflowRuntime async def main(): model_settings = ModelSettings( name="my-mlflow-model", implementation="mlserver_mlflow.MLflowRuntime", parameters={ "uri": model_uri } ) mlflow_runtime = MLflowRuntime(model_settings) await mlflow_runtime.load() # 3. Prepare and send inference request request_input = RequestInput( name="predict", shape=[1, 2], datatype="FP32", data=[[0.5, 0.5]] ) inference_request = InferenceRequest(inputs=[request_input]) response = await mlflow_runtime.predict(inference_request) print("Prediction:", response.outputs[0].data) await mlflow_runtime.unload() temp_dir.cleanup() # Clean up temporary model files asyncio.run(main())

Debug

Known issues

breakingMLServer 0.x to 1.x API changes directly impact `mlserver-mlflow` users. If migrating from older MLServer versions, you'll need to update your `model-settings.json` configuration, `ModelSettings` objects, and client inference request/response structures to align with MLServer 1.x's API.

fix

Refer to the MLServer 1.x documentation for updated `ModelSettings`, `InferenceRequest`, and `InferenceResponse` formats. Ensure `mlserver` itself is `^1.0.0`.

affects: <1.0.0

gotchaMissing dependencies for MLflow models are a common source of errors. MLflow models, especially `pyfunc` types, often define `conda_env` or `pip_requirements`. If these dependencies (e.g., `xgboost`, `tensorflow`, custom packages) are not installed in the environment where `mlserver-mlflow` is running, model loading will fail with `ModuleNotFoundError` or similar.

fix

Explicitly install all required model dependencies in your MLServer environment (e.g., `pip install xgboost`). For complex environments, consider building a custom Docker image for your `mlserver-mlflow` deployment that includes all necessary packages.

affects: All

gotchaIncorrect or inaccessible MLflow model URIs lead to 'model not found' errors. Users sometimes confuse different URI formats (e.g., run-relative artifact URIs, MLflow Model Registry URIs, local file paths).

fix

Carefully verify the `uri` parameter in your `ModelSettings` (or `model-settings.json`). Ensure it's a valid MLflow model URI (e.g., `models:/my_model/Production`, `runs:/<run_id>/path/to/artifact`, or `file:///absolute/path/to/model_dir`). If using `models:/` or `runs:/` schemes, ensure your MLflow tracking server or registry is running and accessible.

affects: All

Errors

Common errors & fixes

ModuleNotFoundError: No module named 'mlserver_mlflow'

The `mlserver-mlflow` package is not installed in the Python environment where MLServer is being run, or the environment is not correctly activated.

fix

Install the package: `pip install mlserver-mlflow`

mlserver.errors.ModelLoadingError: Failed to load model 'my-model': No module named 'scikit-learn'

The MLflow model being loaded requires a specific Python package (e.g., `scikit-learn`, `xgboost`, `tensorflow`) that is not installed in the `mlserver-mlflow` serving environment.

fix

Identify and install the missing dependency. For example, `pip install scikit-learn`. For comprehensive dependency management, ensure your deployment environment matches the MLflow model's `conda_env` or `pip_requirements`.

mlserver.errors.ModelLoadingError: Failed to load model 'my-model': No MLflow model found at URI: 'models:/my-model/Production'

The specified MLflow model URI is incorrect, the model does not exist at the given URI, or the MLflow tracking server/registry is not accessible from the MLServer instance.

fix

Double-check the `uri` in your `ModelSettings` (or `model-settings.json`). Verify the model's existence in your MLflow Tracking Server or file system. Ensure network connectivity to the MLflow Tracking Server if using remote URIs.

Upgrade

Version history

1.7.1latest on PyPI · released Jun 6, 2025

Audit

Dependencies

mlserverrequiredCore MLServer library, required for runtime functionality.

mlflowrequiredMLflow library, required for loading and interpreting MLflow models.

Agent activity

4 hits · last 30 days

node

Resources

packagehttps://pypi.org/project/mlserver-mlflow/ ↗