lmdeploy

library0.13.0pypypi✓ verified 84d ago

LMDeploy is a toolkit for compressing, deploying, and serving large language models (LLMs). It supports efficient inference with quantization, continuous batching, and various backends (e.g., PyTorch, TensorRT-LLM). The current version is 0.12.3, with frequent releases following the development of dependencies and model support.

llm-agents ai-ml devops

pip install lmdeploy

package ↗homepage ↗

INSTALL

IMPORT

SIG · LMDEPLOY

lmdeploy

llm-agentspythonv0.13.0

Install

—

Import

—

Disk

—

Pass rate

0/ 10

Env Coverage0 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v? · pip install

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.95 runs

build_error

glibc

py 3.10–3.95 runs

timeout

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

pipeline

✓ from lmdeploy import pipeline

✗ from lmdeploy.serve import pipeline

The top-level pipeline function is the correct entry point; the 'serve' submodule is for server components.

TurbomildEngineConfig

✓ from lmdeploy import TurbomindEngineConfig

✗ from lmdeploy.turbomind import TurbomindEngineConfig

TurbomindEngineConfig is exported from the main lmdeploy namespace in recent versions.

Initialize a pipeline with a Hugging Face model and engine config, then generate a response.

from lmdeploy import pipeline from lmdeploy import TurbomindEngineConfig engine_config = TurbomindEngineConfig(model_format='hf', tp=1) pipe = pipeline('internlm/internlm2_5-1_8b', engine_config=engine_config) response = pipe('Hello, how are you?') print(response.text)

lmdeploy --version

Debug

Known issues

breakingThe `TurbomindEngineConfig` import path changed. In versions before 0.12.0, it was `from lmdeploy.turbomind import TurbomindEngineConfig`. Now it is `from lmdeploy import TurbomindEngineConfig`.

fix

Update imports to `from lmdeploy import TurbomindEngineConfig`.

affects: >=0.12.0 (change), <0.12.0 (old import)

deprecatedThe `turbomind` backend is deprecated; use `TurbomindEngineConfig` with model_format='hf' or 'awq' instead of direct Turbomind engine creation.

fix

Switch to using the pipeline with `TurbomindEngineConfig`.

affects: >=0.10.0

gotchaWhen using `pipeline`, the model must be in Hugging Face format (HF) or quantized with LMDeploy's format. Passing a model name without the correct format may cause silent fallback or errors.

fix

Explicitly set `model_format` in `TurbomindEngineConfig` (e.g., `model_format='hf'`) or use the `--model-format` argument when using CLI.

affects: all

Errors

Common errors & fixes

ModuleNotFoundError: No module named 'lmdeploy.turbomind'

In recent versions, `turbomind` is not a separate importable module; its classes are moved to `lmdeploy` namespace.

fix

Use `from lmdeploy import TurbomindEngineConfig` instead.

ImportError: cannot import name 'pipeline' from 'lmdeploy.serve'

The `pipeline` function is not in `lmdeploy.serve`; it is in the top-level `lmdeploy` module.

fix

Use `from lmdeploy import pipeline`.

ValueError: Unsupported model format 'xxxx'

The model_format argument in TurbomindEngineConfig expects one of the supported formats (e.g., 'hf', 'awq', 'w4a16', 'w8a8'). An incorrect string causes this error.

fix

Check the model format and use a valid one. For Hugging Face models, use `model_format='hf'`.

Upgrade

Version history

0.13.0latest on PyPI · released May 12, 2026

Audit

Dependencies

No dependency data recorded yet.

Agent activity

15 hits · last 30 days

node

OpenAI (training)

Resources

packagehttps://pypi.org/project/lmdeploy/ ↗

homepagehttps://lmdeploy.readthedocs.io ↗