vllm-omni

library0.20.0pypypiunverified

vLLM-Omni is a framework for efficient model inference with omni-modality models, built on top of vLLM. It supports speech, image, video, audio, and multimodal generation, aligned with upstream vLLM releases. Current version is 0.20.0, with active development and monthly release cadence.

ai-ml llm-agents

pip install vllm-omni

github ↗package ↗docs ↗

INSTALL

IMPORT

SIG · VLLM-OMNI

vllm-omni

ai-mlpythonv0.20.0

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

LLM

✓ from vllm_omni import LLM

✗ from vllm import LLM

Basic inference with a multimodal model using vLLM's LLM interface. Ensure vllm is installed (pip install vllm).

from vllm import LLM, SamplingParams # Load a multimodal model (e.g., Qwen2-VL) llm = LLM(model="Qwen/Qwen2-VL-7B-Instruct", trust_remote_code=True) # Generate with a multimodal prompt prompt = { "role": "user", "content": [ {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg"}}, {"type": "text", "text": "Describe this image."} ] } sampling_params = SamplingParams(temperature=0.7, max_tokens=512) outputs = llm.generate([prompt], sampling_params) print(outputs[0].outputs[0].text)

Debug

Known issues

breakingvLLM-Omni versions must exactly match the upstream vLLM version they are built against. Using mismatched versions (e.g., vllm-omni 0.20.0 with vllm 0.19.0) will cause import errors or silent failures.

fix

Install the exact matching vLLM version, typically vllm==<same_version>. E.g., pip install vllm==0.20.0 vllm-omni==0.20.0

affects: all

gotchaDo not import from vllm_omni directly. All core classes (LLM, SamplingParams, etc.) are re-exported from vllm. Importing from vllm_omni will raise ImportError.

fix

Use import from vllm (e.g., from vllm import LLM) instead of from vllm_omni.

affects: all

deprecatedThe old entrypoint vllm.entrypoints.openai.api_server is deprecated. Use vllm serve command-line or vllm.entrypoints.openai.run_batch for batch inference.

fix

Use `vllm serve` CLI or the new async engine API.

affects: >=0.18.0

Upgrade

Version history

0.20.0latest on PyPI · released May 8, 2026

Audit

Dependencies

vllmrequiredCore inference engine; vllm-omni depends on it and must match versions

Agent activity

68 hits · last 30 days

node

Perplexity

OpenAI (training)

Resources

githubhttps://github.com/vllm-project/vllm-omni ↗

packagehttps://pypi.org/project/vllm-omni/ ↗

docshttps://vllm-omni.readthedocs.io ↗