vllm-tpu

library0.19.0pypypiunverified

vLLM TPU is a variant of vLLM that runs on Google Cloud TPUs (v5e/v5p). It provides a high-throughput and memory-efficient inference and serving engine for large language models, leveraging TPU-specific optimizations like Pallas kernels for attention and quantization. The current version is 0.19.0, following the main vLLM release cadence (monthly).

llm-agents ai-ml gcp

pip install vllm-tpu

github ↗package ↗docs ↗

INSTALL

IMPORT

SIG · VLLM-TPU

vllm-tpu

llm-agentspythonv0.19.0

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

LLM

✓ from vllm import LLM

✗ from vllm import LLM

Basic inference with a small model on TPU. Assumes a TPU VM (v5e/v5p) with torch_xla installed.

import os os.environ['VLLM_TPU'] = '1' # Optional: explicitly enable TPU backend from vllm import LLM, SamplingParams prompts = ["Hello, my name is", "The capital of France is"] sampling_params = SamplingParams(temperature=0.8, top_p=0.95) llm = LLM(model="Qwen/Qwen2.5-1.5B", max_num_seqs=8) outputs = llm.generate(prompts, sampling_params) for output in outputs: print(output.outputs[0].text)

vllm --version

Debug

Known issues

gotchavLLM TPU is experimental and does not support all features of the main vLLM (e.g., tensor parallelism, quantization). Check the official docs for supported model architectures and features.

fix

Verify model compatibility before use; refer to the vLLM TPU docs.

affects: <0.20

gotchaYou must run on a TPU VM (v5e/v5p) with torch_xla installed. Installing vllm-tpu on CPU/GPU will fail.

fix

Provision a TPU VM and install the TPU runtime: https://cloud.google.com/tpu/docs/users-guide-tpu-vm

affects: all

breakingAs of v0.19.0, the vllm-tpu package is a separate PyPI package from vllm. Mixing installations may cause conflicts.

fix

Uninstall vllm first: pip uninstall vllm; then pip install vllm-tpu.

affects: >=0.19.0

Upgrade

Version history

0.19.0latest on PyPI · released May 5, 2026

Audit

Dependencies

torchrequiredRequired for tensor operations on TPU (via torch_xla).

torch_xlarequiredRequired for TPU runtime integration.

jaxrequiredRequired for Pallas kernels and TPU-specific ops.

Agent activity

59 hits · last 30 days

node

Perplexity

OpenAI (training)

Resources

githubhttps://github.com/vllm-project/vllm ↗

packagehttps://pypi.org/project/vllm-tpu/ ↗

docshttps://docs.vllm.ai/en/latest/ ↗