Install & Compatibility
Where this runs
tested against v0.5.2 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
py 3.11
✕ build_error
✕ timeout
py 3.12
✕ build_error
✕ timeout
py 3.13
✕ build_error
✕ build_error
168MB installed
● package 168MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
sglang
✓ import sglang as sgl
Primary import for defining SGLang programs and functions.
OpenAI client
✓ from openai import OpenAI
Used for interacting with an SGLang server running in OpenAI-compatible API mode.
This quickstart demonstrates how to interact with an SGLang server using the OpenAI Python client. First, launch the SGLang server in a separate terminal, specifying the model to serve. Then, use the provided Python script to connect to this server and send a chat completion request. Remember to replace the example model path with a valid one and handle Hugging Face authentication if using gated models.
import os
from openai import OpenAI
import time
# --- Step 1: Launch SGLang Server (Run this in a separate terminal) ---
# Command: python -m sglang.launch_server --model-path meta-llama/Llama-3.1-8B-Instruct --port 30000
# Note: Replace 'meta-llama/Llama-3.1-8B-Instruct' with a model you have access to
# and ensure you have logged into Hugging Face CLI if it's a gated model.
# Server output will indicate when it's ready, e.g., 'Uvicorn running on http://0.0.0.0:30000'
# --- Step 2: Interact with the SGLang server using OpenAI-compatible client ---
# Wait a moment for the server to start, or adjust the sleep duration
time.sleep(5)
client = OpenAI(
base_url=os.environ.get('SGLANG_SERVER_URL', 'http://localhost:30000/v1'),
api_key=os.environ.get('SGLANG_API_KEY', 'EMPTY') # 'EMPTY' is common for local SGLang instances
)
try:
response = client.chat.completions.create(
model="meta-llama/Llama-3.1-8B-Instruct", # Model name must match server's loaded model
messages=[
{"role": "user", "content": "What is the capital of France?"}
],
max_tokens=50,
stream=False
)
print("Response from SGLang server:", response.choices[0].message.content)
except Exception as e:
print(f"Error connecting to SGLang server or making request: {e}")
print("Please ensure the SGLang server is running in a separate terminal.")
sglang --version
Debug
Known issues
breakingSGLang Model Gateway v0.3.0 introduced a complete overhaul of its metrics architecture. Users of the SGLang Gateway must update Prometheus dashboards and alerting rules as metric names and structures have changed significantly.fixReview SGLang Gateway documentation for updated metric names and structure, then update monitoring configurations.
affects: Gateway v0.3.0 and later
gotchaAs of v0.5.10rc0, Piecewise CUDA graph capture is enabled by default. While this generally improves throughput and reduces memory overhead, users with specific performance tuning or those expecting non-graph execution behavior may need to re-evaluate their configurations.fixMonitor performance and memory usage after upgrading. Consult SGLang documentation for options to disable or fine-tune CUDA graph behavior if issues arise.
affects: 0.5.10rc0 and later
breakingSGLang v0.5.10rc0 includes a major upgrade of the `transformers` library from version 4.57.1 to 5.3.0. This significant version jump could lead to compatibility issues with custom models, tokenizers, or code relying on older `transformers` APIs.fixTest existing models and code thoroughly after upgrading. Refer to the `transformers` library's release notes for breaking changes between versions 4.57.1 and 5.3.0.
affects: 0.5.10rc0 and later
gotchaUsers observed a significant performance regression (increased TTFT, reduced cache hit rate) between SGLang 0.5.5 and 0.5.6+ when used with Mooncake 0.3.7 under continuous benchmark workloads, despite a fix for scheduler memory growth.fixIf using Mooncake, benchmark performance carefully after upgrading beyond 0.5.5. Consider pinning SGLang to 0.5.5 if optimal TTFT and cache hit rate are critical for your Mooncake setup, or consult SGLang release notes for further Mooncake-related optimizations in later versions.
affects: 0.5.6 to 0.5.9 (when using Mooncake)
gotchaSGLang heavily relies on NVIDIA CUDA and generally requires a Linux environment for full functionality, particularly for its highly optimized kernels. While some components might run on WSL2 for Windows, macOS is typically not supported due to underlying CUDA dependencies.fixEnsure you are running on a Linux system with a compatible NVIDIA GPU and CUDA Toolkit installed. Refer to the official SGLang documentation for specific hardware and software requirements.
affects: All versions
breakingA `ModuleNotFoundError` for the 'openai' package was encountered. This indicates that the 'openai' library, which may be a direct or indirect dependency for specific SGLang features (e.g., interactions with OpenAI APIs or certain model integrations), is not installed in the environment. Users attempting to utilize such features will experience import errors.fixEnsure the 'openai' package is installed in your environment by running `pip install openai`. Consult SGLang documentation for specific features requiring this dependency.
affects: All versions (when using SGLang features that depend on 'openai')
breakingInstallation of SGLang on Alpine Linux (which uses musl libc) consistently fails during the 'Preparing metadata' step. This is due to the absence of crucial build-time system libraries (like `libgcc_s.so.1`) and a C compiler (`cc`) in the environment, which are necessary for the Rust toolchain (cargo) to compile SGLang's native extensions. The 'Error relocating' messages further indicate potential glibc/musl incompatibility issues with precompiled Rust components.fixBefore attempting to install SGLang on Alpine Linux, ensure the necessary build tools and libraries are installed by running `apk add build-base gcc g++ musl-dev libgcc`. For simpler installation, consider using a glibc-based Linux distribution (e.g., Debian, Ubuntu) instead of Alpine.
affects: All versions (on Alpine Linux)
Upgrade
Version history
0.5.18latest on PyPI · released Aug 21, 2026
Audit
Dependencies
torchrequiredUnderlying deep learning framework for GPU operations and model execution.
transformersrequiredUsed for loading and managing models from Hugging Face, with significant version updates impacting compatibility.