Install & Compatibility
Where this runs
tested against v0.3.2 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.920 runs
installs and imports cleanly · install 0.0s · import 0.000s · 145.4MB
glibcpy 3.10–3.920 runs
installs and imports cleanly · install 7.7s · import 0.000s · 132MB
138MB installed
● package 138MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
Router
✓ from sglang_router import Router
✗ from sglang_router import Router
This quickstart demonstrates how to programmatically initialize the SGLang Router by providing a list of SGLang worker URLs. The router acts as a load balancer for these workers. For actual deployment and to handle incoming requests, the router is often launched as a separate process using the `python -m sglang_router.launch_router` command, or integrated into an ASGI application. Ensure that SGLang worker instances are running and accessible at the specified URLs for the router to function correctly.
import os
from sglang_router import Router
# NOTE: This quickstart assumes SGLang worker instances are running
# at the specified URLs (e.g., http://localhost:8000).
# Replace with actual worker URLs if available.
worker_urls = [
os.environ.get('SGLANG_WORKER_URL_1', 'http://localhost:8000'),
os.environ.get('SGLANG_WORKER_URL_2', 'http://localhost:8001')
]
try:
# Initialize the SGLang Router
# By default, it runs in regular HTTP routing mode.
router = Router(worker_urls=worker_urls)
print(f"SGLang Router initialized with workers: {worker_urls}")
# In a real application, you would typically start the router
# (e.g., in a separate thread or process) and then send requests to it.
# For demonstration, we just show initialization.
# Example of running the router process (requires a running event loop or main function)
# This part is conceptual as `Router` doesn't expose a simple `run()` method directly
# in this programmatic interface; it's often launched via `python -m`.
print("Router instance created. To run, typically use 'python -m sglang_router.launch_router'\n"+
"or integrate into an ASGI app. Refer to SGLang documentation for full deployment.")
except Exception as e:
print(f"Error initializing SGLang Router: {e}")
print("Ensure SGLang worker instances are running and accessible at the provided URLs.")
Debug
Known issues
breakingThe metrics architecture has been redesigned, significantly changing metric names and structures. Users relying on Prometheus dashboards or alerting rules will need to update them when upgrading to newer versions of SGLang (which sglang-router interacts with).fixUpdate Prometheus dashboards and alerting rules to reflect the new 6-layer metrics architecture (protocol, router, worker, streaming, circuit breaker, policy) and unified error codes.
affects: SGLang 0.5.x and later (applies to sglang-router's interaction with SGLang)
breakingWorker resource management now uses UUIDs instead of network endpoints for identification. This is a breaking change for systems that directly manage or monitor workers based on their network addresses.fixUpdate any custom worker management or monitoring logic to use UUIDs for worker identification.
affects: SGLang 0.5.x and later (applies to sglang-router's interaction with SGLang)
gotchaInstalling `sglang-router` in editable mode (`pip install -e .`) can lead to performance degradation. This is generally suitable for development but not recommended for performance testing or production environments.fixFor performance-critical scenarios, always build and install the wheel package (`python -m build && pip install --force-reinstall dist/*.whl`) rather than using an editable install.
affects: All versions
gotchaUnder high concurrent load (~32,768 connections), SGLang, including its router, may experience performance degradation and request failures due to underlying file descriptor limits. This is often a system-level limitation.fixIncrease system-level file descriptor limits (ulimit) for the process running the SGLang Router and workers. Implement proper connection management, capacity limits, and graceful degradation strategies.
affects: All versions
breakingMigrating from SGLang backend versions 0.3.x to 0.5.x (which sglang-router 0.3.2 will likely interact with) may require configuration updates due to backward-incompatible changes in the SGLang core.fixReview SGLang migration guides and release notes when upgrading SGLang backend versions to ensure `sglang-router` configurations remain compatible.
affects: SGLang backend versions 0.3.x to 0.5.x
gotchaThe project is actively evolving from a simple load balancer to a more comprehensive 'SGLang Model Gateway', which aims to provide a full OpenAI API server experience with advanced features like native tool calling, session management, and direct gRPC communication. This ongoing architectural shift may introduce new paradigms and potentially breaking changes in future major versions for users interacting with these advanced features.fixStay informed about the official SGLang documentation and GitHub releases for updates on the Model Gateway's features and any migration paths.
affects: Future major versions (post 0.3.x)
Upgrade
Version history
0.3.2latest on PyPI · released Jan 15, 2026
Audit
Dependencies
Rust and CargooptionalRequired for building from source or using the Rust binary directly; not a Python runtime dependency for wheel installs.
SGLangrequiredThe backend inference engine that sglang-router is designed to route requests to. Not a direct Python dependency, but a functional prerequisite.