Registry /
llm-agents / openinference-instrumentation-litellm
Install & Compatibility
Where this runs
tested against v0.1.34 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.920 runs
installs and imports cleanly · install 0.0s · import 2.176s · 251.4MB
glibcpy 3.10–3.920 runs
installs and imports cleanly · install 20.2s · import 1.978s · 232MB
250MB installed
● package 250MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
LiteLLMInstrumentor
✓ from openinference.instrumentation.litellm import LiteLLMInstrumentor
TracerProvider
✓ from opentelemetry.sdk.trace import TracerProvider
SimpleSpanProcessor
✓ from opentelemetry.sdk.trace.export import SimpleSpanProcessor
OTLPSpanExporter
✓ from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
This quickstart demonstrates how to set up OpenInference LiteLLM Instrumentation with a basic OpenTelemetry configuration, making an LLM call via LiteLLM. It initializes a `TracerProvider`, configures an `OTLPSpanExporter` (e.g., for a local Phoenix collector), instruments LiteLLM, and then executes a sample `completion` and `embedding` call. Remember to set your `OPENAI_API_KEY` environment variable.
import os
import litellm
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from openinference.instrumentation.litellm import LiteLLMInstrumentor
# Configure OpenTelemetry Tracer Provider
resource = Resource.create({
"service.name": "my-litellm-app"
})
tracer_provider = TracerProvider(resource=resource)
# Example: Export traces to a local OpenTelemetry Collector (e.g., Phoenix)
# Ensure a collector is running, e.g., 'python -m phoenix.server.main serve'
OTEL_EXPORTER_OTLP_ENDPOINT = os.environ.get("OTEL_EXPORTER_OTLP_ENDPOINT", "http://127.0.0.1:6006/v1/traces")
tracer_provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter(OTEL_EXPORTER_OTLP_ENDPOINT)))
# Set the global tracer provider
from opentelemetry import trace
trace.set_tracer_provider(tracer_provider)
# Instrument LiteLLM
LiteLLMInstrumentor().instrument(tracer_provider=tracer_provider)
# Set LiteLLM API key (e.g., for OpenAI model)
os.environ["OPENAI_API_KEY"] = os.environ.get("OPENAI_API_KEY", "YOUR_OPENAI_API_KEY_HERE")
try:
print("Making a LiteLLM completion call...")
completion_response = litellm.completion(
model="gpt-3.5-turbo",
messages=[{"content": "What's the capital of France?", "role": "user"}]
)
print("Completion received:", completion_response.choices[0].message.content)
print("\nMaking a LiteLLM embedding call...")
embedding_response = litellm.embedding(
model="text-embedding-ada-002",
input=["Hello, world!"]
)
print("Embedding received (first 10 chars):", str(embedding_response.data[0].embedding)[:10] + "...")
except Exception as e:
print(f"An error occurred: {e}")
print("Please ensure your API key is set correctly and the model is accessible.")
finally:
# It's important to shut down the tracer provider to ensure all spans are exported.
print("\nShutting down tracer provider...")
tracer_provider.shutdown()
print("Traces exported.")
Debug
Known issues
gotchaWhen tracing image generation calls, the instrumentation currently sets the output as a URL attribute rather than rendering the image directly within the trace. Displaying the image requires a UI-side change in the observability tool.fixThis is by design; handle image URLs in your observability tool's UI for visualization.
affects: All versions up to 0.1.30
gotchaThere may be inconsistencies in how output (parsed vs. raw object) is set in span attributes for streamed versus non-streamed LiteLLM calls. While recent updates (v0.1.21) improved full JSON output, manual verification of span attributes for different call types is recommended.fixInspect generated spans in your tracing UI to understand the exact format of 'input' and 'output' attributes for streamed and non-streamed calls.
affects: <0.1.21 (partially resolved), potentially ongoing for specific edge cases
gotchaThe `litellm.responses` function (which is labeled as 'beta' in LiteLLM) is not directly instrumented by `openinference-instrumentation-litellm`.fixIf tracing `litellm.responses` is critical, you may need to add manual OpenTelemetry spans around these calls or await official instrumentation support.
affects: All versions up to 0.1.30
breakingIf you are using LiteLLM version 1.0.0 or higher, be aware that LiteLLM itself introduced breaking changes, including requiring `openai>=1.0.0`, changes to error types (e.g., `openai.InvalidRequestError` to `openai.BadRequestError`), and response objects inheriting from `BaseModel` instead of `OpenAIObject`. While these are LiteLLM's changes, they impact how your application interacts with instrumented LiteLLM calls.fixConsult the LiteLLM v1.0.0+ migration guide and update your application code to handle the new LiteLLM API surface, ensuring compatibility with the instrumentation.
affects: LiteLLM >= 1.0.0
gotchaFor proper auto-instrumentation, the `LiteLLMInstrumentor().instrument()` call and the OpenTelemetry `TracerProvider` setup must occur *before* any LiteLLM calls are made in your application. Loading instrumentation too late can result in untraced operations.fixEnsure the instrumentation setup code is executed at the very beginning of your application's lifecycle, typically immediately after importing necessary modules and before any LiteLLM specific code runs. For complex setups, consider using `-r` flag for module loading.
affects: All versions
Upgrade
Version history
0.1.34latest on PyPI · released May 18, 2026
Audit
Dependencies
litellmrequiredThis library instruments LiteLLM; LiteLLM is required to be installed.
opentelemetry-sdkrequiredRequired for OpenTelemetry tracing infrastructure.
opentelemetry-exporter-otlprequiredCommon exporter for sending OpenTelemetry traces to a collector.