Install & Compatibility
Where this runs
tested against v1.1.39 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.910 runs
installs and imports cleanly · install 0.0s · import 0.000s · 177.9MB
glibcpy 3.10–3.910 runs
installs and imports cleanly · install 11.0s · import 0.000s · 172MB
184MB installed
● package 184MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
TLM
✓ from cleanlab_tlm import TLM
✗ from cleanlab_tlm.tlm import TLMChatCompletion
Eval
✓ from cleanlab_tlm import Eval
TrustworthyRAG
✓ from cleanlab_tlm import TrustworthyRAG
This quickstart demonstrates how to initialize `TLMChatCompletion` with an OpenAI client, send a chat request, and retrieve trust scores and explanation metadata. It highlights the importance of explicitly enabling `trust_scores` and `explanation_metadata` during client initialization.
import os
from cleanlab_tlm.tlm import TLMChatCompletion
from openai import OpenAI
# Initialize the TLM ChatCompletion client
# Ensure OPENAI_API_KEY environment variable is set
tlm_client = TLMChatCompletion(
client=OpenAI(api_key=os.environ.get("OPENAI_API_KEY", "")), # Replace '' with your key if not using env var
trust_scores=True, # enables trust scores
explanation_metadata=True # enables explanation metadata
)
# Example chat interaction
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the capital of France?"}
]
try:
response = tlm_client.chat.completions.create(
model="gpt-4o", # Use an available OpenAI model
messages=messages
)
print(f"LLM Response: {response.choices[0].message.content}")
# Get the trust score
trust_scores = response.get_trust_scores()
print(f"Trust scores: {trust_scores}")
# Get explanation metadata
explanation = response.get_explanation()
print(f"Explanation: {explanation}")
except Exception as e:
print(f"An error occurred: {e}")
print("Please ensure your OPENAI_API_KEY is set and valid, and the model exists.")
Debug
Known issues
gotchaTrust scores and explanation metadata are not enabled by default. You must explicitly set `trust_scores=True` and `explanation_metadata=True` during `TLMChatCompletion` initialization to access these features.fixInitialize `TLMChatCompletion` like: `TLMChatCompletion(client=..., trust_scores=True, explanation_metadata=True)`.
affects: All versions
gotchaPrior to v1.1.33, the presence of `call_id` in formatted responses could lead to incorrectly low trust scores. While fixed, users analyzing historical data or on older client versions should be aware of this potential inaccuracy.fixUpgrade to `cleanlab-tlm>=1.1.33` to ensure accurate trust score calculations, especially when `call_id` is involved in responses.
affects: <1.1.33
gotchaStructured output per-field scoring (added in v1.1.32 and refined in later versions like 1.1.37) requires specific usage patterns and might not be compatible with all models or response structures. Misuse can lead to errors or incorrect scores.fixConsult the official documentation for the correct way to implement per-field scoring for structured outputs, ensuring your model and response types are supported. Always test thoroughly.
affects: <1.1.32 (not available), >=1.1.32 (potential misuse)
Upgrade
Version history
1.1.39latest on PyPI · released Nov 21, 2025
Audit
Dependencies
openairequiredRequired for integrating with OpenAI models and APIs, as TLM wraps OpenAI's functionality.
pydanticrequiredUsed for data validation and settings management, common in modern Python libraries.
pyyamlrequiredLikely used for configuration loading or data serialization.
requestsrequiredStandard library for making HTTP requests, used for internal API communication.