Registry / llm-agents / cleanlab-tlm

cleanlab-tlm

JSON →
library1.1.39pypypi✓ verified 84d ago

The `cleanlab-tlm` library provides a Python client for the Cleanlab Trustworthy Language Model, enabling users to augment LLM interactions with trust scores and explanations. It wraps existing LLM APIs (like OpenAI) to provide a layer of trustworthiness analysis. The current version is 1.1.39, and the library is actively developed with frequent minor releases adding new features and improvements.

pip install cleanlab-tlm
INSTALL
IMPORT
SIG · CLEANLAB-TLM
C
cleanlab-tlm
llm-agentspythonv1.1.39
Install
11.0s avg
Import
Disk
184MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v1.1.39 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.910 runs
installs and imports cleanly · install 0.0s · import 0.000s · 177.9MB
glibc
py 3.103.910 runs
installs and imports cleanly · install 11.0s · import 0.000s · 172MB
184MB installed
● package 184MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

TLM
from cleanlab_tlm import TLM
from cleanlab_tlm.tlm import TLMChatCompletion
Eval
from cleanlab_tlm import Eval
TrustworthyRAG
from cleanlab_tlm import TrustworthyRAG

This quickstart demonstrates how to initialize `TLMChatCompletion` with an OpenAI client, send a chat request, and retrieve trust scores and explanation metadata. It highlights the importance of explicitly enabling `trust_scores` and `explanation_metadata` during client initialization.

import os from cleanlab_tlm.tlm import TLMChatCompletion from openai import OpenAI # Initialize the TLM ChatCompletion client # Ensure OPENAI_API_KEY environment variable is set tlm_client = TLMChatCompletion( client=OpenAI(api_key=os.environ.get("OPENAI_API_KEY", "")), # Replace '' with your key if not using env var trust_scores=True, # enables trust scores explanation_metadata=True # enables explanation metadata ) # Example chat interaction messages = [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What is the capital of France?"} ] try: response = tlm_client.chat.completions.create( model="gpt-4o", # Use an available OpenAI model messages=messages ) print(f"LLM Response: {response.choices[0].message.content}") # Get the trust score trust_scores = response.get_trust_scores() print(f"Trust scores: {trust_scores}") # Get explanation metadata explanation = response.get_explanation() print(f"Explanation: {explanation}") except Exception as e: print(f"An error occurred: {e}") print("Please ensure your OPENAI_API_KEY is set and valid, and the model exists.")
Debug
Known issues
gotchaTrust scores and explanation metadata are not enabled by default. You must explicitly set `trust_scores=True` and `explanation_metadata=True` during `TLMChatCompletion` initialization to access these features.
fix
Initialize `TLMChatCompletion` like: `TLMChatCompletion(client=..., trust_scores=True, explanation_metadata=True)`.
affects: All versions
gotchaPrior to v1.1.33, the presence of `call_id` in formatted responses could lead to incorrectly low trust scores. While fixed, users analyzing historical data or on older client versions should be aware of this potential inaccuracy.
fix
Upgrade to `cleanlab-tlm>=1.1.33` to ensure accurate trust score calculations, especially when `call_id` is involved in responses.
affects: <1.1.33
gotchaStructured output per-field scoring (added in v1.1.32 and refined in later versions like 1.1.37) requires specific usage patterns and might not be compatible with all models or response structures. Misuse can lead to errors or incorrect scores.
fix
Consult the official documentation for the correct way to implement per-field scoring for structured outputs, ensuring your model and response types are supported. Always test thoroughly.
affects: <1.1.32 (not available), >=1.1.32 (potential misuse)
Upgrade
Version history
1.1.39latest on PyPI · released Nov 21, 2025
Audit
Dependencies
openairequiredRequired for integrating with OpenAI models and APIs, as TLM wraps OpenAI's functionality.
pydanticrequiredUsed for data validation and settings management, common in modern Python libraries.
pyyamlrequiredLikely used for configuration loading or data serialization.
requestsrequiredStandard library for making HTTP requests, used for internal API communication.
Agent activity
11 hits · last 30 days
node
10
OpenAI (training)
1
Resources
cleanlab-tlm — pip install cleanlab-tlm · libregistry