Registry / llm-agents / openai-messages-token-helper

openai-messages-token-helper

JSON →
library0.1.13pypypi✓ verified 82d ago

A helper library for estimating token counts for OpenAI Chat Completions messages. Currently at version 0.1.13, it supports GPT-3.5, GPT-4, and GPT-4-turbo models. Active development with monthly releases.

pip install openai-messages-token-helper
INSTALL
IMPORT
SIG · OPENAI-MESSAGES-TO
O
openai-messages-token-helper
llm-agentspythonv0.1.13
Install
7.8s avg
Import
Disk
77MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v0.1.13 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.95 runs
installs and imports cleanly · install 0.0s · import 0.000s · 77.8MB
glibc
py 3.103.95 runs
installs and imports cleanly · install 7.8s · import 0.000s · 78MB
77MB installed
● package 77MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

build_messages
from openai_messages_token_helper import build_messages
from openai_messages_token_helper import messages_to_tokens
count_tokens_for_message
from openai_messages_token_helper import count_tokens_for_message
get_token_limit
from openai_messages_token_helper import get_token_limit

Quick estimate tokens for a list of messages.

import os from openai import AzureOpenAI from openai_messages_token_helper import messages_to_tokens client = AzureOpenAI( api_key=os.environ.get('AZURE_OPENAI_KEY', ''), api_version="2024-02-01", azure_endpoint=os.environ.get('AZURE_OPENAI_ENDPOINT', '') ) system_message = "You are a helpful assistant." user_message = "Hello!" tokens = messages_to_tokens([ {"role": "system", "content": system_message}, {"role": "user", "content": user_message} ], model="gpt-4") print(f"Token count: {tokens}")
Debug
Known issues
gotchaThe token count is an approximation and may differ from actual API usage. Do not rely on exact counts for billing or critical limits.
fix
Use for estimation only; always handle token limit errors in your code.
affects: all
gotchaModel parameter must exactly match the model identifier used in the API (e.g., 'gpt-4', 'gpt-3.5-turbo'). Using an unrecognized model may fall back to a default tokenizer and give inaccurate results.
fix
Always specify the correct model string. Check OpenAI documentation for supported model names.
affects: all
Upgrade
Version history
0.1.13latest on PyPI · released Dec 29, 2025
Audit
Dependencies

No dependency data recorded yet.

Agent activity
28 hits · last 30 days
node
24
OpenAI (training)
1
Resources
openai-messages-token-helper — pip install openai-messages-token-helper · libregistry