Install & Compatibility
Where this runs
tested against v0.1.44 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.980 runs
installs and imports cleanly · install 0.0s · import 0.944s · 94.2MB
glibcpy 3.10–3.980 runs
installs and imports cleanly · install 4.3s · import 0.905s · 90MB
94MB installed
● package 94MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
cache
✓ from gptcache import cache
Commonly used pre-configured global cache instance for quick setup.
GPTCache
✓ from gptcache import GPTCache
The main class for creating a GPTCache instance, allowing custom configuration.
openai
✓ from gptcache.adapter import openai
Adapter to integrate GPTCache with the OpenAI API calls.
This quickstart demonstrates how to integrate GPTCache with the OpenAI API. After initializing the cache, subsequent OpenAI calls will automatically leverage the semantic caching capabilities. The first query will likely go to the LLM, while identical or semantically similar subsequent queries will be served from the cache.
import os
from gptcache import cache
from gptcache.adapter import openai
# Set your OpenAI API key from an environment variable
os.environ["OPENAI_API_KEY"] = os.environ.get("OPENAI_API_KEY", "sk-...")
# Initialize GPTCache
cache.init()
# The gptcache.adapter.openai module automatically wraps the openai library
# Subsequent OpenAI API calls will use the cache
response1 = openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[
{"role": "user", "content": "Hello, what is the capital of France?"}
]
)
print(f"First response (likely from LLM): {response1.choices[0].message.content}")
# A second identical request will hit the cache for faster response and cost savings
response2 = openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[
{"role": "user", "content": "Hello, what is the capital of France?"}
]
)
print(f"Second response (likely from cache): {response2.choices[0].message.content}")
Debug
Known issues
gotchaWhen integrating with LangChain, particularly with Pydantic v2, older versions of GPTCache might have caused 'metaclass conflict errors' or 'LangChain chat pydantic bugs'.fixUpgrade GPTCache to version 0.1.43 or newer to resolve known compatibility issues with Pydantic v2 and LangChain.
affects: <0.1.43
gotchaUsing certain features like remote Redis cache stores or distributed caching might require explicit installation of `redis` and can encounter connection issues in older versions.fixEnsure `pip install gptcache[redis]` if you plan to use Redis. Upgrade to at least 0.1.36 for critical Redis connection fixes, and 0.1.43 to benefit from `redis` being an optional dependency, avoiding unnecessary installs.
affects: <0.1.43 (for optional Redis install), <0.1.36 (for Redis connection fix)
gotchaChanges in external LLM APIs (e.g., OpenAI's API base for embeddings) can cause unexpected behavior or errors if GPTCache is not updated to reflect these changes.fixRegularly update GPTCache to its latest version to ensure compatibility with evolving LLM APIs. Version 0.1.38 addressed specific OpenAI API base changes.
affects: <0.1.38
Errors
Common errors & fixes
gptcache.utils.error.NotInitError: The cache should be inited before using.
This error occurs when an attempt is made to use the GPTCache instance (or an adapter that relies on it) before the cache has been properly initialized using the `cache.init()` method.
fixEnsure that `cache.init()` is called at the beginning of your application or before any operations that interact with the cache. For semantic caching, you might use `init_similar_cache()`.
```python
from gptcache import cache
from gptcache.adapter.api import init_similar_cache
# For basic exact caching
cache.init()
# Or for semantic caching
# init_similar_cache()
# Your code that uses gptcache
```
openai.lib._old_api.APIRemovedInV1: You tried to access openai.ChatCompletion, but this is no longer supported in openai>=1.0.0
This error indicates a version incompatibility between your installed `openai` library (which is version 1.0.0 or higher) and code that expects the older `openai` API syntax (typically pre-1.0.0), often encountered when `gptcache` adapters or examples were written for an older OpenAI library version.
fixYou can either downgrade your `openai` package to a version less than 1.0.0 (e.g., `pip install openai==0.28.1`) or update your `gptcache` integration code to be compatible with `openai` version 1.0.0+ syntax, which often involves using `client.chat.completions.create` instead of `openai.ChatCompletion.create`.
InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: token_type_ids for the following indices index: 1 Got: 1772 Expected: 512 Please fix either the inputs or the model.
This specific ONNXRuntimeError arises when the input tensor's dimensions, particularly for `token_type_ids` during embedding generation, do not match the expected dimensions of the underlying ONNX model used by GPTCache for similarity search, often due to an input text exceeding the model's maximum sequence length.
fixThis often requires ensuring your input text for embedding generation is within the model's supported sequence length. You might need to preprocess the input to truncate or split it, or configure `gptcache` with a preprocessing function that handles input lengths (e.g., `pre_embedding_func` or `pre_func` in `cache.init()`).
TypeError: 'NoneType' object is not subscriptable
This common Python error, in the context of `gptcache` and LangChain integration, often means that a function like `pre_embedding_func` or `post_process_messages_func` has returned `None` when a dictionary or another subscriptable object (like a list) was expected by a subsequent operation. This can happen if the function's logic doesn't cover all input scenarios or is incorrectly configured.
fixReview the custom `pre_embedding_func` or `post_process_messages_func` passed to `cache.init()` or directly to LangChain's `GPTCache` adapter. Ensure these functions always return a valid, subscriptable object (e.g., a dictionary, list, or string) rather than `None`, especially for edge cases or unexpected inputs. Default functions like `get_prompt` or `last_content` are provided by `gptcache` to correctly handle common scenarios.
psycopg2.errors.ProgramLimitExceeded: index row requires X bytes, maximum size is Y
This error, specifically from `psycopg2` (PostgreSQL adapter), occurs when using `gptcache` with a PostgreSQL-backed vector store (like `PGVector`) where the size of an index entry (e.g., for an embedding) exceeds the maximum allowed size for an index row in PostgreSQL. This is often due to very large embedding dimensions combined with other indexed data.
fixReduce the dimension of the embeddings used in your vector store, or adjust PostgreSQL's configuration if possible (though increasing `MAXALIGN` is complex and often not recommended). If using `pgvector`, ensure your `VectorBase` is configured with a suitable `dimension`. Consider alternative vector stores or strategies for handling very high-dimensional embeddings.
Upgrade
Version history
0.1.44latest on PyPI · released Aug 1, 2024
Audit
Dependencies
pythonrequiredRequired Python version.
redisoptionalOptional dependency for distributed caching or using Redis as a cache store.
langchainoptionalOptional dependency for integration with LangChain.
pydanticoptionalTransitive dependency, often related to LangChain integrations; specific versions might cause conflicts.