Install & Compatibility
Where this runs
tested against v0.31 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.910 runs
installs and imports cleanly · install 0.0s · import 1.044s · 55.7MB
glibcpy 3.10–3.910 runs
installs and imports cleanly · install 6.3s · import 0.958s · 56MB
59MB installed
● package 59MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
get_model
✓ import llm
model = llm.get_model('gpt-4o-mini')
Used to retrieve a model instance by ID or alias.
Attachment
✓ from llm import Attachment
For providing multi-modal input (e.g., images) to models that support it.
Tool
✓ from llm import Tool
To define custom tools that LLMs can execute.
This quickstart demonstrates how to use the `llm` Python API to get a model and execute a prompt. It shows both a single prompt and a conversational flow. API keys are preferably managed via environment variables (e.g., `OPENAI_API_KEY`) or the `llm keys set` CLI command. Ensure the relevant model plugin (e.g., `llm-openai`) is installed.
import llm
import os
# Ensure your API key is set as an environment variable (e.g., OPENAI_API_KEY)
# or use llm keys set openai from the CLI
openai_api_key = os.environ.get('OPENAI_API_KEY', 'YOUR_OPENAI_API_KEY_HERE')
if not openai_api_key:
print("Warning: OPENAI_API_KEY environment variable not set. Please configure it.")
# For demonstration, we'll try to proceed, but it might fail.
# In a real application, you'd handle this more robustly.
try:
# Get a specific model (e.g., gpt-4o-mini). Ensure the corresponding plugin is installed.
model = llm.get_model("gpt-4o-mini")
# If the key is not in env, pass it directly (if model plugin supports it)
response = model.prompt(
"Five surprising names for a pet pelican",
key=openai_api_key if openai_api_key != 'YOUR_OPENAI_API_KEY_HERE' else None
)
# Access the generated text (lazy loading)
print(response.text())
# Example with a conversation
conversation = model.conversation()
response1 = conversation.prompt("Tell me a fun fact about pandas.")
print(f"Fact 1: {response1.text()}")
response2 = conversation.prompt("Now, tell me another one.")
print(f"Fact 2: {response2.text()}")
except llm.UnknownModelError as e:
print(f"Error: {e}. Make sure you have installed the necessary plugin, e.g., 'llm install llm-openai'.")
except Exception as e:
print(f"An unexpected error occurred: {e}")
llm --version
Debug
Known issues
breakingLLM 0.28 introduced a minimum Python version requirement of 3.10 or higher. Previous versions supported older Python versions.fixUpgrade your Python environment to 3.10 or newer if using LLM 0.28 or later. Consider using a virtual environment (venv) for project isolation.
affects: >=0.28
gotchaSome LLM plugins that depend on PyTorch (e.g., `llm-sentence-transformers`) may not install cleanly when `llm` itself is installed via Homebrew, due to Python version mismatches with PyTorch's stable releases.fixAs a workaround, manually install PyTorch within the `llm`'s virtual environment using `llm install llm-python` followed by `llm python -m pip install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cpu` before installing the PyTorch-dependent plugin.
affects: All versions when installed via Homebrew with PyTorch-dependent plugins.
gotchaAPI keys for remote LLM providers are crucial and must be configured correctly. The library will look for environment variables (e.g., `OPENAI_API_KEY`) or keys stored via the `llm keys set` CLI command.fixSet API keys as environment variables (e.g., `export OPENAI_API_KEY='sk-...'`) or use `llm keys set <provider_name>` for persistent storage. Pass `key=...` directly to `model.prompt()` only if the specific plugin supports it and it's not sensitive for your use case.
affects: All versions
gotchaThe `Response.text()` method employs lazy loading. If you inspect the `Response` object before calling `.text()`, it will show '... not yet done ...'. The actual API call is made when `.text()` is invoked.fixAlways call `response.text()` (or `response.tool_calls()`, etc.) to trigger the actual model interaction and retrieve the result.
affects: All versions
Errors
Common errors & fixes
llm.UnknownModelError: Unknown model: 'gpt-4o-mini'
The requested model ID or alias is not recognized, often because the corresponding plugin (e.g., llm-openai) has not been installed.
fixInstall the plugin for the desired model provider. For OpenAI models, run `llm install llm-openai`. For Gemini, `llm install llm-gemini`, etc.
AuthenticationError: Invalid API key provided
The API key for the LLM provider is missing, incorrect, or expired.
fixVerify your API key. Set it as an environment variable (e.g., `export OPENAI_API_KEY='sk-...'`) or use the CLI command `llm keys set <provider_name>` to store it securely.
Rate limit reached for gpt-4o in organization org-xxx. Limit: 500 RPM.
You have exceeded the rate limits (requests per minute or tokens per minute) imposed by the LLM provider.
fixImplement retry logic with exponential backoff in your application. Reduce the frequency of your API calls. Check the provider's documentation for current rate limits and consider increasing your quota if necessary.
ModuleNotFoundError: No module named 'llm_openai'
You are trying to import or use an OpenAI model, but the `llm-openai` plugin is not installed.
fixInstall the required plugin using `llm install llm-openai`.
Upgrade
Version history
0.31latest on PyPI · released Apr 24, 2026
Audit
Dependencies
llm-openaioptionalRequired for interacting with OpenAI models. Installed as a plugin.
llm-anthropicoptionalRequired for interacting with Anthropic models. Installed as a plugin.
llm-geminioptionalRequired for interacting with Google Gemini models. Installed as a plugin.
llm-ollamaoptionalRequired for interacting with local Ollama models. Installed as a plugin.