Official Python SDK for GroqCloud API. OpenAI-compatible interface for ultra-low-latency LLM inference on Groq LPU hardware. Model IDs change frequently as models are deprecated and replaced with no versioned aliases.
Install & Compatibility
Where this runs
tested against v1.4.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.925 runs
installs and imports cleanly · install 0.0s · import 0.915s · 33.4MB
glibcpy 3.10–3.925 runs
installs and imports cleanly · install 3.9s · import 0.841s · 33MB
31MB installed
● package 31MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
Groq
✓ from groq import Groq
✗ from groq.cloud.core import ChatCompletion
groq.cloud.core was the original pre-release SDK (v0.3.0, 2024). Completely removed. All current code uses from groq import Groq.
AsyncGroq
✓ from groq import AsyncGroq
Drop-in async version of Groq client. Same interface, use await.
aiohttp backend
✓ from groq import DefaultAioHttpClient
Pass as http_client=DefaultAioHttpClient() to AsyncGroq for better concurrency.
Minimal chat completion
import os
from groq import Groq
client = Groq(api_key=os.environ['GROQ_API_KEY'])
response = client.chat.completions.create(
model='llama-3.3-70b-versatile',
messages=[{'role': 'user', 'content': 'Hello'}]
)
print(response.choices[0].message.content)
Debug
Known issues
breakinggroq.cloud.core (pre-release API) is fully removed. ChatCompletion class no longer exists. Any code from early 2024 tutorials is broken.fixReplace entire client setup with: from groq import Groq; client = Groq(api_key=...)
affects: v0.3.0 and earlier
breakingModels are deprecated and removed with no versioned aliases. gemma-7b-it and mixtral-8x7b-32768 removed. llama-guard-3-8b decommissioned. Hardcoded model IDs break silently.fixNever hardcode model IDs in production. Query https://api.groq.com/openai/v1/models to get current active models. Check https://console.groq.com/docs/deprecations before each release.
affects: all
breakingmax_tokens is deprecated in favor of max_completion_tokens. Still works but may be removed.fixReplace max_tokens= with max_completion_tokens= in all chat.completions.create() calls
affects: current
breakingfunctions and function_call parameters are deprecated in favor of tools and tool_choice respectively.fixMigrate function_call pattern to tools=[{type: 'function', function: {...}}] pattern affects: all
breakingexclude_domains and include_domains parameters deprecated for agentic tooling. Use search_settings parameter instead.fixMove domain filtering into search_settings={include_domains: [...]} or search_settings={exclude_domains: [...]} affects: current
gotchan parameter (number of completions) only supports n=1. Passing any other value returns a 400 error.fixDo not use n > 1. Run multiple requests instead.
affects: all
gotchalogprobs, presence_penalty, and frequency_penalty are listed in the API but not supported by any current models. Passing them does not error but has no effect.fixDo not rely on these parameters for model behavior control
affects: all
gotchaPreview models can be discontinued at short notice. Do not use in production.fixUse only production-tier models. Check model status at console.groq.com/docs/models.
affects: all
gotchaRate limits are per-model and vary significantly. Free tier limits are very low. 429s happen frequently in dev without a paid plan.fixCheck current limits at console.groq.com/settings/limits. Implement exponential backoff on groq.RateLimitError.
affects: all
breakingThe GROQ_API_KEY environment variable is required for client initialization. Failure to set it results in a KeyError.fixEnsure the GROQ_API_KEY environment variable is set before running the application, for example, by using `export GROQ_API_KEY='your_api_key'` or by loading from a .env file.
affects: all
breakingThe GROQ_API_KEY environment variable is not set, leading to a KeyError during client initialization.fixEnsure the GROQ_API_KEY environment variable is correctly set in your environment before running the application.
affects: all
Errors
Common errors & fixes
groq.GroqError: The api_key client option must be set either by passing api_key to the client or by setting the GROQ_API_KEY environment variable
The Groq API key is not being provided to the client, either directly in the code or via the `GROQ_API_KEY` environment variable.
fixSet the `GROQ_API_KEY` environment variable with your actual API key, or pass it directly to the `Groq` client constructor: `import os
from groq import Groq
client = Groq(api_key=os.environ.get("GROQ_API_KEY"))` or `client = Groq(api_key="YOUR_API_KEY")`. ModuleNotFoundError: No module named 'groq'
The `groq` Python library has not been installed in your environment or is not accessible within your current Python path.
fixInstall the `groq` library using pip: `pip install groq`.
The model xxx does not exist or you do not have access to it.
The specified model ID (xxx) is either incorrect, has been deprecated, or your account does not have permissions to access it. Groq model IDs can change frequently.
fixVerify the exact model ID from the Groq console or documentation and ensure it's still available and that your account has access. For example, use a currently available model like `llama-3.1-8b-instant` or `llama-3.3-70b-versatile`.
groq.APIConnectionError: Connection error
The client failed to establish a network connection to the Groq API, possibly due to network issues, a timeout, or an SSL certificate problem.
fixCheck your internet connection, proxy settings, and ensure there are no firewall rules blocking access to `api.groq.com`. If the issue persists, review SSL certificate configurations or try again later as it might be a temporary network issue.
groq.APIStatusError: Error code: 429 - {'error': {'message': 'You are sending requests too quickly. Please retry your request later.'}}
You have exceeded the rate limits imposed by the Groq API for the number of requests you can make within a given timeframe.
fixImplement exponential backoff and retry logic in your application. Reduce the frequency of your API calls or consider upgrading your Groq plan for higher rate limits.
Audit
Dependencies
httpxrequiredDefault HTTP client. Included automatically.
aiohttpoptionalOptional higher-performance async backend. Use DefaultAioHttpClient from groq.