Install & Compatibility
Where this runs
tested against v0.31.3 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.95 runs
build_error
glibcpy 3.10–3.95 runs
installs and imports cleanly · install 15.0s · import 0.000s · 249MB
262MB installed
● package 262MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
load
✓ from mlx_lm import load
✗ from mlx_lm import load
generate
✓ from mlx_lm import generate
Model
✓ from mlx_lm import Model
This quickstart demonstrates how to load an MLX-optimized LLM and tokenizer from the Hugging Face Hub and use it for streaming text generation. Ensure you have an Apple Silicon device for optimal performance.
import mlx_lm as lm
# Load a model and its tokenizer from Hugging Face Hub (MLX community variants are optimized)
# Replace 'mlx-community/Phi-3-mini-4k-instruct-8bit' with your desired model
model, tokenizer = lm.load("mlx-community/Phi-3-mini-4k-instruct-8bit")
# Define a prompt for text generation
prompt_text = "Write a short story about a cat who learns to fly:"
# Generate text
response_stream = lm.generate(
model=model,
tokenizer=tokenizer,
prompt=prompt_text,
verbose=False, # Set to True for detailed generation info
temp=0.7,
max_tokens=200,
stream=True # Stream tokens as they are generated
)
print("Generated text:")
for token in response_stream:
print(token, end="")
print()
mlx_lm --version
Debug
Known issues
gotchaMLX (and thus mlx-lm) is primarily optimized for Apple Silicon (macOS devices with M-series chips). While it can run on CPU, performance will be significantly slower, and larger models might exceed memory limits.fixEnsure you are running on Apple Silicon for best performance. For non-Apple Silicon, manage model size and expectations accordingly.
affects: All versions
breakingCompatibility with the `transformers` library can be sensitive across `mlx-lm` versions. Significant changes to `transformers` (e.g., transition to v5) have required corresponding `mlx-lm` updates.fixAlways check release notes for `mlx-lm` regarding `transformers` compatibility. It's recommended to keep both `mlx-lm` and `transformers` updated to their latest compatible versions or pin versions if specific behavior is needed.
affects: Prior to v0.30.0 for `transformers` v5. Keep `mlx-lm` and `transformers` updated.
gotchaNot all Hugging Face models can be directly loaded or will perform optimally with `mlx-lm`. Many require a conversion step to the MLX format, especially for quantization or specific architectures.fixPrefer models from the `mlx-community` on Hugging Face Hub, which are pre-converted. For other models, use the `mlx_lm.convert` tool to convert them to MLX format before loading.
affects: All versions
gotchaBatch generation and KV caching mechanisms have received numerous improvements and fixes across versions. Older versions might exhibit inefficiencies, incorrect behavior with varying prompt lengths, or issues with specific cache strategies.fixFor critical applications involving batch inference, ensure you are on the latest `mlx-lm` version to benefit from bug fixes and performance enhancements in batching and caching.
affects: Prior to v0.31.2, particularly for complex batching scenarios.
Upgrade
Version history
0.31.3latest on PyPI · released Apr 22, 2026
Audit
Dependencies
mlxrequiredCore MLX framework for efficient array computation on Apple Silicon.
transformersrequiredUsed for tokenizer loading and model configuration compatibility with Hugging Face.