Install & Compatibility
Where this runs
tested against v2.0.1 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
py 3.10
✕ build_error
✓ 77.7s
py 3.11
✕ build_error
✓ 71.5s
py 3.12
✕ build_error
✓ 66.2s
py 3.13
✕ build_error
✓ 56.1s
py 3.9
✕ build_error
✕ timeout
4813MB installed
● package 4813MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
AutoCausalLM
✓ from curated_transformers.models import AutoCausalLM
AutoEncoder
✓ from curated_transformers.models import AutoEncoder
AutoDecoder
✓ from curated_transformers.models import AutoDecoder
AutoTokenizer
✓ from curated_transformers.tokenization import AutoTokenizer
Generator
✓ from curated_transformers.generation import Generator
This quickstart demonstrates how to load a causal language model and its tokenizer from the Hugging Face Hub, and then use the `Generator` to produce new text. This requires `huggingface_hub` and `safetensors` to be installed.
import torch
from curated_transformers.generation import Generator
from curated_transformers.models import AutoCausalLM
from curated_transformers.tokenization import AutoTokenizer
# Example model, replace with your desired model
# Ensure `explosion/mpt-7b-peft-ct2-int4` is accessible or use a local path.
model = AutoCausalLM.from_hf_hub("explosion/mpt-7b-peft-ct2-int4")
tokenizer = AutoTokenizer.from_hf_hub("explosion/mpt-7b-peft-ct2-int4")
generator = Generator(model, tokenizer)
text = "Hello, I'm a language model,"
tokenized = tokenizer([text])
# Generate text (returns an iterator of generated strings)
for generated_text in generator(tokenized, max_length=50, eos_token_id=tokenizer.eos_token_id):
print(generated_text)
Debug
Known issues
breakingIn v2.0.0, the `HFHubRepository` class was renamed to `HFHubModelRepository`. Additionally, `AutoEncoder`, `AutoDecoder`, and `AutoCausalLM` models now require an explicit `repository` object instead of a direct `revision` argument for loading.fixUpdate `curated_transformers.repository.HFHubRepository` to `curated_transformers.repository.HFHubModelRepository`. Modify model loading calls to pass a `repository` object explicitly, e.g., `AutoCausalLM.from_hf_hub("model", repository=HFHubModelRepository())`. affects: <2.0.0
breakingIn v2.0.0, the `MPTModelOutput` class was changed to only output `last_hidden_state`. Previous versions also included `past_key_values`.fixAdjust any code that accesses `MPTModelOutput` to rely solely on the `last_hidden_state` attribute, as `past_key_values` is no longer available directly from this output.
affects: <2.0.0
gotchaPrior to v2.0.0, `curated-transformers` had strict upper bounds on supported PyTorch versions (e.g., v1.3.1 was limited to PyTorch <2.1.0). Attempting to use incompatible PyTorch versions could lead to runtime errors.fixAlways check the `curated-transformers` changelog or documentation for specific PyTorch version compatibility. For PyTorch 2.1.0+, it's recommended to upgrade to `curated-transformers` v2.0.0 or newer.
affects: All, but especially <2.0.0
gotchaVersions `2.0.0`, `1.3.0`, and `1.3.1` of `curated-transformers` contained a bug that caused an activation lookup error when running on Python 3.12.3.fixUsers running on Python 3.12.3 should upgrade to `curated-transformers>=2.0.1` (for the 2.x series) or `curated-transformers>=1.3.2` (for the 1.x series) to resolve this compatibility issue.
affects: 2.0.0, 1.3.0, 1.3.1
gotchaIn v2.0.0, the parameter `n_vocab` in `curated_transformers.models.bertish.BertEmbedding` and `curated_transformers.models.gpt_neo.GPTNeoEmbedding` was renamed to `n_pieces`.fixIf you are defining or configuring these embedding layers directly, update the parameter name from `n_vocab` to `n_pieces`.
affects: <2.0.0
Upgrade
Version history
2.0.1latest on PyPI · released Apr 17, 2024
Audit
Dependencies
torchrequiredCore deep learning framework
safetensorsoptionalRecommended for faster model loading from Hugging Face Hub
huggingface_huboptionalRequired for loading models from Hugging Face Hub and related utilities
fsspecoptionalFor using alternative filesystems as model repositories