spacy-curated-transformers provides efficient and curated transformer models designed for integration into spaCy processing pipelines. It wraps the `curated-transformers` library, offering specialized components and utilities for tasks like wordpiece tokenization and transformer-based embeddings within spaCy's `Doc` and `Span` objects. The library is actively maintained by Explosion, with a focus on compatibility with latest spaCy and Thinc versions, and releases often align with improvements in underlying transformer architectures.
pip install spacy-curated-transformersVerified import paths — ran on the pinned version, not inferred.
This quickstart demonstrates how to use a spaCy model that internally leverages `spacy-curated-transformers` to process text and access the transformer's output data. Users typically interact with the library through a pre-trained spaCy transformer pipeline.
Update your spaCy config or `nlp.add_pipe()` calls to use the factory name `curated_transformer` (or import `CuratedTransformer` from `spacy_curated_transformers.pipeline`).
Adjust custom code to handle `(0, n)` shape for whitespace token transformer outputs, or use spaCy's built-in pooling operations which are designed to correctly handle this.
Review release notes for `curated-transformers` 2.0 and `spacy-curated-transformers` 2.0.0. Ensure compatibility with your existing configurations and custom components.
Remove any code or configuration related to quantization. Monitor future releases for its re-introduction once the API is stable.
Always install `spacy` alongside `spacy-curated-transformers` from scratch in a clean environment, or use `pip install -U spacy-curated-transformers` and then `pip install -U spacy` (or vice-versa) to let pip resolve dependencies. Check `pyproject.toml` or `setup.py` files for specific version requirements for crucial dependencies like `thinc` and `curated-transformers`.