Registry / llm-agents / transformers-cfg

transformers-cfg

library0.2.7pypypiunverified

Extension of the Hugging Face Transformers library for context-free grammar constrained decoding using EBNF grammars. Current version is 0.2.7, released in 2025. Active development with frequent releases.

llm-agents ai-ml

pip install transformers-cfg

github ↗package ↗

INSTALL

IMPORT

SIG · TRANSFORMERS-CFG

transformers-cfg

llm-agentspythonv0.2.7

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

GrammarConstraint

✓ from transformers_cfg import GrammarConstraint

✗ from transformers_cfg.grammar_utils import GrammarConstraint

Basic constrained generation with a JSON grammar.

from transformers import AutoModelForCausalLM, AutoTokenizer from transformers_cfg.grammar_utils import GrammarConstraint from transformers_cfg.generation.logits_process import GrammarLogitsProcessor model_id = "gpt2" tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained(model_id) # Load a grammar (e.g., from a string) import os grammar_str = open(os.path.join(os.path.dirname(__file__), "grammars", "json.ebnf"), "r").read() grammar = GrammarConstraint(grammar_str, tokenizer=tokenizer) logits_processor = GrammarLogitsProcessor(grammar) inputs = tokenizer(["Here is a JSON: "], return_tensors="pt") output = model.generate( **inputs, max_new_tokens=100, logits_processor=[logits_processor], pad_token_id=tokenizer.eos_token_id ) print(tokenizer.decode(output[0], skip_special_tokens=True))

Debug

Known issues

breakingIn v0.2.0, the API was restructured: GrammarConstraint moved from top-level to grammar_utils module and IncrementalGrammarConstraint was added. Older imports will break.

fix

Update imports to from transformers_cfg.grammar_utils import GrammarConstraint

affects: <0.2.0

gotchaGrammarConstraint requires the tokenizer object at initialization; if you pass a tokenizer that is not from the same model, parsing may silently fail or produce incorrect masks.

fix

Always use the same tokenizer that corresponds to the model you are generating with.

affects: all

gotchaThe package reserves special tokens like <|endoftext|> for internal use. If your grammar expects those tokens, use escape sequences or avoid them.

fix

Design grammars without referencing internal special tokens.

affects: all

deprecatedThe old grammar file format using .gbnf is deprecated; use .ebnf instead. The CLI still supports .gbnf but may be removed in a future version.

fix

Convert .gbnf files to .ebnf or use the string-based API.

affects: >=0.2.0

Upgrade

Version history

0.2.7latest on PyPI · released Mar 2, 2025

Audit

Dependencies

transformersrequiredRequired; provides the Hugging Face model interface.

torchrequiredRequired for tensor operations and GPU support.

tokenizersrequiredUsed for tokenization.

absl-pyoptionalUsed for CLI flag parsing.

Agent activity

44 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/epfl-dlab/transformers-CFG ↗

packagehttps://pypi.org/project/transformers-cfg/ ↗