guidance

library0.3.1pypypi✓ verified 25d ago

Microsoft-backed constrained generation framework for LLMs. Programs interleave control flow with generation via lm += gen(...) syntax. Supports regex, CFGs, JSON schema, and select() constraints using a Rust-based llguidance engine. Backends: Transformers, llama.cpp, OpenAI, Azure AI. Model objects are immutable — each += produces a copy.

llm-agents ai-ml

pip install guidance

github ↗package ↗

INSTALL

IMPORT

SIG · GUIDANCE

guidance

llm-agentspythonv0.3.1

Install

—

Import

—

Disk

—

Pass rate

0/ 10

Env Coverage0 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v0.3.1 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

glibc

py 3.10

✕ build_error

2/3 runs

py 3.11

✕ build_error

2/3 runs

py 3.12

✕ build_error

2/3 runs

py 3.13

✕ build_error

2/3 runs

py 3.9

✕ build_error

2/3 runs

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

models.OpenAI

✓ from guidance.models import OpenAI

✗ guidance.llms.OpenAI('gpt-4o')

guidance.llms namespace removed entirely. guidance.llm (singular) also removed. Only guidance.models is valid.

Transformers

✓ from guidance.models import Transformers

✗ from guidance import models; models.Transformers()

Import directly from guidance.models. Requires transformers package installed separately.

Minimal constrained generation with Transformers backend. Model object is immutable — lm is a copy at each step.

from guidance import system, user, assistant, gen from guidance.models import Transformers lm = Transformers('microsoft/Phi-4-mini-instruct') with system(): lm += 'You are a helpful assistant' with user(): lm += 'What is the capital of France?' with assistant(): lm += gen(name='answer', max_tokens=20) print(lm['answer'])

Debug

Known issues

breakingguidance.llms namespace removed. All pre-0.1.x code using guidance.llms.OpenAI() or guidance.llm.OpenAI() raises AttributeError.

fix

Use from guidance.models import OpenAI (or Transformers, LlamaCpp, AzureOpenAI).

affects: < 0.1.0

breakingbench module removed in 0.2.x. from guidance import bench raises ImportError.

fix

JSON schema benchmarking moved to external repo guidance-ai/guidance-bench.

affects: < 0.2.0

breaking@guidance decorator requires explicit stateless=True for pure grammar composition functions that do not call gen() internally.

fix

Annotate stateless functions as @guidance(stateless=True). Pre-0.2 cookbook examples omit this and produce incorrect behavior.

affects: < 0.2.0

breakingllama-cpp-python version must match guidance pin (currently >=0.3.12). Mismatched versions cause AttributeError or silent failures on LlamaCpp model init.

fix

pip install 'llama-cpp-python>=0.3.12' explicitly. Check guidance release notes for updated pin on each upgrade.

affects: all

gotchaModel objects are immutable. lm += ... does not mutate in place — it returns a new copy. Not storing the result silently discards generated output.

fix

Always assign: lm = lm + ... or use lm += ... inside a with block where lm is re-bound.

affects: all

gotchapip install guidance does not install any inference backend. Importing Transformers or LlamaCpp without the backend raises ImportError at runtime.

fix

Install the required backend separately: pip install transformers or pip install llama-cpp-python.

affects: all

gotchaOpenAI and Azure backends do not support token-level constrained decoding (regex/CFG). Constraints fall back to prompt-based steering only. Full constrained generation requires a local backend.

fix

Use Transformers or LlamaCpp backend for hard constraints. OpenAI backend supports JSON schema via OpenAI structured outputs API only.

affects: all

breakingInstalling llama-cpp-python requires system-level C/C++ compilers (like gcc or clang) and CMake to be available in the environment for building its native components. Missing these tools will result in a 'Failed building wheel' error during installation.

fix

Ensure system-level C/C++ compilers (e.g., `apt-get install build-essential cmake` on Debian/Ubuntu, `yum install gcc gcc-c++ make cmake` on RHEL/CentOS, or Xcode Command Line Tools on macOS) are installed before attempting to `pip install llama-cpp-python`.

affects: all

breakingguidance or its Rust-based dependencies (e.g., llguidance) may fail to build on Alpine Linux (musl libc) environments due to missing system libraries or toolchain incompatibilities, leading to errors like 'libgcc_s.so.1 not found' or 'symbol not found' during Rust compilation.

fix

Use a glibc-based Python image (e.g., Debian/Ubuntu), or for Alpine, install 'glibc-compat', 'gcc', 'g++', and 'rustup' and ensure the Rust toolchain is configured to build for musl targets.

affects: all

Errors

Common errors & fixes

ModuleNotFoundError: No module named 'guidance'

The 'guidance' package is not installed in your Python environment, or the environment where your code is running does not have it installed.

fix

Install the 'guidance' package using pip: `pip install guidance`.

AttributeError: module 'guidance' has no attribute 'llms'

The API for accessing language models within the `guidance` library has changed. The `llms` attribute is deprecated or incorrect.

fix

Update your code to use `guidance.models.OpenAI(...)` or set the default language model using `guidance.llm = guidance.models.OpenAI(...)`.

guidance is not callable

You are attempting to call the `guidance` module directly as a function, but it is intended to be used by calling its specific functions, constructors (e.g., `guidance.models.OpenAI`), or as a decorator (`@guidance`).

fix

Ensure you are using the correct `guidance` API, such as defining a program with the `@guidance` decorator for functions, or initializing a model object from `guidance.models`.

AssertionError: The passed tokenizer does have a byte_decoder property and using a standard gpt2 byte_decoder fails!

This error occurs when using `guidance.models.Transformers()` with a Hugging Face tokenizer that does not have the expected `byte_decoder` property, or there's an incompatibility in how `guidance` processes the tokenizer for constrained generation.

fix

Try using a different Hugging Face model known to be compatible with `guidance` (check the official `guidance` documentation for recommended models), or ensure your `transformers` and `guidance` library versions are up to date.

Upgrade

Version history

0.3.1latest on PyPI · released Feb 3, 2026

Audit

Dependencies

llama-cpp-pythonoptionalRequired for LlamaCpp backend. Version-pinned — guidance specifies minimum >=0.3.12. ABI breaks between llama.cpp versions cause silent failures.

transformersoptionalRequired for Transformers backend. Not bundled in base install.

Agent activity

27 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/guidance-ai/guidance ↗

packagehttps://pypi.org/project/guidance/ ↗