ctransformers

library0.2.27pypypi✓ verified 86d ago

Python bindings for Transformer models implemented in C/C++ using the GGML library. Provides a high-level API for inference with popular models (LLaMA, Falcon, GPT-J, StarCoder, etc.) in GGML/GGUF format, with optional GPU acceleration (CUDA, Metal, ROCm). Latest version 0.2.27, active development.

ai-ml llm-agents

pip install ctransformers

github ↗package ↗homepage ↗

INSTALL

IMPORT

SIG · CTRANSFORMERS

ctransformers

ai-mlpythonv0.2.27

Install

8.1s avg

Import

1210ms

Disk

83MB

Pass rate

5/ 10

Env Coverage5 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v0.2.27 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

glibc

py 3.10

3/4 runs

✓ 8.53s

py 3.11

3/4 runs

✓ 8.08s

py 3.12

3/4 runs

✓ 7.73s

py 3.13

3/4 runs

✓ 6.88s

py 3.9

3/4 runs

✓ 9.35s

83MB installed

● package 83MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

AutoModelForCausalLM

✓ from ctransformers import AutoModelForCausalLM

✗ from transformers import AutoModelForCausalLM

ctransformers provides its own AutoModelForCausalLM class, not from Hugging Face transformers.

Load a GGML model from Hugging Face Hub and generate text. Replace model ID with a valid GGML/GGUF model.

from ctransformers import AutoModelForCausalLM llm = AutoModelForCausalLM.from_pretrained('marella/gpt-2-ggml') print(llm('AI is going to', max_new_tokens=50))

Debug

Known issues

deprecatedLLM.reset() method is deprecated since v0.2.27. Use high-level API instead (e.g., create a new instance or reuse with generation parameters).

fix

Remove calls to LLM.reset() and use high-level API (AutoModelForCausalLM).

affects: >=0.2.27

breakingOlder GGML models (v1) may not work with newer versions of ctransformers. GGUF v2 support is added in v0.2.25, but GGML v1 support may be removed gradually.

fix

Use GGUF v2 models or convert your model to GGUF format using llama.cpp convert scripts.

affects: >=0.2.25

gotchaCUDA, Metal, and ROCm support are optional extras; install with appropriate extras like [cuda], [metal], or [rocm] to enable GPU acceleration.

fix

Install with the correct extra, e.g., pip install ctransformers[cuda]. Note: CUDA support is experimental for some model types.

affects: all

gotchaModel types (model_type) must be explicitly specified if the model file name does not follow standard naming conventions. Otherwise loading may fail.

fix

Pass model_type parameter, e.g., AutoModelForCausalLM.from_pretrained('./model.gguf', model_type='llama').

affects: all

Errors

Common errors & fixes

ModuleNotFoundError: No module named 'ctransformers'

ctransformers is not installed or not installed with the correct extra for your platform.

fix

Install with: pip install ctransformers. If you need GPU, use pip install ctransformers[cuda] (or [metal]/[rocm]).

AttributeError: module 'ctransformers' has no attribute 'AutoModelForCausalLM'

Import path is wrong; you are using an older version (<0.2.0) or importing incorrectly.

fix

Use: from ctransformers import AutoModelForCausalLM. Ensure you have ctransformers >=0.2.0.

ValueError: Unknown model type: '...'. Must be one of: ...

The model file type is not recognized, often due to non-standard file name or format.

fix

Specify model_type explicitly, e.g., AutoModelForCausalLM.from_pretrained('path/to/model', model_type='llama').

Upgrade

Version history

0.2.27latest on PyPI · released Sep 10, 2023

Audit

Dependencies

No dependency data recorded yet.

Agent activity

11 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/marella/ctransformers ↗

packagehttps://pypi.org/project/ctransformers/ ↗

homepagehttps://ctransformers.dev ↗