autoawq

library0.2.9pypypi✓ verified 85d ago

AutoAWQ implements the AWQ (Activation-aware Weight Quantization) algorithm for 4-bit quantization of large language models, achieving up to 2x speedup during inference. The library is now deprecated as of v0.2.9 (April 2025), with vLLM having adopted the technology. Last tested with Torch 2.6.0 and Transformers 4.51.3.

ai-ml llm-agents

pip install autoawq

github ↗package ↗

INSTALL

IMPORT

SIG · AUTOAWQ

autoawq

ai-mlpythonv0.2.9

Install

80.6s avg

Import

22525ms

Disk

5325MB

Pass rate

2/ 10

Env Coverage2 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v0.2.9 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

glibc

py 3.10

✕ build_error

✕ timeout

py 3.11

✕ build_error

✕ timeout

py 3.12

✕ build_error

✓ 86.1s

py 3.13

✕ build_error

✓ 75.05s

py 3.9

✕ build_error

✕ timeout

5325MB installed

● package 5325MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

AutoAWQForCausalLM

✓ from awq import AutoAWQForCausalLM

AutoAWQConfig

✓ from awq import AutoAWQConfig

✗ from awq.utils import AutoAWQConfig

AutoAWQConfig is directly importable from awq, not from awq.utils.

Load a pre-quantized AWQ model and generate text.

from awq import AutoAWQForCausalLM, AutoAWQConfig from transformers import AutoTokenizer model_path = 'casperhansen/mixtral-instruct-awq' quant_config = AutoAWQConfig(bits=4, group_size=128, zero_point=True) model = AutoAWQForCausalLM.from_pretrained(model_path, config=quant_config, device_map='auto') tokenizer = AutoTokenizer.from_pretrained(model_path) inputs = tokenizer("Hello, how are you?", return_tensors='pt') outputs = model.generate(**inputs, max_new_tokens=100) print(tokenizer.decode(outputs[0]))

Debug

Known issues

breakingAutoAWQ is officially deprecated as of v0.2.9. No further updates or bug fixes will be provided. Users are advised to migrate to vLLM, which has adopted AWQ natively.

fix

Migrate to vLLM (pip install vllm) and use vLLM's built-in AWQ support.

affects: >=0.2.9

gotchaImport path confusion: Some online examples show 'from auto_gptq import ...' but AutoAWQ is a separate library. Do not confuse with GPTQ (auto_gptq).

fix

Use 'from awq import AutoAWQForCausalLM' (note the lowercase 'awq').

affects: all

gotchaTransformers compatibility is fragile. AutoAWQ v0.2.9 was last tested with Transformers 4.51.3. Using newer versions may cause silent inference errors or import failures.

fix

Pin transformers to <=4.51.3, or upgrade to vLLM which tracks latest transformers versions.

affects: >=0.2.9

Errors

Common errors & fixes

ImportError: cannot import name 'AutoAWQForCausalLM' from 'awq'

Typo: the import uses 'AutoAWQForCausalLM' but the correct symbol may be case-sensitive; also check that the library is installed with correct version.

fix

Run: pip install autoawq --upgrade and use: from awq import AutoAWQForCausalLM

ModuleNotFoundError: No module named 'awq'

AutoAWQ is not installed, or installed but the module name is 'autoawq' (some users mistakenly import 'autoawq' instead of 'awq').

fix

Install the package: pip install autoawq, then use: from awq import ...

Upgrade

Version history

0.2.9latest on PyPI · released May 11, 2025

Audit

Dependencies

torchrequiredCore dependency for tensor computation and GPU support; requires >=2.2.0.

transformersrequiredRequired for model loading and tokenization; compatibility may break with versions >4.51.3.

Agent activity

26 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/casper-hansen/AutoAWQ ↗

packagehttps://pypi.org/project/autoawq/ ↗