Garak is a framework for probing large language models (LLMs) for security vulnerabilities. It automates the process of generating adversarial inputs and evaluating model outputs for issues like jailbreak, hallucination, data leakage, toxicity, and more. Current version is 0.14.1, requiring Python >=3.10. Active development with monthly releases.
pip install garakVerified import paths — ran on the pinned version, not inferred.
Basic import test. Garak is primarily used via CLI; this verifies the installation.
Remove `--generate_autodan` from command lines. Use other probe options instead.
Replace references to 'failure' with 'attack_success' in custom reporting or parsing scripts.
Continue using YAML config files unless you specifically need JSON. Monitor future releases.
Set environment variables like OPENAI_API_KEY or use local generators (e.g., `--model_type test` for testing).
Use `--max_workers 1` to reduce load or increase for faster scanning on capable hardware.
Run `pip install garak` in the correct Python environment (Python >=3.10).
Provide `--model_type` (e.g., openai, huggingface, test) and optionally `--model_name`.
Set environment variable OPENAI_API_KEY before running garak, or use a different model type.
Check the YAML file syntax. Use `--config` with a valid .yml file or remove config flag to use defaults.
Update parsing code to target 'attack_success' instead of 'failure'.
No dependency data recorded yet.