garak

Summary

garak is an open-source, free command-line scanner that probes large language models for weaknesses such as hallucination, prompt injection, jailbreaks, toxicity generation, and data leakage, combining static, dynamic, and adaptive probes to make a target model fail in specific, catalogued ways. It's aimed at red teamers and AI security practitioners assessing a model or chat system before or after deployment, rather than at application developers debugging normal behaviour. Its README compares it to nmap or Metasploit, but for LLMs instead of networks and hosts. Configuration and results follow a defined structure, with a unified spec syntax for selecting probes and buffs and a `report.jsonl` output for downstream analysis. First committed in 2023, garak now has 128 contributors and over a thousand commits in the past year, with a release just 17 days ago, indicating active, ongoing development.

the LLM vulnerability scanner

Command line

Values are placeholders taken from each option’s declared default. Nothing is executed here — the output shown is a recording of a run that already happened.

Release history

1. v0.16.0 Aug 4, 2026 · issue 005

   garak v0.16.0 adds intent/technique-aware scanning, a native Anthropic generator, and an adaptive attacks probe

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch v0.16.0 https://github.com/NVIDIA/garak.git
   # already have the repo? check out this version:
   $ git checkout v0.16.0

   └──▷ TRY IT
   Scan an Anthropic Claude model directly using the new native Anthropic generator.
   copy

   $ python -m garak --target_type anthropic --target_name claude-3-5-haiku-20241022 --probes encoding

   • ›Adds a unified run.spec selection grammar that combines probe, buff, and intent selection into a single specification, replacing separate probe_spec and buff_spec config keys.
   • ›Introduces IntentProbe — a new probe base class enabling context-aware scanning that maps attack techniques to intents and automatically selects detectors based on intent mappings.
   • ›Adds probe_summary entries and an enriched technique_intent_matrix (with names and descriptions) to report.jsonl output, providing a new facet of per-probe and cross-technique/intent reporting.
   • ›Adds a native Anthropic generator plugin, enabling direct scanning of Anthropic Claude models without going through a compatibility shim.
   • ›Adds suppressed_params support to the BedrockGenerator, allowing callers to exclude specific inference parameters from AWS Bedrock requests.
   +2 moreshow less

   • ›Adds Unicode normalisation to StringDetector to improve detection consistency across text encodings.
   • ›Hardens web_injection markdown exfiltration detectors against domain, extension, and parameter evasion.
   └──▷ BREAKING ON UPGRADE

   • !probe_spec and buff_spec configuration keys move into the run definition as part of the new unified spec; existing config files using top-level probe_spec or buff_spec will break.
   • !CLI plugin selection options for probes and buffs are now deprecated (backwards-compatible aliases remain, but dependence on the old forms should be migrated).
   • !report.jsonl output structure changes: new probe_summary entry type added, digest format expanded, and older digest values tied to the removed *_spec configuration keys are no longer emitted.