← all datasets
from the project README
◆ Evals & benchmarks · Prompt injection & jailbreaks
HarmBench
MITA standardised red-teaming evaluation of harmful-behaviour refusals across attack methods and models.
Summary
Automated red teaming holds substantial promise for uncovering and mitigating the risks associated with the malicious use of large language models (LLMs), yet the field lacks a standardized evaluation framework to rigorously assess these methods. To address this issue, we introduce HarmBench, a standardized evaluation framework for automated red teaming. We identify key considerations previously unaccounted for in red teaming evaluations and systematically design HarmBench to meet these criteria.
- Formats
csv- License
- MIT
- Added
- 2026-09-10
Get it
From the project’s own instructions where it documents any; otherwise a plain clone.
-
$ git clone https://github.com/centerforaisafety/HarmBench.git
-
$ cd HarmBench && pip install -r requirements.txt && python -m spacy download en_core_web_sm
Contents
429 files 103.6 MB repository
-
.py123 -
.png113 -
.pkl100 -
.yaml34 -
.json10 -
.csv9 -
.ipynb9 -
.md9
Use it with
Commands are curated, not yet run by us.
Promptfoo — Run your promptfoo config against the dataset's rows as test cases.
$ promptfoo eval -c promptfooconfig.yaml -t sample.csv
DuckDB — Query the file in place with SQL — DuckDB detects CSV, JSON and JSONL by extension.
$ duckdb -c 'SELECT * FROM '\''sample.csv'\'' LIMIT 10'