← all datasets
from the project README
◆ Evals & benchmarks · Prompt injection & jailbreaks
Meta CyberSecEval
license: see sourceMeta's LLM cybersecurity benchmarks (in Purple Llama) — insecure code generation, cyberattack helpfulness, code-interpreter abuse and prompt injection.
Summary
Purple Llama is an umbrella project that over time will bring together tools and evals to help the community build responsibly with open generative AI models. The initial release will include tools and evals for Cyber Security and Input/Output safeguards but we plan to contribute more in the near future.
- Formats
json- License
- license: see source
- Added
- 2026-09-10
Get it
From the project’s own instructions where it documents any; otherwise a plain clone.
-
$ git clone https://github.com/meta-llama/PurpleLlama.git
-
$ cd PurpleLlama && python3 -m venv ~/.venvs/CybersecurityBenchmarks && source ~/.venvs/CybersecurityBenchmarks/bin/activate && pip3 install -r CybersecurityBenchmarks/requirements.txt
Python 3.10. Then `python3 -m CybersecurityBenchmarks.benchmark.run --help`.
Contents
894 files 42.8 MB repository
-
.json341 -
.py160 -
.yaml105 -
.cpp94 -
.md79 -
(no ext)24 -
.png19 -
.js16
Use it with
Commands are curated, not yet run by us.
jq — Stream the records one per line to see their shape.
$ jq -c . sample.json
DuckDB — Query the file in place with SQL — DuckDB detects CSV, JSON and JSONL by extension.
$ duckdb -c 'SELECT * FROM '\''sample.json'\'' LIMIT 10'