← all datasets
from the project README
◆ Evals & benchmarks
BigCodeBench
Apache-2.0Practical Python tasks that call many libraries, with complete and instruct splits (ICLR 2025). The repository is archived.
source ↗
archived
Summary
🎉 Check out our latest work! 🌟 BigCodeArena 🌟 🚀 Open Evaluation Platform on AI for Vibe Coding 🚀 ✨ 100% free to use the latest frontier models! ✨
- Formats
jsonl- License
- Apache-2.0
- Added
- 2026-09-10
Get it
From the project’s own instructions where it documents any; otherwise a plain clone.
-
$ pip install bigcodebench --upgrade
Evaluation runs remotely by default.
Contents
72 files 6.7 MB repository
-
.py40 -
.txt8 -
.dockerfile4 -
.yml4 -
(no ext)3 -
.sh3 -
.json2 -
.md2
Use it with
Commands are curated, not yet run by us.
jq — Stream the records one per line to see their shape.
$ jq -c . sample.jsonl
DuckDB — Query the file in place with SQL — DuckDB detects CSV, JSON and JSONL by extension.
$ duckdb -c 'SELECT * FROM '\''sample.jsonl'\'' LIMIT 10'