Terminal-Bench is a popular benchmark for measuring the capabilities of agents and language models to perform valuable work in containerized environments. Tasks include assembling proteins for synthesis, debugging async code, and resolving security vulnerabilities. It is used by virtually all frontier labs.
from the project README
Formats
tomldockerfile
License
Apache-2.0
Added
2026-09-10
Get it
From the project’s own instructions where it documents any; otherwise a plain clone.
shell
$ uv tool install harbor
Terminal-Bench 2.0 runs on the Harbor harness.
shell
$ uv run harbor run --dataset [email protected] --agent oracle --n-concurrent 4
The oracle agent runs the reference solutions, so it needs no API key. Every task runs in its own Docker container.