Heads up This site is currently under heavy development.
← all tools
◆ AI OBSERVABILITY

Braintrust

1.0.0 commercial

Braintrust is an open-source evals framework for testing and monitoring AI applications with custom test cases and metrics.

source ↗ rss ↗

Summary

Braintrust is an open-source evals framework for testing and monitoring AI applications with custom test cases and metrics.

Release history

  1. docs update Aug 29, 2026 · issue 010

    Braintrust adds dashboards, bt trace CLI, MCP write tools, and new SDK capabilities across JS, Java, and Python

    └──▷ TRY IT
    Set up Claude Code tracing to route all lifecycle events through the Braintrust CLI after upgrading from plugin-specific environment-variable configuration.
    $ bt login
bt trace setup claude
    Trace a single coding-agent run in CI and ensure all trace events are flushed before the script exits.
    $ bt trace run -- <your-agent-command>
    • ›Adds bt trace setup <agent> to install/configure coding-agent tracing plugins (claude, codex, opencode, pi), replacing per-plugin environment-variable and config.json setup.
    • ›Adds bt trace run to trace individual agent runs and wait for trace events to flush before returning — useful for CI and scripted workflows.
    • ›Adds bt trace import to import past agent sessions into Braintrust.
    • ›Adds generateImage call tracing as LLM spans with generated images stored as Braintrust attachments (JS SDK v3.29.0).
    • ›Adds @braintrust/deepseek-harness tracing plugin installable via dsh plugin --profile <profile> add @braintrust/deepseek-harness to trace user turns, LLM steps, tool calls, and nested child sessions.
    +10 moreshow less
    • ›Introduces braintrust/instrumentation package with OpenAIAgentsTraceProcessor and auto-instrumentation via node --import braintrust/hook.mjs, replacing @braintrust/openai-agents.
    • ›Java instrumentation (v0.3.22) adds coverage for OpenAI Responses API via OpenAiResponsesChatModel and OpenAiResponsesStreamingChatModel (LangChain4j 1.14.0+), and captures x-request-id / request-id vendor IDs as span attributes.
    • ›Java instrumentation (v0.3.22) adds AWS Bedrock Converse span capture for streaming tool-use, non-text content blocks, normalized reasoning blocks, prompt caching metrics, and tool definitions.
    • ›Thinking tokens and reasoning tokens are now recorded as span metrics in the JS SDK.
    • ›Monitoring views are now full Dashboards with a dedicated page per dashboard, project-scoped list with search and starring, clone/duplicate/rename/delete via the Dashboard actions menu, and auto-saving chart/filter/grouping changes.
    • ›Adds ability to duplicate dashboards across projects.
    • ›MCP write tools (public preview) enable pipeline end-to-end operations: build dashboards, create alerts and scheduled jobs, run evals, and edit dataset rows using the permissions of your authenticated Braintrust account.
    • ›Applied scorers and classifiers on the Evaluators tab now show recorded rationale inline without requiring the user to open the producing span.
    • ›Adds a standalone Billing address section with an Edit address action to update billing address without replacing the card on file.
    • ›Sandboxes are now available in public preview.
    └──▷ BREAKING ON UPGRADE
    • !bt auth logout is replaced by bt logout; bt auth profiles is replaced by bt profiles; bt auth refresh is replaced by bt login --refresh; the --fresh flag on bt login is renamed to --force.
    • !Coding-agent plugins (claude, codex, opencode, pi) must migrate to bt trace setup <agent> for tracing configuration — previous plugin-specific API key, project, tracing, and config.json/environment-variable settings are broken by this version.
    • !@braintrust/openai-agents is deprecated and has received its final release; users must migrate to braintrust/instrumentation using OpenAIAgentsTraceProcessor or node --import braintrust/hook.mjs.
    • !@braintrust/langchain-js is deprecated and has received its final release; users must migrate to BraintrustLangChainCallbackHandler.
    • !For trace-codex, config.json now takes precedence over environment variables.
  2. docs update Aug 29, 2026 · issue 010

    Braintrust adds Dashboards, token budget policies, Loop observability, and new tracing integrations for LangChain4j, DeepSeek, OpenCode, and pi.

    └──▷ HOW TO FIND IT
    Cap gateway token spend for a specific project to prevent runaway costs in production.
    📍Go to Deploy › Braintrust Gateway › Token Budgets, create a new budget policy, set the scope to 'Project', select your project, and configure the token and spend limits.
    • ›Adds gateway token budget policies scoped to a project, user, or API key to cap spend and token usage on every gateway request — see deploy/gateway-token-budgets.
    • ›Introduces Dashboards in the Observe section: browse, filter, group, and drill into aggregated metrics across logs and experiments over time.
    • ›Adds dashboard creation workflows: clone the built-in dashboard, start empty, or duplicate an existing one, with rename/delete/copy-between-projects support.
    • ›Adds chart types — time series, top list, and big number — with export, duplicate, and move-between-dashboards actions.
    • ›Introduces Loop, an active observability feature for investigating project data, editing Braintrust objects, and detecting recurring patterns via interactive and scheduled analysis.
    +9 moreshow less
    • ›Adds LangChain4j tracing integration to trace LangChain4j calls for debugging prompts, evaluating models, and monitoring production usage.
    • ›Adds DeepSeek Harness agent framework integration to trace agent sessions including user turns, LLM steps, tool calls, and child session interactions.
    • ›Adds OpenCode developer tool integration via the trace-opencode plugin, with access to Braintrust data through built-in tools or MCP.
    • ›Adds pi coding session integration via the pi-extension package, tracing turns, model calls, tool executions, and compactions.
    • ›Publishes OpenAI Agents JS Integration v0.1.6 and OpenTelemetry JS Integration v1.0.0 as new related TypeScript SDK packages.
    • ›Adds a knowledge-base guide for calculating per-trace averages in dashboards with custom measures.
    • ›Adds a knowledge-base guide for creating a blank dashboard via API.
    • ›Adds a knowledge-base guide for setting up Claude Code with OTel tracing.
    • ›Introduces a Feature lifecycle page documenting what private preview, public preview, and general availability mean for Braintrust features.
  3. 1.0.0 Aug 28, 2026 · issue 009

    API surface changed: 65 modified

    API CHANGE

    API surface changed: 65 modified

    • ~ DELETE /v1/acl: request body changed; response schema changed
    • ~ DELETE /v1/acl/{acl_id}: response schema changed
    • ~ DELETE /v1/ai_secret: request body changed
    • ~ DELETE /v1/function/{function_id}: response schema changed
    • ~ DELETE /v1/role/{role_id}: response schema changed
    • ~ DELETE /v1/service_token: request body changed
    • ~ DELETE /v1/view/{view_id}: request body changed; response schema changed
    • ~ GET /v1/acl: response schema changed
    • ~ GET /v1/acl/list_org: response schema changed
    • ~ GET /v1/acl/{acl_id}: response schema changed
    • ~ GET /v1/function: response schema changed
    • ~ GET /v1/function/{function_id}: response schema changed
    • ~ GET /v1/prompt: response schema changed
    • ~ GET /v1/role/{role_id}: response schema changed
    • ~ GET /v1/view: response schema changed
    • ~ GET /v1/view/{view_id}: response schema changed
    • ~ PATCH /v1/env_var/{env_var_id}: request body changed
    • ~ PATCH /v1/function/{function_id}: response schema changed
    • ~ PATCH /v1/role/{role_id}: request body changed; response schema changed
    • ~ PATCH /v1/view/{view_id}: request body changed; response schema changed
    • ~ POST /v1/acl: request body changed; response schema changed
    • ~ POST /v1/acl/batch_update: request body changed; response schema changed
    • ~ POST /v1/agent: request body changed
    • ~ POST /v1/ai_secret: request body changed
    • ~ POST /v1/dataset: request body changed
    • ~ POST /v1/dataset/{dataset_id}/feedback: request body changed
    • ~ POST /v1/dataset/{dataset_id}/insert: request body changed
    • ~ POST /v1/dataset_snapshot: request body changed
    • ~ POST /v1/env_var: request body changed
    • ~ POST /v1/experiment: request body changed
    • ~ POST /v1/experiment/{experiment_id}/feedback: request body changed
    • ~ POST /v1/experiment/{experiment_id}/insert: request body changed
    • ~ POST /v1/function: request body changed; response schema changed
    • ~ POST /v1/group: request body changed
    • ~ POST /v1/mcp_server: request body changed
    • ~ POST /v1/org_automation: request body changed
    • ~ POST /v1/project: request body changed
    • ~ POST /v1/project_automation: request body changed
    • ~ POST /v1/project_group: request body changed
    • ~ POST /v1/project_logs/{project_id}/feedback: request body changed
    • ~ POST /v1/project_logs/{project_id}/insert: request body changed
    • ~ POST /v1/project_score: request body changed
    • ~ POST /v1/project_tag: request body changed
    • ~ POST /v1/prompt: request body changed
    • ~ POST /v1/role: request body changed; response schema changed
    • ~ POST /v1/service_token: request body changed
    • ~ POST /v1/span_iframe: request body changed
    • ~ POST /v1/view: request body changed; response schema changed
    • ~ PUT /v1/agent: request body changed
    • ~ PUT /v1/ai_secret: request body changed
    • ~ PUT /v1/dataset_snapshot: request body changed
    • ~ PUT /v1/env_var: request body changed
    • ~ PUT /v1/function: request body changed; response schema changed
    • ~ PUT /v1/group: request body changed
    • ~ PUT /v1/mcp_server: request body changed
    • ~ PUT /v1/org_automation: request body changed
    • ~ PUT /v1/project_automation: request body changed
    • ~ PUT /v1/project_group: request body changed
    • ~ PUT /v1/project_score: request body changed
    • ~ PUT /v1/project_tag: request body changed
    • ~ PUT /v1/prompt: request body changed
    • ~ PUT /v1/role: request body changed; response schema changed
    • ~ PUT /v1/service_token: request body changed
    • ~ PUT /v1/span_iframe: request body changed
    • ~ PUT /v1/view: request body changed; response schema changed
    • ›DELETE /v1/acl: request body changed; response schema changed
    • ›DELETE /v1/acl/{acl_id}: response schema changed
    • ›DELETE /v1/ai_secret: request body changed
    • ›DELETE /v1/function/{function_id}: response schema changed
    • ›DELETE /v1/role/{role_id}: response schema changed
    +60 moreshow less
    • ›DELETE /v1/service_token: request body changed
    • ›DELETE /v1/view/{view_id}: request body changed; response schema changed
    • ›GET /v1/acl: response schema changed
    • ›GET /v1/acl/list_org: response schema changed
    • ›GET /v1/acl/{acl_id}: response schema changed
    • ›GET /v1/function: response schema changed
    • ›GET /v1/function/{function_id}: response schema changed
    • ›GET /v1/prompt: response schema changed
    • ›GET /v1/role/{role_id}: response schema changed
    • ›GET /v1/view: response schema changed
    • ›GET /v1/view/{view_id}: response schema changed
    • ›PATCH /v1/env_var/{env_var_id}: request body changed
    • ›PATCH /v1/function/{function_id}: response schema changed
    • ›PATCH /v1/role/{role_id}: request body changed; response schema changed
    • ›PATCH /v1/view/{view_id}: request body changed; response schema changed
    • ›POST /v1/acl: request body changed; response schema changed
    • ›POST /v1/acl/batch_update: request body changed; response schema changed
    • ›POST /v1/agent: request body changed
    • ›POST /v1/ai_secret: request body changed
    • ›POST /v1/dataset: request body changed
    • ›POST /v1/dataset/{dataset_id}/feedback: request body changed
    • ›POST /v1/dataset/{dataset_id}/insert: request body changed
    • ›POST /v1/dataset_snapshot: request body changed
    • ›POST /v1/env_var: request body changed
    • ›POST /v1/experiment: request body changed
    • ›POST /v1/experiment/{experiment_id}/feedback: request body changed
    • ›POST /v1/experiment/{experiment_id}/insert: request body changed
    • ›POST /v1/function: request body changed; response schema changed
    • ›POST /v1/group: request body changed
    • ›POST /v1/mcp_server: request body changed
    • ›POST /v1/org_automation: request body changed
    • ›POST /v1/project: request body changed
    • ›POST /v1/project_automation: request body changed
    • ›POST /v1/project_group: request body changed
    • ›POST /v1/project_logs/{project_id}/feedback: request body changed
    • ›POST /v1/project_logs/{project_id}/insert: request body changed
    • ›POST /v1/project_score: request body changed
    • ›POST /v1/project_tag: request body changed
    • ›POST /v1/prompt: request body changed
    • ›POST /v1/role: request body changed; response schema changed
    • ›POST /v1/service_token: request body changed
    • ›POST /v1/span_iframe: request body changed
    • ›POST /v1/view: request body changed; response schema changed
    • ›PUT /v1/agent: request body changed
    • ›PUT /v1/ai_secret: request body changed
    • ›PUT /v1/dataset_snapshot: request body changed
    • ›PUT /v1/env_var: request body changed
    • ›PUT /v1/function: request body changed; response schema changed
    • ›PUT /v1/group: request body changed
    • ›PUT /v1/mcp_server: request body changed
    • ›PUT /v1/org_automation: request body changed
    • ›PUT /v1/project_automation: request body changed
    • ›PUT /v1/project_group: request body changed
    • ›PUT /v1/project_score: request body changed
    • ›PUT /v1/project_tag: request body changed
    • ›PUT /v1/prompt: request body changed
    • ›PUT /v1/role: request body changed; response schema changed
    • ›PUT /v1/service_token: request body changed
    • ›PUT /v1/span_iframe: request body changed
    • ›PUT /v1/view: request body changed; response schema changed
  4. docs update Aug 28, 2026 · issue 009

    Braintrust Playgrounds gain Workflows (public preview) for prompt chaining across chained prompts.

    • ›Adds Workflows (public preview) to Playgrounds, enabling prompt chaining functionality for sequencing multiple prompts together.
  5. docs update Aug 28, 2026 · issue 009

    Braintrust CLI gains unified coding-agent tracing, new scorer creation, profile management, and auto-instrumentation for JS/TS evals

    └──▷ TRY IT
    Set up Claude Code tracing through the unified bt runtime so all trace events are authenticated and routed by bt instead of per-plugin credentials.
    $ bt login
bt trace setup claude
    Create a prompt-based LLM scorer from the CLI without opening the UI.
    $ bt scorers create
    • ›Adds bt trace setup, bt trace run, and bt trace import subcommands to install/configure coding-agent tracing plugins, run traced agent sessions, and import past sessions.
    • ›Adds bt trace setup claude, bt trace setup codex, bt trace setup opencode, and bt trace setup pi to configure per-agent tracing through the unified bt runtime instead of per-plugin credentials.
    • ›Adds bt scorers create CLI subcommand to create prompt-based LLM scorers and classifiers from the command line.
    • ›Adds bt profiles subcommand to list, rename, and delete saved login profiles.
    • ›Adds --no-auto-instrumentation flag to opt out of automatic Braintrust instrumentation for JavaScript and TypeScript evals (v0.16.2), which now runs before eval files load.
    +7 moreshow less
    • ›Adds mise as a supported installation method for the bt CLI.
    • ›Adds bt switch flag to select an organization after logging in, with profiles now decoupled from organizations (log in once per account).
    • ›Tool spans in trace-codex now record tool_approval: "approved" metadata and surface tool output failures in the span.
    • ›Adds cache token metrics, Git repository metadata, skill invocation tracking, and experiment attachment support to trace-codex.
    • ›Adds a Billing address section to update billing address independently of the payment card on file via an Edit address / Update your billing address flow.
    • ›Service accounts now display inherited project permissions from their permission groups, and group membership can be managed directly from the Service tokens page.
    • ›Sandboxes are now in public preview (API, configuration, and behavior subject to change before GA).
    └──▷ BREAKING ON UPGRADE
    • !bt auth subcommands are replaced by top-level commands: bt auth logout → bt logout, bt auth profiles → bt profiles, bt auth refresh → bt login --refresh.
    • !The --fresh flag on bt login is renamed to --force.
    • !bt trace setup claude replaces previous plugin-specific environment-variable setup for Claude Code tracing; old env-var configuration is broken — see Claude Code upgrade notes.
    • !bt trace setup codex replaces previous plugin-specific environment-variable and config.json setup for Codex tracing; old configuration is broken — see Codex upgrade notes. In this version, config.json takes precedence over environment variables.
    • !bt trace setup opencode replaces previous plugin-specific environment-variable setup for OpenCode tracing; old env-var configuration is broken — see OpenCode upgrade notes.
    • !bt trace setup pi replaces previous extension-specific auth and settings for pi tracing; old configuration is broken — see pi upgrade notes.
    • !Older plugin-specific API key, project, tracing, and config-file settings for all coding-agent plugins must be migrated to the unified bt CLI — see the CLI migration guide.
  6. docs update Aug 26, 2026 · issue 008

    Braintrust v0.3.21 adds a full REST API client, richer LLM span nesting, and smarter Topics log filtering

    • ›Adds DefaultBraintrustApiClient.Api, a generated client covering every Braintrust REST endpoint, shipped inside the existing Braintrust.Sdk package with no separate install required.
    • ›Web search, file search, code interpreter, and MCP calls in the OpenAI Responses API now appear as child tool spans nested under the LLM span for cleaner trace visualization.
    • ›server_tool_use blocks in the Anthropic Messages API now appear as child tool spans under the LLM span, and Anthropic spans capture new server_tool_use_* metrics.
    • ›Hovering a row in the List view of a Topics facet card now reveals a 'View logs' link that filters logs to that topic and all nested topics.
    • ›Custom preprocessors whose output is not a recognized message array now render in thread layout as labeled text instead of failing.
    └──▷ BREAKING ON UPGRADE
    • !The Google GenAI provider metadata field name changed; trace queries that filter on the previous value must be updated to use the new field name.
  7. docs update Aug 26, 2026 · issue 008

    Braintrust adds Topics auto-classification, dataset pipelines, remote eval sandboxes, and reasoning model evaluation support.

    └──▷ HOW TO FIND IT
    Enable Topics on a project to automatically classify production logs by intent, sentiment, and issues.
    📍In the Braintrust UI, go to your project › Observe › Topics › Enable Topics, confirm traces render with the Thread preprocessor, then select the facets (Task, Sentiment, Issues) you want active.
    • ›New Topics feature automatically classifies production logs by user intent, sentiment, and issues — with built-in Task, Sentiment, and Issues facets — then surfaces patterns as topic distributions and time-series trends.
    • ›Custom facets let teams define domain-specific classification dimensions using custom preprocessors and prompts beyond the built-in Topics facets.
    • ›Adds dataset pipelines to transform spans or traces from project logs into dataset rows in bulk using declarative, version-controlled pipelines.
    • ›Remote evals and sandboxes allow connecting custom agent code to a playground and running it in an isolated sandbox to test complex agents interactively without modifying deployed code.
    • ›Supports evaluating reasoning models (OpenAI o4, Claude, Gemini) with configurable thinking budgets, capture of intermediate reasoning traces, and scoring of final outputs.
    +10 moreshow less
    • ›New Loop agent enables natural-language interaction to analyze logs, optimize prompts, build datasets, and generate custom React view components.
    • ›Custom views let teams describe a trace/dataset interface in natural language; Loop generates an embeddable, customizable React component.
    • ›Advanced tracing patterns now cover distributed tracing across services, sensitive data masking, custom span rendering, and high-throughput performance tuning.
    • ›Log attachments support images, audio, PDFs, and other binary files attached to traces and experiments for multimodal review in context.
    • ›Adds monitoring dashboards to track request counts, latency, costs, and scores across logs and experiments over time.
    • ›Adds alerts with webhook and Slack notification configuration.
    • ›Topics management includes on-demand pipeline runs, sampling adjustment, pause/resume, and history rewind after facet or filter changes.
    • ›Human review now supports multiple reviewers with configurable score-combination rules and a base value for non-numeric scores.
    • ›Dataset performance tracking shows which experiments used a dataset and how each row performs over time, with chart and SQL filters.
    • ›Adds export of annotated traces and datasets for use in external evaluation frameworks, analysis pipelines, or training workflows.
  8. docs update Aug 25, 2026 · issue 007

    Braintrust adds collapsible SQL query sidebar, per-member permission group editing, span-name log filtering, and searchable filter value picker.

    └──▷ HOW TO FIND IT
    Quickly scope logs to a single span without manually constructing a filter expression.
    📍In the trace panel, click the span name in the header — the logs list automatically filters to that span name.
    Audit or update which permission groups a team member belongs to without hunting through each group.
    📍Go to Settings › Members, open a member's dialog — the panel shows direct group memberships, inherited memberships, and groups available to add.
    • ›The SQL sandbox now organizes queries in a collapsible sidebar with search by name, drag-to-reorder, command-bar navigation, and a per-query menu to rename, duplicate, or delete.
    • ›Manage a member's permission groups directly from the member dialog — showing direct groups, inherited groups, and available groups to add — without opening each group individually.
    • ›Filter the logs list to a specific span name by clicking that span's name in the trace panel header, instead of building the filter manually.
    • ›The filter tab's value picker is now searchable and accepts values not present in the list by typing them directly.
  9. 1.0.0 Aug 24, 2026 · issue 006

    API surface changed: +24 endpoints, 53 modified

    API CHANGE

    API surface changed: +24 endpoints, 53 modified

    • + DELETE /v1/agent/{agent_id}
    • + DELETE /v1/org_automation/{org_automation_id}
    • + DELETE /v1/project_group/{project_group_id}
    • + GET /v1/agent
    • + GET /v1/agent/{agent_id}
    • + GET /v1/org_automation
    • + GET /v1/org_automation/{org_automation_id}
    • + GET /v1/project_group
    • + GET /v1/project_group/{project_group_id}
    • + OPTIONS /v1/agent
    • + OPTIONS /v1/agent/{agent_id}
    • + OPTIONS /v1/org_automation
    • + OPTIONS /v1/org_automation/{org_automation_id}
    • + OPTIONS /v1/project_group
    • + OPTIONS /v1/project_group/{project_group_id}
    • + PATCH /v1/agent/{agent_id}
    • + PATCH /v1/org_automation/{org_automation_id}
    • + PATCH /v1/project_group/{project_group_id}
    • + POST /v1/agent
    • + POST /v1/org_automation
    • + POST /v1/project_group
    • + PUT /v1/agent
    • + PUT /v1/org_automation
    • + PUT /v1/project_group
    • ~ DELETE /v1/acl: request body changed; response schema changed
    • ~ DELETE /v1/acl/{acl_id}: response schema changed
    • ~ DELETE /v1/api_key/{api_key_id}: response schema changed
    • ~ DELETE /v1/function/{function_id}: response schema changed
    • ~ DELETE /v1/project/{project_id}: response schema changed
    • ~ DELETE /v1/project_score/{project_score_id}: response schema changed
    • ~ DELETE /v1/prompt/{prompt_id}: response schema changed
    • ~ DELETE /v1/role/{role_id}: response schema changed
    • ~ DELETE /v1/service_token: response schema changed
    • ~ DELETE /v1/service_token/{service_token_id}: response schema changed
    • ~ DELETE /v1/view/{view_id}: request body changed; response schema changed
    • ~ GET /v1/acl: response schema changed
    • ~ GET /v1/acl/list_org: response schema changed
    • ~ GET /v1/acl/{acl_id}: response schema changed
    • ~ GET /v1/api_key: response schema changed
    • ~ GET /v1/api_key/{api_key_id}: response schema changed
    • ~ GET /v1/function: response schema changed
    • ~ GET /v1/function/{function_id}: response schema changed
    • ~ GET /v1/project: response schema changed
    • ~ GET /v1/project/{project_id}: response schema changed
    • ~ GET /v1/project_score: response schema changed
    • ~ GET /v1/project_score/{project_score_id}: response schema changed
    • ~ GET /v1/prompt: response schema changed
    • ~ GET /v1/prompt/{prompt_id}: response schema changed
    • ~ GET /v1/role/{role_id}: response schema changed
    • ~ GET /v1/service_token: response schema changed
    • ~ GET /v1/service_token/{service_token_id}: response schema changed
    • ~ GET /v1/view: response schema changed
    • ~ GET /v1/view/{view_id}: response schema changed
    • ~ PATCH /v1/function/{function_id}: request body changed; response schema changed
    • ~ PATCH /v1/organization/members: request body changed
    • ~ PATCH /v1/project/{project_id}: response schema changed
    • ~ PATCH /v1/project_score/{project_score_id}: request body changed; response schema changed
    • ~ PATCH /v1/prompt/{prompt_id}: request body changed; response schema changed
    • ~ PATCH /v1/role/{role_id}: request body changed; response schema changed
    • ~ PATCH /v1/view/{view_id}: request body changed; response schema changed
    • ~ POST /v1/acl: request body changed; response schema changed
    • ~ POST /v1/acl/batch_update: request body changed; response schema changed
    • ~ POST /v1/eval: request body changed
    • ~ POST /v1/function: request body changed; response schema changed
    • ~ POST /v1/function/{function_id}/invoke: request body changed
    • ~ POST /v1/project: response schema changed
    • ~ POST /v1/project_score: request body changed; response schema changed
    • ~ POST /v1/prompt: request body changed; response schema changed
    • ~ POST /v1/role: request body changed; response schema changed
    • ~ POST /v1/service_token: request body changed; response schema changed
    • ~ POST /v1/view: request body changed; response schema changed
    • ~ PUT /v1/function: request body changed; response schema changed
    • ~ PUT /v1/project_score: request body changed; response schema changed
    • ~ PUT /v1/prompt: request body changed; response schema changed
    • ~ PUT /v1/role: request body changed; response schema changed
    • ~ PUT /v1/service_token: request body changed; response schema changed
    • ~ PUT /v1/view: request body changed; response schema changed
    • ›New endpoint DELETE /v1/agent/{agent_id}
    • ›New endpoint DELETE /v1/org_automation/{org_automation_id}
    • ›New endpoint DELETE /v1/project_group/{project_group_id}
    • ›New endpoint GET /v1/agent
    • ›New endpoint GET /v1/agent/{agent_id}
    +72 moreshow less
    • ›New endpoint GET /v1/org_automation
    • ›New endpoint GET /v1/org_automation/{org_automation_id}
    • ›New endpoint GET /v1/project_group
    • ›New endpoint GET /v1/project_group/{project_group_id}
    • ›New endpoint OPTIONS /v1/agent
    • ›New endpoint OPTIONS /v1/agent/{agent_id}
    • ›New endpoint OPTIONS /v1/org_automation
    • ›New endpoint OPTIONS /v1/org_automation/{org_automation_id}
    • ›New endpoint OPTIONS /v1/project_group
    • ›New endpoint OPTIONS /v1/project_group/{project_group_id}
    • ›New endpoint PATCH /v1/agent/{agent_id}
    • ›New endpoint PATCH /v1/org_automation/{org_automation_id}
    • ›New endpoint PATCH /v1/project_group/{project_group_id}
    • ›New endpoint POST /v1/agent
    • ›New endpoint POST /v1/org_automation
    • ›New endpoint POST /v1/project_group
    • ›New endpoint PUT /v1/agent
    • ›New endpoint PUT /v1/org_automation
    • ›New endpoint PUT /v1/project_group
    • ›DELETE /v1/acl: request body changed; response schema changed
    • ›DELETE /v1/acl/{acl_id}: response schema changed
    • ›DELETE /v1/api_key/{api_key_id}: response schema changed
    • ›DELETE /v1/function/{function_id}: response schema changed
    • ›DELETE /v1/project/{project_id}: response schema changed
    • ›DELETE /v1/project_score/{project_score_id}: response schema changed
    • ›DELETE /v1/prompt/{prompt_id}: response schema changed
    • ›DELETE /v1/role/{role_id}: response schema changed
    • ›DELETE /v1/service_token: response schema changed
    • ›DELETE /v1/service_token/{service_token_id}: response schema changed
    • ›DELETE /v1/view/{view_id}: request body changed; response schema changed
    • ›GET /v1/acl: response schema changed
    • ›GET /v1/acl/list_org: response schema changed
    • ›GET /v1/acl/{acl_id}: response schema changed
    • ›GET /v1/api_key: response schema changed
    • ›GET /v1/api_key/{api_key_id}: response schema changed
    • ›GET /v1/function: response schema changed
    • ›GET /v1/function/{function_id}: response schema changed
    • ›GET /v1/project: response schema changed
    • ›GET /v1/project/{project_id}: response schema changed
    • ›GET /v1/project_score: response schema changed
    • ›GET /v1/project_score/{project_score_id}: response schema changed
    • ›GET /v1/prompt: response schema changed
    • ›GET /v1/prompt/{prompt_id}: response schema changed
    • ›GET /v1/role/{role_id}: response schema changed
    • ›GET /v1/service_token: response schema changed
    • ›GET /v1/service_token/{service_token_id}: response schema changed
    • ›GET /v1/view: response schema changed
    • ›GET /v1/view/{view_id}: response schema changed
    • ›PATCH /v1/function/{function_id}: request body changed; response schema changed
    • ›PATCH /v1/organization/members: request body changed
    • ›PATCH /v1/project/{project_id}: response schema changed
    • ›PATCH /v1/project_score/{project_score_id}: request body changed; response schema changed
    • ›PATCH /v1/prompt/{prompt_id}: request body changed; response schema changed
    • ›PATCH /v1/role/{role_id}: request body changed; response schema changed
    • ›PATCH /v1/view/{view_id}: request body changed; response schema changed
    • ›POST /v1/acl: request body changed; response schema changed
    • ›POST /v1/acl/batch_update: request body changed; response schema changed
    • ›POST /v1/eval: request body changed
    • ›POST /v1/function: request body changed; response schema changed
    • ›POST /v1/function/{function_id}/invoke: request body changed
    • ›POST /v1/project: response schema changed
    • ›POST /v1/project_score: request body changed; response schema changed
    • ›POST /v1/prompt: request body changed; response schema changed
    • ›POST /v1/role: request body changed; response schema changed
    • ›POST /v1/service_token: request body changed; response schema changed
    • ›POST /v1/view: request body changed; response schema changed
    • ›PUT /v1/function: request body changed; response schema changed
    • ›PUT /v1/project_score: request body changed; response schema changed
    • ›PUT /v1/prompt: request body changed; response schema changed
    • ›PUT /v1/role: request body changed; response schema changed
    • ›PUT /v1/service_token: request body changed; response schema changed
    • ›PUT /v1/view: request body changed; response schema changed
  10. 1.0.0 Aug 20, 2026 · issue 002

    Braintrust now publishes an API — 210 endpoints across 28 areas: Cors, Datasets, Experiments, …

    • ›Cors (65 endpoints) — read
    • ›Datasets (10 endpoints) — create, read, update, delete
    • ›Experiments (10 endpoints) — create, read, update, delete
    • ›Acls (7 endpoints) — create, read, delete
    • ›Aisecrets (7 endpoints) — create, read, update, delete
    +4 moreshow less
    • ›Functions (7 endpoints) — create, read, update, delete
    • ›Datasetsnapshots (6 endpoints) — create, read, update, delete
    • ›Envvars (6 endpoints) — create, read, update, delete
    • ›20 more areas: Groups, Mcpservers, Projectautomations, Projectscores, Projecttags, Prompts, Proxy, Roles, Servicetokens, Spaniframes, Views, Environments, Projects, Logs, Organizations, Apikeys, Users, Crossobject, Evals, Other
  11. snapshot-20260830 Aug 1, 2026 · issue 011
    └──▷ TRY IT
    Set up tracing for Claude Code through the bt CLI, selecting your Braintrust project and writing the plugin's tracing settings in one step.
    $ bt trace setup
    Initialize a Braintrust dataset directly by ID in Python when you know the ID but not the project name.
    python 
    import braintrust

ds = braintrust.init_dataset(dataset_id="<your-dataset-id>")
    • ›Adds bt trace setup, bt trace run, and bt trace import subcommands to the bt CLI for managing tracing plugins for Claude Code, Codex, OpenCode, and pi.
    • ›Adds BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true environment variable (Python SDK v0.32.0) to re-enable LiveKit Agents audio attachments on agent_speaking spans, which are now off by default.
    • ›Adds braintrust eval auto-instrumentation in TypeScript SDK v3.29.0 so supported AI clients and libraries emit spans before eval files load.
    • ›Introduces the AWS Lambda Extension for Python and TypeScript/JavaScript, providing a local handoff path for traces to reduce time spent in flush() during request handling.
    • ›Adds Group scope to online scoring rules, letting you evaluate multi-turn trace sessions as a single unit keyed by a session of your choice, with scores written to the first trace or to every trace.
    +7 moreshow less
    • ›Adds Monitoring views as named, project-scoped Dashboards with per-dashboard pages, search, starring, cloning from the built-in Cost and quality dashboard, and auto-saving of chart, filter, and grouping changes.
    • ›Adds annotated version history for prompts, parameters, and scorers, letting you attach a note when saving a new version and view adjacent versions side by side.
    • ›Adds support for dataset rows referencing a group of up to 64 traces, rendering each inline and flagging unavailable ones.
    • ›Adds Kimi K3 and DeepSeek V4 Flash 0731 as built-in open-source models, requestable as kimi-k3 and deepseek-v4-flash-0731 through the Braintrust Gateway with no AI provider setup required.
    • ›Adds the Harbor job plugin for syncing Harbor evaluation results to Braintrust in Python SDK v0.33.0, defaulting the project name to Harbor so project_name is optional.
    • ›Adds Ollama and @cloudflare/think instrumentation, Anthropic beta sessions tracing via anthropic.beta.sessions.turn and anthropic.beta.sessions.thread.turn, Flue v2 support, and a vitest-evals span input override via meta.eval.input in TypeScript SDK v3.27.0.
    • ›Adds Cloudflare Agents and Cloudflare AI Chat auto-instrumentation in TypeScript SDK v3.26.0.
    └──▷ BREAKING ON UPGRADE
    • !The bt CLI now handles authentication and trace routing for Claude Code, Codex, OpenCode, and pi plugins; older plugin-specific API key, project, tracing, and config-file settings must be migrated using the bt CLI migration guide and per-agent upgrade notes.
    • !LiveKit Agents audio attachments on agent_speaking spans are now disabled by default in Python SDK v0.32.0; set BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true to restore previous behavior.
  12. snapshot-20260829 Aug 1, 2026 · issue 010

    Braintrust adds MCP write tools, bt CLI trace routing, group-scoped scoring, Azure AI Gateway, and new open-source models in August 2026.

    └──▷ TRY IT
    Set up bt CLI tracing for Claude Code once, selecting the Braintrust project and writing tracing settings automatically.
    $ bt trace setup
    Initialize a Braintrust dataset directly by its ID in Python, without needing to know the project name.
    python 
    import braintrust

ds = braintrust.init_dataset(dataset_id="<dataset-id>")
    Restore LiveKit Agents audio attachment capture after upgrading to Python SDK v0.32.0, where it is off by default.
    $ BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true python my_agent.py
    • ›New bt trace setup, bt trace run, and bt trace import subcommands route tracing for Claude Code, Codex, OpenCode, and pi through the bt CLI, handling authentication and trace routing centrally.
    • ›Adds dataset_id parameter to init_dataset() (Python SDK v0.35.0) to initialize a dataset by ID, taking precedence over project, project_id, and name.
    • ›Adds datasetId parameter to initDataset() (TypeScript SDK v3.29.0) to look up a dataset by ID without specifying a project.
    • ›Adds apiKey parameter to loadPrompt() in TypeScript SDK v3.29.0.
    • ›TypeScript SDK v3.29.0: braintrust eval now applies auto-instrumentation so supported AI clients can emit spans before eval files load.
    +17 moreshow less
    • ›Adds invoke_async() as an async counterpart to invoke() in Python SDK v0.32.0.
    • ›Adds Vercel AI SDK for Python auto-instrumentation (enabled by default in auto_instrument()) in Python SDK v0.33.0.
    • ›Adds Cursor SDK Python instrumentation for tracing agent runs, model turns, and tool calls in Python SDK v0.33.0.
    • ›Adds Pipecat auto-instrumentation in Python SDK v0.32.0 for tracing real-time voice AI pipelines, including LLM turns, STT, TTS, and tool calls.
    • ›Adds Voyage AI auto-instrumentation in TypeScript SDK v3.28.0 for embeddings, multimodal embeddings, reranking, and contextualized embeddings.
    • ›TypeScript SDK v3.27.0 adds Ollama and @cloudflare/think instrumentation, Anthropic beta sessions tracing via anthropic.beta.sessions.turn and anthropic.beta.sessions.thread.turn, Flue v2 support, and a vitest-evals span input override via meta.eval.input.
    • ›TypeScript SDK v3.29.0 traces Vercel AI SDK generateImage calls as LLM spans with generated images stored as Braintrust attachments.
    • ›TypeScript SDK v3.29.0 records Anthropic thinking tokens and LangChain reasoning tokens as span metrics.
    • ›The Braintrust MCP server now exposes write tools, enabling coding agents to create and update prompts, scorers, classifiers, dashboards, alerts, scheduled jobs, evals, and dataset rows.
    • ›Adds kimi-k3 and deepseek-v4-flash-0731 as built-in open-source models available through the Braintrust Gateway and provider with no AI provider setup required.
    • ›New AWS Lambda Extension provides a local handoff path for traces in Python and TypeScript/JavaScript Lambda functions, reducing time spent in flush() on the request path.
    • ›Monitoring views are now Dashboards with a dedicated page per dashboard, project-scoped list, search, starring, clone/duplicate/rename/delete via a Dashboard actions menu, and auto-saving chart, filter, and grouping changes.
    • ›Prompt, parameter, and scorer version history now supports annotated saves with a description note pinned to each version, and shows each version alongside the one it replaced.
    • ›A dataset row can now reference a group of up to 64 traces instead of a single one, with each trace rendered inline and unavailable traces flagged.
    • ›Braintrust now supports Azure AI Gateway as an AI provider for backends behind an Azure API Management endpoint, supporting OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages APIs.
    • ›Members' permission groups can now be managed directly from Settings > Members, showing direct groups, inherited groups, and available groups to add.
    • ›TypeScript SDK v3.28.0 passes id and tags of each case to scorer functions in Eval().
    └──▷ BREAKING ON UPGRADE
    • !bt now handles authentication and trace routing for Claude Code, Codex, OpenCode, and pi plugins; older plugin-specific API key, project, tracing, and config-file settings must be migrated per the bt CLI migration guide.
    • !Python SDK v0.32.0: LiveKit Agents audio attachments on agent_speaking spans are now disabled by default; set BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true to restore previous behavior.
  13. snapshot-20260828 Aug 1, 2026 · issue 009

    Braintrust adds bt CLI tracing for coding agents, MCP write tools, new built-in models, Lambda extension, and Group-scope online scoring.

    └──▷ TRY IT
    Set up bt CLI tracing for a Claude Code agent, linking it to a Braintrust project in one step.
    $ bt trace setup
    Route a single coding-agent run through bt tracing without persisting session config.
    $ bt trace run
    • ›Adds bt trace setup, bt trace run, and bt trace import subcommands to the bt CLI to install/update tracing plugins, select a Braintrust project, and manage one-off and saved-session tracing workflows for Claude Code, Codex, OpenCode, and pi.
    • ›Exposes write tools on the Braintrust MCP server, enabling coding agents to create and update prompts, scorers, classifiers, Topics pipeline configuration, monitor views, alerts, scheduled jobs, evals, and dataset rows using the authenticated account's permissions.
    • ›Adds kimi-k3 and deepseek-v4-flash-0731 as built-in open-source models available via the Braintrust provider in playgrounds, prompts, and scorers, and requestable through the Braintrust Gateway with no external AI provider setup.
    • ›Introduces the AWS Lambda Extension, giving Python and TypeScript/JavaScript Lambda functions a local handoff path for traces so the Braintrust SDK's flush() method spends less time in the request path.
    • ›Adds Group scope to online scoring rules, letting you evaluate a set of related multi-turn traces as a single unit using a session key of your choice, without changing your logging.
    +8 moreshow less
    • ›Adds a Summary table layout to the experiments list that shows every experiment as a column with scores and metrics as rows, including an 'All scores (avg)' row and group-based aggregation.
    • ›Allows dataset rows to reference a group of up to 64 traces instead of a single trace, rendering each trace inline for multi-turn sessions or related log sets.
    • ›Adds Azure AI Gateway as a supported AI provider, enabling use of a single provider for all backends behind an Azure API Management endpoint with OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages API support.
    • ›Adds a collapsible query sidebar to the SQL sandbox with search by name, drag-to-reorder, command-bar navigation, per-query rename/duplicate/delete menu, and a 'Copy share link' action that opens the query in a teammate's sandbox without auto-running it.
    • ›Python SDK v0.34.0 adds Hugging Face Transformers auto-instrumentation for local pipelines covering text generation, summarization, translation, feature extraction, and question answering.
    • ›Python SDK v0.33.0 adds Vercel AI SDK for Python auto-instrumentation (enabled by default in auto_instrument()), Cursor SDK Python instrumentation for agent runs/model turns/tool calls, and a native Harbor job plugin for syncing Harbor evaluation results to Braintrust.
    • ›Python SDK v0.34.0 forwards eval case fields to scorer functions and defaults the Harbor plugin's Braintrust project name to Harbor, making project_name optional.
    • ›Go SDK v0.11.1 adds WithProvider and WithModel options on Firebase Genkit's NewMiddleware for explicit model attribution, plus traced tool wrappers DefineTool, DefineToolWithInputSchema, and DefineMultipartTool that auto-replace their untraced counterparts.
    └──▷ BREAKING ON UPGRADE
    • !bt now handles authentication and trace routing for Claude Code, Codex, OpenCode, and pi plugins; older plugin-specific API key, project, tracing, and config-file settings must be migrated per the bt CLI migration guide and each agent's upgrade notes.
    • !Python SDK v0.32.0: LiveKit Agents audio attachments on agent_speaking spans are now disabled by default; set BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true to restore the previous behavior.
    • !Go SDK v0.11.1 (Google GenAI): Provider metadata changed from 'gemini' to 'google'; update trace queries that filter on the previous provider value.
    • !Go SDK v0.11.1 (Eino): ChatModel span output is now an OpenAI-compatible choices array ([{"index": 0, "finish_reason": "...", "message": {...}}]) instead of a flat message map; embedding input is now {"inputs": [{"content": "..."}]} and output is {"count": N}, removing embedding_length and renaming embeddings_count.
  14. snapshot-20260826 Aug 1, 2026 · issue 008
    • ›Adds MCP write tools to the Braintrust MCP server so coding agents can create and update prompts, scorers, classifiers, Topics pipeline config, monitor views, alerts, scheduled jobs, evals, and dataset rows — write tools use your authenticated account permissions; configure your client to require confirmation before running destructive tools.
    • ›Adds kimi-k3 and deepseek-v4-flash-0731 as built-in open-source models served through the Braintrust Gateway alongside GLM-5.2, with no external AI provider setup required; usage draws from monthly model credits shared with Topics.
    • ›Python SDK v0.34.0: forwards eval case fields to scorer functions.
    • ›Python SDK v0.32.0: adds invoke_async() as an async counterpart to invoke().
    • ›TypeScript SDK v3.28.0: scorer functions in Eval() now receive the id and tags of each case.
    +4 moreshow less
    • ›Go SDK v0.11.1: Firebase Genkit gains WithProvider and WithModel options on NewMiddleware for explicit model attribution, plus traced tool wrappers DefineTool, DefineToolWithInputSchema, and DefineMultipartTool; auto-instrumentation now replaces genkit.DefineTool, genkit.DefineToolWithInputSchema, and genkit.DefineMultipartTool calls with their traced equivalents.
    • ›Go SDK v0.11.0: Anthropic spans now capture prompt_cache_creation_5m_tokens and prompt_cache_creation_1h_tokens for TTL-specific prompt caching.
    • ›Go SDK v0.11.0: Bedrock Runtime spans now capture audio and video content blocks; InvokeModelWithResponseStream is now fully instrumented for Anthropic Claude models.
    • ›Go SDK v0.11.0: Google ADK spans now include reasoning and cached token metrics.
    └──▷ BREAKING ON UPGRADE
    • !Python SDK v0.32.0: LiveKit Agents audio attachments on agent_speaking spans are now disabled by default; set BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true to restore the previous behavior.
    • !Go SDK v0.11.1 (Google GenAI): Provider metadata changed from "gemini" to "google"; update trace queries that filter on the previous provider value.
    • !Go SDK v0.11.1 (Eino): ChatModel span output is now an OpenAI-compatible choices array ([{"index": 0, "finish_reason": "...", "message": {...}}]) instead of a flat message map; embedding input is now {"inputs": [{"content": "..."}]} and output is {"count": N}, removing embedding_length and renaming embeddings_count; provider metadata is now lowercase (e.g. "openai" instead of "OpenAI"). Update trace queries that rely on the previous formats.
  15. snapshot-20260825 Aug 1, 2026 · issue 007
    └──▷ USE IT
    Enable Vercel AI SDK auto-instrumentation in a Python service so all agent calls are traced to Braintrust without per-call changes.
    python 
    import braintrust
braintrust.auto_instrument()  # Vercel AI SDK instrumentation enabled by default in v0.33.0
    Restore LiveKit Agents audio capture on agent_speaking spans after the v0.32.0 default change.
    $ BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true python your_livekit_agent.py
    • ›Adds write tools to the Braintrust MCP server so coding agents can create and update prompts, scorers, classifiers, Topics pipeline config, monitor views, alerts, scheduled jobs, evals, and dataset rows — not just read them.
    • ›Adds kimi-k3 and deepseek-v4-flash-0731 as built-in open-source models routable through the Braintrust Gateway with no external AI provider setup; select them under the Braintrust provider in playgrounds, prompts, and scorers.
    • ›Version history for prompts, parameters, and scorers now accepts a text annotation when saving a new version, and shows each version side-by-side with the one it replaced.
    • ›Dataset rows can now reference a group of up to 64 traces instead of a single trace, rendering each inline and flagging any that are no longer available.
    • ›SQL sandbox gains a collapsible query sidebar with search-by-name, drag-to-reorder, command-bar navigation, per-query rename/duplicate/delete menu, and a 'Copy share link' action that generates a shareable URL.
    +9 moreshow less
    • ›Member permission groups are now editable directly from Settings > Members, showing direct memberships, inherited memberships, and available groups in one dialog.
    • ›Python SDK v0.34.0 adds Hugging Face Transformers auto-instrumentation for text generation, summarization, translation, feature extraction, and question answering; adds automatic Braintrust project name defaulting to Harbor in the Harbor plugin (making project_name optional); and forwards eval case fields to scorer functions.
    • ›Python SDK v0.33.0 adds Vercel AI SDK auto-instrumentation (enabled by default in auto_instrument()) and Cursor SDK instrumentation for tracing agent runs, model turns, and tool calls; adds a native Harbor job plugin for syncing evaluation results to Braintrust.
    • ›Python SDK v0.32.0 adds Pipecat auto-instrumentation for real-time voice AI pipelines including LLM turns, STT, TTS, and tool calls; adds invoke_async() as an async counterpart to invoke().
    • ›TypeScript SDK v3.28.0 adds Voyage AI auto-instrumentation for embeddings, multimodal embeddings, reranking, and contextualized embeddings; scorer functions in Eval() now receive the id and tags of each case.
    • ›TypeScript SDK v3.27.0 adds Ollama and @cloudflare/think instrumentation, Anthropic beta sessions tracing (anthropic.beta.sessions.turn and anthropic.beta.sessions.thread.turn), Flue v2 support, and a vitest-evals span input override via meta.eval.input.
    • ›TypeScript SDK v3.26.0 adds auto-instrumentation for Cloudflare Agents, Cloudflare AI Chat, and Hugging Face Transformers.js, plus system prompt capture for Strands Agents SDK spans.
    • ›Go SDK v0.11.1 adds Firebase Genkit WithProvider and WithModel options on NewMiddleware for explicit model attribution, plus traced tool wrappers DefineTool, DefineToolWithInputSchema, and DefineMultipartTool; auto-instrumentation now replaces the un-traced genkit.DefineTool, genkit.DefineToolWithInputSchema, and genkit.DefineMultipartTool calls automatically.
    • ›Go SDK v0.11.0 adds Anthropic span capture for prompt_cache_creation_5m_tokens and prompt_cache_creation_1h_tokens for TTL-specific prompt caching; Bedrock Runtime spans now capture audio and video content blocks and fully instrument InvokeModelWithResponseStream for Anthropic Claude models; Google ADK spans now include reasoning and cached token metrics.
    └──▷ BREAKING ON UPGRADE
    • !Python SDK v0.32.0: LiveKit Agents audio attachments on agent_speaking spans are now disabled by default. Set BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true to restore the previous behavior.
    • !Go SDK v0.11.1 (Google GenAI): Provider metadata changed from 'gemini' to 'google'. Trace queries that filter on the previous provider value must be updated.
    • !Go SDK v0.11.1 (Eino): ChatModel span output is now an OpenAI-compatible choices array ([{"index": 0, "finish_reason": "...", "message": {...}}]) instead of a flat message map. Embedding input is now {"inputs": [{"content": "..."}]} and output is {"count": N}, removing embedding_length and renaming embeddings_count. Provider metadata is now lowercase (e.g. 'openai' instead of 'OpenAI'). Trace queries relying on the previous formats must be updated.
  16. snapshot-20260824 Aug 1, 2026 · issue 006

    Braintrust adds MCP write tools, two new open-source models, Lambda extension, group-scoped online scoring, and more.

    └──▷ TRY IT
    Re-enable LiveKit Agents audio attachment capture after upgrading to Python SDK v0.32.0, where it is now off by default.
    $ BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true python my_livekit_agent.py
    Enable Vercel AI SDK auto-instrumentation for Python to trace all AI calls with no per-call changes.
    python 
    from braintrust import auto_instrument
auto_instrument()

# existing Vercel AI SDK calls are now traced automatically
    • ›Adds write tools to the Braintrust MCP server, enabling coding agents to create and update prompts, scorers, classifiers, monitor views, alerts, scheduled jobs, dataset rows, and Topics pipeline configuration; configure your client to require confirmation before running write tools.
    • ›Adds kimi-k3 and deepseek-v4-flash-0731 as built-in open-source models available through the Braintrust Gateway and provider selector with no AI provider setup required.
    • ›Adds the Braintrust Lambda Extension to give Python and TypeScript/JavaScript Lambda functions a local handoff path for traces, reducing time spent by the SDK's flush() method in the request path.
    • ›Adds Group scope to online scoring rules, letting you evaluate a set of related multi-turn traces as a single unit using a session key of your choice without changing your logging.
    • ›Adds a Summary table layout to the experiments list, comparing every experiment in a project as columns with scores and metrics as rows, including an 'All scores (avg)' row.
    +14 moreshow less
    • ›Adds annotated version history for prompts, parameters, and scorers, pinning a description note to each saved version and showing each version alongside the one it replaced.
    • ›Adds trace group references to dataset rows, allowing a multi-turn session or related set of logs to become one example with up to 64 traces per row.
    • ›Adds Azure AI Gateway as a supported AI provider, supporting models that use the OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages API behind an Azure API Management endpoint.
    • ›Adds Hugging Face Transformers auto-instrumentation in Python SDK v0.34.0 for local pipelines including text generation, summarization, translation, feature extraction, and question answering.
    • ›Adds Vercel AI SDK for Python auto-instrumentation in Python SDK v0.33.0 (enabled by default in auto_instrument()).
    • ›Adds Cursor SDK Python instrumentation in v0.33.0 for tracing agent runs, model turns, and tool calls.
    • ›Adds native Harbor job plugin in Python SDK v0.33.0 for syncing Harbor evaluation results to Braintrust.
    • ›Adds Pipecat auto-instrumentation in Python SDK v0.32.0 for tracing real-time voice AI pipelines including LLM turns, STT, TTS, and tool calls.
    • ›Adds Voyage AI auto-instrumentation in TypeScript SDK v3.28.0 for embeddings, multimodal embeddings, reranking, and contextualized embeddings.
    • ›Adds Ollama and @cloudflare/think instrumentation, Anthropic beta sessions tracing (anthropic.beta.sessions.turn and anthropic.beta.sessions.thread.turn), Flue v2 support, and a vitest-evals span input override via meta.eval.input in TypeScript SDK v3.27.0.
    • ›Adds auto-instrumentation for Cloudflare Agents, Cloudflare AI Chat, and Hugging Face Transformers.js in TypeScript SDK v3.26.0.
    • ›Adds system prompt capture for Strands Agents SDK spans in TypeScript SDK v3.26.0.
    • ›Adds WithProvider and WithModel options on NewMiddleware for explicit model attribution, plus traced tool wrappers DefineTool, DefineToolWithInputSchema, and DefineMultipartTool for Firebase Genkit in Go SDK v0.11.1.
    • ›Adds braintrust.context_json carrying SDK name and version to all spans in Ruby SDK v0.4.1.
    └──▷ BREAKING ON UPGRADE
    • !Python SDK v0.32.0: LiveKit Agents audio attachments on agent_speaking spans are now disabled by default; set BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true to restore the previous behavior.
    • !Go SDK v0.11.1 (Google GenAI): Provider metadata changed from "gemini" to "google"; update trace queries that filter on the previous provider value.
    • !Go SDK v0.11.1 (Eino): ChatModel span output is now an OpenAI-compatible choices array ([{"index": 0, "finish_reason": "...", "message": {...}}]) instead of a flat message map; embedding input is now {"inputs": [{"content": "..."}]} and output is {"count": N}, removing embedding_length and renaming embeddings_count; provider metadata is now lowercase (e.g. "openai" instead of "OpenAI"); update trace queries that rely on the previous formats.
    • !Go SDK v0.11.0 (Anthropic): Span metadata no longer includes endpoint; the output field is now a single message object instead of an array; non-streaming spans no longer emit time_to_first_token.
    • !Go SDK v0.11.0 (Bedrock): Span metadata renames stop_sequences to stop and removes additional_model_request_fields; image, document, and tool block shapes now align with Bedrock's native wire format.
  17. snapshot-20260823 Aug 1, 2026 · issue 005
    └──▷ TRY IT
    Re-enable LiveKit Agents audio capture after upgrading to Python SDK v0.32.0, where it is off by default.
    $ export BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true
python my_livekit_agent.py
    • ›Adds kimi-k3 and deepseek-v4-flash-0731 as requestable model identifiers through the Braintrust Gateway, joining GLM-5.2 as built-in open-source models with no AI provider setup required; usage draws from monthly model credits shared with Topics.
    • ›Adds invoke_async() as an async counterpart to invoke() in the Python SDK (v0.32.0).
    • ›Adds BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true environment variable to the Python SDK to re-enable LiveKit Agents audio attachments on agent_speaking spans (disabled by default as of v0.32.0).
    • ›TypeScript SDK v3.28.0 surfaces id and tags of each eval case to scorer functions inside Eval().
    • ›Go SDK v0.11.0 adds Anthropic span capture for prompt_cache_creation_5m_tokens and prompt_cache_creation_1h_tokens for TTL-specific prompt caching.
    +5 moreshow less
    • ›Go SDK v0.11.0 extends Bedrock Runtime spans to capture audio and video content blocks, and fully instruments InvokeModelWithResponseStream for Anthropic Claude models.
    • ›Go SDK v0.11.0 adds reasoning and cached token metrics to Google ADK spans.
    • ›Exposes MCP write tools on the Braintrust MCP server, enabling coding agents to create and update prompts, scorers, classifiers, monitor views, alerts, scheduled jobs, dataset rows, and the Topics pipeline end to end; write tools use authenticated account permissions and several replace or remove objects.
    • ›Adds a Summary table layout to the experiments list, showing every experiment as a column with scores and metrics as rows, an 'All scores (avg)' row for mean non-pairwise scores, and grouping to compare aggregated scores across groups.
    • ›Adds a collapsible query sidebar to the SQL sandbox with search by name, drag-to-reorder, command-bar jump, and a 'Copy share link' action that generates a URL opening the query in a teammate's sandbox without running it.
    └──▷ BREAKING ON UPGRADE
    • !Python SDK v0.32.0: LiveKit Agents audio attachments on agent_speaking spans are now disabled by default. Set BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true to restore the previous behavior.
    • !Go SDK v0.11.1 (Google GenAI): Provider metadata changed from "gemini" to "google". Update trace queries that filter on the previous provider value.
    • !Go SDK v0.11.1 (Eino): ChatModel span output is now an OpenAI-compatible choices array ([{"index": 0, "finish_reason": "...", "message": {...}}]) instead of a flat message map. Embedding input is now {"inputs": [{"content": "..."}]} and output is {"count": N}, removing embedding_length and renaming embeddings_count. Provider metadata is now lowercase (e.g. "openai" instead of "OpenAI"). Update trace queries that rely on the previous formats.
    • !Go SDK v0.11.0 (Anthropic): Span metadata no longer includes endpoint. The output field is now a single message object instead of an array. Non-streaming spans no longer emit time_to_first_token.
    • !Go SDK v0.11.0 (Bedrock): Span metadata renames stop_sequences to stop and removes additional_model_request_fields. Image, document, and tool block shapes now align with Bedrock's native wire format.
  18. snapshot-20260822 Aug 1, 2026 · issue 004
    └──▷ USE IT
    Enable Vercel AI SDK tracing in Python so every AI call is automatically captured in Braintrust.
    python 
    from braintrust import auto_instrument

auto_instrument()  # Vercel AI SDK for Python instrumentation is enabled by default
    • ›Adds write tools to the Braintrust MCP server, enabling coding agents to create and update prompts, scorers, classifiers, Topics pipeline config, monitor views, alerts, scheduled jobs, evals, and dataset rows using the permissions of the authenticated Braintrust account.
    • ›Adds kimi-k3 and deepseek-v4-flash-0731 as built-in open-source models available under the Braintrust provider in playgrounds, prompts, and scorers, or via the Braintrust Gateway — no AI provider setup required.
    • ›Adds invoke_async() as an async counterpart to the existing invoke() method in the Python SDK v0.32.0.
    • ›Adds WithProvider and WithModel options on NewMiddleware and traced tool wrappers DefineTool, DefineToolWithInputSchema, and DefineMultipartTool for Firebase Genkit in Go SDK v0.11.1; auto-instrumentation now replaces genkit.DefineTool, genkit.DefineToolWithInputSchema, and genkit.DefineMultipartTool calls with their traced equivalents.
    • ›Adds prompt_cache_creation_5m_tokens and prompt_cache_creation_1h_tokens capture for TTL-specific prompt caching on Anthropic spans in Go SDK v0.11.0.
    +5 moreshow less
    • ›Adds audio and video content block capture and full instrumentation for InvokeModelWithResponseStream for Anthropic Claude models in Bedrock Runtime spans in Go SDK v0.11.0.
    • ›Adds annotated version history for prompts, parameters, and scorers, allowing a description note to be pinned to each saved version; history now shows each version side-by-side with the one it replaced.
    • ›Adds support for trace groups in dataset rows, letting a multi-turn session or related set of logs become a single example (up to 64 traces per row), rendered inline with unavailable traces flagged.
    • ›Adds a collapsible query sidebar to the SQL sandbox with search by name, drag-to-reorder, command-bar navigation, and a 'Copy share link' option that opens a query in a teammate's sandbox without running it.
    • ›Forwards eval case fields to scorer functions in Python SDK v0.34.0.
    └──▷ BREAKING ON UPGRADE
    • !Python SDK v0.32.0: LiveKit Agents audio attachments on agent_speaking spans are now disabled by default. Set BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true to restore the previous behavior.
    • !Go SDK v0.11.1 (Google GenAI): Provider metadata changed from 'gemini' to 'google'. Update trace queries that filter on the previous provider value.
    • !Go SDK v0.11.1 (Eino): ChatModel span output is now an OpenAI-compatible choices array ([{"index": 0, "finish_reason": "...", "message": {...}}]) instead of a flat message map. Embedding input is now {"inputs": [{"content": "..."}]} and output is {"count": N}, removing embedding_length and renaming embeddings_count. Provider metadata is now lowercase (e.g. 'openai' instead of 'OpenAI'). Update trace queries that rely on the previous formats.
    • !Go SDK v0.11.0 (Anthropic): Span metadata no longer includes endpoint. The output field is now a single message object instead of an array. Non-streaming spans no longer emit time_to_first_token.
    • !Go SDK v0.11.0 (Bedrock): Span metadata renames stop_sequences to stop and removes additional_model_request_fields. Image, document, and tool block shapes align with Bedrock's native wire format.
  19. snapshot-20260821 Aug 1, 2026 · issue 003

    Braintrust adds MCP write tools, two new open-source models, AWS Lambda extension, group-scoped online scoring, and a raft of SDK instrumentation additions.

    └──▷ USE IT
    Enable Vercel AI SDK auto-instrumentation in a Python service to trace all AI calls without manual span creation.
    python 
    import braintrust

braintrust.auto_instrument()  # Vercel AI SDK for Python instrumented by default in v0.33.0+
    • ›Adds write tools to the Braintrust MCP server so coding agents can create and update prompts, scorers, classifiers, monitor views, alerts, scheduled jobs, dataset rows, and Topics pipeline configuration — not just read them; write tools use your authenticated account permissions and several replace or remove objects, so configure your client to require confirmation.
    • ›Adds kimi-k3 and deepseek-v4-flash-0731 as built-in open-source models served by Braintrust with no AI-provider setup; select them under the Braintrust provider in playgrounds, prompts, and scorers, or request them by name through the Braintrust Gateway; usage draws from monthly model credits shared with Topics.
    • ›Adds the Braintrust Lambda Extension for Python and TypeScript/JavaScript Lambda functions, giving the Braintrust SDK's flush() method a local handoff path for traces to reduce latency in the request path.
    • ›Adds Azure AI Gateway as a supported AI provider, routing calls to any model behind an Azure API Management endpoint that uses the OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages API.
    • ›Adds a Summary table layout to the experiments list that puts each experiment in a column, scores and metrics in rows, and an 'All scores (avg)' row showing the mean of non-pairwise scores, with grouping support.
    +6 moreshow less
    • ›Traces that share a metadata.conversation_id now surface as related traces automatically for end-to-end multi-turn conversation review without requiring grouping configuration; 'Group by' and 'Cluster by' have moved into the row-type selector in the toolbar.
    • ›Adds trace-group references to dataset rows so a multi-turn session or related log set becomes one example, with up to 64 traces per row rendered inline.
    • ›Python SDK v0.33.0 adds Vercel AI SDK for Python auto-instrumentation (enabled by default in auto_instrument()).
    • ›Python SDK v0.33.0 adds Cursor SDK Python instrumentation for tracing agent runs, model turns, and tool calls.
    • ›Go SDK v0.11.1 adds WithProvider and WithModel options on NewMiddleware for explicit model attribution in Firebase Genkit, plus traced tool wrappers DefineTool, DefineToolWithInputSchema, and DefineMultipartTool; auto-instrumentation now replaces genkit.DefineTool, genkit.DefineToolWithInputSchema, and genkit.DefineMultipartTool calls with their traced equivalents.
    • ›Ruby SDK v0.4.1 adds braintrust.context_json carrying SDK name and version to all spans.
    └──▷ BREAKING ON UPGRADE
    • !Python SDK v0.32.0: LiveKit Agents audio attachments on agent_speaking spans are now disabled by default. Set BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true to restore the previous behavior.
    • !Go SDK v0.11.1 (Google GenAI): Provider metadata changed from 'gemini' to 'google'. Update trace queries that filter on the previous provider value.
    • !Go SDK v0.11.1 (Eino): ChatModel span output is now an OpenAI-compatible choices array ([{"index": 0, "finish_reason": "...", "message": {...}}]) instead of a flat message map. Embedding input is now {"inputs": [{"content": "..."}]} and output is {"count": N}, removing embedding_length and renaming embeddings_count. Provider metadata is now lowercase (e.g. 'openai' instead of 'OpenAI'). Update trace queries that rely on the previous formats.
    • !Go SDK v0.11.0 (Anthropic): Span metadata no longer includes endpoint. The output field is now a single message object instead of an array. Non-streaming spans no longer emit time_to_first_token.
    • !Go SDK v0.11.0 (Bedrock): Span metadata renames stop_sequences to stop and removes additional_model_request_fields. Image, document, and tool block shapes now align with Bedrock's native wire format.
  20. snapshot-20260820 Aug 1, 2026 · issue 002

    Braintrust adds Kimi K3 & DeepSeek V4 Flash, AWS Lambda Extension, Group-scoped online scoring, and major SDK auto-instrumentation expansions.

    └──▷ USE IT
    Enable automatic tracing for all Vercel AI SDK for Python calls as part of your existing instrumentation setup.
    python 
    from braintrust import auto_instrument

auto_instrument()  # Vercel AI SDK for Python instrumentation is enabled by default
    Trace async prompt invocations without blocking the request path using the new async counterpart to invoke().
    python 
    result = await prompt.invoke_async({"input": "Classify this support ticket."})
print(result)
    • ›Adds kimi-k3 and deepseek-v4-flash-0731 as built-in open-source models served through the Braintrust Gateway — no AI provider setup required; select them under the Braintrust provider in playgrounds, prompts, and scorers.
    • ›Introduces the Braintrust Lambda Extension, giving Python and TypeScript/JavaScript Lambda functions a local handoff path for traces so the SDK's flush() method spends less time in the request path.
    • ›Adds Group scope to online scoring rules, letting you evaluate a set of related, multi-turn traces as a single unit based on a session key of your choice, without changing logging.
    • ›Adds a Summary table layout to the experiments list that compares every experiment in a project at a glance, with experiments as columns and scores/metrics as rows, plus an 'All scores (avg)' row showing the mean of non-pairwise scores.
    • ›Automatic conversation threading: traces sharing a metadata.conversation_id now surface as related traces automatically for end-to-end review without configuring grouping.
    +18 moreshow less
    • ›Adds annotated version history for prompts, parameters, and scorers — attach a note when saving a new version and view it pinned to that version alongside the replaced version.
    • ›Dataset rows can now reference a group of up to 64 traces instead of a single trace, rendering each inline and flagging unavailable ones.
    • ›Adds Azure AI Gateway as a supported AI provider, supporting models using the OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages API behind an Azure API Management endpoint.
    • ›Python SDK v0.34.0 adds Hugging Face Transformers auto-instrumentation for local pipelines (text generation, summarization, translation, feature extraction, question answering).
    • ›Python SDK v0.33.0 adds Vercel AI SDK for Python auto-instrumentation (enabled by default in auto_instrument()) and Cursor SDK Python instrumentation for tracing agent runs, model turns, and tool calls.
    • ›Python SDK v0.33.0 adds native Harbor job plugin for syncing Harbor evaluation results to Braintrust.
    • ›Python SDK v0.32.0 adds Pipecat auto-instrumentation for real-time voice AI pipelines including LLM turns, STT, TTS, and tool calls.
    • ›Python SDK v0.32.0 adds invoke_async() as an async counterpart to invoke().
    • ›TypeScript SDK v3.28.0 adds Voyage AI auto-instrumentation for embeddings, multimodal embeddings, reranking, and contextualized embeddings.
    • ›TypeScript SDK v3.28.0: scorer functions in Eval() now receive the id and tags of each case.
    • ›TypeScript SDK v3.27.0 adds Ollama and @cloudflare/think instrumentation, Anthropic beta sessions tracing (anthropic.beta.sessions.turn and anthropic.beta.sessions.thread.turn), Flue v2 support, and a vitest-evals span input override via meta.eval.input.
    • ›TypeScript SDK v3.26.0 adds auto-instrumentation for Cloudflare Agents, Cloudflare AI Chat, and Hugging Face Transformers.js, plus system prompt capture for Strands Agents SDK spans.
    • ›Go SDK v0.11.0: Anthropic spans now capture prompt_cache_creation_5m_tokens and prompt_cache_creation_1h_tokens for TTL-specific prompt caching; Bedrock Runtime spans now capture audio and video content blocks with full instrumentation for InvokeModelWithResponseStream on Anthropic Claude models; Google ADK spans now include reasoning and cached token metrics.
    • ›Ruby SDK v0.4.1: all spans now carry braintrust.context_json with SDK name, version, instrumentation scope, and detected runtime environment; override with BRAINTRUST_ENVIRONMENT_NAME and BRAINTRUST_ENVIRONMENT_TYPE.
    • ›Custom views now support writing scores via trace.update, enabling numeric feedback capture without leaving the view.
    • ›The Edit billing information dialog now includes a 'Purchase order' field and 'Tax information' fields (country and tax ID type).
    • ›Starter and Pro plan subscribers can now redeem coupon codes from Settings > Billing by clicking 'Redeem coupon' under Current plan.
    • ›Images pointing to local/private networks or using unsafe URL schemes now require approval before loading, even in Auto-load images mode.
    └──▷ BREAKING ON UPGRADE
    • !Python SDK v0.32.0: LiveKit Agents audio attachments on agent_speaking spans are now disabled by default. Set BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true to restore the previous behavior.
    • !Go SDK v0.11.0 (Anthropic): span metadata no longer includes endpoint; the output field is now a single message object instead of an array; non-streaming spans no longer emit time_to_first_token.
    • !Go SDK v0.11.0 (Bedrock): span metadata renames stop_sequences to stop and removes additional_model_request_fields; image, document, and tool block shapes now align with Bedrock's native wire format.
my-toolchain — 0 tools
paste an install list to detect your tools

A brew list, a Brewfile, requirements.txt, a Dockerfile — or just the product names, free-form. Nothing leaves your browser.

    browse all tools →