This Week's Highlights — issue 004, September 11, 2026
The best tooling updates that shipped this week.
Every feature release from the 354 tools on our watchlist goes into the daily newsletter. Once a week we read the whole field side by side and choose our top picks, judged on two questions: how deep and complete is the single best capability in the release, and how much it changes what you can actually do.
A tool is judged on everything it shipped that week, so a project that releases daily gets credit for the sum — and still only takes one slot. We would rather run a short list than a padded one.
What stands out in this week's picks, grouped by what it lets you do. Every tool named links to its pick below.
Two independent pushes at agent credentials: LangGraph's Connections keeps agent-owned secrets and per-caller OAuth OAuth An open authorization protocol that lets a user grant a third-party application access to their account on another service without sharing their password, using scoped tokens instead of credentials. identities in a LangSmith workspace instead of process env vars, and Arcade collapses per-tool OAuth prompts into one approval per service while logging every tool call. Separately, Pinecone's BM25 full-text search goes GA inside the vector index, so keyword and vector retrieval share one store.
-
govern
Stop hardcoding agent credentials and stop prompting the user once per tool
LangGraph's Connections stores agent-owned secrets and per-caller OAuth OAuth An open authorization protocol that lets a user grant a third-party application access to their account on another service without sharing their password, using scoped tokens instead of credentials. identities in a LangSmith workspace and resolves them at run time via a new CLI and Python API, so a multi-tenant agent no longer needs a homegrown token store or one shared service account for all callers. Arcade bundles pre-authorization so a user approves a service once rather than consenting again at each tool, and Execution Tool Logs give an auditable record of what the agent actually called.
-
build
Run hybrid keyword-plus-vector retrieval without a second search system
BM25 full-text search is now GA inside the vector index, which removes the usual pattern of running Elasticsearch or OpenSearch alongside Pinecone and merging results in application code — the fix for embeddings missing exact identifiers, product codes and rare terms. The expanded admin APIs for organizations, projects, API keys, service accounts and role bindings also make index and key provisioning scriptable rather than console work.
-
deploy
Shed load under pressure instead of letting every request degrade
vLLM v0.29.0 adds request admission-control flags, so an overloaded server can reject or defer arrivals rather than admitting everything and pushing all in-flight requests past their latency budget; deterministic decode tracing also makes a bad generation reproducible for debugging. SGLang v0.5.19 pushes throughput on specific silicon — DeepEP v2 for cross-node MoE MoE Mixture of Experts, a neural network architecture where input is routed to a subset of specialized sub-networks rather than the full model, cutting compute cost while keeping total parameter count high. decoding, W4A8 W4A8 A neural-network quantization scheme where weights are stored at 4-bit precision and activations at 8-bit, reducing model memory and compute cost while preserving enough numerical range for inference accuracy. / FP8 FP8 An 8-bit floating-point numeric format used in AI accelerator hardware to store and compute model weights and activations, trading precision for dramatically lower memory use and faster throughput. MoE on Hopper, decode context parallelism for Blackwell MLA — plus beam search for workloads that need multiple candidate sequences.
-
operate
Have something triage recurring trace failures before you open the dashboard
Braintrust's Patterns runs Loop on a schedule to surface recurring problems across traces and Debugger diagnoses a single one, with Loop now a persistent server-side runtime that can edit prompts, scorers and dashboards itself — the scan-the-logs-for-a-pattern step becomes a standing job. Cursor's Projects adds a persistent coordinator agent delegating to subagents with shared context, subscribed to Slack, schedules and PRs, and self-hosted machine pools let those cloud agents run on your own infrastructure.
Three unrelated things stand out today: Cursor can coordinate long-running coding work across many agents and your own machines; Pinecone now mixes BM25 with vectors in one index; Braintrust starts finding recurring trace failures without a human spelunking dashboards first.
-
build
Hand a coding project to a coordinator instead of a single chat loop
Projects gives teams a persistent agent that can split work across subagents while staying subscribed to Slack, schedules, and PRs. Self-hosted machine pools also let cloud agents run on your infrastructure, so longer coding jobs can use controlled compute instead of a vendor-only runtime.
-
build
Run lexical and semantic retrieval from the same index
BM25 is now GA inside Pinecone’s vector index, so teams can combine exact-term matching with embedding similarity without maintaining a separate search stack. The new admin and access-control APIs also make it easier to automate org, project, API-key, service-account, and role-binding setup instead of wiring them by hand.
-
operate
Let trace investigation run before an engineer opens the dashboard
Patterns runs Loop on a schedule to surface recurring production issues, while Debugger diagnoses individual traces. That turns trace review from a manual hunt through logs into a standing background process that points teams at repeated failures and likely causes.
-
deploy
Push more serving work onto the hardware and runtime path you already picked
vLLM added request admission-control flags and Tenstorrent serving support; SGLang added cross-node MoE decoding, quantized MoE optimizations, and more speculative-decoding backends; llama.cpp improved memory-efficient loading, distributed inference, and KV-cache behavior. Together, these make capacity, latency, and hardware fit less dependent on moving to a different serving stack.
Cursor is an AI-powered code editor built on VS Code that uses large language models to assist with writing, debugging, and refactoring code.
Projects launches a coordinator agent that plans and delegates work across thousands of parallel cloud subagents and keeps running after you close your laptop, with shared context files that sync across every agent and Subscriptions that watch a Slack channel or schedule to act autonomously on new signals.
Cursor's biggest window in a while: Projects brings a persistent coordinator agent that delegates to thousands of subagents with shared context and Slack/schedule/PR subscriptions, alongside self-hosted machine pools for running cloud agents on your own infrastructure, a new Origin code hosting service, and a batch of smaller cloud-agent workflow additions (/goal, Custom Modes, Builds, Publish, live port-forwarding).︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
what's the number?
Each feature carries 0–100 for how completely the vendor documented it — not how big or important the work is. A major capability described in eight words scores low, and that is the finding.
- depth0–40 what it does and how it works or what changed
- specificity0–30 names real surfaces — APIs, flags, formats, limits, numbers
- actionability0–30 enough to go use it — a named endpoint, flag, or config key tops this; a UI path is a starting point
Code and config rank first by construction: a documented endpoint, flag or config key scores at the top of specificity and actionability, so it sorts above a feature described only in prose. Under 60 the notes go thin — everything below the “thinner coverage” line is thinner documentation, not smaller work. Hover any meter for that feature's three sub-scores.
The cursor agent worker start CLI subcommand registers a machine as a self-hosted worker, opening a long-lived outbound HTTPS connection to the Cursor cloud so tool execution, codebase, build outputs, and secrets stay entirely within your own network.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Workers are organized via 'My Machines' (single laptop/VM) or Pools/Team Pools (named queues of workers) that auto-scale and hibernate idle machines, and can run on pre-existing sandbox infrastructure including AWS Lambda, Coder, Cloudflare, Daytona, Modal, Namespace, Vercel, and E2B, or on custom hardware such as GPUs, Apple silicon Macs (for iOS/macOS builds), and Kubernetes. Self-hosted workers on Linux and Mac support computer use — click, type, screenshot, and browser control via Chrome or Chromium — with live desktop viewing and takeover from Cursor.





$ cursor agent worker start
Custom Modes let you pin any skill as an always-on mode in chat, activated via ⌥⏎ (Mac) or Alt+Enter (Windows) from the skill picker.
The /goal command gives a cloud agent a long-lived objective that it pursues until fully complete; pair it with a custom mode or /loop for recurring check-ins.
/goal fix all flaky tests and make CI green$ /goal fix all flaky tests and make CI green
Origin is a built-in code hosting service with repos, pull requests, code browsing, and GitHub sync, available in early beta on all paid plans. The Codebase tab is the home for Origin repos, with a +New button to create repos whose names become part of every repo URL at cursor.com/codebase/<name>; per-repo Settings manage GitHub sync status, access control, and connected apps. Cloud Agents can also start without a connected GitHub or third-party SCM provider via 'Start from scratch' in the repo picker, saving work to a Cursor Origin repo.
Projects, accessible from the left-hand nav, runs a persistent coordinator agent that plans and delegates work across thousands of parallel subagents for large bodies of work spanning months, without requiring the user to stay online. Projects run cloud-by-default on persistent compute so work continues after closing your laptop, spinning up local agents on demand for machine-specific tasks; each subagent runs on its own isolated virtual machine with a clean copy of the project and independent cloud environment. A shared context file set syncs across all cloud and local agents in a Project, growing automatically as agents record codebase knowledge and working preferences.
Subscriptions let a Project or cloud agent coordinator watch a Slack channel, run on a schedule, or follow all your PRs, acting autonomously on detected signals such as CI failures, opens, and merges without manual prompting.
Send a message to redirect an agent mid-run without interrupting it: follow-ups queue at the next tool call rather than cutting the agent off, or send immediately with 'Send now' or by pressing ⏎ twice.
Builds are pre-prepared environment snapshots (repos cloned, dependencies installed) that make cloud agents start 3x faster, with a Builds tab in the Cloud Agents dashboard for inspecting build status, logs, and commit SHAs.
Origin repos gain a per-repo Apps tab for adding app extensions — Vercel, Depot, and Buildkite — enabling preview deployments and CI integration on every PR.
Comments, reactions, and replies made on a pull request in either Origin or GitHub appear in the other within seconds.
Publish connects a Vercel account to a cloud agent and produces a live URL from a built project.
Adds in-browser live port-forwarding of a cloud agent's environment for real-time preview and design mode tooling.
Pinecone is a vector database service that stores and searches high-dimensional embeddings for AI and machine learning applications.
Full-text search is GA with BM25 keyword ranking, Lucene query syntax, and 18-language stemming inside the same vector index — no separate search cluster — plus text-match filters that constrain semantic search to documents matching a literal keyword.
Pinecone shipped full-text search (BM25) as a GA capability inside its vector index, alongside a broad expansion of administrative and access-control APIs covering organizations, projects, API keys, service accounts, and role bindings, plus a paginated document listing endpoint and a doubled pod-to-serverless migration limit.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
what's the number?
Each feature carries 0–100 for how completely the vendor documented it — not how big or important the work is. A major capability described in eight words scores low, and that is the finding.
- depth0–40 what it does and how it works or what changed
- specificity0–30 names real surfaces — APIs, flags, formats, limits, numbers
- actionability0–30 enough to go use it — a named endpoint, flag, or config key tops this; a UI path is a starting point
Code and config rank first by construction: a documented endpoint, flag or config key scores at the top of specificity and actionability, so it sorts above a feature described only in prose. Under 60 the notes go thin — everything below the “thinner coverage” line is thinner documentation, not smaller work. Hover any meter for that feature's three sub-scores.
Adds full CRUD for service accounts: POST /service-accounts to create one with optional initial role bindings (client secret returned only once), GET /service-accounts/{id} and GET /service-accounts to retrieve or list them, PATCH /service-accounts/{id} to update a service account's name, and DELETE /service-accounts/{id} to delete one and revoke all tokens it minted within seconds.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ A rotate-secret endpoint rotates a service account's OAuth client secret, revoking the previous secret and its tokens within seconds and returning the new secret only once.
$ curl -X POST 'https://api.pinecone.io/organizations/<org-id>/service-accounts/<service-account-id>/rotate-secret' \ -H 'Api-Key: <api-key>'
$ curl -X POST https://api.pinecone.io/service-accounts/<service-account-id>/rotate-secret \ -H 'Authorization: Bearer <your-token>'
Full-text search reaches general availability via the Documents API, running BM25 keyword ranking inside the same Pinecone Database index as dense and sparse vectors with no separate search cluster required. It supports Lucene query syntax (boolean operators, phrase queries, and fuzzy matching with the ~ operator) across multiple text fields per index, text-match filters that restrict semantic vector search results to documents matching a literal keyword filter, and tokenization/stemming in 18 languages plus language-agnostic n-gram tokenization for substring- and prefix-matching. Full-text search indexes support Dedicated Read Nodes for provisioned read capacity and Bring Your Own Cloud (BYOC) deployment, and are metered in the same read units and write units as vector indexes with no cluster sizing, sharding, or upgrade maintenance.
Adds a List documents endpoint that returns up to 100 documents per page in a namespace, with an optional prefix parameter to filter by ID prefix and a pagination token for iterating through results.
$ curl -X GET 'https://api.pinecone.io/namespaces/<namespace>/documents?prefix=user-&pagination=<token>' \ -H 'Api-Key: <api-key>'
Adds lifecycle management endpoints across three resource types: organizations (List organizations, Get organization details, Update an organization name, Delete an organization), projects (List projects, Create a new project, Get project details, Update a project including maximum Pod count and customer-managed encryption key (CMEK), Delete a project), and API keys (List API keys, Delete an API key).
Adds POST /role-bindings to grant a role to a principal (such as a service account) at organization or project scope, letting a role be assigned immediately after creating the principal.
$ curl -X POST 'https://api.pinecone.io/organizations/<org-id>/role-bindings' \ -H 'Api-Key: <api-key>' \ -H 'Content-Type: application/json' \ -d '{"principal_id": "<service-account-id>", "role": "editor", "scope": "project", "project_id": "<project-id>"}'
Raises the pod-based index to serverless migration record limit to 50 million records, up from 25 million.
Serves a complete documentation index at /llms.txt for programmatic discovery of all available documentation pages.
Adds GET /invites to list pending and expired invites in the caller's organization.
Anthropic develops Claude, an AI assistant API for developers to build applications with advanced language understanding and reasoning capabilities.
Claude Fable 5.1 launches with a 1M-token context window, 128k max output, and always-on adaptive thinking; ant apply brings declarative, file-based creation of agents, environments, and deployments with a lockfile for reproducible CI.
Anthropic launched Claude Fable 5.1 and Claude Mythos 5.1 with 1M-token context windows and adaptive thinking, added declarative infrastructure-as-code via ant apply, and gave Claude Managed Agents an auto permission policy with live terminal session control, alongside several beta message-control headers, a full Admin API for the ant CLI and SDKs, and multiple breaking changes to tool_choice, thinking-block replay, and SDK beta clients.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
what's the number?
Each feature carries 0–100 for how completely the vendor documented it — not how big or important the work is. A major capability described in eight words scores low, and that is the finding.
- depth0–40 what it does and how it works or what changed
- specificity0–30 names real surfaces — APIs, flags, formats, limits, numbers
- actionability0–30 enough to go use it — a named endpoint, flag, or config key tops this; a UI path is a starting point
Code and config rank first by construction: a documented endpoint, flag or config key scores at the top of specificity and actionability, so it sorts above a feature described only in prose. Under 60 the notes go thin — everything below the “thinner coverage” line is thinner documentation, not smaller work. Hover any meter for that feature's three sub-scores.
Adds auto to Claude Managed Agents permission policies, letting the server evaluate each agent or MCP tool call and automatically run it, deny it, or pause for approval; agent.tool_use and agent.mcp_tool_use events gain evaluation and evaluated_permission fields reporting how each call was evaluated under the policy.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ The ant CLI adds ant beta:sessions connect to attach your terminal to a live session — following it, sending messages, and allowing or denying pending tool calls — plus a --web flag that serves the Claude Console session viewer locally and opens the session in a browser instead.
Makes the Admin API available in the ant CLI and Python, TypeScript, C#, Go, Java, PHP, and Ruby SDKs under client.beta.organization, covering organization info, members, invites, workspaces, API keys, rate limits, service accounts, workload identity federation issuers and rules, and customer-managed encryption keys; reads credentials from ANTHROPIC_API_KEY or ANTHROPIC_AUTH_TOKEN.
Adds per-message effort control in beta on Claude Fable 5.1, Claude Mythos 5.1, and Claude Opus 5: include output_config.effort in a role: "system" message inside messages with the mid-conversation-output-config-2026-07-01 beta header to change effort mid-conversation while preserving the prompt cache. Extended to Google Cloud for the same three models.
Introduces the thinking-binding-controls-2026-08-01 beta header, which reports dropped thinking blocks in an input_transformations response field and adds thinking.block_binding.prefix_mismatch_behavior to choose between rejecting or dropping blocks whose conversation history changed.
Adds turn-scoped system messages in beta via the mid-conversation-system-clear-at-2026-08-21 header: setting clear_at: "next_user_message" on a mid-conversation role: "system" message makes it apply only to the current turn while remaining in history at no token cost.
Extends the Compliance API local session endpoints (beta, Claude Enterprise) to return transcripts for Claude Science sessions (product_surface value claude_science) and Claude for Microsoft 365 sessions in Excel, PowerPoint, Word, and Outlook (product_surface values beginning with office_agents), using the existing Compliance Access Key and read:compliance_user_data scope; also promotes the Compliance API session endpoints for Cowork and Claude Code sessions out of beta.
In Python SDK 1.2.0, TypeScript SDK 0.122.0, Go SDK 1.68.0, Java SDK 2.59.0, Ruby SDK 1.67.0, and C# SDK 12.44.0, client.beta.files and client.beta.skills no longer send the files-api-2025-04-14 and skills-2025-10-02 beta headers and return the same response shapes as client.files and client.skills; client.beta.skills.delete() now deletes a Skill together with all of its versions, and the beta Messages type BetaSkill (the container Skill reference) is renamed BetaContainerSkill.
Adds the ant apply subcommand to the ant CLI (v1.30.0) for declarative, file-based creation and updating of agents, environments, skills, memory stores, and deployments from repository files; each run writes a claude-lock.json lockfile so subsequent runs — locally or in CI — update existing resources instead of creating duplicates.
Launches claude-fable-5-1 with a 1M token context window, 128k max output tokens, always-on adaptive thinking, priced at $10/$50 per MTok input/output with prompt cache reads at $0.25 per MTok (0.025x base input price). claude-mythos-5-1 launches for Project Glasswing participants with the same context window, max output, adaptive thinking, and pricing.
Adds the "updates" value for thinking.display in beta (thinking-display-updates-2026-08-18 header), returning short progress updates Claude writes between tool calls as text, with at most one thinking block before each tool call.
Makes the computer_toolset_20260801 and browser_toolset_20260801 computer use and browser use toolsets available on Google Cloud for Claude Fable 5, Claude Mythos 5, Claude Opus 5, Claude Sonnet 5, and Claude Opus 4.8.
For accounts created on or after August 31, 2026, replaying a thinking block on Claude Fable 5.1 after the system prompt, tools, or an earlier message has changed returns a 400 error; thinking blocks produced by Claude Fable 5.1 and Claude Mythos 5.1 are also dropped by the API if replayed to an earlier model.
On Claude Fable 5.1 and Claude Mythos 5.1, tool_choice types any and tool are not supported and return a 400 error; only auto and none are accepted.
Text output from Claude Fable 5.1 and Claude Mythos 5.1 automatically carries Anthropic's text watermark; image, video, and audio files produced via the code execution tool carry C2PA Content Credentials when retrieved through the Files API, with no request or response changes required.
Adds personal keys and service account keys in the Console, scoped to a specific workspace or active across any workspace or admin endpoints the linked account can access.
The Claude Enterprise Admin API endpoints, Claude Enterprise Analytics API, and Compliance API now require the anthropic-version header on every request.
- !On Claude Fable 5.1 and Claude Mythos 5.1,
tool_choicetypesanyandtoolare not supported and return a 400 error; onlyautoandnoneare accepted. - !For accounts created on or after August 31, 2026, replaying a thinking block on Claude Fable 5.1 after the system prompt, tools, or an earlier message has changed returns a 400 error.
- !Thinking blocks produced by Claude Fable 5.1 and Claude Mythos 5.1 are dropped by the API if replayed to an earlier model.
- !In Python SDK 1.2.0, TypeScript SDK 0.122.0, Go SDK 1.68.0, Java SDK 2.59.0, Ruby SDK 1.67.0, and C# SDK 12.44.0,
client.beta.filesandclient.beta.skillsno longer send thefiles-api-2025-04-14andskills-2025-10-02beta headers and return the same response shapes asclient.filesandclient.skills. - !In the SDK versions above, client.beta.skills.delete() now deletes a Skill together with all of its versions, and the beta Messages type
BetaSkill(the container Skill reference) is renamedBetaContainerSkill. - !The Claude Enterprise Admin API endpoints, Claude Enterprise Analytics API, and Compliance API now require the
anthropic-versionheader on every request.
Braintrust is an AI evaluation platform that helps developers test, benchmark, and monitor AI applications and models in production.
Patterns runs scheduled Loop investigations that surface recurring failure modes and cost trends no scorer covers, saving each finding with supporting traces and a suggested fix, while the Debugger reports likely failure modes for a single trace citing the exact spans, tool calls, and model outputs behind each one.
Braintrust's biggest change this window is a shift toward AI-driven trace investigation — Patterns runs Loop on a schedule to surface recurring problems, Debugger diagnoses single traces, and Loop itself moves to a persistent, server-side runtime that can edit prompts, scorers, and dashboards on its own.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Alongside that, the SDKs picked up OpenAI Batch, ElevenLabs, and Agno instrumentation, the Go SDK gained prompt support, Java evals now run concurrently (with a breaking thread-safety requirement), and a new glm-5.3-flash model, blind human reviews, and dashboard sections shipped too.
what's the number?
Each feature carries 0–100 for how completely the vendor documented it — not how big or important the work is. A major capability described in eight words scores low, and that is the finding.
- depth0–40 what it does and how it works or what changed
- specificity0–30 names real surfaces — APIs, flags, formats, limits, numbers
- actionability0–30 enough to go use it — a named endpoint, flag, or config key tops this; a UI path is a starting point
Code and config rank first by construction: a documented endpoint, flag or config key scores at the top of specificity and actionability, so it sorts above a feature described only in prose. Under 60 the notes go thin — everything below the “thinner coverage” line is thinner documentation, not smaller work. Hover any meter for that feature's three sub-scores.
Adds concurrent eval case execution to the Java SDK (v0.3.23): Eval.start() returns an EvalResult immediately while Eval.run() still awaits completion, with lifecycle APIs getStatus(), isDone(), awaitCompletion(), getStartedAt(), getDuration(), getCasesExecuted(), and getAbortCause().︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Concurrency is controlled via .maxConcurrency(int) on Eval.Builder or Devserver.Builder, or a custom Executor via .executor(Executor) (default: 3 concurrent cases), with global defaults settable via BRAINTRUST_DEFAULT_MAX_CONCURRENCY, BRAINTRUST_OTEL_MAX_QUEUE_SIZE, BRAINTRUST_OTEL_MAX_EXPORT_BATCH_SIZE, and BRAINTRUST_OTEL_EXPORT_INTERVAL_MILLIS. As a result, Task, Scorer, and Classifier implementations must now be thread-safe.
$ export BRAINTRUST_DEFAULT_MAX_CONCURRENCY=10
export BRAINTRUST_OTEL_MAX_QUEUE_SIZE=4096
export BRAINTRUST_OTEL_MAX_EXPORT_BATCH_SIZE=512
export BRAINTRUST_OTEL_EXPORT_INTERVAL_MILLIS=2000
Loop now runs in a Braintrust-managed runtime (server-side, not browser), persisting threads and operating across logs, experiments, and datasets project-wide; it can create and edit prompts, scorers, datasets, facets, custom views, dashboards, and automations, with each write paused for approval unless auto-accept is enabled. It runs on built-in GPT-5.6 Sol, GPT-5.6 Terra, and GPT-5.6 Luna models (no AI provider config required), with GPT-6 Astra later added to the model picker including scheduled automations. An org_name parameter was added to Loop and the MCP server to disambiguate name lookups across organizations, resolving members, API keys, service tokens, patterns, prompts, and scorers by name instead of raw IDs.
Adds OpenAI Batch API instrumentation in the TypeScript SDK (v3.30.0) via openaiFilesCreateTraced, openaiBatchesRetrieveTraced, and completeOpenAIBatchTrace, tracing asynchronous OpenAI Batch requests end-to-end from file upload through submission and retrieval.
import { openaiFilesCreateTraced, openaiBatchesRetrieveTraced, completeOpenAIBatchTrace } from 'braintrust';
const file = await openaiFilesCreateTraced(openai, { file: fs.createReadStream('requests.jsonl'), purpose: 'batch' });
const batch = await openai.batches.create({ input_file_id: file.id, endpoint: '/v1/chat/completions', completion_window: '24h' });
const result = await openaiBatchesRetrieveTraced(openai, batch.id);
await completeOpenAIBatchTrace(result);
import { openaiFilesCreateTraced, openaiBatchesRetrieveTraced, completeOpenAIBatchTrace } from 'braintrust';
const file = await openaiFilesCreateTraced(openai, {
file: fs.createReadStream('requests.jsonl'),
purpose: 'batch',
});
const batch = await openaiBatchesRetrieveTraced(openai, file.id);
await completeOpenAIBatchTrace(batch);
import { openaiFilesCreateTraced, openaiBatchesRetrieveTraced, completeOpenAIBatchTrace } from 'braintrust';
const file = await openaiFilesCreateTraced(openai, {
file: fs.createReadStream('requests.jsonl'),
purpose: 'batch'
});
const batch = await openai.batches.create({
input_file_id: file.id,
endpoint: '/v1/chat/completions',
completion_window: '24h'
});
await openaiBatchesRetrieveTraced(openai, batch.id);
await completeOpenAIBatchTrace(openai, batch.id);
import { openaiFilesCreateTraced, openaiBatchesRetrieveTraced, completeOpenAIBatchTrace } from 'braintrust';
const file = await openaiFilesCreateTraced(openai, { file: fs.createReadStream('requests.jsonl'), purpose: 'batch' });
const batch = await openai.batches.create({ input_file_id: file.id, endpoint: '/v1/chat/completions', completion_window: '24h' });
const result = await openaiBatchesRetrieveTraced(openai, batch.id);
await completeOpenAIBatchTrace(result);
import { openaiFilesCreateTraced, openaiBatchesRetrieveTraced, completeOpenAIBatchTrace } from 'braintrust';
const file = await openaiFilesCreateTraced(openai, { file: fs.createReadStream('requests.jsonl'), purpose: 'batch' });
const batch = await openai.batches.create({ input_file_id: file.id, endpoint: '/v1/chat/completions', completion_window: '24h' });
const result = await openaiBatchesRetrieveTraced(openai, batch.id);
await completeOpenAIBatchTrace(result);
Adds Agno eval instrumentation to the Python SDK (v0.36.0) — AccuracyEval, AgentAsJudgeEval, ReliabilityEval, PerformanceEval, and eval suites are traced automatically via auto_instrument(); eval suite runs open a Braintrust experiment automatically.
import braintrust from agno.eval.accuracy import AccuracyEval braintrust.auto_instrument() eval_run = AccuracyEval(agent=my_agent, dataset=my_dataset) eval_run.run()
from braintrust import auto_instrument
from agno.evals import AccuracyEval, AgentAsJudgeEval, ReliabilityEval, PerformanceEval
auto_instrument()
eval_suite = [
AccuracyEval(agent=my_agent, questions=questions),
AgentAsJudgeEval(agent=my_agent, rubric=rubric),
ReliabilityEval(agent=my_agent, scenarios=scenarios),
PerformanceEval(agent=my_agent, tasks=tasks),
]
for e in eval_suite:
e.run()
from braintrust.integrations.agno import auto_instrument from agno.evals import AccuracyEval, ReliabilityEval auto_instrument() AccuracyEval(agent=my_agent, inputs=test_inputs).run() ReliabilityEval(agent=my_agent, inputs=test_inputs).run()
Adds has_error, metadata, and duration fields to SpanFilters for trace.get_spans() in the Python SDK (v0.39.0), letting trace-level scorers filter spans by error state, metadata values, and execution time bounds.
has_error and duration SpanFilters fields.spans = trace.get_spans(SpanFilters(has_error=True, duration={"gt": 5.0}))
Adds report_scores and report_metrics inputs to the GitHub eval action (v2.1.0) to filter which scores and metrics appear in the PR comment, each accepting a comma- or newline-separated list of names.
- uses: braintrustdata/[email protected] with: api_key: ${{ secrets.BRAINTRUST_API_KEY }} report_scores: accuracy, pass_rate report_metrics: latency, cost
- uses: braintrustdata/eval-action@v2
with:
report_scores: accuracy,faithfulness
report_metrics: latency,cost
- uses: braintrustdata/eval-action@v2
with:
api_key: ${{ secrets.BRAINTRUST_API_KEY }}
report_scores: accuracy, relevance
report_metrics: latency, cost
- uses: braintrust-dev/[email protected] with: api_key: ${{ secrets.BRAINTRUST_API_KEY }} report_scores: accuracy, faithfulness report_metrics: latency, cost
- uses: braintrustdata/[email protected] with: experiment_name: my-experiment report_scores: accuracy, coherence report_metrics: latency, cost
Adds ElevenLabs instrumentation (wrapElevenLabs, TS SDK v3.32.0) for tracing text-to-speech and speech-to-text calls including streaming and timestamped audio, and @google/generative-ai instrumentation (wrapGoogleGenerativeAI, TS SDK v3.32.0) for the legacy Google Generative AI SDK (v0.24.x).
import { wrapElevenLabs } from 'braintrust';
import ElevenLabs from 'elevenlabs';
const client = wrapElevenLabs(new ElevenLabs({ apiKey: process.env.ELEVENLABS_API_KEY }));
Patterns runs Loop against a project on a schedule to surface recurring problems, cost trends, and cohort-level failure modes not covered by any scorer; each finding is saved with supporting traces and a suggested fix, and can be used to create a scorer or classifier, continue investigation in Loop, or be copied as a prompt. Loop automations power Pattern discovery by default, running daily at 9:00 AM with timezone selection for custom cron schedules.
Adds glm-5.3-flash, a multimodal reasoning model served by Braintrust with no AI provider setup required, selectable under the Braintrust provider in playgrounds, prompts, and scorers, or requestable via glm-5.3-flash through the Braintrust Gateway.
$ curl https://api.braintrust.dev/v1/proxy/chat/completions \ -H 'Authorization: Bearer <BRAINTRUST_API_KEY>' \ -H 'Content-Type: application/json' \ -d '{"model": "glm-5.3-flash", "messages": [{"role": "user", "content": "Summarize this trace for me."}]}'
$ curl https://api.braintrust.dev/v1/gateway/chat/completions \ -H 'Authorization: Bearer <BRAINTRUST_API_KEY>' \ -H 'Content-Type: application/json' \ -d '{"model": "glm-5.3-flash", "messages": [{"role": "user", "content": "Score this response for accuracy: ..."}]}'
Loop automations are scheduled Loop runs with their own instruction, model, and write permissions that produce a read-only thread you can Continue, with results deliverable to a Slack channel or webhook; automations support timezone selection for custom cron schedules.
Adds standard verifier output upload to the Harbor plugin in the Python SDK (v0.36.0): test-stdout.txt, test-stderr.txt, and ctrf.json, with attachments and redact_patterns configuration.
Allows scorer functions in the Python SDK (v0.38.0) and TypeScript SDK (v3.31.0) to return {"score": 0.5} without a name field, using the function's name as the score key.
Adds prompt support to the Go SDK (v0.13.0): load a Braintrust prompt, render its variables locally, call any LLM client with the result, drive eval parameters, and link rendered calls back to their prompt in Braintrust traces.
The 'Blind human reviews' project setting hides peer scores, comments, and aggregates until a reviewer submits their own scores; reviewers holding the project Update permission are always exempt.
Adds named, reorderable, collapsible, duplicable dashboard sections with per-browser collapse-state memory (Pro and Enterprise plans), plus a shareable fullscreen link for individual dashboard charts, with chart move and copy actions now in each chart's more-options menu.
Adds faster log search via Brainstore's asynchronous execution engine, streaming results as segments finish for full-text, phrase, and multi-clause ANY_SPAN() (including negated) trace searches — no configuration required.
Adds toolCallId to Vercel AI SDK v6 tool call span metadata (TypeScript SDK v3.31.0), and adds invocation parameters (temperature, max_tokens, and similar model settings) to Pydantic AI LLM span metadata (Python SDK v0.37.0) when model settings are provided.
Debugger reports likely failure modes for a single trace, citing the spans, tool calls, and model outputs behind each one; 'Analyze trace' groups a run into labeled Work sections explaining what the agent did.
Enable the Metric columnstore in project settings alongside log search optimization so Braintrust writes columnar artifacts during compaction, speeding up loading and sorting traces on the Logs page for high-volume projects.
Adds input_audio content type support in Mustache and Nunjucks prompt template rendering in the TypeScript SDK (v3.32.0).
Custom column names that conflict with built-in table fields are now rejected at creation time.
- !Task, Scorer, and Classifier implementations in the Java SDK (
v0.3.23) must now be thread-safe, as eval cases run concurrently. - !Task, Scorer, and Classifier implementations in the Java SDK (v0.3.23) must now be thread-safe due to concurrent eval case execution.
A high-throughput and memory-efficient inference and serving engine for LLMs
vLLM v0.29.0 makes Model Runner V2 the default, adds admission-control queue flags, a sharded P2P weight-sync backend for RL, per-request spec-decode metrics, and five new model families, plus an out-of-tree Tenstorrent plugin that brings LLM serving to a new class of accelerator. The RL weight-sync and admission controls open serving patterns that previously needed custom infrastructure.
vLLM's v0.29.0 release adds request admission-control flags, an RL weight-sync backend, deterministic decode tracing, five new model families, and removes ten legacy model architectures, while a separate Tenstorrent hardware plugin brings vLLM serving to TT-Metal accelerators with its own mesh scheduling and custom model registration.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
what's the number?
Each feature carries 0–100 for how completely the vendor documented it — not how big or important the work is. A major capability described in eight words scores low, and that is the finding.
- depth0–40 what it does and how it works or what changed
- specificity0–30 names real surfaces — APIs, flags, formats, limits, numbers
- actionability0–30 enough to go use it — a named endpoint, flag, or config key tops this; a UI path is a starting point
Code and config rank first by construction: a documented endpoint, flag or config key scores at the top of specificity and actionability, so it sorts above a feature described only in prose. Under 60 the notes go thin — everything below the “thinner coverage” line is thinner documentation, not smaller work. Hover any meter for that feature's three sub-scores.
Introduces the vllm-tt-plugin out-of-tree platform plugin that automatically registers Tenstorrent accelerators as a vLLM platform when ttnn from TT-Metal is importable, supporting TTLlamaForCausalLM, TTMllamaForConditionalGeneration, TTQwen2ForCausalLM, TTQwen3ForCausalLM, TTQwen3_5ForConditionalGeneration, TTQwen2_5_VLForConditionalGeneration, TTQwen3VLForConditionalGeneration, TTMistralForCausalLM, TTMistral3ForConditionalGeneration, TTGemma3ForConditionalGeneration, TTGemma4ForCausalLM, TTGemma4ForConditionalGeneration, TTGemma4UnifiedForConditionalGeneration, TTDeepseekV3ForCausalLM, and TTGptOssForCausalLM.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Tenstorrent mesh targets are selected via the MESH_DEVICE setting (e.g. TG), which replaces --tensor-parallel-size / -tp / -pp (rejected outright on TT targets); --additional-config.tt.sample_on_device_mode (e.g. all) enables on-device sampling so the host receives already-chosen tokens rather than raw logits, and --additional-config.tt.fabric_config configures the mesh fabric topology (FABRIC_1D, FABRIC_2D, FABRIC_1D_RING). Scheduling uses a phase-constrained TTScheduler that issues exclusively prefill-only or decode-only steps (with chunked prefill and interleaved decode), and an alternative TTLaneCoordinator scheduler class (scheduler_config.scheduler_cls) runs one independent TTScheduler per data-parallel lane for Galaxy-class single-execute models, with TT_LLAMA_TEXT_VER (e.g. llama3_70b_galaxy) and TT_QWEN3_TEXT_VER (e.g. qwen3_32b_galaxy) selecting Galaxy-optimised single-execute model variants.
$ vllm serve meta-llama/Llama-3.3-70B-Instruct --additional-config.tt.sample_on_device_mode all --additional-config.tt.fabric_config FABRIC_1D_RING
$ TT_QWEN3_TEXT_VER=qwen3_32b_galaxy vllm serve Qwen/Qwen3-32B --additional-config.tt.fabric_config FABRIC_2D
Adds EXTRA_MODELS_DIR environment variable to register additional model architectures at startup from bundle folders each containing a vllm_metadata.json and an adapter class, without editing plugin source, and TT_VLLM_BUILTIN_MODELS=0 to restrict the architecture registry to only models supplied via EXTRA_MODELS_DIR, excluding all built-in models.
$ EXTRA_MODELS_DIR=/opt/tt-models TT_VLLM_BUILTIN_MODELS=0 vllm serve /opt/tt-models/my-custom-model
Adds --max-num-queued-reqs and --max-num-queued-tokens CLI flags for admission control on the request queue, capping queued load to prevent runaway queue growth during traffic spikes.
$ vllm serve meta-llama/Llama-3.1-8B-Instruct --max-num-queued-reqs 500 --max-num-queued-tokens 131072
Adds support for new models: Hy4-preview (Tencent 770B/49B-active MoE with Gated DeepSeek Sparse Attention and native MTP), Qwen3.8-Flash-Next (BF16/FP8/NVFP4 with MTP), GraniteSWA and GraniteMoeSWA, NemotronH_Omni_Reasoning_V3 with MTP, and Kimi K3 NVFP4 checkpoints.
Adds sharded_rdt P2P backend for RL weight sync, where each worker pulls only its TP/EP slice over NIXL or Ray Direct Transport.
Adds prefix_cache_retention_interval as a CLI argument (default 0) to control Mamba prefix cache retention, makes prefix-cache NONE_HASH deterministic by default (removing the need to pin PYTHONHASHSEED for distributed KV cache users), and introduces a KVCacheLayout enum to standardize KV cache layout across backends.
Adds --linear-backend flashinfer_cutedsl opt-in flag to enable FlashInfer CuTeDSL BF16 low-latency GEMM, and enables FlashInfer all-reduce by default for tensor-parallel CUDA groups, with VLLM_ALLREDUCE_USE_FLASHINFER=0 to opt out.
Removes the PyAV video decoder backend, deprecates python -m vllm.entrypoints.openai.api_server in favor of vllm serve, and removes the VLLM_TEST_FORCE_FP8_MARLIN environment variable.
Adds LoRA support for DeepSeek V4, Qwen3-Omni multimodal, and tower/connector LoRA for LLaVA-NeXT and LFM2-VL.
Model Runner V2 is now the default for all models, completing the rollout begun with pooling models; it also adds extract_hidden_states speculation support and prompt embeds support.
Adds video embeds accepted by the Python frontend, modality-scoped mm_processor_kwargs honored for multimodal inputs, and ViT full CUDA graph support for Idefics3 and SmolVLM.
Adds --per-request-spec-decode-metrics CLI flag to expose per-request speculative decoding acceptance stats in OpenAI API responses.
Adds /v1/messages/render endpoint for the Anthropic Messages API.
Extends --cpu-offload-params support to vision and audio towers.
Adds trace_decode_token_ids for deterministic decode replay.
Adds FP8 quantization support for ModernBERT.
Removes ten deprecated model architectures, and migrates FlexOlmo, Olmo3, and Hunyuan V1/VL to the Transformers modeling backend, removing their native implementations.
Adds bidirectional attention for DeepSeek-backbone embedding models.
Adds a JIT warmup provider registry for attention backends.
Adds rank-local IPC weight updates for RL workflows.
- !Ten deprecated model architectures are removed.
- !FlexOlmo, Olmo3, and Hunyuan V1/VL are migrated to the Transformers modeling backend, removing their native implementations.
- !The PyAV video decoder backend is removed.
- !
python -m vllm.entrypoints.openai.api_serveris deprecated in favor ofvllm serve. - !The
VLLM_TEST_FORCE_FP8_MARLINenvironment variable is removed.