The AI Toolchain — issue 008, August 26, 2026

Groq added openai/gpt-oss-20b (131K token context, 32K max output, ~1000+ TPS, 32-expert MoE) and openai/gpt-oss-120b (131K token context, 32K max output, ~500+ TPS, 128-expert MoE), both with built-in browser search, code execution, and structured output support, available via the existing POST https://api.groq.com/openai/v1/chat/completions endpoint using the model field. GPT-OSS 20B scores 60.7% SWE-Bench Verified, 98.7% AIME 2025, and 85.3% MMLU; GPT-OSS 120B scores 90.0% MMLU, 62.4% SWE-Bench Verified, and 57.6% HealthBench Realistic.

$ curl https://api.groq.com/openai/v1/chat/completions \
  -H "Authorization: Bearer $GROQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "openai/gpt-oss-20b", "messages": [{"role": "user", "content": "Explain why fast inference is critical for reasoning models"}]}'

$ curl https://api.groq.com/openai/v1/chat/completions \
  -H "Authorization: Bearer $GROQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "openai/gpt-oss-120b", "messages": [{"role": "user", "content": "Summarize current treatment guidelines for hypertension"}]}'

Groq added qwen/qwen3.8-27b as a supported reasoning model with a reasoning_effort parameter accepting low, medium, or high values to tune reasoning depth.

$ curl https://api.groq.com/openai/v1/chat/completions \
  -H 'Authorization: Bearer $GROQ_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen/qwen3.8-27b",
    "reasoning_effort": "high",
    "messages": [{"role": "user", "content": "<your prompt>"}]
  }'

Groq added qwen/qwen3.8-27b as a supported vision model — a 27B multimodal model accepting up to 3 images per request, with thinking and instruct modes, tunable reasoning effort, tool use, and JSON mode; each image counts as 2048 input tokens.

The Responses API image_url input now supports a detail: "original" option to preserve full-resolution fidelity when analysing diagrams or screenshots, demonstrated via POST https://api.openai.com/v1/responses with image_url.detail set to original.

$ curl https://api.openai.com/v1/responses \
  -H 'Authorization: Bearer $OPENAI_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{"model": "gpt-5.6", "input": [{"type": "image_url", "image_url": {"url": "https://example.com/diagram.png", "detail": "original"}}]}'

Sets atomic first-token deadline admission to default-on for eligible requests, returning a health-neutral deadline_unreachable status before GPU prompt work begins so the coordinator can retry another provider. To fall back to unlimited partial-prefill interleave (bypassing forecast admission) while keeping hard absolute expiry, run DARKBLOOM_CBV2_MAX_PARTIAL_PREFILLS=0 darkbloom restart.

$ DARKBLOOM_CBV2_MAX_PARTIAL_PREFILLS=0 darkbloom restart

Supports Qwen 3.5 video inference via EngineV2, sampling at most 8 frames at no more than 512×512 pixels per frame and bounding the vision score tensor to 512 MiB, replacing the previous 400 invalid media input failure.

Adds EXIF-orientation-aware full-raster image decode for Qwen vision requests, defaulting enable_thinking=false for media unless the client explicitly sets reasoning.enabled.

Updates earnings calculators to require Mac model, chip family, and unified memory inputs before showing an estimate, grounded in bandwidth-limited capacity; adds M5 Ultra and M6 hardware profiles.

Role and runner-access mappings now bind to emisar's immutable directory-group ID rather than group name, so same-name groups stay distinct, renames preserve grants, and deleting and recreating a group never inherits old access. Upgrading to v0.43.0 requires a one-way migration: stop SSO/SCIM and mapping writes, drain every v0.42 Portal node, run the migration once, then start only the v0.43 image; after the migration commits, v0.42 is not a rollback candidate and the migration must not be run down, so rollback requires an image that understands immutable directory-group mappings.

Custom SSO now supports IdP-initiated SAML login as an opt-in, enabled via the --enable-idp-initiated-sso flag, extending the existing SP-initiated login flow.

Ollama can now be connected to Claude Desktop on macOS as a third-party gateway so Claude Desktop can use local Ollama models, with model selection, auto-mode, and install assistance available from the menu bar. Use ollama launch claude to launch the Claude Code integration through Ollama in place of Anthropic's API.

$ ollama launch claude

RunPod Serverless adds a full asynchronous bulk-inference workflow via REST API v2, running on dedicated workers isolated from standard /run traffic so submitting a batch never delays interactive requests. Endpoints include POST /v2/{endpoint_id}/batch (create a batch, optionally with initial requests as a JSON array), POST /v2/{endpoint_id}/batch/{id}/requests (append requests to a DRAFT batch, up to 10 MiB per call, callable repeatedly to build large batches incrementally), POST /v2/{endpoint_id}/batch/{id}/finalize (lock the batch and move it from DRAFT to FINALIZED), GET /v2/{endpoint_id}/batch/{id} (poll progress via requestTotal, requestInProgress, requestCompleted, requestFailed, with createdAt as a Unix epoch millisecond timestamp), GET /v2/{endpoint_id}/batch/{id}/requests (paginated child-request results with per-request status, output, error, startedAt, completedAt, using offset/limit query params and a hasMore field), POST /v2/{endpoint_id}/batch/{id}/cancel (cancel a batch; queued requests are cancelled and not billed, in-progress requests finish and are billed normally), DELETE /v2/{endpoint_id}/batch/{id}/requests/{requestId} (remove a single request from a DRAFT batch), PUT /v2/{endpoint_id}/batch/{id} (update batch attributes such as display name), and GET /v2/{endpoint_id}/batch (list all batches for an endpoint, newest first). Completion is inferred when requestCompleted + requestFailed equals requestTotal; batch lifecycle states are DRAFT → FINALIZED → FAILED/CANCELLED. Limits are 10 active batches per endpoint, 5,000 requests per batch, and 50,000 queued requests per endpoint (up to 1,000,000 daily for enterprise, configurable for enterprise accounts). Terminal states trigger Console Inbox notifications (batch ID, endpoint name, status, item counts) and optional webhook events, and a new Batch tab in the RunPod console shows batch name, status, progress counts, per-request rows with timestamps/errors, and links to child request detail views.

$ curl -X POST https://api.runpod.io/v2/{endpoint_id}/batch \
  -H 'Authorization: Bearer {api_key}' \
  -H 'Content-Type: application/json' \
  -d '[{"input":{"text":"The quick brown fox"}},{"input":{"text":"Jumped over the lazy dog"}}]'

$ # Finalize
curl -X POST https://api.runpod.io/v2/{endpoint_id}/batch/{batch_id}/finalize \
  -H 'Authorization: Bearer {api_key}'

# Poll until requestCompleted + requestFailed == requestTotal
curl https://api.runpod.io/v2/{endpoint_id}/batch/{batch_id} \
  -H 'Authorization: Bearer {api_key}'

$ curl 'https://api.runpod.io/v2/{endpoint_id}/batch/{batch_id}/requests?offset=0&limit=50' \
  -H 'Authorization: Bearer {api_key}'

$ # 1. Create a batch with initial requests
curl -s -X POST 'https://api.runpod.io/v2/{endpoint_id}/batch' \
  -H 'Authorization: Bearer {api_key}' \
  -H 'Content-Type: application/json' \
  -d '[{"input":{"text":"The quick brown fox"}},{"input":{"text":"Jumped over the lazy dog"}}]'

# Response: {"id": "batch_01j9abc123", "status": "DRAFT"}

# 2. Finalize the batch to start processing
curl -s -X POST 'https://api.runpod.io/v2/{endpoint_id}/batch/batch_01j9abc123/finalize' \
  -H 'Authorization: Bearer {api_key}'

# 3. Poll progress until requestCompleted + requestFailed == requestTotal
curl -s 'https://api.runpod.io/v2/{endpoint_id}/batch/batch_01j9abc123' \
  -H 'Authorization: Bearer {api_key}'

$ # Fetch first 50 results
curl -s 'https://api.runpod.io/v2/{endpoint_id}/batch/batch_01j9abc123/requests?offset=0&limit=50' \
  -H 'Authorization: Bearer {api_key}'

# Fetch next page
curl -s 'https://api.runpod.io/v2/{endpoint_id}/batch/batch_01j9abc123/requests?offset=50&limit=50' \
  -H 'Authorization: Bearer {api_key}'

$ # Add a second chunk of requests to a DRAFT batch
curl -s -X POST 'https://api.runpod.io/v2/{endpoint_id}/batch/batch_01j9abc123/requests' \
  -H 'Authorization: Bearer {api_key}' \
  -H 'Content-Type: application/json' \
  -d '{"requests":[{"input":{"text":"More text to embed"}},{"input":{"text":"Another piece of text"}},{"input":{"text":"And another one"}}]}'

The /review skill gains a --topology minimal flag that runs a single-pass A/B review arm instead of the full multi-arm topology, reducing review overhead.

$ qwen -p "/review --topology minimal"

The resolved default model changes for ClinePass (now GLM 5.3), Z.ai, Hugging Face, evroc, LLM Gateway, NanoGPT, and Weights & Biases — users who use any of these providers without pinning a model will get a different default after upgrading.

Adds a dedicated Marketplace page consolidating plugins, MCP servers, and skills in one list with type-filter and tag chips, linked from sidebar sub-tabs and Customize headers.

Schedule cards are now full click targets that open schedule details, with larger edit, run, pause, and delete controls. Schedule details are now a single scrollable view showing the meta grid, configuration, and most recent runs with a 'Show all N runs' expander — replacing the Overview/Runs tabs. 'Run now' now navigates directly into the session it starts.

installMcpServer no longer treats a -- separator in the install arguments as part of the stdio command.

Improves /cd so the new directory's project settings, hooks, .mcp.json servers, skills, and agents all take effect immediately instead of waiting for --resume.

Adds a startup warning when Bash allow rules contain a wildcard before the subcommand (e.g. Bash(git * main)), flagging rules that also match options injected before the subcommand.

Improves subagent results: a subagent that hits its maxTurns limit now returns output marked as partial with a hint to continue via SendMessage, rather than appearing finished.

Adds an Auto mode tab to /permissions for viewing and editing auto mode classifier rules.

Improves non-interactive sessions (-p, SDK, cloud sessions) to automatically continue a response cut off mid-stream by a server error, connection loss, or stall.

Changes /code-review so Claude can initiate it on Bedrock, Vertex AI, Foundry, the Claude apps gateway, and when telemetry or non-essential traffic is disabled.

Changes /goal idle sessions to start at most three check-ins per goal on long-running background work; the next user message allows three more.

The app server --code-mode-host flag now accepts only http:// and https:// gRPC endpoints; WebSocket (ws://, wss://) host URLs are rejected at CLI parse time, so any configuration passing a WebSocket URL will fail at startup.

Adds an experimental turn/start.cyberAccessProgram option with standard, daybreakBlue, and daybreakRed values, forwarded as access_programs.cyber on Responses, WebSocket, and remote-compaction requests when using ChatGPT authentication.

Adds a hidden --feedback flag to codex doctor that bounds each SQLite database integrity scan to a one-second deadline when collecting diagnostic attachments.

Adds an amazonBedrockAccessKeys login flow that persists SigV4-signed credentials in the configured auth store and reports bedrockAccessKeys as a distinct auth mode, plus Amazon Bedrock setup added to the TUI onboarding flow.

Validates issuer-bound authorization responses, persists registered callback URLs for MCP servers and plugins, and inserts the active listener port into portless loopback redirects.

Adds a full Agent Skills subsystem: skills are installable, on-demand expertise bundles discovered from four precedence-ordered tiers — built-in, extension, user (~/.gemini/skills/ or ~/.agents/skills/), and workspace (.gemini/skills/ or .agents/skills/), the last two using an interoperable .agents/skills/ path alias so directories stay compatible across other AI tools. CLI commands: gemini skills install <url|path> --consent (with --scope user|workspace) installs a skill from a Git repo or local directory; gemini skills list --all lists all discovered skills across every tier including hidden built-ins; gemini skills uninstall <name> --scope workspace removes one. In-session commands: /skills list [all] [nodesc] shows discovered skills (with flags to surface built-ins or suppress descriptions), /skills link <path> [--scope user|workspace] links skills from an arbitrary local directory, /skills disable <name> and /skills enable <name> toggle skills without uninstalling, and /skills reload (alias /skills refresh) rescans all discovery tiers mid-session. A new activate_skill tool is called automatically by the model when a task matches a skill's description, injecting the skill's SKILL.md and granting the agent read access to its bundled assets.

$ gemini skills install https://github.com/user/security-audit-skill.git --scope workspace --consent

$ gemini skills list --all

Installing graphifyy[postgres] now includes the tree-sitter-sql grammar required by the SQL introspection path, and a missing or ABI-incompatible grammar raises an actionable error instead of silently returning zero nodes.

$ pip install graphifyy[postgres]

C# and TypeScript enum members now each emit a graph node with a case_of edge to their parent enum, covering explicit and implicit values, const enum, and quoted TypeScript member names — matching existing Java/Kotlin/Swift enum handling.

Multi-repo community detection now offsets each input's community ids so community 0 of one repo no longer fuses with community 0 of another; the original id is preserved in the local_community field.

bitsandbytes support has been migrated to an out-of-tree plugin, so existing setups using bitsandbytes quantization will break without installing the plugin separately; the Transformers dependency has been bumped to 5.15.0, breaking incompatible models or configurations; the deprecated calculate_kv_scales runtime KV scale calculation has been removed; and override_attention_dtype has been removed, so any configuration or code using these options will fail.

Adds module_path config key for out-of-tree secondary KV cache tier managers, enabling custom offloading backends without forking vLLM; adds disk offloading support for SimpleCPUOffloadConnector, extending tiered KV cache to persistent storage; adds tiering offloading metrics for observability into KV cache tier behavior; adds a canonical CPU layout for parallelism-agnostic KV cache offload; adds partial secondary-tier KV load results support allowing partial cache hits from offload storage; and adds data-parallel topology exposure to offloading backends.

Adds count_reasoning_tokens field to the streaming parser engine for tracking reasoning token counts, content_parts field on /inference/v1/generate for richer multimodal generation requests, request priority parsed from an HTTP header enabling priority-based scheduling, session ID plumbing into requests for stateful session tracking, and makes model optional on all /derender request classes.

Adds FlashInfer XQA decode support on SM12x (Blackwell), a CuTeDSL fused query kernel on SM100, an MLA backend enabling DeepSeek-V2/V3 inference on CPU, GPTQ and AWQ quantization support on s390x, AITER and FP8 inference on GFX120x (AMD ROCm), a triton-cpu wheel for CPU-based Triton kernels, an XPU wheel in the release pipeline, online MXFP4 quantization support, batch-invariant NVFP4 MoE via CUTLASS, a block-wise scaled_mm kernel for quantization, and AMD Quark NVFP4 support with an emulation kernel for DeepSeek-V4.

Model Runner V2 gains E/P/D (encode/prefill/decode) disaggregation support, encoder CUDA graphs, decoder token-wise pooling and Transformers pooling models, attention-free model support, multi-layer MTP KV cache support, weight offloading, and thinking_token_budget support for controlling reasoning token budgets.

Adds sparse MLA end-to-end support for plain decode, MTP, and DSpark speculative decoding in DeepSeek V4; adds DFlash2 speculative decoding with local convolution and a candidate selector; adds DSpark confidence-scheduled verification for speculative decoding; adds async scheduling auto-enabled for draft models; and adds EAGLE3 speculative decoding support on KimiLinear.

Raises default max_num_batched_tokens from 8192 to 16384, raises the Blackwell CUDA graph capture default to 1024, and enables prefix caching by default for Mamba models.

Adds vision tower LoRA support for Gemma4, Keye, and Ultravox models; adds native multimodal support for Dots3 NOTE; adds new models Muse Glimmer, Ling 3.0 Flash (BF16/MTP/FP8/MXFP4), and Interns2mobius; and adds a Transformers modeling backend with MLA support, hardware-agnostic model definition, logit softcapping, and generalized input embedding handling.

Adds CuMemAllocator.discard() for tag-selective GPU memory release in RL workflows, Mooncake store group semantics and tenant ID support with official wheels shipped in the Docker image, and stateful trainer send over NCCL with sparse NCCL support for RL workflows.

Adds a Rust frontend standalone renderer with gRPC multimodal image inference support, explicit data-parallel rank routing and RL lifecycle control, publishes protobuf schemas for the gRPC frontend to Buf, and adds dynamic tools from developer messages in the Rust frontend.

Adds Decode Context Parallel (DCP) support for Kimi-K3, and optional shared-expert sharding for Kimi-K3 saving ~17 GiB of memory per GPU.

Adds NUL byte rejection in structured_outputs.regex and negative token ID rejection for structured output robustness, and adds VLLMValidationError raised from structured output validators.

Giskard v3 is a full rewrite structured around three installable packages — giskard-checks, giskard-agents, and giskard-core — with giskard-scan available via pip install 'giskard[scan]'. The rewrite drops v2 APIs (giskard.Model, giskard.Dataset, giskard.testing, Giskard Hub); those needing v2 must pin pip install 'giskard[llm]>2,<3'. It now requires Python 3.12+, removes Scenario.from_sequence and the scenario() factory in favor of the step-based mutable Scenario constructor, drops templating support in conformity rules, makes Jinja parsing in Workflow.chat opt-in instead of default, and switches the Interact injection to name-based instead of positional.

Adds a vulnerability_scan function in giskard-scan for automated red-teaming across OWASP LLM Top-10 threat categories including prompt injection, harmful content, stereotypes, and misinformation, plus a minimal OWASP LLM suite generator covering LLM01 indirect injection.

import asyncio
from giskard.scan import vulnerability_scan

async def my_agent(inputs: str) -> str:
    return f"Echo: {inputs}"  # replace with your real agent

async def main() -> None:
    await vulnerability_scan(
        target=my_agent,
        description="A customer support chatbot for an e-commerce platform.",
        languages=["en"],
    )

asyncio.run(main())

Introduces Suite for running batches of scenarios with dynamic binding, with a chainable Suite.append(), and allows scenarios to run multiple times within a suite. Adds print_report() to result models for human-readable check output, adds scenario and check error details to suite reports, adds JUnit XML export for SuiteResult for CI integration, and adds annotations support to Scenario/trace objects for attaching metadata to eval steps.

import asyncio
from giskard.checks import Scenario, Groundedness

def get_answer(inputs: str) -> str:
    return my_agent(inputs)  # replace with your model/agent

async def main() -> None:
    scenario = (
        Scenario("test_capital")
        .interact(inputs="What is the capital of France?", outputs=get_answer)
        .check(
            Groundedness(
                name="answer is grounded",
                context="France is in Western Europe. Its capital is Paris.",
            )
        )
    )
    result = await scenario.run()
    result.print_report()

asyncio.run(main())

Adds opt-out telemetry to giskard-core, disabled by setting DO_NOT_TRACK=1 or GISKARD_TELEMETRY_DISABLED=1, collected through optional PostHog analytics.

Adds RegexMatching for asserting outputs match a regular expression with ReDoS mitigation via regex timeout, AllOf, AnyOf, and Not composition operators for combining checks with boolean logic, AnswerRelevance and Toxicity LLM judge checks, a JSON validity check, and a new judges directory for LLM- and prompt-reliant checks. Checks now also accept pydantic-compatible input types.

Introduces giskard-llm as a lean, provider-agnostic LLM routing layer replacing litellm, supporting provider extras such as openai and anthropic installable via pip install 'giskard[openai]'.

set_default_generator now accepts plain model name strings (e.g. 'openai/gpt-4o') in addition to generator objects, reducing boilerplate when configuring the default LLM judge or generator.

from giskard.checks import set_default_generator

set_default_generator('openai/gpt-4o-mini')

Scopes the __robusta_user_approved field to relay-routed remote MCP calls instead of all MCP traffic, wires Kubernetes-remediation MCP bearer-token auth in the Helm chart, and updates the GitLab MCP integration to v2.1.46 with a whoami health check.

Hardens the kubectl-run toolset to execute commands without a host shell, eliminating shell injection paths.

Removes the experimental Azure SQL toolset entirely, and removes the MariaDB MCP integration, consolidating MariaDB support into the native MariaDB datasource; deployments using either must migrate accordingly.

Tool errors that produce oversized results are now capped: a 500-character gist is inlined in the response and the full content is spilled to a file.

Adds openrouter:stealth/ox-alpha to the model matrix, exposing the Ox Alpha stealth/cloaked lab preview via OpenRouter with a 1,048,576-token context window and tool-calling capability. Selectable in the model selector as 'Ox Alpha · via OpenRouter' from the OpenRouter section.

A new public POST /v2/datasets/{dataset_id}/experiment-runs endpoint provides paginated experiment comparison; legacy dataset comparison helpers have been removed from the public OpenAPI spec and generated SDKs, though existing HTTP routes continue to work for LangSmith UI clients only.

A new Chat interface on the Engine page lets practitioners ask natural-language questions about a detected issue and its linked traces, such as 'What went wrong?', 'Why was this flagged?', 'Explain the proposed fix', 'What are my most pressing issues?', and 'How many new issues do I have?'.

Self-hosted LangSmith bulk exports now default to zstd compression instead of gzip; the previous gzip behavior can be retained by setting FF_BULK_EXPORT_DEFAULT_COMPRESSION=gzip.

$ FF_BULK_EXPORT_DEFAULT_COMPRESSION=gzip docker compose up

Adds GET /v1/model_providers exposing built-in model provider entries with a provider field. This is a breaking change: the endpoint no longer returns custom providers or a next_cursor field, and built-in entries now expose provider instead of kind and provider_key; custom providers are now served separately by GET /v1/custom_model_providers.

Adds PATCH /projects/{project_identifier}/retention REST endpoint to configure per-project trace retention policies.

Go SDK v0.11.1 (Eino): ChatModel span output is now an OpenAI-compatible choices array ([{"index": 0, "finish_reason": "...", "message": {...}}]) instead of a flat message map; embedding input is now {"inputs": [{"content": "..."}]} and output is {"count": N}, removing embedding_length and renaming embeddings_count; provider metadata is now lowercase (e.g. "openai" instead of "OpenAI"). Update trace queries that rely on the previous formats.

Topics automatically classifies production logs by user intent, sentiment, and issues using built-in Task, Sentiment, and Issues facets, surfacing patterns as topic distributions and time-series trends; custom facets let teams define domain-specific classification dimensions using custom preprocessors and prompts beyond the built-in facets. Topics management adds on-demand pipeline runs, sampling adjustment, pause/resume, and history rewind after facet or filter changes, and hovering a row in the List view of a Topics facet card reveals a 'View logs' link that filters logs to that topic and all nested topics. Custom preprocessors whose output is not a recognized message array now render in thread layout as labeled text instead of failing.

Go SDK v0.11.1 adds WithProvider and WithModel options on NewMiddleware for explicit model attribution with Firebase Genkit, plus traced tool wrappers DefineTool, DefineToolWithInputSchema, and DefineMultipartTool; auto-instrumentation now replaces genkit.DefineTool, genkit.DefineToolWithInputSchema, and genkit.DefineMultipartTool calls with their traced equivalents.

Adds DefaultBraintrustApiClient.Api, a generated REST API client covering every Braintrust REST endpoint, shipped inside the existing Braintrust.Sdk package with no separate install required.

Go SDK v0.11.0 expands span instrumentation across providers: Anthropic spans capture prompt_cache_creation_5m_tokens and prompt_cache_creation_1h_tokens for TTL-specific prompt caching; Bedrock Runtime spans capture audio and video content blocks and InvokeModelWithResponseStream is now fully instrumented for Anthropic Claude models; Google ADK spans now include reasoning and cached token metrics.

Web search, file search, code interpreter, and MCP calls in the OpenAI Responses API now appear as child tool spans nested under the LLM span for cleaner trace visualization; server_tool_use blocks in the Anthropic Messages API are similarly nested as child tool spans, with Anthropic spans now capturing new server_tool_use_* metrics.

Adds kimi-k3 and deepseek-v4-flash-0731 as built-in open-source models served through the Braintrust Gateway alongside GLM-5.2, with no external AI provider setup required; usage draws from monthly model credits shared with Topics.

Python SDK v0.32.0 disables LiveKit Agents audio attachments on agent_speaking spans by default; set BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true to restore the previous behavior.

The Google GenAI provider metadata field name changed from "gemini" to "google" (Go SDK v0.11.1); trace queries that filter on the previous value must be updated to use the new field name.

The new Loop agent enables natural-language interaction to analyze logs, optimize prompts, build datasets, and generate custom React view components; custom views let teams describe a trace/dataset interface in natural language and have Loop generate an embeddable, customizable React component.

Braintrust now supports evaluating reasoning models (OpenAI o4, Claude, Gemini) with configurable thinking budgets, capture of intermediate reasoning traces, and scoring of final outputs.