Backfill issue -164 — The AI Toolchain

EDITIONS tail grep head diff uniq

The daily firehose — everything the toolchain shipped today, already filtered.

// HOW THIS ISSUE IS MADE

We read every release from the 174 tools on our watchlist at the source — GitHub and GitLab release notes, vendor release pages and changelogs, project blogs and feeds, vendor press releases, and the source code behind the tag. Bug-fix-only releases and non-product newsroom noise are dropped; what's left is summarized down to the new capability, how to try it, and any screenshots or videos the release itself published. Every entry links to the sources it was built from.

VIEW

ISSUE VIEW full issue

Do you prefer this view?

$ tct list   # 8 tools matched

AI & LLM Tooling

◆ AI Coding Agents

Anthropic Claude Code

v2.1.71 NOTES

Claude Code is an agentic coding tool that lives in your terminal, understands your codebase, and helps you code faster by executing routine tasks, explaining complex code, and handling git workflows - all through natural language commands.

Claude Code v2.1.71 adds /loop scheduling, rebindable push-to-talk voice, and expands bash auto-approval allowlist

└──▷ GET THIS VERSION

$ git clone --branch v2.1.71 https://github.com/anthropics/claude-code.git
# already have the repo? check out this version:
$ git checkout v2.1.71

└──▷ TRY IT

Continuously poll a deploy status every 5 minutes without manually re-running the prompt — useful for watching long-running CI/CD pipelines.

$ /loop 5m check the deploy

Rebind push-to-talk to a modifier+letter combo that won't interfere with typing, then activate voice hands-free.

json

// In keybindings.json
{ "voice:pushToTalk": "meta+k" }

›New /loop command runs a prompt or slash command on a recurring interval (e.g. /loop 5m check the deploy)
›Adds cron scheduling tools for recurring prompts within a session
›New voice:pushToTalk keybinding makes the voice activation key rebindable in keybindings.json (default: space); supports modifier+letter combos like meta+k
›Expands bash auto-approval allowlist with fmt, comm, cmp, numfmt, expr, test, printf, getconf, seq, tsort, and pr
›Updates /debug to toggle debug logging on mid-session without requiring a restart

+2 moreshow less

›Improves /plugin uninstall to disable project-scoped plugins in .claude/settings.local.json instead of .claude/settings.json, keeping changes out of shared teammate config
›Plugin-provided MCP servers that duplicate a manually-configured server (same command/URL) are now automatically skipped, with suppressions shown in the /plugin menu

Was this useful?

SST OpenCode

v1.2.21 NOTES

The open source coding agent.

OpenCode v1.2.21 adds a project git init API, interactive timeline visualization, and sidebar reveal animation with hover peek overlay.

└──▷ GET THIS VERSION

$ git clone --branch v1.2.21 https://github.com/sst/opencode.git
# already have the repo? check out this version:
$ git checkout v1.2.21

›Adds project git init API for initializing Git repositories from within OpenCode.
›Adds sidebar reveal animation, hover peek overlay, and weaker dividers to the desktop app.
›Enables auto-accept keybind regardless of permission config.
›New session now uses the agent model/variant by default; model selection sticks to the session after being changed.
›Speeds up share loads.

+1 moreshow less

›Logs stack trace when schema validation fails, aiding debugging of configuration issues.

Was this useful?

Earendil Works Pi

v0.57.1 2 RELEASES · 2026-03-07 NOTES STABLE

AI agent toolkit: unified LLM API, agent loop, TUI, coding agent CLI

Pi v0.57.1 adds /tree branch folding, a session_directory extension event, and digit keybindings to the TUI.

└──▷ GET THIS VERSION

$ git clone --branch v0.57.1 https://github.com/earendil-works/pi.git
# already have the repo? check out this version:
$ git checkout v0.57.1

›Adds branch folding and segment-jump navigation in /tree via Ctrl+←/Ctrl+→ (segment jump) and Alt+←/Alt+→ (fold/unfold), alongside existing ←/→ and Page Up/Page Down paging.
›New session_directory extension event fires before session manager creation, letting extensions customize the session directory path based on cwd or other factors (CLI --session-dir still takes precedence).
›Digit keys (0-9) now supported in the TUI keybinding system, including modified combos like ctrl+1, with Kitty CSI-u and xterm modifyOtherKeys protocol support.

1 more release in this issue · 2026-03-07

v0.57.0 NOTES STABLE

Pi v0.57.0 adds extension payload interception, non-capturing overlay focus control, and strict JSONL framing in RPC mode.

└──▷ GET THIS VERSION

$ git clone --branch v0.57.0 https://github.com/earendil-works/pi.git
# already have the repo? check out this version:
$ git checkout v0.57.0

›Adds before_provider_request extension hook so extensions can inspect or replace provider request payloads before they are sent.
›Enables non-capturing overlays in extension UIs with explicit focus control via OverlayOptions.nonCapturing and OverlayHandle.focus() / unfocus() / isFocused().
›RPC mode now uses strict LF-only JSONL framing for more robust payload handling.

└──▷ BREAKING ON UPGRADE

!RPC mode now uses strict LF-delimited JSONL framing: clients must split records on \n only and can no longer use generic line readers such as Node readline, which also split on Unicode separators (U+2028, U+2029) inside JSON payloads.

Was this useful?

◆ Local LLM Runtimes

llama.cpp

b8233 3 RELEASES · 2026-03-07 NOTES STABLE

llama.cpp b8233 adds the GATED_DELTA_NET op to ggml with fused backend support, enabling Qwen3.5 dense model inference.

└──▷ GET THIS VERSION

$ git clone --branch b8233 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8233

›Adds GATED_DELTA_NET op to ggml, enabling support for Gated DeltaNet-based architectures including Qwen3.5 dense models.
›Adds fused gated delta net backend support with a corresponding check in the llama model loader.

2 more releases in this issue · 2026-03-07

b8227 NOTES STABLE

llama.cpp b8227 introduces a fully refactored Autoparser architecture with Kimi 2.5 model support.

└──▷ GET THIS VERSION

$ git clone --branch b8227 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8227

›Adds a Kimi 2.5 model parser as part of the new Autoparser architecture.
›Complete refactoring of the parser architecture into a new Autoparser system.

b8225 NOTES STABLE

llama.cpp server now preserves Anthropic thinking blocks during message conversion

└──▷ GET THIS VERSION

$ git clone --branch b8225 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8225

›Server preserves Anthropic thinking blocks when converting messages, maintaining chain-of-thought content across API interactions.

Was this useful?

oobabooga's Text Generation WebUI (textgen)

v4.0 NOTES

oobabooga textgen v4.0 ships parallel API requests, a tool-calling overhaul, N-gram speculative decoding, new CLI generation-parameter flags, and major security hardening.

└──▷ GET THIS VERSION

$ git clone --branch v4.0 https://github.com/oobabooga/textgen.git
# already have the repo? check out this version:
$ git checkout v4.0

└──▷ TRY IT

Enable N-gram speculative decoding in llama.cpp for faster generation without a draft model.

$ python server.py --loader llama.cpp --spec-type ngram-mod --spec-ngram-size-n 8 --spec-ngram-size-m 2 --spec-ngram-min-hits 2

Run concurrent API requests through llama.cpp with 4-way parallelism, multiplying context by 4 to provide enough KV-cache headroom.

$ python server.py --loader llama.cpp --parallel 4 --n-ctx 32768

Set default API generation parameters and a custom jinja2 chat template at server startup so every API caller inherits them.

$ python server.py --temperature 0.7 --min-p 0.05 --top-k 40 --repetition-penalty 1.1 --chat-template-file my_template.jinja

›Enables parallel API requests via --parallel N for llama.cpp (multiply context length by N), ExLlamaV3, and TensorRT-LLM loaders for maximum throughput.
›Adds --spec-type, --spec-ngram-size-n, --spec-ngram-size-m, and --spec-ngram-min-hits parameters to llama.cpp for N-gram speculative decoding — set --spec-type ngram-mod to activate faster generation without a draft model.
›Adds --fit-target parameter to control per-GPU VRAM headroom (default: 1024 MiB) alongside automatic GPU-layer selection (-1 auto) for llama.cpp.
›Adds new CLI flags --temperature, --min-p, --top-k, --repetition-penalty, --enable-thinking, --reasoning-effort, and --chat-template-file to set default API generation parameters; --chat-template-file accepts .jinja or .yaml files.
›Adds --user-data-dir flag to customize the user data directory location; also auto-detects a ../user_data folder in portable mode for easier updates.

+17 moreshow less

›Restricts model loading over API to block extra_flags and trust_remote_code parameters, reducing attack surface.
›Migrates TensorRT-LLM from the old ModelRunner API to the new LLM API, which accepts any Transformers model as input and exposes more sampling parameters.
›Tool-calling now works correctly for Qwen 3.5, Devstral 2, GPT-OSS, DeepSeek V3.2, GLM 5, MiniMax M2.5, Kimi K2/K2.5, and Llama 4 models with strict OpenAI format compliance.
›Adds 'Target all linear layers' checkbox in the Training tab to apply LoRA to every nn.Linear layer except lm_head, working across any model architecture.
›Adds checkpoint resumption to the Training tab: HF Trainer checkpoint directories are detected automatically and training resumes with full optimizer/scheduler state.
›Adds dynamic padding for chat training datasets — batches pad to the longest sequence in the batch rather than always padding to cutoff_len.
›Conversations exceeding the cutoff length are now dropped instead of silently truncated (configurable).
›Introduces an adaptive-p sampler for llama.cpp, Transformers, ExLlamaV3, and ExLlamaV3_HF loaders that reshapes the logit distribution to favor tokens near a target probability.
›Adds a user persona dropdown in the Character settings tab to save and load user profiles (name, bio, profile picture) for switching personas.
›Adds Qwen 3.5 and Solar Open thinking block support to the UI.
›Replaces PyPDF2 with pymupdf for more accurate conversion of PDF inputs to text.
›Adds ROCm portable builds for Windows and CUDA 13.1 portable builds; restores macOS x86_64 (Intel) portable builds.
›llama.cpp binaries now autodetect CPU instruction set (AVX, AVX2, AVX-512) at runtime, replacing separate AVX/AVX2 builds.
›llama-server now spawns on port 5005 by default instead of a random port.
›SSE separator for streaming over the API changed from \r\n to \n to match OpenAI.
›Replaces Gradio with a custom fork (oobabooga/gradio) delivering faster startup, instant SSE message delivery (replacing 50 ms polling), and a new zero-rendering gr.Headless component that reduces chat-streaming overhead.
›Removes the ExLlamaV2 loader (archived upstream); EXL2 users must migrate to EXL3.

└──▷ BREAKING ON UPGRADE

!The ExLlamaV2 loader has been removed; EXL2 model users must migrate to the EXL3 loader.
!The Training_PRO extension has been removed after being made obsolete by the Training tab rewrite.
!The SSE streaming separator over the API changed from \r\n to \n; clients that relied on \r\n framing will break.
!Conversations exceeding cutoff_len during training are now dropped rather than silently truncated, changing training dataset behavior for affected entries.
!Model loading over API no longer accepts extra_flags or trust_remote_code parameters; callers that pass these will be blocked.
!TensorRT-LLM has been migrated from the old ModelRunner API to the new LLM API; existing TensorRT-LLM integrations built against ModelRunner will break.

Was this useful?

vLLM

v0.17.0 NOTES

vLLM v0.17.0 adds FlashAttention 4, --performance-mode, Anthropic count_tokens, quantized LoRA, and PyTorch 2.10.

└──▷ GET THIS VERSION

$ git clone --branch v0.17.0 https://github.com/vllm-project/vllm.git
# already have the repo? check out this version:
$ git checkout v0.17.0

└──▷ TRY IT

Optimize a vLLM server for interactive (low-latency) use cases without manually tuning individual parameters.

$ vllm serve meta-llama/Llama-3.1-8B-Instruct --performance-mode interactivity

Optimize a vLLM server for maximum throughput batch workloads.

$ vllm serve meta-llama/Llama-3.1-8B-Instruct --performance-mode throughput

Select a specific MoE kernel backend explicitly when serving a Mixture-of-Experts model.

$ vllm serve deepseek-ai/DeepSeek-V3 --moe-backend trtllm

›Adds --performance-mode {balanced, interactivity, throughput} flag to simplify performance tuning for common deployment scenarios.
›Adds --moe-backend flag for explicit MoE kernel selection.
›Adds --language-model-only flag for hybrid models.
›Adds Anthropic count_tokens API support.
›Adds Anthropic tool_choice=none support.

+53 moreshow less

›Adds Anthropic thinking blocks support.
›Integrates FlashAttention 4 backend for next-generation attention performance.
›Supports loading quantized LoRA adapters (e.g. QLoRA) directly.
›Adds AOT compile support with PyTorch 2.10.
›Enables AR+RMSNorm fusion by default at -O2 and SiLU+FP4 quant fusion by default at O1+ for torch.compile.
›Adds MFU Prometheus counters for observability.
›Adds unrecognized environment variable warnings.
›Adds per-head KV cache scales in the attention selector.
›Adds sleep level 0 mode with enqueue/wait pattern.
›Adds native weight syncing API via IPC for RL workflows.
›Adds repetitive token pattern detection flags.
›Adds Triton-based top-k and top-p sampler kernels.
›Adds FlashInfer Sparse MLA backend.
›Adds Helion kernel framework with autotuning infrastructure.
›Adds NIXL Token-based IPC API.
›Adds Elastic Expert Parallelism Milestone 2 for dynamic GPU scaling on MoE models.
›Adds MoERunner abstraction with modular kernel architecture.
›Adds min_tokens support with speculative decoding.
›Adds Nemotron-H MTP and Mamba speculative decoding.
›Adds Eagle3 speculative decoding with CUDA graphs on Model Runner V2.
›Adds Pipeline Parallel and Decode Context Parallel support to Model Runner V2.
›Adds weight offloading V2 with prefetching to hide onloading latency, selective CPU weight offloading, and CPU offloading without pinned memory doubling.
›Adds support for new model architectures: Qwen3.5 (with GDN, FP8 quantization, MTP speculative decoding, and reasoning parser), COLQwen3, ColModernVBERT, Ring 2.5, skt/A.X-K1, Ovis 2.6, nvidia/llama-nemotron-embed-vl-1b-v2, nvidia/llama-nemotron-rerank-vl-1b-v2, nvidia/nemotron-colembed.
›Adds ASR model support: FunASR, FireRedASR2, Qwen3-ASR realtime streaming.
›Adds OpenPangu-VL video input, audio chunking for offline LLM, and Parakeet audio encoder.
›Adds Whisper automatic language detection.
›Adds structured outputs support to the Responses API.
›Adds structured output JSON feature validation.
›Adds explicit validation error for tool calls.
›Adds sparse embedding IO process plugin.
›Adds IO Processor plugin simplification.
›Adds LoRA support for LFM2 and Llama 4 Vision tower/connector; increases max LoRA vocab size to 258048.
›Adds SM100 MXFP8 blockscaled grouped MM and quantization kernels.
›Adds AITER fused RoPE+KVCache for AMD ROCm.
›Adds CUDA graph support for Intel XPU.
›Adds ARM BF16 cross-compilation and s390x FP16 support for CPU builds.
›CPU release now supports AVX2, AVX-512, VNNI, AVX512BF16, and AMX via a multi-ISA dispatcher.
›Upgrades to PyTorch 2.10.0.
›Adds Async EPLB rebalance algorithm.
›Adds Ray environment variable propagation to workers.
›Adds mixed precision support for ModelOpt.
›Adds Llama-4 attention quantization (int8, fp8).
›Adds bitsandbytes quantization support on AMD ROCm.
›Adds GPT-OSS Quark format support on AMD ROCm.
›Adds generation_config max_tokens treated as a default rather than a ceiling.
›Adds validation of non-text content in system messages.
›Adds ColBERT extension to non-standard BERT backbones and multimodal scoring for late-interaction models.
›Adds KV cache update ops extraction from FlashInfer forward and MLA backends.
›Adds TRTLLM DSV3 Router GEMM kernel with 6% batch-1 speedup.
›Adds pooling model support to Model Runner V2 with 13.9% throughput improvement.
›Pipeline Parallel async send/recv yields 2.9% E2E throughput improvement.
›Adds Reasoning API streaming reasoning_part events.
›Adds amd-aiter package (renamed from aiter) for AMD ROCm.

└──▷ BREAKING ON UPGRADE

!PyTorch upgraded to 2.10.0, which is a breaking change for environment dependencies.
!KV load failure policy default changed from 'recompute' to 'fail' (controlled in large-scale serving configuration).
!AMD ROCm aiter package renamed to amd-aiter; any environment or install referencing aiter by name must be updated.

Was this useful?

Other / Uncategorized

◆ AI OBSERVABILITY

Arize Phoenix

arize-phoenix-v13.9.0 NOTES

Phoenix 13.9.0 adds a trace_id filter to GET spans REST endpoints and filetype-agnostic dataset uploads.

└──▷ GET THIS VERSION

$ git clone --branch arize-phoenix-v13.9.0 https://github.com/Arize-ai/phoenix.git
# already have the repo? check out this version:
$ git checkout arize-phoenix-v13.9.0

└──▷ TRY IT

Retrieve all spans belonging to a specific trace by filtering directly in the REST API call.

$ curl -X GET 'https://<phoenix-host>/v1/spans?trace_id=<trace_id>' \
  -H 'Authorization: Bearer <api-key>'

›Adds trace_id filter parameter to GET spans REST endpoints, enabling callers to retrieve spans scoped to a specific trace.
›New filetype-agnostic dataset upload in the UI allows uploading dataset files regardless of file extension.
›Adds a record icon to the UI.

Was this useful?

◆ VECTOR DB RAG

Chroma

1.5.3 NOTES

Chroma 1.5.3 adds delete-with-limit, updated Gemini embedding functions, new compactor endpoints, and OpenTelemetry metrics.

└──▷ GET THIS VERSION

$ git clone --branch 1.5.3 https://github.com/chroma-core/chroma.git
# already have the repo? check out this version:
$ git checkout 1.5.3

›Adds fragment_storage config key for a dedicated fragment fetcher storage configuration.
›Adds fetch_log_concurrency semaphore to the worker for controlling concurrent log fetch operations.
›Adds ReadLevel support to the count operation in the backend.
›Adds compaction_failure_count gauge metric in sysdb for tracking compaction failures via OpenTelemetry.
›Adds ListInProgressJobs endpoint to the compactor for querying active compaction jobs.

+11 moreshow less

›Adds a compaction endpoint that returns where a collection would be assigned.
›Adds pointer-based log fetch via ScoutLogFragments for more efficient log retrieval.
›Adds tracing spans to the log fetch path for improved observability.
›Adds OpenTelemetry metrics to the system crate.
›Adds ResourceExhausted error code for log backpressure signaling.
›Supports delete-with-limit in both server and clients, enabling bounded deletes.
›Updates Gemini embedding functions (EFs) with new capabilities.
›Parallelizes segment reader initialization in filter and IDF operators for faster query startup.
›Skips record load when only the document ID is requested, reducing unnecessary I/O.
›Removes PostHog as a dependency and makes telemetry a no-op.
›Drops pydantic v1 compatibility layer, enabling Python 3.14 support.

└──▷ BREAKING ON UPGRADE

!The pydantic v1 compatibility layer has been dropped; setups relying on pydantic v1 behavior will break on upgrade.
!Telemetry is now a no-op and PostHog is removed as a dependency; any configuration or integrations depending on PostHog telemetry will no longer function.

Was this useful?

// END OF TRANSMISSION

The AI Toolchain · curated daily · March 7, 2026
Releases & features from the open-source and commercial AI tools that practitioners actually run.

The AI Toolchain — issue -164, March 7, 2026