Backfill issue -095 — The AI Toolchain

EDITIONS tail grep head diff uniq

The daily firehose — everything the toolchain shipped today, already filtered.

// HOW THIS ISSUE IS MADE

We read every release from the 174 tools on our watchlist at the source — GitHub and GitLab release notes, vendor release pages and changelogs, project blogs and feeds, vendor press releases, and the source code behind the tag. Bug-fix-only releases and non-product newsroom noise are dropped; what's left is summarized down to the new capability, how to try it, and any screenshots or videos the release itself published. Every entry links to the sources it was built from.

VIEW

ISSUE VIEW full issue

Do you prefer this view?

$ tct list   # 9 tools matched

AI & LLM Tooling

◆ AI Coding Agents

Cline

cli-v3.0.5 NOTES

Autonomous coding agent as an SDK, IDE extension, or CLI assistant.

Cline CLI v3.0.5 surfaces plugin-provided tools and slash commands directly in the settings dialog.

└──▷ GET THIS VERSION

$ git clone --branch cli-v3.0.5 https://github.com/cline/cline.git
# already have the repo? check out this version:
$ git checkout cli-v3.0.5

›Displays plugin-provided tools and slash commands in the CLI settings dialog, hydrated through the sandbox.
›Preserves hydrated plugin tools and config reload options across settings toggles so they remain visible after changes.

Was this useful?

SST OpenCode

v1.15.1 NOTES

The open source coding agent.

OpenCode v1.15.1 adds inline collapsed thinking view and pinned sessions with quick-switch slots in the TUI.

└──▷ GET THIS VERSION

$ git clone --branch v1.15.1 https://github.com/sst/opencode.git
# already have the repo? check out this version:
$ git checkout v1.15.1

›Adds a collapsed thinking view in the TUI that can be expanded inline to inspect model reasoning without cluttering the chat.
›Adds pinned sessions with quick-switch slots in the session picker for fast context switching between active investigations.

Was this useful?

Earendil Works Pi

v0.74.1 NOTES

AI agent toolkit: unified LLM API, agent loop, TUI, coding agent CLI

Pi v0.74.1 adds image generation, Together AI provider, and Windows ARM64 binaries

└──▷ GET THIS VERSION

$ git clone --branch v0.74.1 https://github.com/earendil-works/pi.git
# already have the repo? check out this version:
$ git checkout v0.74.1

└──▷ TRY IT

Authenticate with Together AI to start using its models without manual config.

$ pi /login

›Adds image generation support via OpenRouter, including image generation APIs and image model metadata.
›Adds Together AI as a built-in provider with /login API-key authentication and default model resolution.
›Adds standalone release binaries for Windows ARM64.
›Improves terminal markdown rendering with list indentation and task-list checkbox display.

Was this useful?

◆ AI Agent Frameworks

Nous Research Hermes

v2026.5.16 NOTES

The agent that grows with you

Hermes v2026.5.16 adds SuperGrok OAuth, an OpenAI-compatible local proxy, X search, Teams integration, LINE/SimpleX platforms, and dozens of new capabilities.

└──▷ GET THIS VERSION

$ git clone --branch v2026.5.16 https://github.com/NousResearch/hermes-agent.git
# already have the repo? check out this version:
$ git checkout v2026.5.16

└──▷ TRY IT

Route Aider (or any OpenAI-compatible tool) through your Claude Pro or SuperGrok subscription — no API key needed.

$ hermes proxy &
aider --openai-api-base http://localhost:<port> --openai-api-key dummy

Install Hermes from PyPI and launch immediately — no repo clone required.

$ pip install hermes-agent && hermes

›Adds xAI Grok as a SuperGrok OAuth provider with grok-4.3 bumped to a 1M-token context window — no API key required.
›New hermes proxy command exposes an OpenAI-compatible local endpoint backed by Claude Pro, ChatGPT Pro, or SuperGrok OAuth, letting tools like Codex, Aider, Cline, and Continue consume any subscription without an API key.
›Adds x_search as a first-class built-in tool for searching X (Twitter) via OAuth or API key — no skill install or custom integration needed.
›Adds full Microsoft Teams integration: Microsoft Graph auth, webhook listener, pipeline plugin runtime, and outbound delivery.
›Hermes is now installable via pip install hermes-agent from PyPI — no repo clone or shell installer required.

+18 moreshow less

›Adds cross-session 1-hour Claude prompt caching (system prompt, skills, memory) for Anthropic, OpenRouter, and Nous Portal backends.
›Browser CDP calls are 180× faster via a persistent Chrome DevTools connection shared across browser_console evaluations.
›Cold-start launch time reduced by ~19 seconds through deferred adapter loading, disk-cached model catalogs, parallel doctor checks, and a chat -q banner-skip flag.
›Adds LINE and SimpleX Chat as first-class messaging platforms, bringing the total to 22.
›/handoff now transfers the full active session — messages, tool calls, and context — live to a new model, persona, or profile.
›Native platform button UI for clarify multiple-choice prompts on Telegram and Discord.
›Discord channel history is now backfilled by default when the agent joins a channel or thread for the first time.
›vision_analyze now passes raw pixels directly to vision-capable models (GPT-5, Claude, Gemini, Grok-vision) instead of converting to a text description.
›Adds a per-turn file-mutation verifier footer summarizing paths, line counts, and deltas after every turn that writes or edits files.
›Adds unified video_generate tool with pluggable provider backends, supporting new video models via a one-file plugin.
›computer_use cua-driver backend now works with non-Anthropic models.
›Heavy backends (Slack, Matrix, Feishu, DingTalk, image-gen SDKs, voice/TTS providers) now lazy-install on first use, with a supply-chain advisory checker scanning every install.
›Adds Zed ACP Registry integration via uvx.
›Adds native Windows beta support.
›Adds 9 new optional skills.
›Adds OpenRouter Pareto Code router.
›Adds huggingface/skills as a trusted default skill tap.
›Clickable URLs now render in any terminal.

└──▷ BREAKING ON UPGRADE

!The [all] extras no longer include dependencies now covered by lazy-installation — packages previously pulled in by pip install hermes-agent[all] may no longer be present at install time and will instead be fetched on first use.

Was this useful?

◆ Local LLM Runtimes

KoboldCpp

v1.113.2 NOTES

KoboldCpp v1.113.2 adds --splitmode, --reqtimeout, image LoRA directory selection, and customizable multiuser queue limits.

└──▷ GET THIS VERSION

$ git clone --branch v1.113.2 https://github.com/LostRuins/koboldcpp.git
# already have the repo? check out this version:
$ git checkout v1.113.2

└──▷ TRY IT

Run KoboldCpp on multiple GPUs using the new independent split-mode flag with experimental tensor splitting.

$ koboldcpp --usecuda --splitmode tensor --model my-model.gguf

Set a custom request timeout for router mode to avoid hanging requests in multi-user deployments.

$ koboldcpp --model my-model.gguf --reqtimeout 120

›Adds --splitmode (layer/row/tensor) flag to control GPU split mode independently, including experimental tensor split support (replaces the old combined --usecuda rowsplit syntax).
›Adds --reqtimeout flag to specify request timeouts for router mode and similar requests.
›Adds --musiclowvram flag to allow Ace Step VAE decoding to fall back gracefully when the larger chunk size does not fit in VRAM.
›Adds runtime image LoRA directory selection, enabling all image-gen LoRAs in a chosen directory to be loaded via the <lora:filename:weight> prompt syntax.
›Allows customizing the multiuser queue limit via the GUI.

+4 moreshow less

›Supports loading MTP models (speculative decoding pass not yet enabled).
›Speeds up Ace Step VAE decoding by increasing chunk size.
›Switches wav output from ulaw encoding to pcm16.
›Delivers Qwen3TTS inference speedups by switching back to original precision.

└──▷ BREAKING ON UPGRADE

!Split mode is no longer set as an argument to --usecuda; it must now be specified separately with --splitmode (layer/row/tensor). The old syntax --usecuda rowsplit will no longer work.

Was this useful?

LocalAI

v4.2.6 2 RELEASES · 2026-05-16 NOTES STABLE

LocalAI v4.2.6 bumps llama.cpp to MTP-merge SHA and automatically configures MTP defaults.

└──▷ GET THIS VERSION

$ git clone --branch v4.2.6 https://github.com/mudler/LocalAI.git
# already have the repo? check out this version:
$ git checkout v4.2.6

›Upgrades llama-cpp to the MTP-merge SHA and automatically sets Multi-Token Prediction (MTP) defaults, enabling MTP inference without manual configuration.

1 more release in this issue · 2026-05-16

v4.2.5 NOTES STABLE

LocalAI v4.2.5 exposes 12 additional llama.cpp common_params and updates Swagger docs.

└──▷ GET THIS VERSION

$ git clone --branch v4.2.5 https://github.com/mudler/LocalAI.git
# already have the repo? check out this version:
$ git checkout v4.2.5

›Exposes 12 previously missing common_params via options[] for the llama-cpp backend, giving practitioners finer-grained inference control.
›Updates Swagger API documentation to reflect current endpoint definitions.
›Validates archive member paths before extraction, closing a path-traversal exposure during model downloads.
›Validates video image URLs before download, preventing unexpected outbound requests.
›Updates bundled vllm cu130 wheel to version 0.21.0.

Was this useful?

SGLang

v0.5.12 NOTES

SGLang v0.5.12 adds DeepSeek V4 inference, TokenSpeed MLA backend, HiCache/UnifiedRadixTree, and a wave of new model support.

└──▷ GET THIS VERSION

$ git clone --branch v0.5.12 https://github.com/sgl-project/sglang.git
# already have the repo? check out this version:
$ git checkout v0.5.12

└──▷ TRY IT

Skip KV pool allocation to run a prefill-only benchmark without wasting GPU memory on a KV cache.

$ python -m sglang.launch_server --model-path deepseek-ai/DeepSeek-V3 --prefill-only-disable-kv-cache

Force every radix-cache lookup to miss, useful for isolating cold-start latency during profiling.

$ SGLANG_RADIX_FORCE_MISS=1 python -m sglang.launch_server --model-path deepseek-ai/DeepSeek-V3

Tokenize a chat-formatted message via the new /v1/tokenize endpoint to inspect token counts before committing to a full generate call.

$ curl http://localhost:30000/v1/tokenize -H 'Content-Type: application/json' -d '{"model": "deepseek-ai/DeepSeek-V3", "messages": [{"role": "user", "content": "Hello!"}]}'

›Adds --prefill-only-disable-kv-cache flag to skip KV pool allocation entirely for prefill-only workloads.
›Adds SGLANG_TRACE_LEVEL environment variable to control startup trace verbosity.
›Adds SGLANG_MAX_KV_CHUNK_CAPACITY environment variable to cap KV chunk capacity.
›Adds SGLANG_RADIX_FORCE_MISS environment variable to force radix-cache misses.
›Adds --enable-strict-thinking server flag enabling two-phase reasoning grammar for structured thinking output.

+41 moreshow less

›Adds --random-input-len argument to send_one.py for synthetic load generation.
›Renames SGLANG_USE_JIT_ALL_REDUCE to SGLANG_OPT_USE_CUSTOM_ALL_REDUCE_V2; the old name no longer applies.
›Enables SGLANG_OPT_FP8_WO_A_GEMM by default, activating FP8 weight-only GEMM optimizations without manual configuration.
›Adds /v1/tokenize endpoint with chat-completion-style request support.
›Adds sglang:get_loads_duration_seconds Prometheus metric for load-duration observability.
›Adds fwd_occupancy metric to SchedulerStats plus a matching Prometheus collector.
›Adds per-iteration forward-pass metrics published over ZMQ PUB socket.
›Adds SWA and Mamba radix-cache KV event metrics.
›Adds full inference path for DeepSeek V4, including Tensor/Expert/Context/Data Parallel Attention, Prefill-Decode Disaggregation, HiSparse KV offload, reasoning and tool-call parsers, DeepGemm and FlashMLA kernels, W4A4/W4A8 MoE kernels, and TP16 on H100/H20.
›Adds unified Docker image tag lmsysorg/sglang:v0.5.12 covering all Nvidia GPU variants.
›Adds TokenSpeed MLA attention backend for Blackwell (SM100) with FP8 KV cache support for low-latency MLA serving.
›Adds HiCache framework support for UnifiedRadixTree (including SWA) with SSD offload via Mooncake store.
›Adds custom speculative-algorithm registry, enabling user-defined spec-decoding strategies.
›Adds EagleDraftExtendInput split for Spec V1 draft-extend paths.
›Adds trtllm decode kernel for draft extend in speculative decoding.
›Adds Adaptive Spec V2, EAGLE-3 SWA support, Kimi K2.5 EAGLE-3 MLA, and Gemma 3/4 + EAGLE-3 speculative decoding.
›Adds MLA attention LoRA support for q_b_proj and kv_b_proj projections.
›Adds CSGMV backend with virtual experts for MoE LoRA.
›Adds LoRADrainer to reduce P99 TTFT under heavy LoRA workloads.
›Adds Torch-Native LoRA embedding and graph optimization.
›Adds NIXL staging buffer for heterogeneous-TP KV transfer in PD disaggregation.
›Adds NIXL async transfer support.
›Adds Mooncake incremental transfer and SSD offload for PD disaggregation.
›Adds PrefillDelayer NCCL all-gather for cross-DP info sync.
›Adds adaptive queue-based prefill-delayer trigger.
›Adds retry-with-backoff for prefill bootstrap registration.
›Adds Azure Blob Storage connector supporting az:// URIs and *.blob.core.windows.net endpoints.
›Adds Crusoe managed-inference backend.
›Adds structural tags for strict tool calling and reasoning across additional models.
›Adds auto-detection of reasoning and tool-call parser from chat template.
›Adds OpenAI reasoning.enabled mapping to thinking and enable_thinking fields.
›Adds multi-detokenizer support for concurrent detokenization.
›Enforces rejection of repetition_penalty=0 in SamplingParams.verify().
›Adds new model support: Intern-S2-Preview, MiniCPM-V 4.6, Laguna-XS.2, Ring-2.6-1T (1T-parameter reasoning model), Gemma 4 MTP, Trinity-mini (Ascend NPU), HunyuanVideo ModelOpt FP8, and Qwen Image ModelOpt FP8.
›Adds CFG parallelism framework with multi-branch CFG for LTX-2 in SGLang-Diffusion.
›Adds initial dynamic batching for SGLang-Diffusion.
›Adds dit_precision config key for diffusion models, replacing hardcoded bf16.
›Adds component attention-backend override CLI for SGLang-Diffusion.
›Adds AMD online MXFP4 and FP8 diffusion quantization, aiter RMSNorm, and dual-stream MoE for ROCm.
›Migrates DeepEP source to deepseek-ai/DeepEP@hybrid-ep for clean CUDA 13 builds; pins FlashInfer at 0.6.11.post1.
›Adds TMA bulk-store set_mla_kv_buffer kernel delivering up to 12x throughput over baseline.

└──▷ BREAKING ON UPGRADE

!SGLANG_USE_JIT_ALL_REDUCE is renamed to SGLANG_OPT_USE_CUSTOM_ALL_REDUCE_V2; configurations or scripts referencing the old name will no longer take effect.
!SGLANG_OPT_FP8_WO_A_GEMM is now enabled by default; deployments that previously relied on it being off must explicitly disable it.
!VAE CPU offload is disabled by default in SGLang-Diffusion; workflows that depended on automatic CPU offload must re-enable it explicitly.

Was this useful?

llama.cpp

b9180 2 RELEASES · 2026-05-16 NOTES STABLE

llama.cpp b9180 adds Multi-Token Prediction (MTP) speculative decoding support across CPU, Metal, and Vulkan backends.

└──▷ GET THIS VERSION

$ git clone --branch b9180 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9180

›Adds MTP (Multi-Token Prediction) support for speculative decoding, with draft models identified by the draft-mtp prefix in model naming.
›Adds partial sequence rollback for GDN (Gated Delta Net) recurrent models during speculative decoding, storing intermediates up to draft_max tokens to avoid full restart on draft rejection.
›Extends Metal backend with GDN partial rollback support via a new kernel function constant K (snapshot slot count) controlling the 3D state tensor slot behavior.
›Extends Vulkan backend with GDN partial rollback support for speculative decoding.
›Enables checkpointing with partial rollback in llama-memory for recurrent model contexts.

+1 moreshow less

›Server disables RS-based MTP when other speculative decoding types are active, preventing incompatible combinations.

1 more release in this issue · 2026-05-16

b9174 NOTES STABLE

llama.cpp b9174 renames WebUI surfaces to UI: new --ui, --ui-config, --ui-config-file, and --ui-mcp-proxy flags plus matching env vars.

└──▷ GET THIS VERSION

$ git clone --branch b9174 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9174

└──▷ TRY IT

Start the llama-server with the web UI and MCP proxy enabled using the new canonical flags.

$ llama-server --ui --ui-mcp-proxy --ui-config-file /etc/llama/ui-config.json -m model.gguf

Enable the web UI via environment variable in a containerised deployment instead of passing CLI flags.

$ LLAMA_ARG_UI=1 LLAMA_ARG_UI_MCP_PROXY=1 llama-server -m model.gguf

›Adds --ui / --no-ui CLI flags (replacing deprecated --webui / --no-webui) to control the built-in web interface.
›Adds --ui-config CLI flag (replacing deprecated --webui-config) for passing UI configuration.
›Adds --ui-config-file CLI flag (replacing deprecated --webui-config-file) for specifying the UI config file path.
›Adds --ui-mcp-proxy / --no-ui-mcp-proxy CLI flags (replacing deprecated --webui-mcp-proxy) to toggle MCP proxy support in the UI.
›Adds environment variables LLAMA_ARG_UI, LLAMA_ARG_UI_CONFIG, LLAMA_ARG_UI_CONFIG_FILE, and LLAMA_ARG_UI_MCP_PROXY as alternatives to the corresponding CLI flags.

+3 moreshow less

›Renames CMake build variable LLAMA_BUILD_WEBUI to LLAMA_BUILD_UI and LLAMA_USE_PREBUILT_WEBUI to LLAMA_USE_PREBUILT_UI (old names kept as deprecated aliases).
›Moves UI source tree from tools/server/public / tools/server/webui/ to tools/ui/ and build output to build/tools/ui/dist.
›JSON API now emits both ui / ui_settings and legacy webui / webui_settings keys for backward compatibility.

Was this useful?

vMLX

v1.5.37 NOTES

vMLX - JANGTQ Uber Compressed MLX Models - L2 Disk Cache (survives restart) + L1 Paged (super fast ttft) + Hybrid SSM Scheduler + Cont Batching + etc!

vMLX 1.5.37 adds max_prompt_tokens, max_context_tokens, and max_context aliases for prompt context admission across OpenAI, Anthropic, Ollama, and gateway paths.

└──▷ GET THIS VERSION

$ git clone --branch v1.5.37 https://github.com/jjang-ai/vmlx.git
# already have the repo? check out this version:
$ git checkout v1.5.37

└──▷ TRY IT

Cap how much prompt context is admitted per request when calling through the OpenAI-compatible gateway, to stay within model token limits.

$ max_prompt_tokens=4096 max_context_tokens=8192

›Adds max_prompt_tokens, max_context_tokens, and max_context aliases for controlling request/session prompt context admission across OpenAI, Anthropic, Ollama, and gateway paths.
›Applies model-declared generation defaults at session startup, removing panel-side sampling fallbacks.
›Scopes sampling overrides to the current chat or API request instead of carrying stale per-model state across requests.

Was this useful?

// END OF TRANSMISSION

The AI Toolchain · curated daily · May 16, 2026
Releases & features from the open-source and commercial AI tools that practitioners actually run.

The AI Toolchain — issue -095, May 16, 2026