Backfill issue -337 — The AI Toolchain

EDITIONS tail grep head diff uniq

The daily firehose — everything the toolchain shipped today, already filtered.

// HOW THIS ISSUE IS MADE

We read every release from the 174 tools on our watchlist at the source — GitHub and GitLab release notes, vendor release pages and changelogs, project blogs and feeds, vendor press releases, and the source code behind the tag. Bug-fix-only releases and non-product newsroom noise are dropped; what's left is summarized down to the new capability, how to try it, and any screenshots or videos the release itself published. Every entry links to the sources it was built from.

VIEW

ISSUE VIEW full issue

Do you prefer this view?

$ tct list   # 2 tools matched

AI & LLM Tooling

◆ AI Coding Agents

Cline

v3.28.3 NOTES

Autonomous coding agent as an SDK, IDE extension, or CLI assistant.

Cline v3.28.3 adds AI-generated commit messages for staged (or unstaged fallback) changes.

└──▷ GET THIS VERSION

$ git clone --branch v3.28.3 https://github.com/cline/cline.git
# already have the repo? check out this version:
$ git checkout v3.28.3

›Generates commit messages for staged changes, falling back to unstaged changes when no staged files are present.

Was this useful?

◆ Local LLM Runtimes

vLLM

v0.10.2 NOTES

vLLM v0.10.2 adds aarch64/GB200 support, 20+ new model families, PyTorch 2.8, and major API/quantization expansions with several breaking changes.

└──▷ GET THIS VERSION

$ git clone --branch v0.10.2 https://github.com/vllm-project/vllm.git
# already have the repo? check out this version:
$ git checkout v0.10.2

└──▷ TRY IT

Load a model using the Terratorch backend for geospatial or semantic segmentation tasks instead of standard LLM inference.

$ vllm serve <model> --model-impl terratorch

Speed up model weight loading when checkpoints are on a slow NFS mount.

$ vllm serve <model> --safetensors-load-strategy <strategy>

Install vLLM with aarch64 wheels on a GB200 or other ARM64 system.

$ uv pip install vllm==0.10.2 --extra-index-url https://wheels.vllm.ai/0.10.2/ --torch-backend=auto

›Adds --model-impl terratorch flag to enable the Terratorch backend for non-language model tasks like semantic segmentation and geospatial applications.
›Adds --safetensors-load-strategy flag for accelerating model weight loading from NFS-based file systems.
›Adds --log-error-stack flag to surface full error stack traces in the frontend.
›Adds --mm-encoder-tp-mode flag (now formalized) to control multimodal encoder tensor-parallelism mode.
›Adds VLLM_DISABLE_PAD_FOR_CUDAGRAPH environment variable to control CUDA graph padding behavior.

+40 moreshow less

›Adds FlashInfer cubin directory environment variable for offline environments and pre-cached CUDA binaries.
›Adds return_token_ids parameter to the OpenAI-compatible API.
›Adds EPLB configuration parameter for expert-parallel load balancing.
›Adds native aarch64 support, enabling vLLM on GB200 platforms; multiplatform vllm/vllm-openai Docker image and wheels available via --extra-index-url https://wheels.vllm.ai/0.10.2/ --torch-backend=auto.
›Extends V1 engine support to GPUs with compute capability below 8.0.
›Adds cross-attention KV cache support for encoder-decoder models in the V1 engine.
›Adds request-level logits processor integration in V1.
›Adds user-configurable KV cache memory size.
›Adds support for all prompt logprobs and final logprobs in sampling.
›Enables full CUDA graph support by default for hybrid and Mamba models.
›Adds V1 engine as the default for Mamba models.
›Adds multimodal cache tracking via mm_hash and UUID-based multimodal identifiers.
›Adds Apple Silicon bfloat16 support for M2+ chips.
›Adds IBM Z V1 engine support (s390x).
›Adds Intel XPU torch.compile, MoE data parallelism, Triton attention, and FP8 quantization support.
›Adds ROCm pipeline parallelism with Ray and TorchAO quantization enablement.
›Adds FP8 MLA support with CUTLASS backend for NVIDIA Blackwell/SM100 GPUs.
›Adds MXFP4 fused CUTLASS MoE support, enabled by default on Blackwell.
›Adds W4A8 support on Hopper GPUs.
›Adds per-layer quantization routing.
›Adds GGUF quantization with layer-skipping capability.
›Adds NFP4+FP8 MoE support.
›Adds AMD CDNA2/CDNA3 FP4 quantization support.
›Adds compressed tensors transforms for linear operations, enabling SpinQuantR1R2R4 and QuIP quantization methods.
›Adds LoRA support for Voxtral, Qwen-2.5-Omni, and DeepSeek V2/V3/R1-0528 models, with significantly faster LoRA startup.
›Adds multi-label classification support and sigmoid normalization for pooling/classification models.
›Adds Support for Decode Context Parallel (DCP) for MLA in distributed serving.
›Adds new model families: Apertus, LFM2, MiDashengLM, Motif-1-Tiny, Seed-Oss, Google EmbeddingGemma-300m, GTE sequence classification, Donut OCR, KeyeVL-1.5-8B, R-4B vision, Ernie4.5 VL, MiniCPM-V 4.5, Ovis2.5, Qwen3-Next with hybrid attention, InternVL3.5 with video, Qwen2Audio embeddings, NemotronH Nano VLM, BLOOM V1, and Whisper encoder-decoder for V1.
›Adds pipeline parallelism support for Hunyuan, Ovis2.5, GPT-OSS, and Kimi-VL-A3B-Thinking-2506.
›Enables data parallelism for ViT across Qwen2.5VL, MiniCPM-V, Kimi-VL, and GLM-4.5V.
›Adds Gemma3n audio transcription/translation endpoints to the OpenAI-compatible API.
›Adds transcription response usage statistics to the API.
›Adds streaming support for non-harmony responses in the Response API.
›Adds non-streaming logprobs to the Response API.
›Adds MCP tool background mode and streaming+background support.
›Adds tool output token reporting to the Response API.
›Adds collective RPC endpoint.
›Adds embedding endpoint support for chat requests.
›Upgrades to PyTorch 2.8.0 and FlashInfer v0.3.0.
›Delivers 13.7x speedup for token conversion and symmetric memory all-reduce enabled by default.

└──▷ BREAKING ON UPGRADE

!PyTorch 2.8.0 upgrade requires updated CUDA versions and environment dependency changes.
!FlashMLA is disabled on Blackwell GPUs due to compatibility issues.
!V0 Neuron backend removed.
!V0 pooling model support removed.
!V0 FlashInfer attention backend removed.
!Original Marlin quantization format removed.
!Quantized Mixtral hack implementation removed.
!prompt_token_ids fallback removed from LLM.generate and LLM.embed.
!Metrics name TPOT renamed to ITL — dashboards and alerting rules referencing TPOT will break.

Was this useful?

// END OF TRANSMISSION

The AI Toolchain · curated daily · September 13, 2025
Releases & features from the open-source and commercial AI tools that practitioners actually run.

The AI Toolchain — issue -337, September 13, 2025

Cline

vLLM