The daily firehose — everything the toolchain shipped today, already filtered.
// HOW THIS ISSUE IS MADE
We read every release from the 174 tools on our watchlist at the source — GitHub and GitLab release notes, vendor release pages and changelogs, project blogs and feeds, vendor press releases, and the source code behind the tag. Bug-fix-only releases and non-product newsroom noise are dropped; what's left is summarized down to the new capability, how to try it, and any screenshots or videos the release itself published. Every entry links to the sources it was built from.
LocalAI v3.9.0 adds Agent Jobs panel, LRU model eviction, smart memory reclaimer, VibeVoice TTS, and CUDA 13 support.
└──▷ GET THIS VERSION
$ git clone --branch v3.9.0 https://github.com/mudler/LocalAI.git
# already have the repo? check out this version:$ git checkout v3.9.0
›Adds single_active_backend mode now defaults to LRU eviction with a limit of 1, automatically unloading the least recently used model when a new one is loaded — configurable to any model count limit.
›New Memory Reclaimer watchdog monitors GPU/VRAM consumption across all backends and evicts LRU models when usage exceeds a configured threshold, preventing out-of-memory crashes.
›New Agent Jobs panel in the web UI and API lets you create, schedule (via cron syntax), and trigger background agentic tasks programmatically, with optional webhook delivery of results and dynamic variable-populated prompts.
›New vibevoice TTS backend adds real-time text-to-speech with voice cloning support, installable from the model gallery.
›Adds CUDA 13 support in Docker images and L4T builds.
›Whisper backend adds a prompt parameter to condition transcription output.
›StableDiffusion-GGML gains LoRA support for image generation.
›Hugging Face API now returns files in nested directories, enabling model imports from URLs with subfolders (e.g., huggingface://user/model/GGUF/model.gguf).
›New UI mask for installing custom backends from the web interface.
›UI search results can now be ordered by user selection.
›Agent Jobs support multimedia inputs.
└──▷ BREAKING ON UPGRADE
!Data storage path changed from /usr/share to /var/lib; existing deployments must migrate mutable data to the new location.
!x86_64 macOS (darwin-x86_64) support dropped; ARM64 (M1/M2/M3/M4) is now the only supported Mac architecture.
SGLang Gateway v0.3.0 adds unified IGW mode, tokenize/detokenize endpoints, parser endpoints, gRPC embeddings, TLS, and a Go implementation.
└──▷ GET THIS VERSION
$ git clone --branch gateway-v0.3.0 https://github.com/sgl-project/sglang.git
# already have the repo? check out this version:$ git checkout gateway-v0.3.0
›Adds /parse/reasoning and /parse/function_call HTTP endpoints for parsing reasoning outputs and function call responses, including a GLM-4 function call parser.
›Adds tokenize/detokenize HTTP endpoints with a dynamic tokenizer control plane (add, list, get, remove tokenizers on-the-fly) backed by a TokenizerRegistry.
›Adds native embeddings endpoint for the gRPC router, expanding the gateway beyond text generation to embedding workloads.
›Adds server-side TLS support with graceful shutdown for both TLS and non-TLS servers.
›Adds X-SMG-Error-Code response headers and unified error codes across all router types.
+8 moreshow less
›Adds streaming metrics (TTFT, TPOT, tokens, duration) for the gRPC router, plus smg_worker_*, smg_router_*, MCP, and discovery metric layers in a new 6-layer Prometheus metrics architecture.
›Adds custom Prometheus duration buckets.
›Introduces Unified Inference Gateway (IGW) mode: a single gateway instance now supports gRPC (PD and regular), HTTP (PD and regular), and OpenAI routers simultaneously, auto-enabled when Kubernetes service discovery is turned on.
›Adds retry and circuit breaker support for OpenAI and gRPC routers; returns HTTP 503 when all workers are circuit-broken.
›Adds a complete Go implementation of the SGLang Model Gateway with an OpenAI-compatible API server, contributed by the iFlytek MaaS team.
›Adds multi-architecture Docker builds supporting Linux, macOS, Windows, and ARM.
›Introduces a DAG-based parallel workflow engine with pre-computed dependency graphs and concurrent event processing for worker lifecycle orchestration.
›Replaces PolicyRegistryRwLock with DashMap for lock-free policy lookups and adds lock-free router snapshots to reduce CPU overhead.
└──▷ BREAKING ON UPGRADE
!Prometheus metric names and structure have changed (complete 6-layer architecture redesign) — update all Prometheus dashboards and alerting rules.
!Workers are now identified by UUIDs instead of endpoints — update any tooling or scripts that interact with the worker API.
!SmgMetrics is renamed to Metrics and smg_labels is renamed to metrics_labels — update any code referencing these names.
!Legacy RouterMetrics has been removed — migrate to the new unified metrics layer.
llama.cpp b7531 adds support for the LlamaBidirectionalModel architecture, enabling llama-embed-nemotron embedding models.
└──▷ GET THIS VERSION
$ git clone --branch b7531 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:$ git checkout b7531
›Adds support for the LlamaBidirectionalModel architecture (llama-embed arch), enabling use of bidirectional embedding models such as llama-embed-nemotron.
›Adds LLAMA_ARG_OVERRIDE_TENSOR environment variable as an alternative way to supply the -ot (override tensor) argument, enabling tensor-routing control without modifying CLI invocations.
Pi v0.27.5 adds syntax highlighting and full markdown rendering to HTML exports.
└──▷ GET THIS VERSION
$ git clone --branch v0.27.5 https://github.com/earendil-works/pi.git
# already have the repo? check out this version:$ git checkout v0.27.5
›Adds syntax highlighting to code blocks in HTML exports using highlight.js, with theme-aware colors matching the TUI.
›Adds server-side markdown rendering in HTML exports (tables, headings, code blocks, image rendering for user messages, TUI-style language markers, light/dark theme support).
Qwen Code v0.6.0-preview.0 adds Gemini as a provider and removes legacy Google OAuth.
└──▷ GET THIS VERSION
$ git clone --branch v0.6.0-preview.0 https://github.com/QwenLM/qwen-code.git
# already have the repo? check out this version:$ git checkout v0.6.0-preview.0
›Adds Gemini as a supported LLM provider, replacing legacy Google OAuth authentication.
└──▷ BREAKING ON UPGRADE
!Legacy Google OAuth support has been removed; existing setups relying on Google OAuth will need to migrate to the new Gemini provider authentication.
Qwen Code v0.6.0-nightly adds Gemini as a provider and removes legacy Google OAuth.
└──▷ GET THIS VERSION
$ git clone --branch v0.6.0-nightly.20251224.bc2a7efc https://github.com/QwenLM/qwen-code.git
# already have the repo? check out this version:$ git checkout v0.6.0-nightly.20251224.bc2a7efc
›Adds Gemini as a supported LLM provider, enabling Qwen Code users to route requests through Gemini models.
›Removes legacy Google OAuth authentication in favour of the new Gemini provider integration.
└──▷ BREAKING ON UPGRADE
!Legacy Google OAuth support has been removed; users relying on Google OAuth for authentication must migrate to the new Gemini provider.
Qwen Code v0.5.1-preview.1 adds Gemini as a provider and removes legacy Google OAuth.
└──▷ GET THIS VERSION
$ git clone --branch v0.5.1-preview.1 https://github.com/QwenLM/qwen-code.git
# already have the repo? check out this version:$ git checkout v0.5.1-preview.1
›Adds Gemini as a supported LLM provider and removes legacy Google OAuth authentication.
›Adds a link to Gemini CLI Desktop for users who prefer a desktop UI over the CLI.
└──▷ BREAKING ON UPGRADE
!Legacy Google OAuth is removed; any working setup relying on Google OAuth for authentication will break on upgrade.
PydanticAI v1.39.0 adds embedding model support, agent run metadata on results and spans, and a new BedrockModelSettings tier field.
└──▷ GET THIS VERSION
$ git clone --branch v1.39.0 https://github.com/pydantic/pydantic-ai.git
# already have the repo? check out this version:$ git checkout v1.39.0
└──▷ USE IT
Set a specific AWS Bedrock service tier on a model to control throughput or priority for an agent run.
python
from pydantic_ai.models.bedrock import BedrockModelSettings
settings = BedrockModelSettings(bedrock_service_tier='standard')
result = await agent.run('Summarize this document', model_settings=settings)
›Adds bedrock_service_tier setting to BedrockModelSettings for controlling AWS Bedrock service tier per agent run.
›Adds agent and agent run metadata, exposed on result objects and OpenTelemetry span attributes.
›Introduces embedding model support via new embedding model classes and APIs.
›Supports ThinkingPart in MCP Sampling, enabling reasoning-aware model responses over the Model Context Protocol.
›Allows system prompt functions to return None, treating it as a no-op rather than an error.
Triton v2.64.0 promotes OpenAI-compatible frontend to stable, adds multi-LoRA and echo parameter support.
└──▷ GET THIS VERSION
$ git clone --branch v2.64.0 https://github.com/triton-inference-server/server.git
# already have the repo? check out this version:$ git checkout v2.64.0
›Adds echo request parameter for TensorRT-LLM and Python backends to the OpenAI-compatible API frontend v1/completions endpoint.
›Enables multi-LoRA support for the TensorRT-LLM backend via the OpenAI-compatible API frontend.
›Adds TRITONBACKEND_ModelInstanceReady function for backends to implement, enabling accurate model readiness reporting.
›Promotes the OpenAI-compatible frontend from beta to stable release.
›Updates the Python backend to accurately report model readiness status.
Chroma 1.4.0 adds group-by search, CMEK support, and a new Rust sysdb service to Python, JS, and Rust clients.
└──▷ GET THIS VERSION
$ git clone --branch 1.4.0 https://github.com/chroma-core/chroma.git
# already have the repo? check out this version:$ git checkout 1.4.0
└──▷ USE IT
Use a self-hosted or Azure OpenAI-compatible endpoint with the JS client embedding function by specifying a custom base URL.
javascript
import { OpenAIEmbeddingFunction } from 'chromadb';
const embedder = new OpenAIEmbeddingFunction({
openai_api_key: process.env.OPENAI_API_KEY,
openai_model: 'text-embedding-3-small',
base_url: 'https://my-openai-proxy.example.com/v1'
});
›Adds group_by operator to collection search in the Python and JS clients, with quota enforcement, enabling faceted or clustered retrieval in a single query.
›Adds CMEK (Customer-Managed Encryption Key) support in the Python and JS clients.
›Adds base_url specification support in the JS client OpenAI embedding function, allowing custom or self-hosted OpenAI-compatible endpoints.
›Adds count_collections to the Rust client (released as Rust client 0.10.0).
›Introduces a new Rust sysdb service with a gRPC server, config file support, Spanner emulator integration, schema migration runner, checksummed migration manifests, and a feature-flag-gated migration service.
THE AI TOOLCHAIN