https://aitoolchain.io/tools/mlx-serve New releases and features in mlx-serve, tracked by The AI Toolchain. en Sat, 29 Aug 2026 14:03:23 GMT https://github.com/ddalcu/mlx-serve/releases/tag/v26.8.11 https://github.com/ddalcu/mlx-serve/releases/tag/v26.8.11 Sat, 29 Aug 2026 14:03:23 GMT mlx-serve v26.8.11 adds Qwen 3.8 Flash Next, multimodal video input, sparse attention kernels, and batch-concurrent inference. • Adds `--mtp` flag (and MoE toggle in Settings) to opt into speculative decoding on Flash Next: +41% on code generation, with speculative decoding now also applied to image questions. • Adds native support for Qwen 3.8 Flash Next (125B), delivering ~60 tok/s on M4 Max, ~78 tok/s with speculative decoding, using ~70 GB RAM with the 4-bit pack (`ddalcu/Qwen3.8-Flash-Next-MLX-Serve-4bit`). • Adds image and video understanding with follow-up questions on the same image answered instantly by reusing the cached visual context. • Adds sparse attention past 2k tokens via custom kernels, keeping 8k-token prompt throughput competitive even under speculative decoding. • Adds concurrent request batching on Flash Next: 2 parallel streams yield 1.3x total throughput, 4 streams yield 1.8x, with no regression on single-stream latency. • Adds native support for `gpt-oss` 20B and 120B (OpenAI MoE, harmony format). • Updates to MLX 0.32.2, delivering up to +3% speed improvement on MoE models at long context.