Heads up This site is currently under heavy development.
← all tools
◆ Local LLM Runtimes

llama.cpp

v0.3.0 open-source

LLM inference in C/C++

source ↗ website ↗ rss ↗ last repo push Aug 22, 2026

Summary

llama.cpp is an open-source local-llm-runtime that implements LLM inference in C/C++ using the MIT license. It operates as a command-line tool, allowing users to run quantization and inference using GGML/GGUF formats. This cyber tool is aimed at application developers needing to run models locally, and its documentation positions it alongside tools that handle LLM inference. The project is actively maintained, with clear release badges for current and nightly builds.

LLM inference in C/C++

What llama.cpp answers

Which formats does it use for models?

GGML and GGUF formats

What type of operation does it perform?

LLM inference in C/C++

Does it offer a way to run it as a service?

It includes a server component for REST API access

What is the required compilation base?

The core logic is built with C/C++

Can it be used in automated development workflows?

It provides workflows for Docker and Winget installation

What kind of access does it require to operate?

It operates as a command-line tool

Release history

  1. v0.3.0 Aug 25, 2026 · issue 007

    llama.cpp v0.3.0 adds dots3-note multimodal model, DeepSeek 4 tensor-split via -sm tensor, WebP support, and a new LLAMA_SERVER_SLOTS_N_DIFF debug env var.

    └──▷ GET THIS VERSION
    $ git clone --branch v0.3.0 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout v0.3.0
    └──▷ TRY IT
    Run DeepSeek 4 across multiple GPUs using tensor-split mode for better VRAM distribution.
    $ llama-cli -m deepseek4.gguf -sm tensor
    Widen the slot debug diff window when diagnosing slot assignment issues in llama-server.
    $ LLAMA_SERVER_SLOTS_N_DIFF=20 llama-server -m model.gguf
    • ›Adds -sm tensor flag to enable tensor-split mode for DeepSeek 4 multi-GPU inference.
    • ›Adds LLAMA_SERVER_SLOTS_N_DIFF environment variable to widen the slot debug diff window in the llama-server.
    • ›Adds the dots3-note multimodal model with a new DSA-ISWA KV cache type, including vision and audio support.
    • ›Adds MTP (multi-token prediction) support for the GLM-4.5-Air model.
    • ›Adds WebP image decoding support in the multimodal subsystem via ffmpeg.
    +4 moreshow less
    • ›Adds DSpark architecture support for the bailingmoe3 model.
    • ›Adds tabbed navigation for chat conversations in the built-in web UI.
    • ›Upgrades ggml to v0.22.0, bringing meta-backend tensor split, per-op Metal kernels with parallel compilation, and non-in-place ggml_clamp.
    • ›Improves mamba2 throughput by flattening in/out projections to dispatch GEMM instead of GEMV.
    └──▷ BREAKING ON UPGRADE
    • !The -no-cnv CLI option has been removed.
  2. v0.2.0 Aug 21, 2026 · issue 004

    llama.cpp v0.2.0 introduces stable semantic versioning, new model support, backend kernels, and server/multimodal improvements.

    └──▷ GET THIS VERSION
    $ git clone --branch v0.2.0 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout v0.2.0
    • ›Adds --mmproj-device argument to specify the device for multimodal projector inference.
    • ›Adds dedup-cache-models preset option to the server.
    • ›Adds mtmd_bitmap_set_mergeable API to the multimodal subsystem.
    • ›Makes /metrics endpoint accessible during server sleep state.
    • ›Makes models endpoints private when authentication is enabled on the server.
    +16 moreshow less
    • ›Adds SME2 F32 GEMV kernel support via KleidiAI on ARM.
    • ›Ports fused ssm_scan kernel (Mamba-2, d_state 128/256) to GPU via OpenCL.
    • ›Enables tensor split support for LFM2 and LFM2MOE models.
    • ›Adds support for GraniteSWAForCausalLM and GraniteMoeSWAForCausalLM model architectures.
    • ›Adds support for DSpark architecture for LFM2 models.
    • ›Adds tiled transpose for 0<->2 permuted CONT on the Vulkan backend.
    • ›Adds mulmat with overlapping src0/src1 (e.g. for MiniMax-01) on the WebGPU backend.
    • ›Adds RPC use_count population to enable operation fusion inside backends.
    • ›Optimises quantisation memory usage by evicting weights after processing each layer.
    • ›Adds CUDA switch points per hardware and quantisation type to tune the MVQ-to-MMQ decode crossover.
    • ›Introduces stable semantic versioning (vX.Y.Z tags) alongside existing nightly b[NUM] tags for easier downstream distribution.
    • ›Adds graceful fallback for unsupported regex patterns in JSON schema on the common layer.
    • ›Adds size guards to GGUFReader in the gguf-py library.
    • ›Adds GGML_OPENCL_A7X_LMHEAD_CPU environment variable to override automatic CPU offload of vocab-scale K-quant lm_head on Adreno A7X GPUs.
    • ›Adds Windows ARM64 CUDA support to the manual CI workflow.
    • ›Updates OpenVINO backend to version 2026.3.
  3. b10549 Aug 21, 2026 · issue 003

    llama.cpp b10549 adds --split-mode tensor support for LFM2 and LFM2MOE model families

    └──▷ GET THIS VERSION
    $ git clone --branch b10549 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10549
    • ›Adds --split-mode tensor support for LFM2 and LFM2MOE model families, enabling multi-GPU tensor parallelism for these architectures.
    • ›Extends ggml_rope_set_offset support to the OpenCL, SYCL, WebGPU, and Hexagon backends.
  4. b10541 Aug 21, 2026 · issue 003

    llama.cpp b10541 adds --mmproj-device / -mmdev flag and MTMD_BACKEND_DEVICE env var for per-device multimodal projector offloading, plus DSpark speculative decoding support for LFM2 models.

    └──▷ GET THIS VERSION
    $ git clone --branch b10541 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10541
    └──▷ TRY IT
    Offload the multimodal projector to a specific GPU device (e.g. iGPU) while the main model uses a different device — useful on systems with mixed GPU setups.
    $ llama-server -m model.gguf --mmproj mmproj.gguf --mmproj-device CUDA1
    Use the MTMD_BACKEND_DEVICE env var to set the mmproj device without changing command-line args — handy for preset or scripted deployments.
    $ MTMD_BACKEND_DEVICE=CUDA1 llama-cli -m model.gguf --mmproj mmproj.gguf
    Run LFM2 with DSpark speculative decoding to achieve multi-fold token generation speedup on supported hardware.
    $ llama-server -m LFM2.5-1.2B-Instruct-F16.gguf -md LFM2.5-1.2B-Instruct-DSpark-5L-draft-f16.gguf -ngl 99 -ngld 99 -fa on --spec-draft-n-max 7 --spec-draft-n-min 0
    • ›Adds --mmproj-device (short flag: -mmdev) CLI argument and MTMD_BACKEND_DEVICE environment variable to select which compute device handles the multimodal projector (mmproj), enabling scenarios like offloading mmproj to an iGPU while the main model runs on a dGPU; accepts none to disable GPU offload entirely.
    • ›Adds DSpark speculative decoding support for LFM2 models, including partial rollback for LFM2 recurrent state, enabling draft-model-accelerated inference for LFM2.5 series models.
  5. b10534 Aug 21, 2026 · issue 003

    llama.cpp b10534 adds per-hardware CUDA MMVQ→MMQ switch points, yielding up to 41% faster K-quant decode on Blackwell/Ada GPUs.

    └──▷ GET THIS VERSION
    $ git clone --branch b10534 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10534
    • ›Adds per-hardware and per-quantization-type batch crossover thresholds for the CUDA mul_mat_vec_q → MMQ (int8 tensor-core) dispatch path, with tuned switch points for Blackwell (RTX 5090, DGX Spark) and Ada (RTX 4090) GPU families, delivering +23–41% decode throughput at batch size 8 for K-quants (Q2_K, Q3_K, Q4_K, Q5_K, Q6_K) on dense models with no low-batch regression.
  6. b10532 Aug 21, 2026 · issue 003

    Metal backend now dequantizes Q4_0/Q4_1/Q5_0/Q5_1/Q8_0 KV caches to F16 before flash attention, improving accuracy and performance on Apple Silicon.

    └──▷ GET THIS VERSION
    $ git clone --branch b10532 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10532
    └──▷ TRY IT
    Run perplexity evaluation on Apple Silicon with a quantized KV cache — the Metal backend now dequantizes to F16 automatically before flash attention, matching F16 KV accuracy.
    $ llama-perplexity -m model-q8_0.gguf -fa 1 -ctk q8_0 -ctv q8_0
    • ›Adds a preprocessing pass on the Metal backend for GGML_OP_FLASH_ATTN_EXT that dequantizes quantized KV caches (Q4_0, Q4_1, Q5_0, Q5_1, Q8_0) into contiguous F16 scratch buffers before running the existing F16 flash-attention kernels, instead of performing in-kernel dequantization.
    • ›Adds a new Metal kernel kernel_flash_attn_ext_dequant_to_f16 dispatched separately for K and V into per-op scratch memory, with stride-aware dequantization supporting permuted KV layouts.
    • ›Skips the redundant V dequantization when V is a view of K (as in MLA-based models), reading dequantized V from the K F16 buffer with K's strides instead.
    • ›Adds new quantized-KV flash-attention test cases in test-backend-ops, including the MLA shape (576/512, V is a view of K), pad path (kv=113), and large context (kv=16384).
  7. b10519 Aug 20, 2026 · issue 003

    llama-server now enforces auth on /v1/models and exposes /metrics during sleep mode

    └──▷ GET THIS VERSION
    $ git clone --branch b10519 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10519
    • ›The /v1/models endpoint is now private when server authentication is enabled, blocking unauthenticated access to model file paths (which may contain PII such as home directory names).
    • ›The /metrics endpoint is now accessible while the server is in sleep/idle state, allowing monitoring systems to scrape metrics without waking the server.
    └──▷ BREAKING ON UPGRADE
    • !When authentication is enabled, unauthenticated clients that previously could query /v1/models without a key will now receive an auth error — any tooling or scripts relying on unauthenticated access to /v1/models will break.
  8. b10514 Aug 20, 2026 · issue 003

    Adds support for GraniteSWAForCausalLM and GraniteMoeSWAForCausalLM models with interleaved Sliding Window Attention

    └──▷ GET THIS VERSION
    $ git clone --branch b10514 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10514
    • ›Adds conversion and inference support for GraniteSWAForCausalLM and GraniteMoeSWAForCausalLM model architectures, enabling IBM Granite models with interleaved Sliding Window Attention (SWA) and Attention Sinks.
    • ›Adds per-layer RoPE/NoPE determination via llama_hparams::has_rope, allowing models with mixed attention layers (some with rotary embeddings, some without) to load and run correctly.
    • ›SYCL backend now reports zero devices instead of aborting when no SYCL-capable hardware is present, allowing tools like llama-quantize to run on non-SYCL hosts without crashing.
  9. b10509 Aug 20, 2026 · issue 002

    Adds ggml_rope_set_offset operation with CPU, Metal, CUDA, and Vulkan backend support for DSA and 2D RoPE use cases.

    └──▷ GET THIS VERSION
    $ git clone --branch b10509 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10509
    • ›Adds ggml_rope_set_offset ggml operation enabling DSA-indexer RoPE ('nope | rope' style with offset = n_dims_nope) and 2D RoPE for multimodal models, with support across CPU, Metal, CUDA, and Vulkan backends.
  10. b10505 Aug 20, 2026 · issue 002

    llama.cpp b10505 adds dedup-cache-models preset option to hide duplicate cached models from the server model list.

    └──▷ GET THIS VERSION
    $ git clone --branch b10505 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10505
    └──▷ USE IT
    Prevent duplicate model entries in the GET /models list when multiple presets resolve to the same underlying model file.
    preset.ini 
    [global]
dedup-cache-models = 1
    • ›Adds dedup-cache-models (values 0|1) as a server router preset option in preset.ini: when set in global scope, hides cached models from GET /models when a preset resolves to the same model file, deduplicating the model list.
  11. b10502 Aug 19, 2026 · issue 002

    llama.cpp b10502 adds signed release artifact attestations and reduces quantization memory use via layer-weight eviction.

    └──▷ GET THIS VERSION
    $ git clone --branch b10502 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10502
    • ›Adds GitHub Actions attestation for signed release artifacts, enabling cryptographic verification of official binaries via the attestations page.
    • ›Reduces peak memory usage during quantization by evicting processed layer weights from memory pages once each layer is done (benefits llama-quantize and similar workflows).
    • ›SYCL backend now supports GGML_HINT_SRC0_IS_HADAMARD, porting the Fast Walsh-Hadamard Transform kernel from CUDA to SYCL for accelerated Hadamard matrix-multiply on Intel GPUs.
    • ›Optimizes thread pool usage when -t and -tb differ: tools now share a single pool sized to the larger count rather than allocating two separate pools.
  12. b10470 Aug 17, 2026 · issue 002

    Adds BailingMoE3/Ling 3.0 Flash model support and removes get_datetime from built-in agent tools

    └──▷ GET THIS VERSION
    $ git clone --branch b10470 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10470
    • ›Adds support for BailingMoE3 model architecture (Ling 3.0 Flash) with MTP support, enabling inference on the new Ling 3.0 Flash model.
    • ›Removes get_datetime from the list of available --tools built-in agent tools; the remaining available tools are read_file, file_glob_search, grep_search, exec_shell_command, write_file, edit_file, and get_info.
    └──▷ BREAKING ON UPGRADE
    • !The get_datetime tool is no longer listed as an available option for the --tools flag; any agent configuration specifying get_datetime will silently receive no matching tool.
  13. b10455 Aug 17, 2026 · issue 002

    SYCL backend gains AdamW and SGD optimizer step support for on-device training workloads.

    └──▷ GET THIS VERSION
    $ git clone --branch b10455 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10455
    • ›Adds OPT_STEP_ADAMW and OPT_STEP_SGD operation support to the SYCL backend, enabling GPU-accelerated optimizer steps for AdamW and SGD on Intel GPUs.
  14. b10448 Aug 15, 2026 · issue -004

    llama.cpp b10448 adds Kimi-K3 model support with hybrid KDA+MLA attention, latent MoE, and a dedicated chat format.

    └──▷ GET THIS VERSION
    $ git clone --branch b10448 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10448
    • ›Adds common_chat_params_init_kimi_k3 (PEG_NATIVE) chat format for Kimi-K3 with reasoning extraction, response unwrapping, and typed tool-call parsing using the tools/call/argument tag structure.
    • ›Adds kda_gate_lower_bound GGUF key (emitted by the model saver) so save/load roundtrips correctly preserve Kimi-K3's KDA gate activation; K3 sets this to -5.0.
    • ›Increases LLAMA_MAX_EXPERTS from 512 to 1024 to accommodate Kimi-K3's expert count.
    • ›Adds Kimi-K3 text model support with hybrid KDA (linear) + MLA (full) attention, cross-layer residual attention (attn_res_block_size), latent MoE (n_expert_latent), situ activation, MLA output gate, and full-rank KDA gate (ssm_g).
    • ›Adds lossless MXFP4 repack for Kimi-K3's compressed-tensors 'mxfp4-pack-quantized' routed experts — bit-compatible with ggml MXFP4, repacked rather than dequantized, avoiding a ~5.5 TB bf16 round-trip; the repack_mxfp4_blocks helper is now shared with DeepSeek-V4.
    +3 moreshow less
    • ›Adds per-role message-start delimiters (message_delimiters) for Kimi-K3, enabling token-level span splitting for user, assistant, system, and tool messages.
    • ›Routes Kimi-K3 model architecture on the top-level model name (not text_config's KimiLinearForCausalLM) via get_model_architecture.
    • ›CPU and CUDA backends support Kimi-K3's cross-layer residuals via ggml_dsv4_hc_pre; Metal and Vulkan fall back per-node until dedicated kernels are added.
  15. b10448 Aug 15, 2026 · issue 002

    llama.cpp b10448 adds Kimi-K3 model support with hybrid KDA+MLA attention, latent MoE, and MXFP4 lossless weight repack

    └──▷ GET THIS VERSION
    $ git clone --branch b10448 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10448
    • ›Adds common_chat_params_init_kimi_k3 chat format (PEG_NATIVE) with thinking_forced_open handling, reasoning extraction, response unwrapping, and typed tool-call parsing for Kimi-K3's XTML-ish tagged output format.
    • ›Increases LLAMA_MAX_EXPERTS from 512 to 1024 to accommodate Kimi-K3's latent MoE expert count.
    • ›Adds support for the Kimi-K3 text model architecture, featuring hybrid KDA (linear) + MLA (full) attention, cross-layer residual attention (attn_res_block_size), latent MoE (n_expert_latent), situ activation (replacing SwiGLU), MLA output gate, and full-rank KDA gate (ssm_g).
    • ›Adds kda_gate_lower_bound config key (set to -5.0 for Kimi-K3) controlling KDA gate activation swap — from -exp(A_log)*softplus(x) to lower_bound*sigmoid(exp(A_log)*x) — now correctly emitted by the model saver to survive save/load roundtrips.
    • ›Adds lossless MXFP4 weight repack for Kimi-K3's compressed-tensors 'mxfp4-pack-quantized' routed experts, repacking nibble positions without a bf16 dequantization round-trip; shared repack helper with DeepSeek-V4.
    +2 moreshow less
    • ›Adds per-role message delimiters for Kimi-K3 chat format, enabling token-level span splitting for user, assistant, system, and tool messages using the K3 tiktoken vocabulary.
    • ›Routes Kimi-K3 model architecture detection on the top-level model name rather than text_config's KimiLinearForCausalLM to correctly distinguish K3 from the older Kimi-Linear-48B architecture.
  16. b10444 Aug 15, 2026 · issue -004

    llama.cpp b10444 adds --models-dir support for automatically loading MTP assistant (draft) models.

    └──▷ GET THIS VERSION
    $ git clone --branch b10444 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10444
    • ›Extends --models-dir to automatically discover and load MTP assistant models using strict prefix matching, enabling speculative decoding setups without manually specifying draft model paths.
  17. b10444 Aug 15, 2026 · issue 002

    llama.cpp b10444 adds automatic MTP assistant model discovery via --models-dir

    └──▷ GET THIS VERSION
    $ git clone --branch b10444 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10444
    • ›Extends --models-dir to automatically detect and load MTP assistant models (files matching the mtp-*.gguf prefix) as companion models, similar to existing mmproj auto-discovery, preventing them from being incorrectly loaded as the main model.
  18. b10441 Aug 15, 2026 · issue -004

    llama.cpp b10441 unifies --mmap, --no-mmap, --mlock, and --direct-io into a single --load-mode flag.

    └──▷ GET THIS VERSION
    $ git clone --branch b10441 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10441
    • ›Replaces --mmap, --no-mmap, --mlock, and --direct-io flags with the unified --load-mode argument across scripts, examples, and documentation.
    └──▷ BREAKING ON UPGRADE
    • !The --mmap, --no-mmap, --mlock, and --direct-io flags are deprecated and replaced by --load-mode; existing scripts using those flags will trigger a warning and may break in future releases.
  19. b10441 Aug 15, 2026 · issue 002

    llama.cpp b10441 replaces deprecated --mmap/--no-mmap/--mlock/--direct-io flags with unified --load-mode argument

    └──▷ GET THIS VERSION
    $ git clone --branch b10441 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10441
    • ›Replaces deprecated --mmap, --no-mmap, --mlock, and --direct-io CLI flags with the unified --load-mode argument across scripts, examples, and documentation.
  20. b10437 Aug 15, 2026 · issue -004

    llama.cpp b10437 adds support for MiniMaxText01ForCausalLM and MiniMaxM1ForCausalLM models with Jinja chat templates.

    └──▷ GET THIS VERSION
    $ git clone --branch b10437 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10437
    • ›Adds support for MiniMaxText01ForCausalLM and MiniMaxM1ForCausalLM architectures, enabling local inference of MiniMax-Text-01 and MiniMax-M1 models.
    • ›Adds a Jinja chat template for MiniMax-M1 to enable structured conversation formatting with the new model family.
    • ›Adds converter support for MiniMaxM1ForCausalLM during GGUF conversion, treating it as equivalent to MiniMaxText01ForCausalLM.
  21. b10437 Aug 15, 2026 · issue 002

    llama.cpp b10437 adds inference support for MiniMax-Text-01 and MiniMax-M1 lightning-attention models

    └──▷ GET THIS VERSION
    $ git clone --branch b10437 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10437
    • ›Adds support for MiniMaxText01ForCausalLM and MiniMaxM1ForCausalLM model architectures, enabling inference for MiniMax-Text-01 and MiniMax-M1 lightning-attention hybrid recurrent models.
    • ›Adds Jinja chat template for MiniMax-M1, automatically applied during conversion (stored in models/templates/MiniMax-M1.jinja).
    • ›Adds conversion support for MiniMaxM1ForCausalLM in the GGUF converter, treating it as equivalent to MiniMaxText01ForCausalLM.
    └──▷ BREAKING ON UPGRADE
    • !GGUF files for MiniMax-Text-01 created with the previous (January 2025) implementation are incompatible with b10437 due to a model architecture rename.
  22. b10434 Aug 14, 2026 · issue -005

    llama.cpp b10434 adds reasoning_effort to chat templates and generation params, enabling model-specific thinking control via the OpenAI-compatible API.

    └──▷ GET THIS VERSION
    $ git clone --branch b10434 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10434
    • ›Adds reasoning_effort field to common_chat_templates_inputs, making OpenAI Chat Completions reasoning_effort available to Jinja templates with model-specific translations.
    • ›Adds reasoning_effort to generation params, positioned alongside enable_thinking for consistent reasoning control.
    • ›Server now reads reasoning_effort from the request body, with server_chat_convert_responses_to_chatcmpl handling conversion of Responses API reasoning.effort to reasoning_effort.
  23. b10434 Aug 14, 2026 · issue 002

    Adds --reasoning-effort CLI flag and LLAMA_ARG_REASONING_EFFORT env var to pass reasoning effort levels to chat templates

    └──▷ GET THIS VERSION
    $ git clone --branch b10434 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10434
    └──▷ TRY IT
    Run the CLI with a thinking-capable model and cap its reasoning to minimal effort to reduce token usage.
    $ llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF --reasoning-effort minimal
    Serve a reasoning model via the API server with maximum reasoning effort enabled by default for all completions.
    $ llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF --reasoning-effort max
    • ›Adds --reasoning-effort flag (with LLAMA_ARG_REASONING_EFFORT env var) accepting levels 'minimal', 'low', 'medium', 'high', 'xhigh', 'max', or 'default' to control reasoning effort passed to the chat template; available in server, completion, and CLI examples.
    • ›Exposes reasoning_effort as a field in common_chat_templates_inputs so Jinja chat templates (including model-specific translations) can consume it during generation.
  24. b10431 Aug 14, 2026 · issue 002

    Adds recurrent state rollback support for ggml_ssm_scan on CPU and CUDA, enabling speculative decoding with Nemotron/SSM models.

    └──▷ GET THIS VERSION
    $ git clone --branch b10431 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10431
    • ›Adds recurrent state rollback to ggml_ssm_scan for CPU and CUDA backends, enabling speculative decoding on SSM-based models (e.g. Nemotron) by saving per-draft-token state copies and rolling back to the last accepted token's state.
    • ›Extends ggml_ssm_scan to support K > 1 across all backends, with automatic CPU fallback when the fused SSM op is not supported by the backend.
  25. b10430 Aug 14, 2026 · issue -005

    llama.cpp b10430 adds support for virtual iGPU devices.

    └──▷ GET THIS VERSION
    $ git clone --branch b10430 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10430
    • ›Adds support for virtual iGPU devices, enabling selection and use of integrated GPU virtual device surfaces in the llama runtime.
  26. b10430 Aug 14, 2026 · issue 002

    llama.cpp b10430 enables virtual iGPU device support for multi-GPU setups like DGX Spark.

    └──▷ GET THIS VERSION
    $ git clone --branch b10430 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10430
    └──▷ TRY IT
    Run inference split across two virtual CUDA devices on a DGX Spark (or similar hardware with virtual iGPU devices) to utilize both virtual device slices.
    $ GGML_CUDA_DEVICES=2 ./bin/llama-completion -hf ggml-org/Qwen3-0.6B-GGUF:Q8_0 -p "I believe the meaning of life is" -n 32 --sampling-seq "k" --top-k 1 -no-cnv -lv 4
    • ›Supports multiple virtual iGPU devices of the same backend (e.g. CUDA) via the GGML_CUDA_DEVICES environment variable, enabling workloads across virtual CUDA devices on hardware like DGX Spark.
  27. b10429 Aug 14, 2026 · issue -005

    llama.cpp server now serves /metrics and /slots endpoints without blocking during active llama_decode() calls

    └──▷ GET THIS VERSION
    $ git clone --branch b10429 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10429
    • ›Enables concurrent access to /metrics and /slots endpoints while llama_decode() is executing, eliminating observability blind spots during active inference.
  28. b10429 Aug 14, 2026 · issue 002

    llama.cpp server now serves /metrics and /slots endpoints concurrently during active llama_decode() calls

    └──▷ GET THIS VERSION
    $ git clone --branch b10429 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10429
    • ›The llama-server /metrics and /slots endpoints are now accessible during active llama_decode() calls, eliminating request blocking while inference is running.
  29. b10427 Aug 14, 2026 · issue 002

    SYCL backend gains fused dense-FFN kernel for Q4_K models, boosting batched throughput up to +12% on Intel Arc GPUs.

    └──▷ GET THIS VERSION
    $ git clone --branch b10427 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10427
    • ›Fuses mul_mat(gate) + mul_mat(up) + GLU into a single q4_K reorder mat-vec kernel on the SYCL backend, delivering up to +12.4% token generation throughput at batch size 8 on Intel Arc Pro B70 (Battlemage) hardware.
  30. b10423 Aug 13, 2026 · issue -006

    llama.cpp b10423 applies CPU parameters consistently across all tools in the suite.

    └──▷ GET THIS VERSION
    $ git clone --branch b10423 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10423
    • ›Applies CPU parameters (such as thread counts and affinity settings) consistently across all llama.cpp tools, not just the main inference binary.
  31. b10423 Aug 13, 2026 · issue 002

    CPU parameters --cpu-mask, --cpu-range, and --prio now apply consistently across all llama.cpp tools

    └──▷ GET THIS VERSION
    $ git clone --branch b10423 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10423
    • ›Extends --cpu-mask, --cpu-range, and --prio CPU parameters to llama-cli, llama-server, and other tools — previously only available in llama-completion.
  32. b10419 Aug 13, 2026 · issue -006

    OpenVINO backend gains Qwen3.5 support, new env-var memory controls, and a frontend model cache cutting compile time ~2.4x.

    └──▷ GET THIS VERSION
    $ git clone --branch b10419 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10419
    └──▷ TRY IT
    Reduce steady-state GPU memory after model compile on Arc iGPU by releasing host weight pages once the device copy is ready.
    $ GGML_OPENVINO_RELEASE_WEIGHTS=1 ./llama-cli -m Llama-3.2-1B-Q4_K_M.gguf -p 'Hello' --n-gpu-layers 999
    Cut compile-time peak RAM by ~1-2 GB on large models by enabling streaming requantization and weight-node caching.
    $ GGML_OPENVINO_REDUCE_COMPILE_MEM=1 ./llama-cli -m Llama-3.1-8B-Q4_K_M.gguf -p 'Hello' --n-gpu-layers 999
    Skip requant + convert + compile on subsequent runs by caching the compiled OpenVINO model to disk, reducing startup from ~15 s to ~6 s for an 8B model.
    $ GGML_OPENVINO_MODEL_CACHE_DIR=/tmp/ov-cache ./llama-cli -m Llama-3.1-8B-Q4_K_M.gguf -p 'Hello' --n-gpu-layers 999
    • ›Adds GGML_OPENVINO_RELEASE_WEIGHTS env var to madvise(MADV_DONTNEED) host weight buffers after GPU compile, dropping steady-state RSS from ~1555 MB to ~710 MB for a 1B Q4_K_M model on Arc iGPU.
    • ›Adds GGML_OPENVINO_REDUCE_COMPILE_MEM env var to enable streaming weight requantization and cross-compile weight-node caching, cutting compile-time peak RSS by ~1 GB (1B model) to ~2 GB (8B model).
    • ›Adds GGML_OPENVINO_MODEL_CACHE_DIR env var to enable a frontend model cache that imports a previously compiled CompiledModel, reducing compile phase from 15.3 s to 6.3 s (~2.4x) on an 8B Q4_K_M GPU run.
    • ›Adds cache-key coverage for GGML_OPENVINO_DISABLE_KV_SLICE and GGML_OPENVINO_MANUAL_GQA_ATTN to prevent the frontend model cache from reusing blobs compiled under different attention-graph configurations.
    • ›Enables Qwen3.5 (including dense multi-sequence and MoE variants) on the OpenVINO backend.
    +7 moreshow less
    • ›Enables OpenVINO fallback to CPU backend when GPU ops are unsupported.
    • ›Adds GGML_OPENVINO_RELEASE_WEIGHTS hard-abort (GGML_ABORT) on cache-miss recompile or second model load to prevent silent wrong-model or zeroed-weight inference.
    • ›Optimizes MoE expert aggregation in the OpenVINO backend using ReduceSum.
    • ›Enables GGML_UNARY_OP_SIGMOID, SQR, SQRT, FILL, and GET_ROWS with multi-dimensional support in the OpenVINO backend.
    • ›Adds gpt-oss MoE and mxfp4 support to the OpenVINO backend.
    • ›Enables zero-size copy for view ops in the OpenVINO backend.
    • ›Adds per-node runtime offsets (rs_slot_begin_*, rs_src_begin_*) to the OpenVINO backend for correct recurrent state rollback CPY writeback across any kv head, sequence count, and snapshot slot.
  33. b10419 Aug 13, 2026 · issue 002

    OpenVINO backend gains Qwen3.5 Dense+MoE support, GPU memory optimizations, and three new opt-in env flags

    └──▷ GET THIS VERSION
    $ git clone --branch b10419 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10419
    └──▷ TRY IT
    Reclaim ~845 MB of host RAM after model weights are uploaded to an Intel GPU — useful when running a 1B+ model on an Arc iGPU with tight memory.
    $ GGML_OPENVINO_RELEASE_WEIGHTS=1 llama serve -hf ggml-org/Llama-3.2-1B-GGUF
    Cut the OpenVINO compile-time memory peak by ~1-2 GB on large models, making cold-start feasible on memory-constrained Intel GPU systems.
    $ GGML_OPENVINO_REDUCE_COMPILE_MEM=1 llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
    Skip full requantization and recompilation on repeated starts by caching the compiled OpenVINO model — ~2.4x faster subsequent launches for 8B models.
    $ GGML_OPENVINO_MODEL_CACHE_DIR=/tmp/ov-model-cache GGML_OPENVINO_REDUCE_COMPILE_MEM=1 GGML_OPENVINO_RELEASE_WEIGHTS=1 llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
    • ›Adds GGML_OPENVINO_RELEASE_WEIGHTS environment variable to reclaim host RSS after GPU model compilation — measured ~845 MB reduction on Llama-3.2-1B-Q4_K_M on Arc iGPU.
    • ›Adds GGML_OPENVINO_REDUCE_COMPILE_MEM environment variable to enable streaming weight requantization and a cross-compile weight-node cache, cutting compile-time peak RSS by ~1 GB (1B model) to ~2 GB (8B model).
    • ›Adds GGML_OPENVINO_MODEL_CACHE_DIR environment variable for a frontend model cache that skips requantization, graph conversion, and compilation on warm starts — measured ~2.4x faster compile phase on an 8B Q4_K_M model on GPU.
    • ›Adds GGML_OPENVINO_DISABLE_KV_SLICE and GGML_OPENVINO_MANUAL_GQA_ATTN environment variables included in the frontend model cache key for correctness across different OpenVINO backend configurations.
    • ›Enables Qwen3.5 Dense and MoE model inference on the OpenVINO CPU and GPU backends.
    +2 moreshow less
    • ›Adds support for the GGML_UNARY_OP_SIGMOID, SQR, SQRT, and FILL ops in the OpenVINO backend.
    • ›Enables CPU backend fallback in the OpenVINO backend when an op is unsupported on the primary device.
  34. b10418 Aug 13, 2026 · issue -006

    llama.cpp b10418 adds SYCL host pinned memory support via ggml_backend_sycl_host_buffer_type_get_max_size to improve Host-to-Device memory access.

    └──▷ GET THIS VERSION
    $ git clone --branch b10418 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10418
    • ›Adds ggml_backend_sycl_host_buffer_type_get_max_size to support host pinned memory on SYCL backends, improving Host-to-Device memory access performance.
  35. b10418 Aug 13, 2026 · issue 002

    SYCL backend gains host pinned memory support via GGML_SYCL_ENABLE_HOST_PINNED_MEM, boosting Host-to-Device throughput by up to 13.5%.

    └──▷ GET THIS VERSION
    $ git clone --branch b10418 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10418
    └──▷ TRY IT
    Enable host pinned memory on a SYCL-backed system to improve Host-to-Device throughput, especially for large quantized models.
    $ GGML_SYCL_ENABLE_HOST_PINNED_MEM=1 llama-cli -m model.gguf -p 'Hello, world'
    • ›Adds GGML_SYCL_ENABLE_HOST_PINNED_MEM environment variable to enable host pinned memory for the SYCL backend, improving Host-to-Device memory access throughput — benchmarks show up to +13.5% prompt-processing speed on Intel Arc B70 with IQ2_XXS quantized models.
  36. b10415 Aug 13, 2026 · issue -006

    llama.cpp b10415 adds auto-detection of MTP draft model type for speculative decoding.

    └──▷ GET THIS VERSION
    $ git clone --branch b10415 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10415
    • ›Adds auto-detection of the MTP draft model type in the speculative decoding spec, removing the need to manually identify the draft model.
  37. b10415 Aug 13, 2026 · issue 002

    llama.cpp b10415 adds auto-detection of MTP draft model type by checking for the NextN tensor.

    └──▷ GET THIS VERSION
    $ git clone --branch b10415 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10415
    • ›Adds auto-detection of the draft-mtp model type by inspecting the presence of the NextN tensor in the draft model, eliminating the need to manually specify the speculative decoding mode.
  38. b10414 Aug 13, 2026 · issue -006

    llama.cpp b10414 adds TQ2_0 ternary quantization support to the Metal backend for Apple Silicon.

    └──▷ GET THIS VERSION
    $ git clone --branch b10414 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10414
    • ›Adds GGML_TYPE_TQ2_0 (ternary, 2 bits per element) quantization support to the Metal backend, enabling GPU-accelerated inference with TQ2_0 models on Apple Silicon.
  39. b10414 Aug 13, 2026 · issue 002

    llama.cpp b10414 adds TQ2_0 (ternary, 2-bit) quantization support to the Metal backend for Apple Silicon

    └──▷ GET THIS VERSION
    $ git clone --branch b10414 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10414
    • ›Adds GGML_TYPE_TQ2_0 (ternary, 2 bits per element) support to the Metal backend, enabling GPU-accelerated inference of TQ2_0-quantized models on Apple Silicon.
  40. b10413 Aug 13, 2026 · issue -006

    llama.cpp b10413 auto-detects speculative decoding type from local draft GGUF metadata, fixing silent no-op when using -md without --spec-type.

    └──▷ GET THIS VERSION
    $ git clone --branch b10413 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10413
    • ›Adds common_speculative_types_from_gguf() in speculative.cpp/.h to auto-detect --spec-type from a local draft model's general.architecture GGUF header when -md is used without --spec-type, mapping dflash + markov_w1.weight tensor to draft-dspark and dflash alone to draft-dflash.
    • ›Emits a LOG_INF message when spec-type auto-detection fires, giving users visibility into why speculative decoding was enabled.
  41. b10413 Aug 13, 2026 · issue 002

    llama.cpp b10413 auto-detects speculative decoding type from local draft GGUF metadata, eliminating the need for explicit --spec-type.

    └──▷ GET THIS VERSION
    $ git clone --branch b10413 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10413
    • ›Adds common_speculative_types_from_gguf() in speculative.cpp/.h to read general.architecture from a local draft GGUF header and automatically map it to draft-dspark (dflash + markov_w1.weight tensor) or draft-dflash (dflash without markov head), activating speculative decoding when -md loads a local draft model without --spec-type.
  42. b10408 Aug 13, 2026 · issue -006

    llama.cpp b10408 adds SYCL ESIMD kernels for Q3_K, Q4_K, and Q6_K quantized inference on Intel GPUs

    └──▷ GET THIS VERSION
    $ git clone --branch b10408 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10408
    • ›Adds DMMV ESIMD kernels for Q3_K, Q4_K, and Q6_K quantization formats under the SYCL backend, enabled at build time with -DGGML_SYCL_ESIMD=ON and used by default at runtime when available on the device.
  43. b10408 Aug 13, 2026 · issue 002

    SYCL backend gains ESIMD kernels for Q3_K, Q4_K, Q6_K quantizations and concat support for Q4_0–Q8_0.

    └──▷ GET THIS VERSION
    $ git clone --branch b10408 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10408
    • ›Adds SYCL ESIMD DMMV kernels for Q3_K, Q4_K, and Q6_K quantization types, enabled automatically at runtime when available on Intel GPU hardware.
    • ›Extends SYCL OP_CONCAT to support Q4_0, Q4_1, Q5_0, Q5_1, and Q8_0 quantized tensor concatenation.
  44. b10400 Aug 13, 2026 · issue 002

    llama.cpp b10400 adds system-level config files, a UI read_media tool, slot save/restore with media, and Vulkan TQ2_0 support.

    └──▷ GET THIS VERSION
    $ git clone --branch b10400 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10400
    • ›Adds system-level config file support at /etc/llama.cpp/config.ini (Linux/macOS), %PROGRAMDATA%\llama.cpp\config.ini (Windows), and ${XDG_CONFIG_HOME:-~/.config}/llama.cpp/config.ini (user-level) — later files override earlier ones, enabling org-wide or per-user llama.cpp defaults without touching command-line flags.
    • ›Adds read_media tool to the built-in UI, enabling multimodal media input directly from the web interface.
    • ›Adds slot save/restore support for sessions that include media inputs in llama-server.
    • ›Adds TQ2_0 (ternary quantization) support to the Vulkan backend.
    • ›Adds a warp-per-row WKV7 CUDA kernel for single-token decode on head-size-64 RWKV7 models, improving throughput at batch size 1.
    +4 moreshow less
    • ›Adds KleidiAI runtime feature detection for aarch64 — enables dynamic dispatch to optimized routines without a separate build.
    • ›Adds per_layer_config handling in the Gemma4 model converter for compatibility with transformers 5.15.
    • ›Hardens the GGUF loader against malformed tensor dimensions and metadata types.
    • ›Refines CUDA graph disabling logic so graphs remain active for quantized and bf16/f16 MoE models that use the sync-free MMQ/MMF path, improving parallel decode performance.
  45. b10369 Aug 12, 2026 · issue -007

    llama.cpp b10369 adds pocket-tts support with 80% faster CUDA decoding and new mmproj audio config keys

    └──▷ GET THIS VERSION
    $ git clone --branch b10369 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10369
    • ›Adds clip.gen.audio.frames_after_eos and clip.gen.audio.pad_short_text keys to mmproj files, letting language packs tune end-of-speech padding and short-prompt padding per pack (e.g. french_24l requests 8 tail frames, english_2026-01 pads short prompts with spaces).
    • ›Adds clip.gen.audio.model_variant key to mmproj files for per-pack model variant selection.
    • ›Adds mtmd_gen_inp_default API function for pocket-tts input generation.
    • ›Adds pocket-tts (PocketTTS) model support to the mtmd multimodal subsystem, enabling on-device TTS inference.
    • ›Replaces grouped transposed convolutions in the Seanet decoder with a GEMM + col2im_1d formulation, cutting generation time per frame by 80% on CUDA and 50% on CPU.
    └──▷ BREAKING ON UPGRADE
    • !Existing mmproj files must be re-converted to carry the new clip.gen.audio.frames_after_eos and clip.gen.audio.pad_short_text keys; mmproj files without them fall back to previous defaults.
  46. b10369 Aug 12, 2026 · issue 002

    llama.cpp b10369 adds pocket-tts TTS support and a new auto default for --load-mode that avoids mmap on iGPUs.

    └──▷ GET THIS VERSION
    $ git clone --branch b10369 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10369
    └──▷ TRY IT
    Load a model on a system with an iGPU without triggering double-RAM usage — auto is now the default but can be set explicitly.
    $ llama serve -m model.gguf --load-mode auto
    • ›Adds auto value to the --load-mode / -lm flag (now the default), which automatically disables mmap on iGPUs (CUDA/ROCm, Vulkan) to avoid doubling RAM usage during model load.
    • ›Adds pocket-tts support to the multimodal (mtmd) subsystem, enabling local text-to-speech generation via llama.cpp.
    • ›Adds clip.gen.audio.frames_after_eos and clip.gen.audio.pad_short_text keys to mmproj files, carrying per-language-pack TTS tuning (tail-frame padding and short-prompt padding); existing mmproj files must be reconverted to carry these keys.
    • ›Adds clip.gen.audio.model_variant key to mmproj files to identify the pocket-tts model variant.
    • ›Adds mtmd_gen_inp_default to the mtmd API for default TTS input generation.
    +1 moreshow less
    • ›Implements pocket-tts transposed convolutions as GEMM + col2im_1d, reducing TTS generation time per frame by 80% on CUDA and 50% on CPU.
    └──▷ BREAKING ON UPGRADE
    • !The default value of --load-mode changes from mmap to auto; setups relying on mmap behavior by default on iGPU systems will now skip mmap automatically.
    • !Existing mmproj files for pocket-tts must be reconverted to include the new clip.gen.audio.frames_after_eos and clip.gen.audio.pad_short_text keys.
  47. b10355 Aug 10, 2026 · issue -009

    llama.cpp b10355 enables multi-output backend sampling with token speculation support across CPU and GPU backends.

    └──▷ GET THIS VERSION
    $ git clone --branch b10355 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10355
    • ›Adds multi-output backend sampling, enabling GPU-accelerated sampling across multiple output sequences simultaneously, including support for token speculation workflows.
    • ›Introduces a numeric context parameter declaring the maximum outputs per sequence, allowing fine-grained control over multi-output sampling capacity.
  48. b10355 Aug 10, 2026 · issue 002

    llama.cpp b10355 adds multi-output backend sampling, enabling GPU-accelerated speculative decoding with ~8% throughput gain.

    └──▷ GET THIS VERSION
    $ git clone --branch b10355 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10355
    • ›Adds n_sampling_outputs_per_seq_max to context parameters, separating the total output budget from the per-sequence backend-sampling limit to support multiple sampler graph rows per output.
    • ›Enables backend sampling during speculative decoding (token speculation), extending GPU-accelerated sampling from one output per sequence to multiple outputs.
    • ›Makes distribution sampling deterministic between CPU and backend paths by using a temporary RNG for multi-output sampling so rejected speculative tokens do not advance the RNG state.
  49. b10353 Aug 10, 2026 · issue 002

    Adds podman and ssh tool isolation runtimes to --tools-runtime, plus Muse Glimmer model support and mmproj quantization restored

    └──▷ GET THIS VERSION
    $ git clone --branch b10353 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10353
    └──▷ TRY IT
    Run llama-server tool calls in a rootless Podman container to isolate executed code from the host environment
    $ llama-server -m model.gguf --tools-runtime podman:python:3.12-slim
    Run llama-server tool calls on a remote host over SSH instead of locally, keeping the server machine clean
    $ llama-server -m model.gguf --tools-runtime ssh:user@remote-host
    • ›Adds podman:<image> and podman-container:<id> options to the --tools-runtime flag, enabling tool isolation in rootless Podman containers alongside the existing Docker options
    • ›Adds ssh:<target> option to --tools-runtime for running tools on a remote POSIX host over SSH (requires key-based auth and a trusted host key)
    • ›Adds support for the Muse Glimmer model architecture
    • ›Restores quantization support for mmproj (multimodal projector) GGUF files, which was inadvertently broken in a prior refactor
    • ›Adds Windows ARM64 CUDA 13.4 pre-built binary packages, enabling GPU-accelerated inference on ARM64 Windows devices (e.g. RTX Spark)
  50. b10344 Aug 10, 2026 · issue -009

    llama.cpp b10344 adds Multi-Token Prediction (MTP) support for the Nemotron Nano model.

    └──▷ GET THIS VERSION
    $ git clone --branch b10344 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10344
    • ›Adds mtp_flags configuration and Multi-Token Prediction (MTP) support for the Nemotron Nano model, enabling speculative decoding workflows with this architecture.
  51. b10344 Aug 10, 2026 · issue 002

    llama.cpp b10344 adds Multi-Token Prediction (MTP) support for the Nemotron Nano model.

    └──▷ GET THIS VERSION
    $ git clone --branch b10344 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10344
    • ›Adds Multi-Token Prediction (MTP) support for the Nemotron Nano model, including mtp_flags configuration for Nemotron-specific MTP behaviour.
  52. b10342 Aug 10, 2026 · issue -009

    llama.cpp b10342 adds the Granite-Switch architecture: a dense Granite-4.1 model with per-token LoRA adapter switching via in-graph router attention.

    └──▷ GET THIS VERSION
    $ git clone --branch b10342 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10342
    • ›Adds graniteswitch arch registration (in llama-arch.h/cpp, llama-model.h/cpp) and src/models/granite_switch.cpp implementing per-token switched-LoRA inference via ggml_mul_mat_id over stacked lora_a/lora_b tensors.
    • ›Adds conversion/granite.py GraniteSwitchModel converter that stacks N LoRA adapters plus a zero base slot into per-projection A/B tensors and emits switch metadata into GGUF.
    • ›Introduces llm_graph_input_switch in granite_switch.cpp providing stateless per-token adapter-index resolution via a single-head causal router attention layer (hparams.router_layer) stored in the KV cache, isolating concurrent requests automatically.
    • ›Adds hparams.has_rope(il) predicate to skip k-shift RoPE on the router layer, preventing corruption of the literal-magnitude router K signal.
    • ›Granite-Switch conversion honors --outtype for LoRA tensor precision instead of forcing F16.
    +1 moreshow less
    • ›Validates substitute (adapter) token IDs against n_vocab and bounds adapter count and LoRA rank from GGUF at load time, throwing on bad metadata.
  53. b10342 Aug 10, 2026 · issue 002

    Adds graniteswitch architecture support for IBM Granite-Switch 4.1 models with per-token switched LoRA adapters

    └──▷ GET THIS VERSION
    $ git clone --branch b10342 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10342
    • ›Adds graniteswitch architecture registration, enabling llama.cpp to load and run IBM Granite-Switch 4.1 models (dense all-attention models with N embedded LoRA adapters selected per-token via control tokens); runs on CPU and Metal (Apple Silicon).
    • ›Adds conversion/granite.py GraniteSwitchModel converter that stacks N adapters plus a zero base slot into per-projection .lora_a/.lora_b tensors and emits switch metadata into GGUF.
    • ›Adds GGUF schema extensions for the graniteswitch arch including KV keys under the {arch}.adapters.* namespace and router_gain metadata key (read from GGUF instead of hardcoded), with stacked LoRA tensor names using .lora_a/.lora_b suffixes.
    • ›Implements an in-graph router attention mechanism (stored in the model KV cache at layer index router_layer) that isolates adapter selection per sequence, enabling safe concurrent multi-request inference without adapter-index leakage.
    • ›Adds llm_graph_input_switch input class in src/models/granite_switch.cpp providing stateless per-token adapter-index mapping with adapter_token_* control token substitution and validation of substitute token IDs against n_vocab.
    +2 moreshow less
    • ›Adds hparams.has_rope(il) predicate to skip KV-shift RoPE for the router layer, preventing corruption of the literal magnitude stored in router K dim-0.
    • ›Supports all three preview Granite-Switch model sizes in the converter and architecture handler.
  54. b10329 Aug 8, 2026 · issue -011

    llama.cpp server tools now declare working-directory support, so the WebUI /cwd control only appears when a relevant tool is active.

    └──▷ GET THIS VERSION
    $ git clone --branch b10329 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10329
    • ›Built-in tools now publish a working-directory flag in the /tools listing, indicating whether they resolve paths against the working directory — the WebUI shows the working-directory chip and enables the /cwd command only when at least one such tool is both served and enabled.
  55. b10328 Aug 8, 2026 · issue -011

    llama.cpp server gains Docker-backed tool isolation via x-tool-runtime for sandboxed LLM tool execution.

    └──▷ GET THIS VERSION
    $ git clone --branch b10328 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10328
    • ›Adds initial tool isolation support to the server via Docker, separating tool execution into tools_io_sandbox and tools_io_docker modes, exposed through the x-tool-runtime request header.
  56. b10328 Aug 8, 2026 · issue 002

    llama-server gains --tools-runtime flag to sandbox LLM tool calls inside Docker containers

    └──▷ GET THIS VERSION
    $ git clone --branch b10328 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10328
    └──▷ TRY IT
    Run an agentic server session where all tool calls execute inside a fresh Ubuntu container, keeping the host environment untouched.
    $ llama-server --agent --tools-runtime docker:ubuntu:jammy
    Reuse a pre-existing Docker container for tool isolation — useful when you want to pre-stage dependencies or persist state across requests.
    $ llama-server --agent --tools-runtime docker-container:<id>
    • ›Adds --tools-runtime flag (env: LLAMA_ARG_TOOLS_RUNTIME) to llama-server to run tool calls in an isolated runtime environment; supports docker:<image> (spins up a fresh container, cleaned up on server exit) and docker-container:<id> (reuses an existing container by ID, not stopped on exit) as values.
  57. b10318 Aug 7, 2026 · issue 002

    WebUI gains ?model=xxx&load=true URL parameter to pre-load models on page open; npm supply-chain hardening added.

    └──▷ GET THIS VERSION
    $ git clone --branch b10318 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10318
    └──▷ TRY IT
    Pre-load a specific model as soon as the WebUI page opens, so inference is ready before the first message is sent.
    $ http://localhost:8080/?model=my-model.gguf&load=true
    • ›Adds ?load=true URL parameter to the WebUI so that a model specified via ?model= is loaded immediately when the page opens, rather than waiting for the first message.
    • ›Sets npm min-release-age in the WebUI build to reduce exposure to supply-chain attacks by avoiding newly-published packages.
  58. b10313 Aug 7, 2026 · issue -012

    llama.cpp server gains an LRU scheduler for smarter request routing across slots

    └──▷ GET THIS VERSION
    $ git clone --branch b10313 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10313
    • ›Adds lru_sched LRU (Least Recently Used) scheduler to the server router, improving slot assignment for concurrent inference requests.
  59. b10313 Aug 7, 2026 · issue 002

    llama.cpp server gains an LRU scheduler for smarter multi-model request queuing and eviction.

    └──▷ GET THIS VERSION
    $ git clone --branch b10313 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10313
    • ›Adds an LRU scheduler to the llama-server router, queuing eviction requests when server capacity is exhausted and unblocking all waiting requests for the same model simultaneously once it is loaded.
    • ›Disconnected requests now automatically remove themselves from the LRU queue, preventing stale waiters from holding slots.
  60. b10305 Aug 7, 2026 · issue -012

    SYCL backend gains DSv4 operator support: LIGHTNING_INDEXER, DSV4_HC_COMB, DSV4_HC_POST, DSV4_HC_PRE

    └──▷ GET THIS VERSION
    $ git clone --branch b10305 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10305
    • ›Adds SYCL backend support for four DSv4 operations: LIGHTNING_INDEXER, DSV4_HC_COMB, DSV4_HC_POST, and DSV4_HC_PRE, expanding hardware-accelerated inference coverage on Intel SYCL devices.
  61. b10305 Aug 7, 2026 · issue 002

    SYCL backend gains four DSv4 ops — LIGHTNING_INDEXER, DSV4_HC_COMB, DSV4_HC_POST, DSV4_HC_PRE — paving the way for DeepSeek-V4-Flash inference on Intel GPUs.

    └──▷ GET THIS VERSION
    $ git clone --branch b10305 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10305
    • ›Adds SYCL backend support for four DSv4 compute ops: LIGHTNING_INDEXER, DSV4_HC_COMB, DSV4_HC_POST, and DSV4_HC_PRE, enabling models that depend on these ops (including DeepSeek-V4-Flash) to run on Intel GPUs via the SYCL path.
  62. b10298 Aug 6, 2026 · issue 002

    llama.cpp b10298 adds save/load API for multimodal input chunks in the mtmd subsystem.

    └──▷ GET THIS VERSION
    $ git clone --branch b10298 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10298
    • ›Adds chunk save/load functions to the mtmd multimodal subsystem, enabling serialization and deserialization of mtmd_input_chunk objects.
  63. b10293 Aug 6, 2026 · issue 002

    Adds --model-name flag to model conversion scripts and onboards AMD ROCm CI with gfx1151 (Strix Halo) support.

    └──▷ GET THIS VERSION
    $ git clone --branch b10293 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10293
    • ›Adds --model-name flag to the causal and embedding model conversion scripts to explicitly set the general.name metadata field, overriding the default (basename of the model path).
    • ›Onboards AMD ROCm CI pipeline with gfx1151 (Strix Halo / integrated RDNA3.5) support, enabling automated testing of the HIP backend on that architecture.
  64. b10285 Aug 5, 2026 · issue 002

    llama.cpp b10285 adds multi-row batching support for DeepSeek-OCR in the multimodal subsystem.

    └──▷ GET THIS VERSION
    $ git clone --branch b10285 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10285
    • ›Adds multi-row batching for DeepSeek-OCR in the mtmd multimodal subsystem, grouping same-size rows into a single batch pass to improve throughput (overview images with differing sizes are excluded from batching).
  65. b10282 Aug 5, 2026 · issue -014

    llama.cpp b10282 adds speculative-decoding counters to the /metrics endpoint, aligned with vLLM naming.

    └──▷ GET THIS VERSION
    $ git clone --branch b10282 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10282
    └──▷ TRY IT
    Scrape speculative-decoding acceptance stats from a running llama.cpp server to feed into a Prometheus/Grafana monitoring stack.
    $ curl http://localhost:8080/metrics | grep spec
    • ›Adds spec-decode counters to the /metrics endpoint on the llama.cpp server, with parameter names aligned to vLLM conventions for drop-in compatibility.
  66. b10282 Aug 5, 2026 · issue 002

    llama.cpp server gains speculative-decoding Prometheus counters on /metrics, aligned with vLLM schema

    └──▷ GET THIS VERSION
    $ git clone --branch b10282 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10282
    └──▷ TRY IT
    Scrape speculative-decoding efficiency metrics from a running llama-server to measure draft acceptance rates in Prometheus.
    $ curl http://localhost:8080/metrics | grep llamacpp:spec_decode
    • ›Adds four speculative-decoding counters to the /metrics Prometheus endpoint: llamacpp:spec_decode_num_draft_tokens, llamacpp:spec_decode_num_accepted_tokens, llamacpp:spec_decode_num_drafts, and llamacpp:spec_decode_num_accepted_tokens_per_pos{position="N"} (per-draft-position labeled counter, absent until the first completed speculative request), aligned with the vLLM counter schema.
    • ›Extends gguf_convert_endian.py to support endianness conversion for Q1 and TQ2 quantization formats (swapping only the fp16 values, since sub-byte quantized values require no byte-swap).
  67. b10280 Aug 5, 2026 · issue 002

    llama.cpp b10280 enables generation statistics by default in the built-in chat UI

    └──▷ GET THIS VERSION
    $ git clone --branch b10280 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10280
    • ›Built-in web UI now shows generation statistics by default in chat settings.
  68. b10271 Aug 4, 2026 · issue -015

    llama.cpp b10271 adds per-conversation working directory picker with path navigation to the server UI agent.

    └──▷ GET THIS VERSION
    $ git clone --branch b10271 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10271
    • ›Adds per-conversation working directory picker to the server UI, supporting path navigation via POSIX (/, ~), Windows drive roots (C:), and UNC shares (//host/share) in addition to glob-matching.
    • ›Extends file_glob_search on the server with hardened directory listing including kind enum, timeout, symlink guard, and absolute base enforcement.
    • ›Persists per-call tool cwd on tool result messages in the database, so working directory context survives across conversation turns.
    • ›Adds get_info tool call block rendering in the UI.
  69. b10270 Aug 4, 2026 · issue -015

    llama.cpp b10270 adds Qwen3-TTS support via a revamped llama-tts binary and new mtmd_helper_gen_audio API.

    └──▷ GET THIS VERSION
    $ git clone --branch b10270 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10270
    • ›Adds mtmd_helper_gen_audio API for multimodal audio generation, enabling programmatic TTS workflows.
    • ›Adds llama_model_get_tok_embd function to the llama library API.
    • ›Adds mtmd_helper_model_can_chat helper to query whether a loaded model supports chat-style interaction.
    • ›Adds step_prompt() method to the mtmd generation API for incremental prompt stepping.
    • ›Supports Qwen3-TTS model in the mtmd multimodal framework, including text model, speaker encoder, code predictor, and GEN_WAV (audio generation) subgraph.
    +2 moreshow less
    • ›Revamps llama-tts binary with updated CLI interface and Qwen3-TTS support, including voice clone demo capability.
    • ›Adds suppress_tokens support to the TTS backbone vocabulary pipeline.
    └──▷ BREAKING ON UPGRADE
    • !The llama-tts binary interface has breaking changes as part of the Qwen3-TTS integration revamp — existing invocations will need to be updated.
  70. b10255 Aug 4, 2026 · issue -015

    llama.cpp b10255 extends SYCL oneDNN SDPA to Q4_0–Q8_0 and FP32 KV caches on Intel hardware.

    └──▷ GET THIS VERSION
    $ git clone --branch b10255 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10255
    • ›Extends the oneDNN Scaled Dot-Product Attention (SDPA) path on SYCL to support quantized KV cache types Q4_0, Q4_1, Q5_0, Q5_1, Q8_0, and F32 in addition to the existing F16 path, by dequantizing K/V to dense FP16 on-device before the fused systolic kernel runs.
    • ›Non-F16 KV cache SDPA is gated to prefill scenarios only: requires sequence length K >= 1024 and query count Q >= 32; F16 KV continues to run at any length.
  71. b10254 Aug 4, 2026 · issue -015

    llama.cpp b10254 adds a DeepSeek V4 Flash 0731 chat template with reasoning-effort mapping and structured output support.

    └──▷ GET THIS VERSION
    $ git clone --branch b10254 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10254
    • ›Adds a separate chat template for DeepSeek-V4-Flash-0731 with updated high and max reasoning-effort mappings.
    • ›Adds structured output response-format instructions to DeepSeek V4 templates, passing the schema into template rendering.
    • ›Defaults drop_thinking for DeepSeek V4 conversation history so prior reasoning is omitted unless preserve_reasoning is requested or tools are present.
  72. b10251 Aug 4, 2026 · issue -015

    llama.cpp b10251 adds Multi-Token Prediction (MTP) support for the GLM-4.7-Flash model.

    └──▷ GET THIS VERSION
    $ git clone --branch b10251 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10251
    • ›Adds Multi-Token Prediction (MTP) support for the GLM-4.7-Flash model.
  73. b10249 Aug 4, 2026 · issue -015

    llama.cpp server gains a get_info tool for probing host OS and environment diagnostics.

    └──▷ GET THIS VERSION
    $ git clone --branch b10249 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10249
    • ›Adds get_info tool to the server, exposing host OS identity and environment diagnostics via a structured probe mechanism.
  74. b10242 Aug 3, 2026 · issue -016

    llama.cpp b10242 adds GPU-accelerated CUDA backend sampler for penalty handling including frequency, presence, and top-k penalties.

    └──▷ GET THIS VERSION
    $ git clone --branch b10242 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10242
    • ›Adds CUDA backend sampler for llama_sampler_penalties, offloading frequency, presence, and repeat penalty computation to GPU via the llama_sampler_backend infrastructure.
    • ›Adds llama_n_ctx parameter to common_sampler_init so penalty_last_n defaults to the model context length when not explicitly set.
    • ›Adds support for top-k penalties in backend sampling, with configurable positions in the sampler chain.
    • ›Adds vocabulary-sized count tensor to replace the per-candidate penalty comparison matrix, improving scalability for large history windows.
    • ›Adds validation that repeat penalty is finite and greater than 0, and preserves masked logits as -Inf to eliminate NaN generation.
    └──▷ BREAKING ON UPGRADE
    • !The signature of common_sampler_init now requires an additional llama_n_ctx parameter; callers that do not pass it will fail to compile.
  75. b10238 Aug 3, 2026 · issue -016

    llama.cpp b10238 adds Multi-Token Prediction (MTP) support for Qwen3-Next models.

    └──▷ GET THIS VERSION
    $ git clone --branch b10238 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10238
    • ›Adds Multi-Token Prediction (MTP) support for Qwen3-Next, including load_mtp flags and opt_num_mtp_layers defined in the model mixin.
  76. b10237 Aug 3, 2026 · issue -016

    llama.cpp b10237 adds Multi-Token Prediction (MTP) support for DeepSeek V3.2 models.

    └──▷ GET THIS VERSION
    $ git clone --branch b10237 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10237
    • ›Adds Multi-Token Prediction (MTP) support for DeepSeek V3.2, enabling faster speculative decoding with that model family.
  77. b10232 Aug 2, 2026 · issue -017

    llama.cpp b10232 adds Metal GPU kernels for DeepSeek V4 hyper-connections via three new GGML ops.

    └──▷ GET THIS VERSION
    $ git clone --branch b10232 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10232
    • ›Implements GGML_OP_DSV4_HC_COMB, GGML_OP_DSV4_HC_PRE, and GGML_OP_DSV4_HC_POST Metal kernels for DeepSeek V4 hyper-connection layers, using SIMDgroup register and shuffle optimizations for GPU-accelerated inference on Apple Silicon.
  78. b10231 Aug 2, 2026 · issue -017

    llama.cpp b10231 adds DSpark speculative sidecar support with Markov-head auto-selection priority over DFlash.

    └──▷ GET THIS VERSION
    $ git clone --branch b10231 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10231
    • ›Adds DSpark speculative sidecar resolution to common, matching the existing sidecar discovery rules: the -hfd tag applies, a requested sidecar resolves without a full model at the tag, and an explicit -md selection disables discovery.
    • ›DSpark auto-ranks above DFlash in sidecar auto-selection when no type is explicitly requested, because the DSpark sidecar carries an extra Markov head.
  79. b10227 Aug 2, 2026 · issue -017

    llama.cpp b10227 adds a specialized Qwen3 tool-call chat parser with tagged thinking and <tool_call> omission support.

    └──▷ GET THIS VERSION
    $ git clone --branch b10227 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10227
    • ›Adds a specialized Qwen3 chat parser that handles tagged thinking blocks and supports optional <tool_call> omission, enabling cleaner agentic tool-use workflows with Qwen3 and Qwen3-Coder models.
  80. b10219 Aug 1, 2026 · issue -018

    llama-cli now persists reasoning_content in chat history so --reasoning-preserve can re-inject prior thoughts on later turns.

    └──▷ GET THIS VERSION
    $ git clone --branch b10219 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10219
    • ›Stores reasoning_content in the chat message history within llama-cli, enabling --reasoning-preserve to re-inject prior model thoughts on subsequent conversation turns.
  81. b10218 Aug 1, 2026 · issue -018

    llama.cpp b10218 adds MiniCPM-V 4.6 multimodal downsample support via the mtmd image preprocessor.

    └──▷ GET THIS VERSION
    $ git clone --branch b10218 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10218
    • ›Adds MiniCPM-V 4.6 downsample mode to the mtmd multimodal image preprocessor, enabling inference with the MiniCPM-V 4.6 vision model via mtmd_image_preprocessor_llava_uhd.
    • ›Stores the downsample mode inside the GGUF file metadata, allowing model files to self-describe their preprocessing requirements.
    • ›Adds a 4x ignore ViT merger path for MiniCPM-V 4.6 image tiling.
  82. b10217 Aug 1, 2026 · issue -018

    llama.cpp b10217 enables tool call support during thinking mode for DeepSeek v4 (DS4) models.

    └──▷ GET THIS VERSION
    $ git clone --branch b10217 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10217
    • ›Enables tool call execution within the thinking/reasoning phase for DeepSeek v4 (DS4) models in chat mode.
  83. b10216 Jul 31, 2026 · issue -019

    llama.cpp b10216 adds full GGML_OP_POOL_1D support to the Vulkan backend via a new compute shader.

    └──▷ GET THIS VERSION
    $ git clone --branch b10216 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10216
    • ›Adds GGML_OP_POOL_1D support to the Vulkan backend, including vk_op_pool1d_push_constants, pipeline_pool1d_f32, and the pool1d.comp compute shader, enabling 1D pooling operations on Vulkan-accelerated hardware.
  84. b10208 Jul 31, 2026 · issue -019

    SYCL backend gains oneMKL GEMM flash attention for XMX-accelerated prompt processing, delivering up to 1.97x throughput on Intel Battlemage GPUs.

    └──▷ GET THIS VERSION
    $ git clone --branch b10208 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10208
    • ›Adds GGML_SYCL_ENABLE_MKL_FA environment variable to enable/disable oneMKL GEMM flash attention on the SYCL backend (set to 0 to disable); activates automatically when flash-attn is on and Q tokens >= 32.
    • ›Adds GGML_SYCL_MKL_FA_DEBUG environment variable to enable an FA watchdog with timing diagnostics for the MKL flash attention path.
    • ›Adds GGML_SYCL_MKL_FA_DIAG environment variable to emit FA-DIAG output fingerprints for A/B testing the MKL vs. TILE attention paths.
    • ›New oneMKL GEMM flash attention kernel accelerates multi-token prompt processing via XMX hardware on Intel Arc/Battlemage GPUs, supporting all KV cache types (F16, BF16, F32, and quantized) with head dimensions 64–512 (multiples of 64) and GQA >= 2.
    • ›XMX-accelerated path delivers up to 1.97x prompt throughput over the TILE path: Gemma-4-26B reaches ~1473 t/s vs. ~746 t/s, Qwen3.6-27B reaches ~609 t/s vs. ~330 t/s, on Intel B70/Battlemage at 32K context with q8_0 KV cache.
  85. b10204 Jul 31, 2026 · issue -019

    SYCL backend gains device-to-device memcpy support via DEV2DEV_MEMCPY_FORWARD for Intel GPU acceleration.

    └──▷ GET THIS VERSION
    $ git clone --branch b10204 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10204
    • ›Adds DEV2DEV_MEMCPY_FORWARD support to the SYCL backend, enabling direct device-to-device memory copies between Intel GPU devices.
  86. b10203 Jul 31, 2026 · issue -019

    llama.cpp b10203 adds SYCL support for Q2_0 quantized matrix multiplication

    └──▷ GET THIS VERSION
    $ git clone --branch b10203 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10203
    • ›Adds q2_0 quantization support to mul_mat on the SYCL backend, enabling Q2 models to run on Intel GPUs and other SYCL-capable accelerators.
  87. b10199 Jul 30, 2026 · issue -020

    llama.cpp server gains support for input embeddings to drive next-token sampling

    └──▷ GET THIS VERSION
    $ git clone --branch b10199 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10199
    • ›Adds inp embd (input embedding) support to the server, enabling callers to supply raw embedding vectors directly to drive sampled token generation instead of text prompts.
  88. b10198 Jul 30, 2026 · issue -020

    llama.cpp b10198 adds quantized concat support for the Vulkan backend.

    └──▷ GET THIS VERSION
    $ git clone --branch b10198 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10198
    • ›Adds quantized concat operation support to the Vulkan backend, enabling GPU-accelerated concatenation of quantized tensors on Vulkan devices.
  89. b10174 Jul 29, 2026 · issue -021

    llama.cpp b10174 adds NextN/MTP speculative decoding support for GLM-5.2 (GLM_DSA) with --spec-type draft-mtp and --mtp/--no-mtp convert flags.

    └──▷ GET THIS VERSION
    $ git clone --branch b10174 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10174
    └──▷ TRY IT
    Export a GLM-5.2 model without the NextN/MTP block to produce a leaner standard GGUF for non-speculative inference.
    $ python convert_hf_to_gguf.py --no-mtp path/to/GLM-5.2
    • ›Adds --spec-type draft-mtp support for GLM_DSA (GLM-5.2) models, enabling NextN/MTP speculative decoding with a dense MLA draft head, sigmoid-gated MoE with shared expert, and NVFP4 _s scale tensor support.
    • ›Adds --mtp and --no-mtp export flags to the converter for GlmMoeDsaForCausalLM (GLM-5.2): --no-mtp drops the NextN block (blk.78) and its nextn_predict_layers KV; --mtp keeps only the NextN block plus shared embeddings, norm, and lm_head.
  90. b10164 Jul 28, 2026 · issue -022

    llama.cpp b10164 adds chunked SSD matrix multiplication on CUDA/HIP/MUSA to accelerate Mamba-2 prefill.

    └──▷ GET THIS VERSION
    $ git clone --branch b10164 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10164
    • ›Adds chunked SSD (State Space Duality) matmul CUDA kernel to accelerate Mamba-2 prefill, with support across CUDA, HIP, and MUSA backends.
  91. b10159 Jul 28, 2026 · issue -022

    llama.cpp b10159 adds a Fast Walsh-Hadamard Transform (FWHT) kernel for the Metal backend on Apple Silicon.

    └──▷ GET THIS VERSION
    $ git clone --branch b10159 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10159
    • ›Adds a Fast Walsh-Hadamard Transform (FWHT) compute kernel to the Metal backend, enabling GPU-accelerated FWHT operations on Apple Silicon devices.
  92. b10155 Jul 27, 2026 · issue -023

    llama.cpp b10155 adds MiMo-V2.5 audio input support via RVQ-based model in the multimodal backend.

    └──▷ GET THIS VERSION
    $ git clone --branch b10155 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10155
    • ›Adds MiMo-V2.5 audio input support (RVQ-based model) to the mtmd multimodal backend, including a GGUF converter for MiMo audio.
  93. b10154 Jul 27, 2026 · issue -023

    llama.cpp b10154 adds common_print_available_devices() to list usable compute devices at runtime.

    └──▷ GET THIS VERSION
    $ git clone --branch b10154 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10154
    • ›Adds common_print_available_devices() function to the common library for enumerating available compute devices at runtime.
  94. b10153 Jul 27, 2026 · issue -023

    llama.cpp b10153 adds support for the Nanbeige4.2 model architecture.

    └──▷ GET THIS VERSION
    $ git clone --branch b10153 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10153
    • ›Adds support for the Nanbeige4.2 model architecture.
  95. b10145 Jul 27, 2026 · issue -023

    llama.cpp b10145 adds -lm mlock mode to lock model weights in RAM without mmap.

    └──▷ GET THIS VERSION
    $ git clone --branch b10145 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10145
    └──▷ TRY IT
    Pin a model entirely in physical RAM (no mmap) to prevent swap-induced latency spikes during inference.
    $ llama-cli -lm mlock -m model.gguf -p "Explain buffer overflow attacks"
    • ›Adds -lm mlock argument to lock model weights in RAM via mlock without using mmap, preventing swap-out of model data.
    └──▷ BREAKING ON UPGRADE
    • !The meaning of --mlock is reverted to non-mmap'ed mlock behavior, which may change behavior for setups that relied on its previous semantics.
  96. b10142 Jul 27, 2026 · issue -023

    llama.cpp b10142 adds multimodal vision support for MiniMax-M3, including sparse attention and GPU-accelerated inference.

    └──▷ GET THIS VERSION
    $ git clone --branch b10142 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10142
    • ›Adds vision support for MiniMax-M3 via mtmd (multimodal) framework, including mmproj and clip graph construction for the vision tower.
    • ›Implements MiniMax-M3 sparse attention (MSA) with CUDA-native indexer op and flash attention for sparse layers, with dense fallback for multi-sequence --kv-unified mode.
    • ›Sets default KV cache type to F32 for MiniMax-M3 and allocates in-cache k_idx only for architectures that opt in, reducing compute buffer from ~6.8 GiB to ~4.2 GiB at ub2048/62k context.
    • ›Supports prompt caching and multi-stream inference (-np N when kv_unified=false) for MiniMax-M3, with context shift explicitly disallowed for this architecture.
    • ›Adds MiniMax-M3 end-of-sequence token to the llama vocabulary.
    +1 moreshow less
    • ›Changes MiniMax-M3 image resize padding to 'none' and resize algorithm to Bicubic (Pillow) during conversion.
    └──▷ BREAKING ON UPGRADE
    • !All MiniMax-M3 GGUFs generated before this release must be regenerated due to the renaming of MSA index tensors to the indexer naming convention.
  97. b10105 Jul 24, 2026 · issue -026

    llama.cpp b10105 consolidates --mmap, --mlock, and --direct-io into a single --load-mode argument

    └──▷ GET THIS VERSION
    $ git clone --branch b10105 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10105
    └──▷ TRY IT
    Lock model weights into RAM to prevent paging during inference on a memory-constrained host.
    $ llama-cli --model model.gguf --load-mode mlock -p 'Explain quantum entanglement'
    • ›Replaces the separate --mmap, --mlock, and --no-mmap flags with a unified --load-mode argument that accepts modifier-based values (e.g. mmap, mlock, dio) controlling how model weights are loaded from disk.
    └──▷ BREAKING ON UPGRADE
    • !The previous --mmap, --no-mmap, and --mlock flags are now deprecated; combining them with the new --load-mode flag triggers a deprecation warning and may produce undefined behaviour — users should migrate to --load-mode.
  98. b10099 Jul 23, 2026 · issue -027

    CUDA NVFP4 W4A4 activation quantization improved with fused kernels and intrinsics for faster inference on NVIDIA GPUs.

    └──▷ GET THIS VERSION
    $ git clone --branch b10099 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10099
    • ›Improves NVFP4 W4A4 activation quantization on CUDA with fused per-channel amax and quantization kernels, 32-byte loads, and nvfp4x4 intrinsic support where available, reducing overhead during quantized matrix multiplication.
  99. b10094 Jul 23, 2026 · issue -027

    llama-server now auto-detects mtp/dflash/eagle3 speculative draft sidecars from -hfd repos without requiring --spec-type.

    └──▷ GET THIS VERSION
    $ git clone --branch b10094 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10094
    └──▷ TRY IT
    Run llama-server with speculative decoding against a draft repo that ships sidecars — no --spec-type flag needed; the type is inferred automatically.
    $ llama-server -hf repo:Q3_K_M -hfd repo:Q8_0
    • ›Adds automatic speculative decoding type inference when using -hfd with a repo that ships mtp-, dflash-, or eagle3- sidecars — selects the first available following priority order mtp > dflash > eagle3 and sets --spec-type automatically, so llama-server -hf repo:Q3_K_M -hfd repo:Q8_0 works without any extra flag. An explicit --spec-type disables the inference.
  100. b10090 Jul 22, 2026 · issue -028

    llama.cpp b10090 adds GGML_OP_CONV_2D_DW (depthwise conv2d) support to the WebGPU backend.

    └──▷ GET THIS VERSION
    $ git clone --branch b10090 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10090
    • ›Implements GGML_OP_CONV_2D_DW (depthwise convolution 2D) kernel for the WebGPU backend, ported from the Vulkan backend's conv2d_dw.comp shader.
  101. b10089 Jul 22, 2026 · issue -028

    CUDA GET_ROWS now handles all quantized GGML types on-device, eliminating host fallback for k-quants, i-quants, and mxfp4.

    └──▷ GET THIS VERSION
    $ git clone --branch b10089 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10089
    • ›Adds full k-quant support (q2_K through q6_K) to the CUDA GET_ROWS kernel via a new k_get_rows_kq kernel, keeping embedding lookups on-device instead of falling back to the host and copying the full embedding matrix per token.
    • ›Extends CUDA GET_ROWS device-side coverage to all nine i-quants, completing quantized type support alongside k-quants.
    • ›Adds mxfp4 support to CUDA GET_ROWS, closing the final gap so every quantized GGML type now takes the direct device path in single-device graphs.
  102. b10087 Jul 22, 2026 · issue -028

    llama.cpp b10087 adds support for Laguna XS.2 & M.1 hardware.

    └──▷ GET THIS VERSION
    $ git clone --branch b10087 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10087
    • ›Adds support for Laguna XS.2 & M.1 accelerator hardware.
  103. b10069 Jul 20, 2026 · issue -030

    OpenCL Adreno backend gains broadcast support for MUL_MAT and correct view-offset handling for Q8_0 in multi-stream llama-server

    └──▷ GET THIS VERSION
    $ git clone --branch b10069 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10069
    • ›Adds broadcast support for Adreno GEMM and GEMV (noshuffle) kernels in the OpenCL backend, enabling multi-stream llama-server workloads that require tensor broadcasting.
    • ›Adds correct view_offs handling for Adreno Q8_0 MUL_MAT (noshuffle GEMM/GEMV) in the OpenCL backend, fixing correctness for non-zero-offset tensor views in multi-stream inference.
  104. b10058 Jul 17, 2026 · issue -033

    llama.cpp b10058 adds Q2_0 quantization support to the Vulkan backend for GPU-accelerated mat-vec-mul.

    └──▷ GET THIS VERSION
    $ git clone --branch b10058 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10058
    • ›Adds Q2_0 quantization format support to the Vulkan backend, enabling GPU-accelerated matrix-vector multiplication for Q2_0 models.
  105. b10045 Jul 16, 2026 · issue -034

    llama.cpp server gains text-only slot save/restore support for multimodal (mtmd) sessions

    └──▷ GET THIS VERSION
    $ git clone --branch b10045 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10045
    • ›Adds text-only slot save/restore capability to the server when running with multimodal (mtmd) support, enabling KV-cache slot persistence for text portions of multimodal sessions.
  106. b10043 Jul 16, 2026 · issue -034

    llama.cpp b10043 adds CUDA Virtual Device support, enabling multi-GPU splits across virtual device boundaries.

    └──▷ GET THIS VERSION
    $ git clone --branch b10043 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10043
    • ›Adds CUDA Virtual Device support, allowing llama.cpp to treat virtual CUDA devices as distinct GPU backends (NCCL path is automatically disabled when virtual devices are in use).
  107. b10042 Jul 16, 2026 · issue -034

    llama.cpp b10042 enables CUDA graphs on Volta and Turing GPUs for improved inference throughput.

    └──▷ GET THIS VERSION
    $ git clone --branch b10042 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10042
    • ›Enables CUDA graphs support on Volta and Turing GPU architectures, unlocking lower kernel-launch overhead and higher inference throughput on older NVIDIA hardware.
  108. b10037 Jul 16, 2026 · issue -034

    llama.cpp b10037 lets --pure quantization mode coexist with manual per-tensor type overrides.

    └──▷ GET THIS VERSION
    $ git clone --branch b10037 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10037
    • ›Allows manual tensor type overrides to be combined with --pure during quantization, so practitioners can pin specific tensors to a chosen quant type while still enforcing a pure quantization pass for the rest of the model.
  109. b10032 Jul 15, 2026 · issue -035

    llama.cpp b10032 adds CUDA implementation of GGML_OP_LIGHTNING_INDEXER with generic vector and WMMA kernels (Turing+).

    └──▷ GET THIS VERSION
    $ git clone --branch b10032 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10032
    • ›Adds CUDA implementation of GGML_OP_LIGHTNING_INDEXER with a generic vector kernel and a WMMA (Tensor Core) kernel, requiring Turing-class GPUs or newer.
  110. b10016 Jul 15, 2026 · issue -035

    llama.cpp b10016 adds SYCL Flash Attention via oneDNN XMX engine on Intel Battlemage (Xe2), delivering up to 4.26× prefill speedup at 80k context.

    └──▷ GET THIS VERSION
    $ git clone --branch b10016 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10016
    • ›Adds F16 Flash Attention for SYCL backends using the oneDNN graph API's XMX engine, scoped to Intel Battlemage (Xe2/BMG) GPUs; other architectures fall back to the existing FA kernel.
    • ›Multi-GPU synchronization for the oneDNN FA path via stream wait_and_throw(), enabling validated multi-device inference on Battlemage.
    • ›Delivers up to 4.26× prefill throughput increase at 80k context (pp512: ~32% gain) for Q8_0 models on supported hardware, unlocking long-context use cases that were previously throughput-constrained.
  111. b10010 Jul 14, 2026 · issue -036

    llama.cpp b10010 adds --cors-* CLI options to the server for fine-grained CORS control.

    └──▷ GET THIS VERSION
    $ git clone --branch b10010 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10010
    • ›Adds --cors-* options to the server command, including a special localhost value, enabling fine-grained cross-origin resource sharing control.
  112. b10003 Jul 14, 2026 · issue -036

    llama.cpp b10003: tokenize tool gains standard model-sourcing flags including HuggingFace and offline support

    └──▷ GET THIS VERSION
    $ git clone --branch b10003 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b10003
    └──▷ TRY IT
    Tokenize text piped from stdin using a HuggingFace-hosted model without downloading it manually.
    $ echo 'Analyze this payload' | llama-tokenize --stdin -hf <org/model> --show-count
    Tokenize a file in an air-gapped environment using a local model, suppressing BOS token and showing token IDs only.
    $ llama-tokenize -m /models/llama.gguf --offline --no-bos --ids -f input.txt
    • ›Exposes -m, -mu, -dr, -hf, -hff, --offline, and HF_TOKEN model-sourcing flags to the tokenize tool via LLAMA_EXAMPLE_TOKENIZE, enabling HuggingFace-hosted and offline model use directly from the tokenizer.
    • ›Registers --ids, --stdin, --no-bos, --no-parse-special, and --show-count as standard common args in the tokenize tool, replacing its hand-rolled argument parser.
  113. b9994 Jul 14, 2026 · issue -036

    llama.cpp b9994 adds Q2_0 quantization support for Metal (Apple GPU) inference.

    └──▷ GET THIS VERSION
    $ git clone --branch b9994 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9994
    • ›Adds Q2_0 quantization format support to the Metal backend, enabling Apple Silicon GPU-accelerated inference with Q2_0 models.
  114. b9993 Jul 13, 2026 · issue -037

    llama.cpp b9993 adds Tencent Hunyuan 3 (hy_v3) architecture support with MTP speculative decoding.

    └──▷ GET THIS VERSION
    $ git clone --branch b9993 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9993
    • ›Adds hy_v3 (HunyuanV3 / HYV3ForCausalLM) architecture support: a MoE decoder with per-head Q/K RMSNorm, sigmoid router with expert selection bias, an always-active ungated shared expert, and leading dense blocks (first_k_dense_replace), enabling MTP speculative decoding for Tencent Hunyuan 3 models in GGUF format.
  115. b9990 Jul 13, 2026 · issue -037

    llama.cpp b9990 adds Minimax2 EAGLE3 speculative decoding support.

    └──▷ GET THIS VERSION
    $ git clone --branch b9990 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9990
    • ›Adds EAGLE3 speculative decoding support for Minimax2 models.
  116. b9987 Jul 13, 2026 · issue -037

    llama.cpp b9987 adds gguf_get_tensor_ne accessor for retrieving tensor shapes from GGUF files.

    └──▷ GET THIS VERSION
    $ git clone --branch b9987 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9987
    • ›Adds gguf_get_tensor_ne to the GGUF API, returning tensor shape as const int64_t * for programmatic inspection of model tensor dimensions.
  117. b9985 Jul 13, 2026 · issue -037

    llama.cpp b9985 adds fused top-k MoE kernel for SYCL backends, controllable via GGML_SYCL_ENABLE_FUSION.

    └──▷ GET THIS VERSION
    $ git clone --branch b9985 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9985
    └──▷ TRY IT
    Enable fused top-k MoE kernel on a SYCL-backed build to accelerate MoE model inference on Intel GPUs.
    $ GGML_SYCL_ENABLE_FUSION=1 ./llama-cli -m <model.gguf> -p '<prompt>'
    • ›Adds GGML_SYCL_ENABLE_FUSION environment variable to enable fused top-k Mixture-of-Experts (MoE) kernel dispatch on SYCL backends, printed at startup alongside other env vars.
  118. b9972 Jul 12, 2026 · issue -038

    llama.cpp server gains streaming support for the exec_shell_command tool

    └──▷ GET THIS VERSION
    $ git clone --branch b9972 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9972
    • ›Adds streaming output support for exec_shell_command in the llama.cpp server, enabling real-time token delivery when the shell tool is invoked.
  119. b9970 Jul 12, 2026 · issue -038

    llama.cpp b9970 adds GGML_OP_LIGHTNING_INDEXER for native DeepSeek V3.2 and V4 model support.

    └──▷ GET THIS VERSION
    $ git clone --branch b9970 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9970
    • ›Adds GGML_OP_LIGHTNING_INDEXER ggml operation implementing the DeepSeek V3.2 and V4 lightning indexer, with f16 mask parameter and mask broadcasting support.
    • ›Enables use of GGML_OP_LIGHTNING_INDEXER in DeepSeek V3.2 and DeepSeek V4 model inference paths.
    • ›Bumps the RPC version to reflect the new GGML_OP_LIGHTNING_INDEXER operation.
  120. b9968 Jul 12, 2026 · issue -038

    llama.cpp b9968 adds int8 DP4 dense and MoE prefill optimizations for Adreno GPUs via OpenCL

    └──▷ GET THIS VERSION
    $ git clone --branch b9968 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9968
    • ›Adds int8 DP4 dense GEMM and MoE GEMM prefill optimizations for Qualcomm Adreno GPUs via OpenCL, accelerating both dense and Mixture-of-Experts model inference on mobile hardware.
  121. b9967 Jul 12, 2026 · issue -038

    llama.cpp server now accepts null sampling params (temperature, top_p, …) to request server defaults, matching the OpenAI spec.

    └──▷ GET THIS VERSION
    $ git clone --branch b9967 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9967
    • ›Server API now treats null values for nullable sampling parameters (e.g., temperature, top_p) as absent, falling back to server defaults — matching the OpenAI spec behavior.
  122. b9963 Jul 11, 2026 · issue -039

    llama.cpp b9963 adds DeepSeek-OCR v1 multi-tile dynamic resolution support with unified image preprocessors.

    └──▷ GET THIS VERSION
    $ git clone --branch b9963 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9963
    • ›Adds DeepSeek-OCR v1 multi-tile dynamic resolution support in mtmd, unifying image preprocessors across DeepSeek-OCR v1 and v2.
  123. b9957 Jul 10, 2026 · issue -040

    llama.cpp b9957 improves server tool-use with a new tools_io abstraction and enhanced edit tool.

    └──▷ GET THIS VERSION
    $ git clone --branch b9957 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9957
    • ›Adds tools_io and tools_io_basic abstraction layer to the server, giving tool-use I/O a dedicated interface.
    • ›Improves the server edit tool with enhanced functionality.
    • ›Removes apply_diff from the server tool set.
    └──▷ BREAKING ON UPGRADE
    • !The apply_diff tool has been removed from the server; any workflow relying on it will break on upgrade.
  124. b9951 Jul 10, 2026 · issue -040

    llama.cpp b9951 adds an initial ExecuTorch (ET) backend with hardware-accelerated kernels for inference on ET-SoC silicon.

    └──▷ GET THIS VERSION
    $ git clone --branch b9951 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9951
    └──▷ TRY IT
    Build llama.cpp with the ET backend enabled and run the llama-server against an ET-SoC device.
    $ cmake -B build -DGGML_CUDA=OFF -DGGML_ET=ON -DLLAMA_CURL=OFF -DGGML_CCACHE=ON
cmake --build build --config Release -j $(nproc)
./build/bin/llama-server --model Qwen3-0.6B-Q8_0.gguf --alias Qwen3-0.6B-Q8_0 -fa 0 --ctx-size 1024 --no-warmup --host 127.0.0.1 --port 8080
    Build with ET emulator support to develop and validate ET backend kernels without physical ET-SoC hardware.
    $ cmake -B build -DGGML_ET=ON -DGGML_ET_SYSEMU=ON
cmake --build build --config Release -j $(nproc)
./build/bin/test-backend-ops
    • ›Adds new GGML_ET backend (ExecuTorch) enabled at build time with -DGGML_ET=ON, supporting MUL_MAT, ROPE, RMS_NORM, GLU, SOFT_MAX, GET_ROWS, SET_ROWS, MUL_MAT_ID, FlashAttention, and many more operators offloaded to ET-SoC hardware.
    • ›Adds -DGGML_ET_SYSEMU=ON compile-time flag to enable ET-SoC emulator (sysemu) support for development and testing without physical hardware.
    • ›Supports Q4_0, Q4_K, Q8_0, F16, and F32 quantization formats in the ET backend, including matrix-engine (TensorFMA) accelerated paths for Q4_0 and Q8_0 MUL_MAT.
    • ›Supports kernel fusion in the ET backend, initially fusing RMS_NORM + MUL into a single accelerated pass.
    • ›Adds FlashAttention support (F32 and F16 via matrix engine) in the ET backend, enabling full offload of attention for models such as LLaMA 3.1 and Gemma 4.
    +6 moreshow less
    • ›Adds MROPE (multi-rope) operator support in the ET backend, broadening compatible model architectures.
    • ›Adds uberkernel support in the ET backend, batching multiple operator kernels into a single dispatch for reduced launch overhead.
    • ›Adds WKV*, SSM_CONV, SSM_SCAN, GATED_DELTA_NET, and GROUP_NORM operators to the ET backend, enabling inference for Mamba/RWKV-style and hybrid SSM architectures.
    • ›Embeds ET kernels directly into the build via CMake (with filesystem fallback), eliminating the need for runtime kernel import hacks.
    • ›Adds DeepSeek-V2-Lite RoPE test coverage to test-backend-ops.
    • ›Supports Gemma 4 model family via the ET backend.
  125. b9949 Jul 10, 2026 · issue -040

    llama.cpp b9949 adds cluster-parallel Flash Attention decode for Adreno GPUs via OpenCL.

    └──▷ GET THIS VERSION
    $ git clone --branch b9949 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9949
    • ›Adds cluster-parallel decode Flash Attention for Adreno GPUs via the OpenCL backend, improving inference throughput on Qualcomm Adreno hardware.
  126. b9947 Jul 10, 2026 · issue -040

    llama.cpp b9947 adds --output option to the CLI.

    └──▷ GET THIS VERSION
    $ git clone --branch b9947 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9947
    └──▷ TRY IT
    Redirect model inference output to a file instead of stdout for downstream processing or logging.
    $ llama-cli --output result.txt -m model.gguf -p "Summarize this document:"
    • ›Adds --output flag to the llama-cli command for directing output to a specified destination.
  127. b9931 Jul 8, 2026 · issue -042

    llama.cpp b9931 adds OpenCL ragged-tile MoE prefill GEMM optimizations with new GGML_OPENCL_MOE_RAGGED_GRAN env var control.

    └──▷ GET THIS VERSION
    $ git clone --branch b9931 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9931
    └──▷ TRY IT
    Run MoE inference with finer-grained tile skipping disabled (half-granularity) to trade off skip overhead vs. padding waste on larger batch sizes.
    $ GGML_OPENCL_MOE_RAGGED_GRAN=16 ./llama-cli -m mixtral.gguf -p 'Hello world'
    Opt out of all ragged-tile MoE FP16 skipping on OpenCL to revert to the baseline GEMM path for comparison or debugging.
    $ GGML_OPENCL_MOE_RAGGED_FP16=0 ./llama-cli -m mixtral.gguf -p 'Hello world'
    • ›Adds GGML_OPENCL_MOE_RAGGED_GRAN={8,16,32} environment variable to control OpenCL MoE prefill GEMM tile-skip granularity (quarter/half/off), defaulting to quarter (8-column skip-groups) across all eight *_f32_ns MoE GEMMs.
    • ›Adds GGML_OPENCL_MOE_RAGGED_FP16=0 opt-out environment variable to disable ragged-tile half-tile skipping for OpenCL MoE FP16 prefill GEMM.
    • ›New OpenCL ragged-tile MoE prefill optimization skips fully-padded expert tiles during GEMM, reducing wasted compute on low tokens-per-expert workloads while remaining numerically identical to the non-skipped path.
  128. b9927 Jul 8, 2026 · issue -042

    llama.cpp b9927 moves the CLI to an HTTP-based implementation with router mode support.

    └──▷ GET THIS VERSION
    $ git clone --branch b9927 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9927
    • ›Migrates the cli to an HTTP-based implementation, enabling it to connect to a remote server and operate in router mode (multi-model routing).
    • ›Adds router mode support to the cli, including handling the single-model router edge case and displaying model aliases.
  129. b9913 Jul 8, 2026 · issue -042

    llama.cpp b9913 adds Q2_0 quantization type with CPU backend support.

    └──▷ GET THIS VERSION
    $ git clone --branch b9913 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9913
    • ›Adds Q2_0 quantization type definition and CPU backend, enabling a new 2-bit quantization format for model compression.
  130. b9909 Jul 8, 2026 · issue -042

    llama.cpp b9909 adds timings and progress fields to the /responses API stream endpoint.

    └──▷ GET THIS VERSION
    $ git clone --branch b9909 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9909
    • ›Adds timings and progress data to the /responses API streaming endpoint, enabling clients to monitor inference performance and completion progress in real time.
  131. b9902 Jul 7, 2026 · issue -043

    SYCL backend gains support for cross_entropy_loss and cross_entropy_loss_back operations in llama.cpp b9902.

    └──▷ GET THIS VERSION
    $ git clone --branch b9902 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9902
    • ›Adds cross_entropy_loss and cross_entropy_loss_back operator support to the SYCL backend, enabling GPU-accelerated training-related operations on Intel SYCL devices.
  132. b9885 Jul 6, 2026 · issue -044

    llama.cpp b9885 enables tiled matmul on AIX, delivering ~2x prompt-processing speed gains for FP32, Q4_0, and Q8_0 models.

    └──▷ GET THIS VERSION
    $ git clone --branch b9885 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9885
    • ›Enables the matmul_tiled path on AIX by reducing stack buffer footprint for A_pack and B_pack, unlocking ~2x gains in prompt-processing speed for FP32, Q4_0, and Q8_0 models on that platform.
  133. b9882 Jul 6, 2026 · issue -044

    Download scripts now use HF_TOKEN to authenticate when fetching UI assets from Hugging Face.

    └──▷ GET THIS VERSION
    $ git clone --branch b9882 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9882
    • ›Adds HF_TOKEN environment variable support to the UI asset download scripts, enabling authenticated downloads from Hugging Face.
  134. b9874 Jul 5, 2026 · issue -045

    llama.cpp b9874 adds CUDA concat support for quantized tensor types.

    └──▷ GET THIS VERSION
    $ git clone --branch b9874 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9874
    • ›Adds CUDA concat kernel implementation for quantized types, enabling GPU-accelerated concatenation operations on quantized tensors without falling back to CPU.
  135. b9867 Jul 3, 2026 · issue -047

    llama.cpp b9867 adds spec-draft-p-min support to DFlash speculative decoding.

    └──▷ GET THIS VERSION
    $ git clone --branch b9867 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9867
    • ›Adds spec-draft-p-min parameter support to DFlash speculative decoding, with n_min and n_max guards to bound draft token counts.
  136. b9864 Jul 3, 2026 · issue -047

    Adds sse_ping_interval as a per-request body field so slow prefill never drops healthy SSE connections

    └──▷ GET THIS VERSION
    $ git clone --branch b9864 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9864
    • ›Adds sse_ping_interval as a typed per-request body field in the server's request schema, accepting positive integers or -1 (disabled), with hard limits of -1 to INT32_MAX; the global CLI default remains 30 so existing API clients see no behavior change.
    • ›Server now pings silent SSE streams every 1 second and kicks connections only after 3 seconds, preventing slow prefill from dropping healthy long-running connections.
    • ›WebUI automatically sends sse_ping_interval: 1 in each request body to enforce the 3-second visibility-kick contract without requiring server-side reconfiguration.
  137. b9860 Jul 2, 2026 · issue -048

    llama.cpp b9860 adds llama_ftype_name() to the public C API for querying model quantization type names.

    └──▷ GET THIS VERSION
    $ git clone --branch b9860 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9860
    • ›Adds llama_ftype_name() to the public C API, returning a human-readable quantization name (e.g. Q8_0, Q4_K - Medium) for a given llama_ftype enum value; the returned pointer is valid for the lifetime of the model and allocation-free.
  138. b9859 Jul 1, 2026 · issue -049

    OpenCL backend gains support for loading precompiled binary kernels from an external library, including q8_0, q4_0, q4_1, q4_k, and MoE MX-FP4 GEMM kernels.

    └──▷ GET THIS VERSION
    $ git clone --branch b9859 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9859
    • ›Adds OpenCL support for loading precompiled binary kernels (including gemm_moe_mxfp4_f32_ns, q8_0 GEMM, q4_0/q4_1/q4_k MoE GEMM) from an external kernel library at runtime, enabling vendor-optimized kernel delivery without recompilation.
  139. b9844 Jun 30, 2026 · issue -050

    llama.cpp b9844 adds NVFP4 support to the WebGPU backend.

    └──▷ GET THIS VERSION
    $ git clone --branch b9844 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9844
    • ›Adds NVFP4 quantization support to the ggml-webgpu backend, enabling NVIDIA FP4 model inference via WebGPU.
  140. b9840 Jun 29, 2026 · issue -051

    llama.cpp b9840 adds DeepSeek V4 model support including conversion, multi-sequence inference, and Flash Attention.

    └──▷ GET THIS VERSION
    $ git clone --branch b9840 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9840
    • ›Adds dsv4 architecture support to the convert tooling, enabling conversion of DeepSeek V4 (and Pro) models to GGUF format.
    • ›Supports DeepSeek V4 inference with Flash Attention (FA) enabled, graph reuse, multi-sequence support, and save/load state.
    • ›Adds compatibility with antirez-format DeepSeek V4 GGUFs, broadening which existing GGUF files can be loaded.
    • ›Uses expert_gating_func in place of moe.score_func for DeepSeek V4 MoE gating configuration.
  141. b9837 Jun 29, 2026 · issue -051

    llama.cpp b9837 adds --reasoning-preserve flag for controlling reasoning content in chat output.

    └──▷ GET THIS VERSION
    $ git clone --branch b9837 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9837
    • ›Adds --reasoning-preserve flag to control whether reasoning/chain-of-thought content is preserved in chat and Jinja template output.
  142. b9833 Jun 28, 2026 · issue -052

    llama.cpp b9833 adds a dedicated MiniCPM5 tool-call parser with XML tool-call support via autoparser.

    └──▷ GET THIS VERSION
    $ git clone --branch b9833 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9833
    • ›Adds a dedicated MiniCPM5 tool-call parser using the openbmb-MiniCPM5-1B.jinja template, with XML tool-call handling via autoparser and strict JSON argument parsing.
  143. b9831 Jun 28, 2026 · issue -052

    llama.cpp b9831 adds DFlash v2 support with per-layer sliding window attention.

    └──▷ GET THIS VERSION
    $ git clone --branch b9831 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9831
    • ›Adds DFlash v2 support, including per-layer sliding window attention via layer_types configuration.
  144. b9830 Jun 28, 2026 · issue -052

    llama.cpp b9830 exposes --offline to llama download for air-gapped and cache-check workflows.

    └──▷ GET THIS VERSION
    $ git clone --branch b9830 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9830
    └──▷ TRY IT
    Check whether a model is already cached before serving it in an air-gapped or network-restricted environment.
    $ llama download --offline --model <model-name>
    • ›Adds --offline flag support to llama download, letting scripts verify whether a model is already cached locally without making any network requests.
  145. b9828 Jun 27, 2026 · issue -053

    OpenCL Flash Attention gains f16/f32/q4_0/q8_0 support with prefill prepass kernels and tile-skip optimizations.

    └──▷ GET THIS VERSION
    $ git clone --branch b9828 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9828
    • ›Adds OpenCL Flash Attention prefill prepass kernels (flash_attn_kv_pad_f16, flash_attn_mask_pad_f16, flash_attn_blk_f16) that classify KV tiles as fully masked, mixed, or fully unmasked — enabling the main kernel to skip fully-masked tiles and bypass mask lookups for fully-unmasked ones.
    • ›Adds OpenCL FA kernels for q4_0 and q8_0 quantization formats, extending Flash Attention beyond f16/f32 on Adreno and compatible OpenCL backends.
    • ›Adds OpenCL set_rows operation supporting f32-to-q8_0/q4_0 conversion.
    • ›Adds OpenCL dequantization kernels for q4_0 and q8_0.
    • ›Adds an OpenCL FA tile tuning table with a per-device override mechanism.
  146. b9821 Jun 26, 2026 · issue -054

    llama.cpp b9821 adds --version, --licenses, and --help flags to app binaries.

    └──▷ GET THIS VERSION
    $ git clone --branch b9821 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9821
    └──▷ TRY IT
    Check the build version of a llama.cpp binary at a glance, useful for confirming which release is deployed.
    $ ./llama-cli --version
    • ›Adds --version, --licenses, and --help flags to llama.cpp app binaries.
  147. b9817 Jun 26, 2026 · issue -054

    OpenVINO backend updated to OV 2026.2.1 with self-contained release packages and expanded operator support.

    └──▷ GET THIS VERSION
    $ git clone --branch b9817 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9817
    • ›Updates OpenVINO backend to OV 2026.2.1 with self-contained release packages (no external dependencies required at runtime).
    • ›Adds GLU/SwiGLU OAI operator support (glu_swiglu_oai) to the OpenVINO backend.
    • ›Enables softmax with sink input in the OpenVINO backend.
    • ›Expands add_id in the OpenVINO backend to support 2D and 4D tensor shapes.
    • ›Optimizes mul_mat_id conversion processing in the OpenVINO backend for large sizes.
    +1 moreshow less
    • ›Removes hardcoded compute_op_type sets in the OpenVINO backend, allowing dynamic operator type resolution.
  148. b9813 Jun 26, 2026 · issue -054

    llama.cpp b9813 adds Vulkan cooperative matrix support for Intel Xe-LPG Plus (Xe1-ARLH) GPUs

    └──▷ GET THIS VERSION
    $ git clone --branch b9813 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9813
    • ›Adds INTEL_XE1 architecture enum to the Vulkan backend and enables coopmat1 (cooperative matrix) acceleration on Intel Xe-LPG Plus (Xe1-ARLH) hardware, including a Windows driver compatibility check.
  149. b9788 Jun 25, 2026 · issue -055

    SYCL backend gains --split-mode tensor for dual-GPU tensor parallelism with BF16-compressed all-reduce

    └──▷ GET THIS VERSION
    $ git clone --branch b9788 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9788
    • ›Adds --split-mode tensor support to the SYCL backend, enabling tensor parallelism across dual Intel Arc GPUs without OneCCL or new build dependencies.
    • ›Implements a two-path all-reduce: FP32 direct memcpy for small tensors (nelem < 32768) and BF16-compressed cross-device transfer for large tensors (nelem >= 32768), halving PCIe bandwidth for the common large-tensor case.
    • ›Delivers up to +78.6% token-generation throughput on Llama-3.3-70B Q4_K_M versus layer-split mode on dual Intel Arc Pro B70 hardware.
  150. b9784 Jun 24, 2026 · issue -056

    Hexagon NPU backend gains 32×32 tiled weight repack, kernel-param caching, and GGML_HEXAGON_MM_SELECT for matmul dispatch control.

    └──▷ GET THIS VERSION
    $ git clone --branch b9784 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9784
    • ›Adds GGML_HEXAGON_MM_SELECT environment variable to control matmul kernel selection on Hexagon NPU, selectable from the host.
    • ›Introduces simple graph caching on the Hexagon backend to avoid recomputing kernel-params on repeated graph executions.
    • ›Implements 32×32 tiled weight repack layout for HMX matmul kernels, aligning tiles with DMA for higher throughput.
    • ›Adds non-tiled MUL_MAT fallback path on Hexagon, falling back to HVX flat kernels when HMX tiling is not viable.
    • ›Drops support for Hexagon arch < v73; HMX is now required for the backend — non-v73+ devices will no longer work.
    └──▷ BREAKING ON UPGRADE
    • !Support for Hexagon architecture versions below v73 has been removed; HMX is now required, so devices running arch < v73 will no longer function with the Hexagon backend.
  151. b9777 Jun 24, 2026 · issue -056

    llama.cpp b9777 adds support for LFM2.5-ColBERT-350M and LFM2.5-Embedding-350M models.

    └──▷ GET THIS VERSION
    $ git clone --branch b9777 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9777
    • ›Adds support for the LFM2.5-ColBERT-350M and LFM2.5-Embedding-350M model architectures.
  152. b9768 Jun 23, 2026 · issue -057

    llama.cpp b9768 adds conversion and inference support for Granite Speech Plus with multi-layer audio concatenation.

    └──▷ GET THIS VERSION
    $ git clone --branch b9768 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9768
    • ›Adds conversion support for IBM Granite Speech Plus models, extending the granite_speech architecture to handle multi-layer (feature_layers) audio concatenation for richer speech understanding.
  153. b9763 Jun 22, 2026 · issue -058

    llama.cpp server now includes an id field in tool call responses.

    └──▷ GET THIS VERSION
    $ git clone --branch b9763 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9763
    • ›Adds id field to tool call responses in the server API, enabling callers to correlate tool call requests and responses.
  154. b9761 Jun 22, 2026 · issue -058

    llama.cpp server gains real-time model download progress tracking via a new /models/sse SSE endpoint.

    └──▷ GET THIS VERSION
    $ git clone --branch b9761 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9761
    └──▷ TRY IT
    Stream live model download progress from the server without polling — useful in CI pipelines or admin dashboards that need to know when a model is ready.
    $ curl -N http://localhost:8080/models/sse
    • ›Adds /models/sse Server-Sent Events endpoint to the llama.cpp server for real-time model load and download progress tracking.
    • ›Moves model downloading to a dedicated child process in the server router, isolating download work from the main server process.
  155. b9760 Jun 22, 2026 · issue -058

    llama.cpp b9760 generalizes the server input file schema to support input_video and raw base64 intake.

    └──▷ GET THIS VERSION
    $ git clone --branch b9760 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9760
    • ›Adds input_video field to the server input file schema, enabling video input alongside existing file types.
    • ›Server now accepts raw base64-encoded input directly in the input file schema.
  156. b9748 Jun 21, 2026 · issue -059

    llama.cpp b9748 adds a verbose field to the server schema for richer response metadata.

    └──▷ GET THIS VERSION
    $ git clone --branch b9748 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9748
    • ›Adds verbose field to the llama.cpp server response schema, enabling clients to request detailed inference metadata.
  157. b9747 Jun 21, 2026 · issue -059

    llama.cpp b9747 adds real-time model load progress tracking via a new /models/sse SSE endpoint.

    └──▷ GET THIS VERSION
    $ git clone --branch b9747 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9747
    └──▷ TRY IT
    Stream model load progress in real time while the server is initializing a model — useful for monitoring long loads in automation or a UI.
    $ curl -N http://localhost:8080/models/sse
    • ›Adds GET /models/sse Server-Sent Events endpoint to the llama.cpp server for real-time model load progress tracking.
  158. b9745 Jun 21, 2026 · issue -059

    llama.cpp b9745 adds speculative decoding support for Step3.5/3.7 flash MTP via new llama_set_mtp_layer_offset and llama_model_n_nextn_layer APIs.

    └──▷ GET THIS VERSION
    $ git clone --branch b9745 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9745
    • ›Adds llama_set_mtp_layer_offset and llama_model_n_nextn_layer API functions to support speculative multi-token prediction (MTP) for Step3.5 and Step3.7 flash models.
    • ›Adds mtp_layer_offset configuration and nextn flags with graph reuse support, enabling speculative multi-head draft and process pipelines for MTP blocks.
  159. b9728 Jun 19, 2026 · issue -061

    llama.cpp b9728 adds comment line support to --api-key-file for cleaner API key management.

    └──▷ GET THIS VERSION
    $ git clone --branch b9728 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9728
    └──▷ TRY IT
    Document entries in your API key file with comment lines so teammates know which key belongs to which client or service.
    $ # api-keys.txt
# Production client key
sk-prod-abc123
# Dev/test key
sk-dev-xyz789

llama-server --api-key-file api-keys.txt
    • ›Adds comment line support to --api-key-file, allowing inline documentation inside API key files.
  160. b9726 Jun 19, 2026 · issue -061

    llama.cpp server gains --agent argument for agentic inference modes.

    └──▷ GET THIS VERSION
    $ git clone --branch b9726 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9726
    └──▷ TRY IT
    Launch the llama.cpp server in agent mode to enable agentic inference workflows.
    $ llama-server --model <model.gguf> --agent
    • ›Adds --agent argument to the llama-server CLI to enable agent-mode operation.
  161. b9716 Jun 19, 2026 · issue -061

    llama.cpp b9716 adds batching support for InternVL in the multimodal (mtmd) subsystem.

    └──▷ GET THIS VERSION
    $ git clone --branch b9716 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9716
    • ›Adds batching support for InternVL models in the mtmd (multimodal) subsystem, enabling more efficient multi-image and batch inference with InternVL.
  162. b9714 Jun 19, 2026 · issue -061

    llama.cpp server now emits X-Accel-Buffering: no on streaming endpoints to prevent Nginx reverse-proxy buffering.

    └──▷ GET THIS VERSION
    $ git clone --branch b9714 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9714
    • ›Adds X-Accel-Buffering: no response header to all streaming endpoints in llama-server, preventing Nginx from buffering streamed output when deployed behind a reverse proxy.
  163. b9713 Jun 19, 2026 · issue -061

    llama.cpp b9713 adds batching support and video tests to the mtmd-cli multimodal tool.

    └──▷ GET THIS VERSION
    $ git clone --branch b9713 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9713
    • ›Adds batching support to mtmd-cli, enabling more efficient multimodal (image/video) inference throughput.
    • ›Adds video tests to the mtmd multimodal framework.
  164. b9707 Jun 19, 2026 · issue -061

    llama.cpp server gains a schema field with input validation for structured output enforcement.

    └──▷ GET THIS VERSION
    $ git clone --branch b9707 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9707
    • ›Adds schema field to the llama.cpp server API for structured output, with validation that includes field-level error messages when constraints are violated.
  165. b9699 Jun 18, 2026 · issue -062

    SYCL backend gains MUL_MAT and OUT_PROD support for Q1_0 quantization format

    └──▷ GET THIS VERSION
    $ git clone --branch b9699 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9699
    • ›Adds MUL_MAT and OUT_PROD operation support for the Q1_0 quantization format in the SYCL backend.
  166. b9689 Jun 17, 2026 · issue -063

    Metal backend concat operator gains f16, bf16, i8, i16, and i64 tensor type support on Apple Silicon.

    └──▷ GET THIS VERSION
    $ git clone --branch b9689 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9689
    • ›Extends the Metal backend kernel_concat operator to support f16, bf16, i8, i16, and i64 tensor types in addition to the existing f32 and i32, enabling broader model architectures to run GPU-accelerated on Apple Silicon.
  167. b9688 Jun 17, 2026 · issue -063

    llama.cpp b9688 adds a server-side model management API with download, delete, and SSE real-time update endpoints.

    └──▷ GET THIS VERSION
    $ git clone --branch b9688 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9688
    • ›Adds a model management API to the llama.cpp server router, including endpoints to download models, delete models, and stream real-time status updates via SSE (Server-Sent Events).
  168. b9675 Jun 17, 2026 · issue -063

    llama.cpp b9675 extends SYCL FP16 support to SQR, SQRT, LOG, SIN, COS, and CLAMP operations.

    └──▷ GET THIS VERSION
    $ git clone --branch b9675 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9675
    • ›Enables FP16 support in SYCL backend for the SQR, SQRT, LOG, SIN, COS, and CLAMP operations, broadening half-precision GPU acceleration on Intel SYCL devices.
  169. b9673 Jun 17, 2026 · issue -063

    llama.cpp b9673 adds GGML_SYCL_USM_SYSTEM env var to enable VRAM overcommit via USM system allocations on SYCL devices.

    └──▷ GET THIS VERSION
    $ git clone --branch b9673 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9673
    └──▷ TRY IT
    Run a model that exceeds available VRAM on a SYCL device (e.g. Intel B580) by enabling USM system allocations for automatic host/device memory migration.
    $ GGML_SYCL_USM_SYSTEM=1 ./examples/sycl/test.sh -m Qwen3.5-27B-Q3_K_M.gguf -lv 4
    • ›Adds GGML_SYCL_USM_SYSTEM environment variable to enable optional USM system allocations for large GPU buffers (≥ 1 GB) on SYCL devices, allowing VRAM overcommit by letting the OS manage host/device memory migration; falls back to regular allocations if unsupported.
  170. b9669 Jun 16, 2026 · issue -064

    llama.cpp b9669 adds backend sampling support for Eagle3 speculative decoding.

    └──▷ GET THIS VERSION
    $ git clone --branch b9669 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9669
    • ›Adds backend sampling support for Eagle3, enabling hardware-accelerated speculative decoding with the Eagle3 draft model architecture.
  171. b9665 Jun 16, 2026 · issue -064

    llama.cpp b9665 adds --offline flag to the bench command.

    └──▷ GET THIS VERSION
    $ git clone --branch b9665 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9665
    └──▷ TRY IT
    Run llama.cpp benchmarks in an air-gapped or network-restricted environment without triggering remote calls.
    $ llama-bench --offline
    • ›Adds --offline flag to bench to run benchmarks without network access, with a default value included.
  172. b9664 Jun 16, 2026 · issue -064

    SYCL backend gains reordered Q4_K, Q5_K, and Q6_K MoE MUL_MAT_ID support for fused expert inference

    └──▷ GET THIS VERSION
    $ git clone --branch b9664 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9664
    • ›Extends SYCL reordered-weight handling to fused MoE MUL_MAT_ID operations for Q4_K, Q5_K, and Q6_K expert tensors, enabling GPU-accelerated MoE inference with these quantization types on SYCL backends.
    • ›Adds Q5_K reordered DMMV (dequantize matrix-vector multiply) coverage on SYCL.
    • ›Unsupported 3D reorder cases now fall back gracefully instead of aborting.
  173. b9654 Jun 15, 2026 · issue -065

    llama.cpp b9654 adds a post-decode callback to the multimodal (mtmd) subsystem.

    └──▷ GET THIS VERSION
    $ git clone --branch b9654 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9654
    • ›Adds a post-decode callback to the mtmd (multimodal) subsystem, enabling callers to hook into the decode pipeline after each multimodal token batch is processed.
  174. b9647 Jun 15, 2026 · issue -065

    SYCL backend gains pool_1d support in llama.cpp b9647

    └──▷ GET THIS VERSION
    $ git clone --branch b9647 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9647
    • ›Adds pool_1d operation support to the SYCL backend, with pool_1d and pool_2d code consolidated into pool.cpp/pool.hpp.
  175. b9637 Jun 14, 2026 · issue -066

    llama.cpp b9637 adds a dedicated chat parser for the Cohere2MoE (North Code) model.

    └──▷ GET THIS VERSION
    $ git clone --branch b9637 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9637
    • ›Adds a dedicated Cohere2MoE (North Code) chat template parser for correct prompt formatting with that model family.
  176. b9626 Jun 13, 2026 · issue -067

    llama.cpp b9626 adds architecture support for the cohere2-MoE (cohere2moe) model family.

    └──▷ GET THIS VERSION
    $ git clone --branch b9626 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9626
    • ›Adds cohere2moe architecture support, enabling inference for Cohere2 Mixture-of-Experts models (including North-Mini-Code-1.0) with iSWA-based sliding window attention, sigmoid expert selection, and LayerNorm support for Command variants.
    • ›Adds cohere2moe to the Llama Model Saver supported architecture list, enabling GGUF conversion of cohere2moe models.
  177. b9624 Jun 13, 2026 · issue -067

    llama.cpp b9624 adds build-time gzip compression to the web UI for faster asset delivery.

    └──▷ GET THIS VERSION
    $ git clone --branch b9624 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9624
    • ›Adds build-time gzip compression for web UI assets, reducing payload size for browser clients.
  178. b9606 Jun 12, 2026 · issue -068

    llama.cpp b9606 adds EAGLE3 speculative decoding support, including Gemma4 EAGLE3 models from RedHatAI.

    └──▷ GET THIS VERSION
    $ git clone --branch b9606 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9606
    • ›Adds EAGLE3 speculative decoding support, enabling faster inference via a third-generation draft model architecture, with support for Gemma4 EAGLE3 models from RedHatAI.
    └──▷ BREAKING ON UPGRADE
    • !The output_layer_inp cparams field is renamed to embeddings_layer_inp.
  179. b9603 Jun 12, 2026 · issue -068

    llama.cpp b9603 adds q5_0 and q5_1 GEMM/GEMV OpenCL kernels for Qualcomm Adreno GPUs.

    └──▷ GET THIS VERSION
    $ git clone --branch b9603 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9603
    • ›Adds q5_0 and q5_1 GEMM and GEMV OpenCL kernels for Qualcomm Adreno GPUs, enabling accelerated inference with 5-bit quantized models on Adreno hardware.
  180. b9577 Jun 9, 2026 · issue -071

    llama.cpp b9577 adds --log-prompts-dir to capture every server prompt to disk for auditing.

    └──▷ GET THIS VERSION
    $ git clone --branch b9577 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9577
    └──▷ TRY IT
    Capture every prompt sent to the inference server to disk for later review or audit.
    $ llama-server --model <model.gguf> --log-prompts-dir /var/log/llama-prompts/
    • ›Adds --log-prompts-dir flag to the server to write each incoming prompt to a separate text file in a specified directory, enabling prompt logging and audit trails.
  181. b9575 Jun 9, 2026 · issue -071

    llama.cpp b9575 adds GGML_OP_COL2IM_1D, enabling efficient 1D transposed convolution for vocoder-style audio models.

    └──▷ GET THIS VERSION
    $ git clone --branch b9575 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9575
    • ›Adds GGML_OP_COL2IM_1D CPU operator (F32, F16, BF16 with F32 accumulator) implementing the overlap-add scatter step of a 1D transposed convolution, allowing ConvTranspose1d to be factorized as a plain mul_mat followed by col2im_1d so the heavy work stays on optimized, quantizable matmul kernels.
    • ›Bumps RPC_PROTO_PATCH_VERSION to cover the new op (GGML_OP_COUNT rises from 96 to 97); the op is appended so no existing op codes shift.
  182. b9568 Jun 8, 2026 · issue -072

    llama.cpp b9568 adds MTP support for Gemma-4 E2B and E4B assistant models.

    └──▷ GET THIS VERSION
    $ git clone --branch b9568 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9568
    • ›Adds multi-token prediction (MTP) support for Gemma-4 E2B and E4B assistant model variants, including masked_embd tensors in the gemma4-assist architecture and converter updates to handle masked_embedding tensor filtering during conversion.
  183. b9562 Jun 8, 2026 · issue -072

    llama.cpp b9562 adds video input support via --video CLI arg, base64 server input, and MTMD_VIDEO config.

    └──▷ GET THIS VERSION
    $ git clone --branch b9562 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9562
    • ›Adds --video CLI argument to enable video input for multimodal inference from the command line, with auto-completion support.
    • ›Adds MTMD_VIDEO configuration option to control video input support.
    • ›Supports video input on the server via base64-encoded input payloads.
    • ›Introduces mtmd_helper_video helper for video handling in the multimodal (mtmd) subsystem.
  184. b9556 Jun 8, 2026 · issue -072

    llama.cpp b9556 adds HIP support for AMD RDNA3.5 GPUs gfx1152 and gfx1153.

    └──▷ GET THIS VERSION
    $ git clone --branch b9556 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9556
    • ›Adds gfx1152 and gfx1153 GPU targets to the HIP backend under the RDNA3.5 architecture class, enabling GPU-accelerated inference on those AMD chips.
  185. b9549 Jun 7, 2026 · issue -073

    llama.cpp b9549 adds Gemma4 MTP (multi-token prediction) support.

    └──▷ GET THIS VERSION
    $ git clone --branch b9549 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9549
    • ›Adds Gemma4 MTP (multi-token prediction) support to the llama model backend.
  186. b9547 Jun 7, 2026 · issue -073

    llama.cpp b9547 skips redundant mmproj downloads when a user-supplied mmproj is already provided.

    └──▷ GET THIS VERSION
    $ git clone --branch b9547 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9547
    • ›Skips automatic mmproj download when the user has already supplied an mmproj via argument, avoiding redundant network fetches.
  187. b9543 Jun 6, 2026 · issue -074

    llama.cpp b9543 adds video support for Qwen3.5 via 'frame merge' for Qwen-VL-based models.

    └──▷ GET THIS VERSION
    $ git clone --branch b9543 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9543
    • ›Adds 'frame merge' multimodal video support for Qwen-VL-based models, enabling video input for Qwen3.5 via the mtmd layer.
  188. b9522 Jun 5, 2026 · issue -075

    llama.cpp b9522 adds dynamic chunk-based scheduling for hybrid CPU/GPU execution via KleidiAI.

    └──▷ GET THIS VERSION
    $ git clone --branch b9522 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9522
    • ›Adds dynamic chunk-based scheduling for hybrid execution in the KleidiAI backend, enabling more efficient mixed CPU/GPU inference workloads.
  189. b9521 Jun 5, 2026 · issue -075

    CUDA PDL enrollment for MoE matrix-vector ops boosts MTP speculative decoding throughput by ~5–6% on Blackwell GPUs.

    └──▷ GET THIS VERSION
    $ git clone --branch b9521 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9521
    • ›Enrolls mul_mat_vec_q_moe into CUDA PDL (Programmatic Dependent Launch), improving Multi-Token Prediction (MTP) speculative decoding throughput by roughly 5–6% on Blackwell-architecture GPUs (e.g. B4500), with tok/s gains observed across code generation, summarization, QA, translation, and math workloads.
  190. b9499 Jun 4, 2026 · issue -076

    llama.cpp b9499 refactors WebGPU FlashAttention and standardizes quantization support for k/v tensors on the tile path.

    └──▷ GET THIS VERSION
    $ git clone --branch b9499 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9499
    • ›Refactors the WebGPU FlashAttention implementation with split k/v quantization and abstracted quantization logic shared between flash_attn and mul_mat operations.
    • ›Adds quantization support to the WebGPU tile path for FlashAttention, broadening the model formats that can run accelerated attention on WebGPU backends.
  191. b9494 Jun 3, 2026 · issue -077

    llama.cpp b9494 enables non-causal vision support for Gemma 4 unified multimodal inference.

    └──▷ GET THIS VERSION
    $ git clone --branch b9494 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9494
    • ›Enables non-causal vision attention in the mtmd (multimodal) subsystem for Gemma 4 unified model inference.
  192. b9489 Jun 3, 2026 · issue -077

    llama.cpp b9489 reserves CUDA memory for quantized KV-cache at startup to avoid runtime allocation failures.

    └──▷ GET THIS VERSION
    $ git clone --branch b9489 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9489
    • ›CUDA backend now reserves GPU memory for quantized KV-cache at startup, preventing runtime out-of-memory errors when using KV-cache quantization.
  193. b9488 Jun 3, 2026 · issue -077

    llama.cpp b9488 adds Qwen3 SSM architecture support and a new LLM_KV_ATTENTION_RECURRENT_LAYERS key.

    └──▷ GET THIS VERSION
    $ git clone --branch b9488 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9488
    • ›Adds LLM_KV_ATTENTION_RECURRENT_LAYERS architecture key to support recurrent layer configuration in GGUF models.
    • ›Extends test coverage to include Qwen3 SSM (state-space model) architectures.
  194. b9482 Jun 2, 2026 · issue -078

    llama.cpp b9482 adds support for the Mellum model architecture.

    └──▷ GET THIS VERSION
    $ git clone --branch b9482 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9482
    • ›Adds support for the Mellum model architecture, enabling inference on Mellum-family models.
  195. b9481 Jun 2, 2026 · issue -078

    llama.cpp b9481 adds support for IBM Granite multilingual embedding R2 models and a new <arch>.hidden_activation GGUF key.

    └──▷ GET THIS VERSION
    $ git clone --branch b9481 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9481
    • ›Adds new GGUF key <arch>.hidden_activation (internal key LLM_KV_HIDDEN_ACT) and a writer, enabling models to declare their hidden-layer activation function (e.g. SwiGLU) in the GGUF metadata.
    • ›Adds llm_ffn_op_type_from_string() helper and a new LLM_FFN_NONE = 0 sentinel to llm_ffn_op_type, centralising hidden-activation mapping in llama-model.cpp.
    • ›Adds support for ibm-granite/granite-embedding-97m-multilingual-r2 and ibm-granite/granite-embedding-311m-multilingual-r2 embedding models, including SwiGLU FFN, a fixed-regex GPT-4o tokenizer variant (97m), and the Gemma4 tokenizer (311m).
    • ›Extends the GGUF converter to emit hidden_activation from config.json and adds model hashes for the Granite Embedding Multilingual R2 models.
  196. b9480 Jun 2, 2026 · issue -078

    llama.cpp b9480 adds support for the StepFun 3.5 MTP model architecture.

    └──▷ GET THIS VERSION
    $ git clone --branch b9480 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9480
    • ›Adds support for the StepFun 3.5 MTP model architecture.
  197. b9478 Jun 2, 2026 · issue -078

    llama.cpp server gains configurable SSE ping interval to keep streaming connections alive.

    └──▷ GET THIS VERSION
    $ git clone --branch b9478 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9478
    • ›Adds SSE ping interval configuration to the server, allowing clients to maintain long-lived streaming connections without timing out.
  198. b9474 Jun 2, 2026 · issue -078

    llama.cpp b9474 adds a Thinking mode toggle with reasoning effort levels to the chat UI.

    └──▷ GET THIS VERSION
    $ git clone --branch b9474 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9474
    • ›Adds a 'Thinking' toggle and status icon to the chat UI, enabling chain-of-thought / reasoning mode for supported models.
    • ›Adds a reasoning effort levels selector in the model dropdown, letting users tune how much reasoning the model applies per request.
    • ›Adds a chat template thinking detector utility and thinking support detection to the models store, so the UI automatically surfaces thinking controls only for capable models.
    • ›Makes the server origin configurable via an environment variable for the UI build.
  199. b9468 Jun 2, 2026 · issue -078

    llama.cpp b9468 adds a POST /v1/chat/completions/control endpoint to interrupt model reasoning mid-generation in real time.

    └──▷ GET THIS VERSION
    $ git clone --branch b9468 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9468
    • ›Adds reasoning_control opt-in flag on completion slots that arms the budget sampler on demand, enabling the new control endpoint to call common_sampler_reasoning_budget_force and halt thinking at any point during streaming.
    • ›Web UI gains a 'skip' button that appears only during the active reasoning phase (tracked via isReasoning streaming state), posts the streamed completion id back to the control endpoint, and is wired through the agentic flow.
  200. b9459 Jun 1, 2026 · issue -079

    Metal GLU kernels now support f16 inputs, reducing memory bandwidth on Apple Silicon.

    └──▷ GET THIS VERSION
    $ git clone --branch b9459 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9459
    • ›Metal GLU kernels (geglu/swiglu) now accept f16 tensor inputs in addition to f32, loading and storing in the native tensor type to reduce memory bandwidth while keeping ALU compute in float for numerical stability.
  201. b9455 Jun 1, 2026 · issue -079

    llama.cpp b9455 adds quantized KV cache support for tensor parallelism (TP) deployments.

    └──▷ GET THIS VERSION
    $ git clone --branch b9455 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9455
    • ›Adds quantized KV cache support for tensor parallelism (TP) configurations, enabling reduced memory footprint in multi-GPU inference setups.
  202. b9453 Jun 1, 2026 · issue -079

    llama.cpp b9453 adds EXAONE 4.5 model support including multimodal vision capabilities.

    └──▷ GET THIS VERSION
    $ git clone --branch b9453 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9453
    • ›Adds EXAONE 4.5 model support with GQA and multimodal vision via <vision> and </vision> image boundary markers, routed through the Qwen2.5-VL-style encode path with window attention and optional mmproj input norm.
    • ›Adds convert_hf_to_gguf support for EXAONE 4.5 mmproj export, enabling conversion of EXAONE 4.5 vision projector weights to GGUF format.
  203. b9442 May 31, 2026 · issue -080

    llama.cpp b9442 adds tokenizer support for jina-embeddings-v2-base-zh with a whitespace tokenizer.

    └──▷ GET THIS VERSION
    $ git clone --branch b9442 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9442
    • ›Adds whitespace tokenizer support for jina-embeddings-v2-base-zh, with lowercase defaulting to true.
  204. b9437 May 30, 2026 · issue -081

    llama-bench gains -fa auto support and -ngl now defaults to -1.

    └──▷ GET THIS VERSION
    $ git clone --branch b9437 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9437
    • ›Adds -fa auto option to llama-bench, enabling automatic flash-attention mode selection during benchmarking.
    • ›Changes the default value of -ngl in llama-bench to -1, aligning it with the behavior of other llama.cpp tools.
  205. b9436 May 30, 2026 · issue -081

    llama.cpp b9436 adds bf16 support for OpenCL backend via f16 conversion

    └──▷ GET THIS VERSION
    $ git clone --branch b9436 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9436
    • ›Adds bf16 support to the OpenCL backend by converting to f16 at runtime, enabling bf16 model inference on OpenCL-capable GPUs.
  206. b9415 May 29, 2026 · issue -082

    llama.cpp b9415 adds a skip_download option to prevent automatic model downloads.

    └──▷ GET THIS VERSION
    $ git clone --branch b9415 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9415
    • ›Adds skip_download flag to the download subsystem, allowing operators to prevent automatic model fetching — including when the target file does not exist locally.
  207. b9414 May 29, 2026 · issue -082

    llama.cpp b9414 adds DeepSeek-OCR 2 support with multi-tile dynamic resolution in the mtmd subsystem.

    └──▷ GET THIS VERSION
    $ git clone --branch b9414 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9414
    • ›Adds DeepSeek-OCR 2 model support in mtmd with multi-tile dynamic resolution via tools/mtmd/models/deepseekocr2.cpp.
  208. b9411 May 29, 2026 · issue -082

    llama.cpp b9411 adds DeepSeek V3.2 model support with sparse attention and NVFP4 quantization.

    └──▷ GET THIS VERSION
    $ git clone --branch b9411 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9411
    • ›Adds support for the DeepseekV32ForCausalLM architecture (LLM_ARCH_DEEPSEEK32) with a generic DeepSeek Sparse Attention (DSA) implementation including a lightning indexer cache.
    • ›Adds convert tooling support for the DeepseekV32ForCausalLM architecture to enable GGUF conversion of DeepSeek V3.2 model weights.
    • ›Adds NVFP4 quantization support for DeepSeek V3.2 models.
    • ›Adds llama_kv_cache_dsa memory type combining KV cache with a lightning indexer cache to accelerate DSA inference.
    • ›Adds f16 support for the GGML_OP_FILL operation in the ggml backend.
  209. b9391 May 29, 2026 · issue -082

    llama.cpp b9391 adds LLAMA_ARG_API_KEY_FILE env var to supply API keys via file instead of inline flag.

    └──▷ GET THIS VERSION
    $ git clone --branch b9391 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9391
    └──▷ TRY IT
    Keep API keys out of shell history or process lists by pointing the server at a key file through the environment.
    $ export LLAMA_ARG_API_KEY_FILE=/etc/llama/api_keys.txt
llama-server --model /models/my-model.gguf
    • ›Adds LLAMA_ARG_API_KEY_FILE environment variable as an alternative to passing --api-key-file directly, letting operators supply API keys via file path through the environment.
  210. b9389 May 29, 2026 · issue -082

    llama.cpp b9389 automatically applies the iGPU flag for integrated CUDA/HIP devices

    └──▷ GET THIS VERSION
    $ git clone --branch b9389 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9389
    • ›Automatically applies the integrated GPU (iGPU) flag for CUDA and HIP backends when an integrated device is detected, removing the need for manual configuration.
  211. b9387 May 29, 2026 · issue -082

    llama.cpp b9387 adds per-quantization MMQ dispatch thresholds on AMD MFMA hardware, yielding up to +76% prompt throughput.

    └──▷ GET THIS VERSION
    $ git clone --branch b9387 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9387
    • ›Introduces ggml_cuda_should_use_mmvq(type, cc, ne11) to apply per-quantization MMVQ/MMQ batch thresholds on AMD CDNA (MFMA) hardware, replacing the single global MMVQ_MAX_BATCH_SIZE = 8 cutoff: K-quants Q3_K/Q4_K/Q5_K switch to MMQ at batch≥4, Q2_K/Q6_K at batch≥6, legacy and IQ quants retain the original threshold — delivering up to +76% pp512 throughput on MI250X.
    • ›Non-AMD-MFMA paths (NVIDIA, RDNA, CDNA1 without MFMA) are unchanged by this dispatch logic update.
  212. b9383 May 28, 2026 · issue -083

    llama.cpp b9383 adds the Granite 4.1 chat template for IBM Granite model support.

    └──▷ GET THIS VERSION
    $ git clone --branch b9383 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9383
    • ›Adds Granite 4.1 chat template, enabling correct prompt formatting for IBM Granite 4.1 models.
  213. b9380 May 28, 2026 · issue -083

    llama-server gains HTTP ETag support for UI asset caching in b9380.

    └──▷ GET THIS VERSION
    $ git clone --branch b9380 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9380
    • ›Adds HTTP ETag support to llama-server, enabling browser-side caching of UI elements via FNV hash-based ETags set on every response.
  214. b9370 May 27, 2026 · issue -084

    llama.cpp b9370 adds Q4_1 quantization support for Hexagon MUL_MAT and MUL_MAT_ID operations via HVX and HMX.

    └──▷ GET THIS VERSION
    $ git clone --branch b9370 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9370
    • ›Adds Q4_1 quantization support to Hexagon MUL_MAT and MUL_MAT_ID operations, with both HVX-only and HMX paths, enabling the Hexagon backend to offload nearly the entire compute graph.
    • ›Adds early-wake support and op-batch completion polling on the Hexagon backend, reducing inference latency when the backend claims full graph ownership.
  215. b9366 May 27, 2026 · issue -084

    llama.cpp b9366 adds Vulkan REPEAT op support for f16-to-f16 tensors.

    └──▷ GET THIS VERSION
    $ git clone --branch b9366 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9366
    • ›Adds Vulkan GPU pipeline support for the REPEAT operation on f16-to-f16 tensor types, enabling models that use this op to run fully on Vulkan backends.
  216. b9354 May 27, 2026 · issue -084

    llama.cpp b9354 adds MiniCPM5 tokenizer support for GGUF conversion

    └──▷ GET THIS VERSION
    $ git clone --branch b9354 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9354
    • ›Adds MiniCPM5 pre-tokenizer support to convert_hf_to_gguf_update.py, enabling conversion of MiniCPM5 models to GGUF format with correct BPE tokenization via hardcoded regex handling in llama-vocab.cpp.
  217. b9319 May 25, 2026 · issue -086

    llama.cpp b9319 adds gguf_init_from_callback and gguf_init_from_buffer for flexible GGUF loading from memory and streaming sources.

    └──▷ GET THIS VERSION
    $ git clone --branch b9319 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9319
    • ›Adds gguf_init_from_buffer to initialize a GGUF context directly from an in-memory buffer, enabling model loading without file I/O.
    • ›Adds gguf_init_from_callback with a gguf_reader_callback_t callback type to stream GGUF data chunk-by-chunk, with max_chunk_read == 0 meaning SIZE_MAX (unlimited chunk size) and max_expected_size and offsets typed as uint64_t for large-model safety.
  218. b9310 May 25, 2026 · issue -086

    llama.cpp b9310 adds --checkpoint-min-step flag and autoparser message-barrier detection for smarter server checkpointing.

    └──▷ GET THIS VERSION
    $ git clone --branch b9310 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9310
    └──▷ TRY IT
    Prevent excessive checkpoint creation during long prompts by setting a minimum token step between checkpoints.
    $ llama-server --checkpoint-min-step 256 -m model.gguf
    • ›Adds --checkpoint-min-step CLI flag to control minimum spacing between server context checkpoints.
    • ›Adds common_chat_split_by_role to split chat output by role, enabling message-span extraction from chat templates.
    • ›Supports autoparser detection for message barriers, allowing the server to automatically identify prompt split points without manual configuration.
    • ›Enables multimodal prompt support when mapping text/template positions to server prompt tokens during checkpoint creation.
  219. b9291 May 22, 2026 · issue -089

    llama.cpp b9291 improves SYCL MoE prefill throughput via counting sort-based expert routing.

    └──▷ GET THIS VERSION
    $ git clone --branch b9291 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9291
    • ›Improves SYCL MoE prefill throughput by replacing an O(n_as * n_routed_rows) expert-routing loop in k_copy_src1_to_contiguous with a counting sort-based procedure of O(n_as + n_routed_rows) complexity, enabling faster batch inference on SYCL backends.
  220. b9279 May 22, 2026 · issue -089

    Vulkan backend gains fused snake activation kernel for audio decoder models (BigVGAN, Vocos) across F32/F16/BF16.

    └──▷ GET THIS VERSION
    $ git clone --branch b9279 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9279
    • ›Adds a fused Vulkan kernel (snake.comp) that collapses the 5-op snake activation decomposition (mul, sin, sqr, mul, add) into a single elementwise dispatch via ggml_vk_snake_dispatch_fused, accelerating audio decoder models such as BigVGAN and Vocos on Vulkan backends across F32, F16, and BF16 precisions.
  221. b9276 May 22, 2026 · issue -089

    llama.cpp server now exposes prompt token counts in the /slots endpoint for client-side progress monitoring.

    └──▷ GET THIS VERSION
    $ git clone --branch b9276 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9276
    • ›Adds n_prompt_tokens, n_prompt_tokens_processed, and n_prompt_tokens_cache fields to the GET /slots JSON response, enabling clients to monitor prompt evaluation progress during processing.
  222. b9272 May 21, 2026 · issue -090

    llama.cpp b9272 adds batched-bench, fit-params, quantize, and perplexity as bundled app commands.

    └──▷ GET THIS VERSION
    $ git clone --branch b9272 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9272
    • ›Adds batched-bench, fit-params, quantize, and perplexity as built-in app subcommands.
  223. b9270 May 21, 2026 · issue -090

    llama.cpp b9270 adds native HybridDNATokenizer support for HuggingFaceBio Carbon models via a new LLAMA_VOCAB_TYPE_HYBRIDDNA vocab type.

    └──▷ GET THIS VERSION
    $ git clone --branch b9270 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9270
    • ›Adds LLAMA_VOCAB_TYPE_HYBRIDDNA = 7 in include/llama.h as a first-class vocab type (peer to SPM/WPM/UGM/RWKV/PLAMO2) for the HybridDNATokenizer used by HuggingFaceBio/Carbon-{500M,3B,8B}.
    • ›Adds llm_tokenizer_hybriddna and its session in src/llama-vocab.cpp, routing text through a DNA-aware splitter that chunks sequences inside <dna>...</dna> regions into fixed 6-mers (right-padded with A), maps non-ACGT bases to <oov>, and delegates non-DNA text to an embedded BPE tokenizer.
    • ›Adds _set_vocab_hybriddna in conversion/base.py that writes tokenizer.ggml.model = 'hybriddna' during GGUF conversion, dispatched from conversion/llama.py when tokenizer_config.json["tokenizer_class"] == "HybridDNATokenizer".
    └──▷ BREAKING ON UPGRADE
    • !The short-lived LLAMA_VOCAB_PRE_TYPE_CARBON (introduced during this same development branch) is removed from src/llama-vocab.h in favour of LLAMA_VOCAB_TYPE_HYBRIDDNA; any code referencing LLAMA_VOCAB_PRE_TYPE_CARBON will fail to compile.
  224. b9263 May 21, 2026 · issue -090

    llama.cpp b9263 merges HunyuanOCR into the HunyuanVL projector path, restoring correct vision precision.

    └──▷ GET THIS VERSION
    $ git clone --branch b9263 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9263
    • ›Merges HunyuanOCR into the HUNYUANVL projector and HUNYUAN_VL text architecture, consolidating OCR and vision-language into a single code path with the +0.1 bilinear sampler applied correctly.
  225. b9254 May 20, 2026 · issue -091

    llama.cpp b9254 adds Programmatic Dependent Launch (PDL) for NVIDIA Hopper+ GPUs, toggled via GGML_CUDA_PDL.

    └──▷ GET THIS VERSION
    $ git clone --branch b9254 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9254
    └──▷ TRY IT
    Disable PDL on a Hopper GPU if you suspect it causes regressions or need a reproducible baseline.
    $ GGML_CUDA_PDL=0 ./llama-cli -m model.gguf -p 'Hello'
    • ›Adds GGML_CUDA_PDL environment variable to enable or disable Programmatic Dependent Launch (PDL) on NVIDIA Hopper+ GPUs; set GGML_CUDA_PDL=0 to disable.
    • ›Introduces PDL support for NVIDIA Hopper+ (SM90+) GPUs, overlapping CUDA kernel execution to improve inference throughput; enabled by default on qualifying devices, excluded on Ada and below.
  226. b9253 May 20, 2026 · issue -091

    llama.cpp b9253 introduces a unified llama executable with a serve subcommand for the server.

    └──▷ GET THIS VERSION
    $ git clone --branch b9253 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9253
    • ›Introduces the llama unified executable, consolidating multiple tools under a single binary with subcommands including serve (for the server) and help.
  227. b9244 May 20, 2026 · issue -091

    llama.cpp b9244 adds OpenCL MoE support for q4_k, q5_k, and q6_k quantizations on Adreno GPUs.

    └──▷ GET THIS VERSION
    $ git clone --branch b9244 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9244
    • ›Adds OpenCL Mixture-of-Experts (MoE) inference support for q4_k, q5_k, and q6_k quantization formats on Adreno GPUs.
  228. b9180 May 16, 2026 · issue -095

    llama.cpp b9180 adds Multi-Token Prediction (MTP) speculative decoding support across CPU, Metal, and Vulkan backends.

    └──▷ GET THIS VERSION
    $ git clone --branch b9180 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9180
    • ›Adds MTP (Multi-Token Prediction) support for speculative decoding, with draft models identified by the draft-mtp prefix in model naming.
    • ›Adds partial sequence rollback for GDN (Gated Delta Net) recurrent models during speculative decoding, storing intermediates up to draft_max tokens to avoid full restart on draft rejection.
    • ›Extends Metal backend with GDN partial rollback support via a new kernel function constant K (snapshot slot count) controlling the 3D state tensor slot behavior.
    • ›Extends Vulkan backend with GDN partial rollback support for speculative decoding.
    • ›Enables checkpointing with partial rollback in llama-memory for recurrent model contexts.
    +1 moreshow less
    • ›Server disables RS-based MTP when other speculative decoding types are active, preventing incompatible combinations.
  229. b9174 May 16, 2026 · issue -095

    llama.cpp b9174 renames WebUI surfaces to UI: new --ui, --ui-config, --ui-config-file, and --ui-mcp-proxy flags plus matching env vars.

    └──▷ GET THIS VERSION
    $ git clone --branch b9174 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9174
    └──▷ TRY IT
    Start the llama-server with the web UI and MCP proxy enabled using the new canonical flags.
    $ llama-server --ui --ui-mcp-proxy --ui-config-file /etc/llama/ui-config.json -m model.gguf
    Enable the web UI via environment variable in a containerised deployment instead of passing CLI flags.
    $ LLAMA_ARG_UI=1 LLAMA_ARG_UI_MCP_PROXY=1 llama-server -m model.gguf
    • ›Adds --ui / --no-ui CLI flags (replacing deprecated --webui / --no-webui) to control the built-in web interface.
    • ›Adds --ui-config CLI flag (replacing deprecated --webui-config) for passing UI configuration.
    • ›Adds --ui-config-file CLI flag (replacing deprecated --webui-config-file) for specifying the UI config file path.
    • ›Adds --ui-mcp-proxy / --no-ui-mcp-proxy CLI flags (replacing deprecated --webui-mcp-proxy) to toggle MCP proxy support in the UI.
    • ›Adds environment variables LLAMA_ARG_UI, LLAMA_ARG_UI_CONFIG, LLAMA_ARG_UI_CONFIG_FILE, and LLAMA_ARG_UI_MCP_PROXY as alternatives to the corresponding CLI flags.
    +3 moreshow less
    • ›Renames CMake build variable LLAMA_BUILD_WEBUI to LLAMA_BUILD_UI and LLAMA_USE_PREBUILT_WEBUI to LLAMA_USE_PREBUILT_UI (old names kept as deprecated aliases).
    • ›Moves UI source tree from tools/server/public / tools/server/webui/ to tools/ui/ and build output to build/tools/ui/dist.
    • ›JSON API now emits both ui / ui_settings and legacy webui / webui_settings keys for backward compatibility.
  230. b9161 May 15, 2026 · issue -096

    llama.cpp b9161 adds Codex CLI compatibility by gracefully skipping unsupported Responses API tools.

    └──▷ GET THIS VERSION
    $ git clone --branch b9161 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9161
    • ›Adds support for Codex CLI by skipping unsupported Responses tools instead of failing, with a warning emitted for each skipped tool.
  231. b9158 May 14, 2026 · issue -097

    llama.cpp b9158 adds RDNA3 tensor core support for flash attention and extends CDNA head size support to 256.

    └──▷ GET THIS VERSION
    $ git clone --branch b9158 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9158
    • ›Adds RDNA3 tensor core (MMA) support to the CUDA flash attention kernel, using FP16 accumulation with 32-unit tiles for head sizes evenly divisible by 32.
    • ›Extends CDNA1 flash attention support to attention head sizes up to 256.
    • ›Improves AMD matrix transposition efficiency for RDNA3 and RDNA4 via wider 32-unit tiles that align with the warp size of 32.
    • ›Adds a new ggml_cuda_mma::data_layout entry to guard against accumulator data layout misuse introduced by the new RDNA3/4 tiling scheme.
    • ›Tunes flash attention kernel parameters for RDNA3, RDNA4, and CDNA1 architectures.
  232. b9145 May 14, 2026 · issue -097

    SYCL backend gains Level Zero memory allocation to eliminate multi-GPU host RAM mirroring on Intel Arc systems.

    └──▷ GET THIS VERSION
    $ git clone --branch b9145 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9145
    └──▷ TRY IT
    Disable Level Zero memory allocation at runtime to fall back to standard SYCL paths, useful for diagnosing allocation issues on Intel Arc multi-GPU systems.
    $ GGML_SYCL_ENABLE_LEVEL_ZERO=0 ./llama-cli -m model.gguf -p 'Hello'
    • ›Adds GGML_SYCL_ENABLE_LEVEL_ZERO runtime environment variable (default 1) to switch the SYCL backend between Level Zero (zeMemAllocDevice) and standard SYCL (sycl::malloc_device) memory APIs, eliminating the 1:1 VRAM-to-host-RAM mirroring that caused OOM crashes on multi-GPU Intel Arc systems.
    • ›Adds GGML_SYCL_SUPPORT_LEVEL_ZERO CMake build option (default ON) to compile in Level Zero support when the SDK headers and loader are present, with automatic fallback to the SYCL allocation path when Level Zero is unavailable.
    • ›Adds Windows Level Zero SDK path detection via LEVEL_ZERO_V1_SDK_PATH in CMakeLists.txt for building with Level Zero on Windows.
    • ›Level Zero allocation path automatically disables itself with a warning at runtime if the detected devices do not support the Level Zero backend, preserving compatibility on non-dGPU or CPU-only systems.
  233. b9142 May 14, 2026 · issue -097

    OpenCL backend gains q5_0 and q5_1 MoE quantization support for Adreno GPUs.

    └──▷ GET THIS VERSION
    $ git clone --branch b9142 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9142
    • ›Adds q5_0 and q5_1 MoE (Mixture-of-Experts) kernel support to the OpenCL backend, enabling quantized MoE inference on Adreno GPUs.
  234. b9141 May 14, 2026 · issue -097

    llama.cpp b9141 adds continue_final_message body flag for vLLM/transformers API compatibility in the server and WebUI.

    └──▷ GET THIS VERSION
    $ git clone --branch b9141 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9141
    └──▷ TRY IT
    Trigger assistant prefill via the vLLM-compatible flag instead of server-side prefill_assistant — useful when integrating llama.cpp into pipelines that already target the vLLM API.
    $ curl -s http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{"model": "local", "messages": [{"role": "user", "content": "Hello"}, {"role": "assistant", "content": "Sure, I"}], "add_generation_prompt": false, "continue_final_message": true}'
    • ›Adds continue_final_message request body flag to the server API; when set with add_generation_prompt: false, it triggers the assistant prefill code path — aliasing the existing prefill_assistant heuristic for vLLM/transformers API compatibility.
    • ›Enforces mutual exclusion between continue_final_message: true and add_generation_prompt: true, returning HTTP 400 on conflict, matching vLLM/transformers behavior.
    • ›WebUI 'Continue' button now sends continue_final_message and add_generation_prompt: false automatically, with a matching opt-in option on the chat service.
  235. b9133 May 13, 2026 · issue -098

    llama.cpp server and WebUI now support continuing generation on reasoning models with chain-of-thought preservation.

    └──▷ GET THIS VERSION
    $ git clone --branch b9133 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9133
    • ›Server and WebUI support continued generation on reasoning models by orchestrating thinking_start_tag / thinking_end_tag pairs around prefilled messages, so the stream parser routes chunks correctly after a resume.
    • ›Server rejects reasoning prefill on channel-based templates (e.g. GPT-OSS) with an explicit error, limiting scope to simple thinking-tag templates.
  236. b9129 May 13, 2026 · issue -098

    llama.cpp b9129 adds GGML_ZENDNN_ADAPTIVE_FALLBACK env var to control ZenDNN-to-CPU fallback for small batch sizes.

    └──▷ GET THIS VERSION
    $ git clone --branch b9129 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9129
    • ›Adds GGML_ZENDNN_ADAPTIVE_FALLBACK environment variable (default: enabled) to control whether the ZenDNN backend adaptively falls back to the CPU backend for small batch sizes.
  237. b9127 May 12, 2026 · issue -099

    llama.cpp b9127 adds opt-in Adreno cross-memory F16xF32 GEMM kernel for faster prefill on OpenCL.

    └──▷ GET THIS VERSION
    $ git clone --branch b9127 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9127
    • ›Adds opt-in Adreno cross-memory (xmem) F16xF32 GEMM kernel for the OpenCL backend, accelerating prefill on Adreno GPUs.
  238. b9124 May 12, 2026 · issue -099

    llama.cpp b9124 exposes model modalities via mtmd_caps on the /v1/models API endpoint.

    └──▷ GET THIS VERSION
    $ git clone --branch b9124 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9124
    • ›Adds mtmd_caps field to the /v1/models API response, exposing the modalities (e.g. text, vision) supported by a loaded model.
  239. b9123 May 12, 2026 · issue -099

    WebGPU backend gains support for running the gpt-oss-20b model via a refactored mulmat-q kernel.

    └──▷ GET THIS VERSION
    $ git clone --branch b9123 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9123
    • ›Enables running the gpt-oss-20b model on the WebGPU backend (ggml-webgpu), expanding large-model support without CUDA or Vulkan.
  240. b9116 May 12, 2026 · issue -099

    llama.cpp b9116 adds multimodal vision support for MiMo v2.5 models.

    └──▷ GET THIS VERSION
    $ git clone --branch b9116 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9116
    • ›Adds vision (multimodal) support for MiMo v2.5 models via the mtmd multimodal subsystem, including fused QKV for the vision encoder.
  241. b9109 May 11, 2026 · issue -100

    llama.cpp b9109 adds parallel speculative drafting with support for multiple chained speculators in the server and CLI.

    └──▷ GET THIS VERSION
    $ git clone --branch b9109 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9109
    • ›Introduces common_speculative_type_from_names() to parse user-provided speculator types as a vector, enabling multiple speculative decoding strategies to be specified at once.
    • ›Introduces common_get_enabled_speculative_impls() to determine which speculative decoding implementations are active based on the provided type vector.
    • ›Introduces common_speculative_process() as a unified entry point for running multiple speculators sequentially, selecting the best draft by maximizing expected accepted tokens.
    • ›Replaces the single type field in common_params_speculative with a vector of common_speculative_type values, allowing multiple speculator types to be configured in one pass.
    • ›Adds parallel drafting support to the server, enabling concurrent speculative decoding across multiple request slots via a shared draft context.
    +3 moreshow less
    • ›Adds draft prompt cache and checkpoints to the server for more efficient speculative decoding across slots.
    • ›Supports chaining multiple speculators where all run sequentially and the best-performing draft (by expected accepted tokens) is verified and committed.
    • ›Reuses device buffers across draft and main contexts when possible, reducing memory overhead during parallel speculative decoding.
    └──▷ BREAKING ON UPGRADE
    • !The type field of type common_speculative_type in common_params_speculative is replaced with a vector of common_speculative_type; any code or config directly setting a single type value will need to be updated to use a vector.
  242. b9095 May 10, 2026 · issue -101

    llama.cpp b9095 adds a NCCL-free internal AllReduce CUDA kernel for 2-GPU tensor parallelism, selectable via GGML_CUDA_ALLREDUCE or --reduction-provider.

    └──▷ GET THIS VERSION
    $ git clone --branch b9095 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9095
    └──▷ TRY IT
    Force the internal NCCL-free AllReduce provider for a 2-GPU tensor-parallel run to avoid NCCL dependencies.
    $ GGML_CUDA_ALLREDUCE=internal llama-bench --model <model.gguf> -ts 1,1
    Benchmark tensor-parallel inference with the NCCL provider explicitly selected to isolate regressions from the internal provider.
    $ llama-bench --model <model.gguf> -ts 1,1 --reduction-provider nccl
    • ›Adds GGML_CUDA_ALLREDUCE environment variable accepting 'nccl' or 'internal' to select the AllReduce provider for LLAMA_SPLIT_MODE_TENSOR multi-GPU inference.
    • ›Adds --reduction-provider / -rp flag to llama-bench accepting auto, nccl, or internal to control the AllReduce provider per benchmark run.
    • ›Adds GGML_CUDA_NCCL_STATIC CMake option (default OFF) to switch NCCL linking between static and dynamic, with updated FindNCCL.cmake searching Windows source-build layouts including cmake/lib/Release, cmake/src/Release, and src/include.
    • ›Introduces a NCCL-free internal AllReduce CUDA kernel for 2-GPU FP32 tensor parallelism (tensors up to 256 KB) that pipelines D2H copy, cross-GPU handshake via pinned-memory volatile flags, and reduction in a single kernel launch per GPU — no NCCL dependency required.
    • ›Adds an optional AllReduce hang watchdog (enabled by setting #define GGML_CUDA_AR_WATCHDOG 1 in allreduce.cu) with runtime tuning via GGML_CUDA_AR_WATCHDOG (ms timeout) and GGML_CUDA_AR_MAX_SPIN (kernel bailout) environment variables; zero overhead when disabled.
  243. b9093 May 9, 2026 · issue -102

    llama.cpp b9093 adds support for the Sarvam MoE model architecture.

    └──▷ GET THIS VERSION
    $ git clone --branch b9093 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9093
    • ›Adds sarvam_moe architecture support, enabling inference on Sarvam mixture-of-experts models.
  244. b9082 May 8, 2026 · issue -103

    llama.cpp b9082 adds an L2_NORM HVX kernel for the Hexagon backend.

    └──▷ GET THIS VERSION
    $ git clone --branch b9082 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9082
    • ›Adds an L2_NORM HVX kernel for the Hexagon backend, enabling hardware-accelerated L2 normalization on Qualcomm Hexagon DSPs.
  245. b9080 May 8, 2026 · issue -103

    llama.cpp b9080 adds GGUF conversion support for the Gemma4 26B A4B NVFP4 (NVIDIA FP4) checkpoint format.

    └──▷ GET THIS VERSION
    $ git clone --branch b9080 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9080
    • ›Adds GGUF conversion support for the Gemma4_26B_A4B_NVFP4 HuggingFace checkpoint format, enabling the Gemma 4 26B mixture-of-experts model with NVIDIA FP4 quantization to be used with llama.cpp.
  246. b9077 May 8, 2026 · issue -103

    llama.cpp server gains Vertex AI compatible API mode, activated via the AIP_MODE environment variable.

    └──▷ GET THIS VERSION
    $ git clone --branch b9077 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9077
    • ›Adds Vertex AI compatible API support to the server, enabled when the AIP_MODE environment variable is set (along with other AIP_* env vars).
  247. b9076 May 8, 2026 · issue -103

    llama.cpp b9076: router server now exposes child model info via GET /v1/models

    └──▷ GET THIS VERSION
    $ git clone --branch b9076 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9076
    └──▷ TRY IT
    Discover all child models registered behind a llama.cpp router instance.
    $ curl http://localhost:8080/v1/models
    • ›The router server's GET /v1/models endpoint now exposes metadata for child models, enabling clients to discover all models behind a router instance.
  248. b9063 May 7, 2026 · issue -104

    Adds OpenCL operation-filter regex for targeted GPU kernel debugging in llama.cpp b9063.

    └──▷ GET THIS VERSION
    $ git clone --branch b9063 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9063
    • ›Adds an opfilter regex to the OpenCL backend to selectively enable or disable specific GPU operations during debugging.
  249. b9060 May 7, 2026 · issue -104

    llama.cpp b9060 adds FILL, CUMSUM, DIAG, SOLVE_TRI, SSM_SCAN, and GATED_DELTA_NET ops to the SYCL backend.

    └──▷ GET THIS VERSION
    $ git clone --branch b9060 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9060
    • ›Adds FILL, CUMSUM, DIAG, SOLVE_TRI, SSM_SCAN, and GATED_DELTA_NET operations to the SYCL backend, expanding GPU-accelerated model coverage on Intel hardware.
  250. b9055 May 7, 2026 · issue -104

    llama.cpp b9055 adds Mimo v2.5 model support including MTP weights in GGUF

    └──▷ GET THIS VERSION
    $ git clone --branch b9055 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9055
    • ›Adds support for the Mimo v2.5 model architecture, including tensor-parallel (TP) handling and MTP weights embedded in GGUF output.
  251. b9049 May 6, 2026 · issue -105

    llama.cpp b9049 adds multimodal support for MiniCPM-V 4.6 with flash attention compatibility.

    └──▷ GET THIS VERSION
    $ git clone --branch b9049 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9049
    • ›Adds support for the MiniCPM-V 4.6 multimodal model via the TYPE_MINICPMV4_6 model type, including flash attention support through build_attn.
  252. b9045 May 6, 2026 · issue -105

    llama.cpp b9045 adds multimodal speech support for IBM Granite 4.0 1B Speech via the mtmd layer.

    └──▷ GET THIS VERSION
    $ git clone --branch b9045 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9045
    • ›Adds ibm-granite/granite-4.0-1b-speech model support in mtmd, enabling Conformer-encoder speech-to-text with QFormer projector, 80-bin log-mel spectrogram, and 2x frame stacking (80→160 mel dims).
    • ›GGUF converter (convert) now handles batch-norm folding, fused K/V split, and Conv1d weight reshaping for Granite Speech models at export time, and registers GraniteModel for GraniteSpeechForConditionalGeneration.
  253. b9038 May 6, 2026 · issue -105

    llama.cpp b9038 improves OpenCL memory estimation for --fit using CL_DEVICE_GLOBAL_MEM_SIZE.

    └──▷ GET THIS VERSION
    $ git clone --branch b9038 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9038
    • ›Uses CL_DEVICE_GLOBAL_MEM_SIZE to report estimated OpenCL device memory when using --fit, enabling more accurate model-to-device fitting decisions on OpenCL backends.
  254. b9023 May 4, 2026 · issue -107

    llama.cpp server gains GET /models?reload=1 endpoint to reload the model list at runtime.

    └──▷ GET THIS VERSION
    $ git clone --branch b9023 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9023
    └──▷ TRY IT
    Trigger a live reload of the available models list from a running llama.cpp server instance.
    $ curl -s 'http://localhost:8080/models?reload=1'
    • ›Adds reload=1 query parameter to the GET /models server endpoint, enabling on-demand model list reloads without restarting the server.
  255. b9018 May 4, 2026 · issue -107

    llama.cpp b9018 adds a built-in get_datetime server tool for time-aware LLM interactions.

    └──▷ GET THIS VERSION
    $ git clone --branch b9018 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9018
    • ›Adds get_datetime as a built-in server tool, giving the llama.cpp server a native way to expose current date/time to the model without custom tool implementations.
  256. b9014 May 4, 2026 · issue -107

    llama.cpp b9014 adds layer norm ops to the WebGPU backend shader.

    └──▷ GET THIS VERSION
    $ git clone --branch b9014 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9014
    • ›Adds layer norm ops to the ggml-webgpu shader backend, enabling WebGPU-accelerated normalization layers.
  257. b9000 May 2, 2026 · issue -109

    llama.cpp b9000 adds HMX-accelerated flash attention for Hexagon HTP prefill, with multi-threaded Q/O phases and FP16 softmax.

    └──▷ GET THIS VERSION
    $ git clone --branch b9000 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b9000
    • ›Adds HMX-accelerated flash attention (op_hmx_flash_attn_ext) for the Qualcomm Hexagon HTP backend, enabling hardware-accelerated prefill across GQA, ALiBi, softcap, and masked attention shapes.
    • ›Enables multi-threaded Q-load and O-store phases via fa_phase_q_load and fa_phase_o_store worker-pool helpers, parallelising F32↔F16 conversion across row ranges on HVX threads.
    • ›Adds experimental FP16 softmax path (FA_EXP_F16, controlled via CMake host-side option) using hvx_exp2_hf directly — ~22 ALU ops per 64 lanes versus ~44 for the F32 round-trip path.
    • ›Relaxes the HMX matmul pipeline gate to cover k > n shapes (e.g. FFN_down layers), selecting pipeline vs. sequential layout based on actual chunker output rather than a static m>=128 && n>=256 threshold.
    • ›Adds HMX flash-attention test coverage in test-backend-ops over the full {mask, ALiBi, logit_softcap} cross-product for prefill — head dims 64/128, GQA 4×4, kv=512/nb=64 and kv=113/nb=32 non-aligned cases, F16 KV.
  258. b8995 May 1, 2026 · issue -110

    Vulkan coopmat2 path gains asymmetric Flash Attention support, including Q1_0 quantization type.

    └──▷ GET THIS VERSION
    $ git clone --branch b8995 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8995
    • ›Adds asymmetric Flash Attention support to the Vulkan coopmat2 shader path, enabling mixed quantization types for FA workloads.
    • ›Adds Q1_0 quantization type support in the Vulkan coopmat2 Flash Attention shader path.
  259. b8980 Apr 29, 2026 · issue -112

    llama.cpp b8980 adds configurable vmem size and buffer counts for Hexagon/Snapdragon backends.

    └──▷ GET THIS VERSION
    $ git clone --branch b8980 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8980
    • ›Adds host-side control over maximum vmem size for the Hexagon backend, with a new default of 3.2 GB and support for autodetection of available vmem space.
    • ›Increases Hexagon op buffer count to 16 (matching the maximum number of mmaps) and moves pinned mmap management to the host.
  260. b8966 Apr 28, 2026 · issue -113

    llama.cpp b8966 adds CUDA flash-attention support for Mistral Small 4's non-standard head sizes (DKQ=320/DV=256) with GQA=32.

    └──▷ GET THIS VERSION
    $ git clone --branch b8966 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8966
    • ›Adds CUDA flash-attention (MMA-f16) kernel support for head sizes DKQ=320/DV=256 with ncols2=32, enabling GPU-accelerated inference for Mistral Small 4's GQA ratio of 32.
  261. b8956 Apr 28, 2026 · issue -113

    llama.cpp b8956 adds seven new CANN ops and fused GLU/cross-entropy kernels for Ascend NPU backends.

    └──▷ GET THIS VERSION
    $ git clone --branch b8956 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8956
    • ›Adds GGML_OP_SET to the CANN backend, implemented via aclnnInplaceCopy on the target region.
    • ›Adds GGML_OP_CUMSUM to the CANN backend, implemented via aclnnCumsum.
    • ›Adds GGML_OP_FILL to the CANN backend, implemented via aclnnInplaceFillScalar.
    • ›Adds GGML_OP_DIAG to the CANN backend, implemented via aclnnInplaceCopy on diagonal strides.
    • ›Adds GGML_OP_TRI (lower/lower_diag/upper_diag/upper modes) to the CANN backend via aclnnTril and aclnnTriu with appropriate diagonal offsets.
    +5 moreshow less
    • ›Adds GGML_OP_SOLVE_TRI to the CANN backend via aclnnTriangularSolve.
    • ›Adds GGML_UNARY_OP_SOFTPLUS to the CANN backend via aclnnSoftplus.
    • ›Fuses GLU variants (SwiGLU, GeGLU, GeGLU_ERF, GeGLU_QUICK) into single aclnnSwiGlu/aclnnGeGluV3 calls on the CANN backend when applicable.
    • ›Replaces the five-kernel cross-entropy sequence (LogSoftmax→Mul→ReduceSum×2→Muls) with a single aclnnSoftmaxCrossEntropyWithLogits call on the CANN backend.
    • ›Implements CANN backend memset_tensor via aclrtMemset, replacing a NULL stub.
  262. b8952 Apr 27, 2026 · issue -114

    llama.cpp server router now forwards form-data, enabling /v1/audio/transcriptions in router mode.

    └──▷ GET THIS VERSION
    $ git clone --branch b8952 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8952
    • ›Enables the server router to forward multipart/form-data requests to model servers, unlocking use of the /v1/audio/transcriptions endpoint in router mode.
  263. b8935 Apr 26, 2026 · issue -115

    llama.cpp b8935 adds IQ4_NL quantization support for OpenCL, including optimized GEMM/GEMV kernels for Adreno GPUs.

    └──▷ GET THIS VERSION
    $ git clone --branch b8935 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8935
    • ›Adds iq4_nl quantization format support to the OpenCL backend, enabling inference with IQ4_NL-quantized models on OpenCL-capable devices.
    • ›Adds optimized iq4_nl GEMM and GEMV kernels specifically for Adreno GPUs via OpenCL, improving throughput for quantized model workloads on Qualcomm mobile hardware.
  264. b8929 Apr 25, 2026 · issue -116

    llama.cpp b8929 changes the default quantization type in llama_model_quantize_params from Q5_1 to Q8_0.

    └──▷ GET THIS VERSION
    $ git clone --branch b8929 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8929
    • ›Changes the default ftype in llama_model_quantize_params from LLAMA_FTYPE_MOSTLY_Q5_1 to LLAMA_FTYPE_MOSTLY_Q8_0, so programs using default quantization params now produce higher-quality Q8_0 models instead of the older Q5_1 format.
    └──▷ BREAKING ON UPGRADE
    • !The default ftype field in llama_model_quantize_params is changed from LLAMA_FTYPE_MOSTLY_Q5_1 to LLAMA_FTYPE_MOSTLY_Q8_0; any code that relies on the default quantization type without explicitly setting ftype will now produce Q8_0 output instead of Q5_1.
  265. b8911 Apr 24, 2026 · issue -117

    llama.cpp b8911 adds configurable Hexagon op profiling with PMU counter support via the GGML_HEXAGON_PROFILE environment variable.

    └──▷ GET THIS VERSION
    $ git clone --branch b8911 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8911
    └──▷ TRY IT
    Post-process a saved Hexagon profiler log (or pipe it live) through the new profile tool to get a human-readable per-op summary.
    $ cat hexagon_profile.log | python3 scripts/snapdragon/ggml-hexagon-profile.py
    • ›Adds GGML_HEXAGON_PROFILE environment variable to enable and configure Hexagon HTP op profiling, including optional PMU counter collection (made optional because PMU counters are expensive on older devices).
    • ›Adds support for setting profiler PMU events from the environment on Hexagon HTP backends.
    • ›Adds scripts/snapdragon/ggml-hexagon-profile.py, a post-processing tool for Hexagon profiler output, with support for reading logs from stdin.
  266. b8902 Apr 23, 2026 · issue -118

    llama.cpp b8902 enables the transcriptions API for LFM2-Audio models in the server.

    └──▷ GET THIS VERSION
    $ git clone --branch b8902 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8902
    • ›Enables the transcriptions API for LFM2-Audio models in the server.
  267. b8893 Apr 23, 2026 · issue -118

    llama.cpp b8893 enables HIP graphs by default on ROCm, boosting AMD GPU inference throughput.

    └──▷ GET THIS VERSION
    $ git clone --branch b8893 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8893
    • ›Enables GGML_HIP_GRAPHS by default on ROCm backends, delivering improved inference performance on AMD GPUs without any configuration change.
    └──▷ BREAKING ON UPGRADE
    • !GGML_HIP_GRAPHS now defaults to on; users who previously relied on the off-by-default behavior to work around ROCm graph issues may need to explicitly set GGML_HIP_GRAPHS=0 to restore the old behavior.
  268. b8891 Apr 22, 2026 · issue -119

    WebGPU backend gains fused RMS_NORM + MUL kernel with optional GGML_WEBGPU_DISABLE_FUSION escape hatch.

    └──▷ GET THIS VERSION
    $ git clone --branch b8891 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8891
    • ›Adds fused rms_norm_mul kernel for the WebGPU backend, combining RMS normalisation and element-wise multiply into a single GPU dispatch to reduce overhead.
  269. b8885 Apr 22, 2026 · issue -119

    llama.cpp b8885 adds HunyuanVL vision-language model support with M-RoPE and GGUF conversion.

    └──▷ GET THIS VERSION
    $ git clone --branch b8885 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8885
    • ›Adds LLM_ARCH_HUNYUAN_VL architecture with M-RoPE (XD-RoPE) multi-dimensional position encoding support for HunyuanVL image tokens.
    • ›Adds PROJECTOR_TYPE_HUNYUANVL with PatchMerger vision encoder for HunyuanVL multimodal inference.
    • ›Adds GGUF conversion support for HunyuanVL vision and text models (HunyuanVL and HunyuanOCR) via convert_hf_to_gguf.py, verified with F16 and Q8_0 quantization on Metal.
  270. b8882 Apr 22, 2026 · issue -119

    llama.cpp b8882 adds WebGPU conv2d shader kernels with f32 and f16 support.

    └──▷ GET THIS VERSION
    $ git clone --branch b8882 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8882
    • ›Adds conv2d shader kernels to the WebGPU (Dawn) backend, passing f32 and f16 tests.
  271. b8875 Apr 21, 2026 · issue -120

    llama.cpp b8875 adds Reka Edge 2603 multimodal support with Yasa2 (ConvNeXtV2) vision encoder and reka-edge chat template.

    └──▷ GET THIS VERSION
    $ git clone --branch b8875 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8875
    • ›Adds PROJECTOR_TYPE_YASA2 (ConvNeXtV2-based) vision encoder support for Reka Edge, wired into clip.cpp and mtmd.cpp init_vision, using mtmd_image_preprocessor_fixed_size with image_resize_algo = RESIZE_ALGO_BICUBIC and image_resize_pad = false.
    • ›Adds chat-reka.cpp/chat-reka.h implementing a PEG-based parser for the reka-edge chat format, including tool-use and thinking support, with a new Reka-Edge.jinja chat template detected automatically via try_specialized_template().
  272. b8874 Apr 21, 2026 · issue -120

    llama.cpp b8874 adds the --spec-default CLI argument for speculative decoding configuration.

    └──▷ GET THIS VERSION
    $ git clone --branch b8874 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8874
    • ›Adds --spec-default CLI argument to configure default speculative decoding behavior.
  273. b8842 Apr 19, 2026 · issue -122

    llama.cpp server gains speculative decoding with checkpoints, including multimodal (mtmd) support.

    └──▷ GET THIS VERSION
    $ git clone --branch b8842 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8842
    • ›Adds speculative decoding with checkpointing to the server, enabling faster token generation by saving and restoring draft model state across speculative sessions.
    • ›Enables speculative decoding support for multimodal (mtmd) workloads in the server.
    └──▷ BREAKING ON UPGRADE
    • !The --spec-use-checkpoints argument has been removed.
  274. b8840 Apr 18, 2026 · issue -123

    llama.cpp b8840 exposes media_tag on the /props server endpoint.

    └──▷ GET THIS VERSION
    $ git clone --branch b8840 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8840
    └──▷ TRY IT
    Query the server properties endpoint to retrieve the newly exposed media_tag value for a running llama.cpp server instance.
    $ curl http://localhost:8080/props | jq '.media_tag'
    • ›Exposes media_tag field on the /props endpoint of the llama.cpp server API.
  275. b8831 Apr 17, 2026 · issue -124

    llama.cpp b8831 adds official Android arm64 CPU builds and fixes server EOS flag handling.

    └──▷ GET THIS VERSION
    $ git clone --branch b8831 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8831
    • ›Adds official Android arm64 (CPU) binary release (llama-b8831-bin-android-arm64.tar.gz) via new CI pipeline.
    • ›Server now respects the ignore EOS flag, enabling proper end-of-sequence control in server mode.
  276. b8828 Apr 17, 2026 · issue -124

    llama.cpp b8828 adds Gemma4 model type detection support.

    └──▷ GET THIS VERSION
    $ git clone --branch b8828 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8828
    • ›Adds Gemma4 model type detection, enabling llama.cpp to identify and load Gemma4 architecture models.
  277. b8821 Apr 17, 2026 · issue -124

    llama.cpp server gains LLAMA_MEDIA_MARKER env var to pin the media marker used in multimodal prompts.

    └──▷ GET THIS VERSION
    $ git clone --branch b8821 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8821
    • ›Adds LLAMA_MEDIA_MARKER environment variable to pin the server's media marker token, overriding the randomly generated one; useful for scripted or automated multimodal prompt workflows.
  278. b8815 Apr 16, 2026 · issue -125

    llama.cpp b8815 adds Metal ROLL op implementation for Apple Silicon GPU acceleration.

    └──▷ GET THIS VERSION
    $ git clone --branch b8815 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8815
    • ›Implements the ROLL operation for the Metal backend, enabling GPU-accelerated tensor rolling on Apple Silicon and iOS devices.
  279. b8814 Apr 16, 2026 · issue -125

    Adds 128-bit RISC-V Vector (RVV) implementations for i-quants, ternary quants, iq2_xs, iq3_s, iq3_xxs, and tq2_0 quantization dot products.

    └──▷ GET THIS VERSION
    $ git clone --branch b8814 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8814
    • ›Adds 128-bit RVV (RISC-V Vector) kernel implementations for quantization vector dot products, covering iq2_xs, iq3_s, iq3_xxs, tq2_0, i-quants, and ternary quants — enabling hardware-accelerated inference on 128-bit RVV-capable RISC-V CPUs.
  280. b8813 Apr 16, 2026 · issue -125

    llama.cpp b8813 adds a SIMD GEMM kernel for the RISC-V Vector extension, accelerating inference on RVV hardware.

    └──▷ GET THIS VERSION
    $ git clone --branch b8813 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8813
    • ›Adds a simd_gemm kernel for the RISC-V Vector (RVV) extension in ggml, enabling hardware-accelerated matrix multiplication on RVV-capable CPUs.
  281. b8806 Apr 15, 2026 · issue -126

    llama.cpp b8806 adds initial CUDA backend support for the Q1_0 quantization format.

    └──▷ GET THIS VERSION
    $ git clone --branch b8806 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8806
    • ›Adds initial CUDA backend support for the Q1_0 quantization format, including dp4a acceleration and AMD MMA compatibility.
  282. b8802 Apr 15, 2026 · issue -126

    llama.cpp b8802 adds native RDMA transport (RoCEv2) for the RPC backend.

    └──▷ GET THIS VERSION
    $ git clone --branch b8802 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8802
    • ›Adds native RDMA transport (RoCEv2) to the RPC backend, enabling high-throughput, low-latency distributed inference over RDMA-capable network fabric.
  283. b8794 Apr 14, 2026 · issue -127

    llama.cpp b8794 adds mtmd_image_tokens_get_decoder_pos() API for multimodal token position queries.

    └──▷ GET THIS VERSION
    $ git clone --branch b8794 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8794
    • ›Adds mtmd_image_tokens_get_decoder_pos() to the mtmd API for retrieving the decoder position of image tokens in multimodal inference pipelines.
  284. b8785 Apr 14, 2026 · issue -127

    Vulkan backend gains GGML_TYPE_NVFP4 support for get_rows, dequant, and mul_mat operations

    └──▷ GET THIS VERSION
    $ git clone --branch b8785 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8785
    • ›Adds GGML_TYPE_NVFP4 support to the Vulkan backend for get_rows, dequant, and mul_mat/mul_mat_id operations via fp16/fp32 paths.
  285. b8784 Apr 14, 2026 · issue -127

    llama.cpp server gains OpenAI-compatible POST /v1/audio/transcriptions API endpoint.

    └──▷ GET THIS VERSION
    $ git clone --branch b8784 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8784
    └──▷ TRY IT
    Transcribe an audio file using the new OAI-compatible endpoint on a running llama.cpp server.
    $ curl http://localhost:8080/v1/audio/transcriptions -F [email protected] -F model=whisper
    • ›Adds POST /v1/audio/transcriptions endpoint to the llama.cpp server, implementing the OpenAI-compatible audio transcription API.
  286. b8781 Apr 13, 2026 · issue -128

    llama.cpp b8781 adds a dedicated DeepSeek v3.2 chat parser and its official prompt template.

    └──▷ GET THIS VERSION
    $ git clone --branch b8781 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8781
    • ›Adds a dedicated DeepSeek v3.2 chat parser and 'official' prompt template for correct message formatting with that model.
  287. b8779 Apr 13, 2026 · issue -128

    Vulkan backend gains Flash Attention DP4A shader for quantized KV cache, accelerating inference on compatible GPUs.

    └──▷ GET THIS VERSION
    $ git clone --branch b8779 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8779
    • ›Adds Vulkan Flash Attention DP4A integer dot-product shader for quantized KV cache, enabling faster attention computation on Vulkan-capable GPUs with quantized key/value types.
  288. b8777 Apr 13, 2026 · issue -128

    llama.cpp server now exposes build_info in router mode for deployment introspection.

    └──▷ GET THIS VERSION
    $ git clone --branch b8777 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8777
    • ›Exposes build_info via the server in router mode, enabling operators to query build metadata from a routed llama.cpp deployment.
  289. b8769 Apr 12, 2026 · issue -129

    llama.cpp b8769 adds Qwen3 audio support, enabling Qwen3-Omni and Qwen3-ASR models for speech recognition.

    └──▷ GET THIS VERSION
    $ git clone --branch b8769 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8769
    • ›Adds multimodal audio support for Qwen3-Omni and Qwen3-ASR models via the mtmd subsystem, enabling automatic speech recognition (ASR) workflows with Qwen3 architecture.
  290. b8766 Apr 12, 2026 · issue -129

    llama.cpp b8766 adds Gemma 4 audio conformer encoder support via a USM-style 12-layer Conformer with dedicated mel preprocessor.

    └──▷ GET THIS VERSION
    $ git clone --branch b8766 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8766
    • ›Adds mtmd_audio_preprocessor_gemma4a — a dedicated mel preprocessor for Gemma 4 audio using HTK mel scale, 128 bins, magnitude STFT, and semicausal left-padding, enabling audio transcription with Gemma 4 E2B/E4B models on CPU and Vulkan backends.
  291. b8762 Apr 11, 2026 · issue -130

    llama.cpp b8762 adds MERaLiON-2 audio-language model support for speech transcription and spoken QA

    └──▷ GET THIS VERSION
    $ git clone --branch b8762 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8762
    • ›Adds PROJECTOR_TYPE_MERALION projector type enabling MERaLiON-2 multimodal audio-language models (3B and 10B) via the --mmproj flag on convert_hf_to_gguf.py with architecture MERaLiON2ForConditionalGeneration.
    • ›Supports speech transcription (EN/ZH/MS/TA), translation, and spoken QA tasks using a Whisper large-v2 encoder with a gated MLP adaptor and Gemma2 decoder.
  292. b8761 Apr 11, 2026 · issue -130

    llama.cpp b8761 adds OpenCL support for Q5_K quantization format (matrix-vector and matrix-matrix ops).

    └──▷ GET THIS VERSION
    $ git clone --branch b8761 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8761
    • ›Adds OpenCL backend support for Q5_K quantization, including general matrix-vector (mv) and flattened matrix-vector multiply, plus general matrix-matrix (mm) operations.
  293. b8755 Apr 11, 2026 · issue -130

    llama.cpp b8755 adds Hexagon backend support for Linux on Snapdragon (Debian on ex2).

    └──▷ GET THIS VERSION
    $ git clone --branch b8755 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8755
    • ›Adds Hexagon backend support for Linux on Snapdragon, including Debian on ex2 hardware, with -fvectorize enabled for C/C++ CMake builds.
  294. b8754 Apr 11, 2026 · issue -130

    llama.cpp b8754 improves Hexagon NPU op batching, buffer management, and replaces GGML_HEXAGON_EXPERIMENTAL with GGML_HEXAGON_OPFILTER

    └──▷ GET THIS VERSION
    $ git clone --branch b8754 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8754
    └──▷ TRY IT
    Disable a specific Hexagon backend op by regex during debugging, replacing the removed GGML_HEXAGON_EXPERIMENTAL variable.
    $ GGML_HEXAGON_OPFILTER='MUL_MAT' ./llama-cli -m model.gguf -p 'Hello'
    • ›Replaces the GGML_HEXAGON_EXPERIMENTAL environment variable with GGML_HEXAGON_OPFILTER, a regex-based op filter for selectively disabling Hexagon backend ops during debugging or validation.
    • ›Introduces op request batching on the Hexagon backend: the host now prepares batches of up to 256 requests and dispatches them via a single dspqueue message, reducing inter-op overhead.
    • ›Adds configurable opbatch sizing (OB and OQ params) and a vmem limit for op batching on the Hexagon backend.
    • ›Introduces pinned mmap-based shared buffers for packing op batches, with explicit per-tensor L2 cache management (flush/invalidate) replacing coarse global flushes.
    • ›Enables Flash Attention during token generation (TG) on the Hexagon backend to reduce graph splits.
    +1 moreshow less
    • ›Improves VTCM acquisition on the Hexagon backend to remove inter-op overhead, with full compatibility with QNN-HTP coexistence.
    └──▷ BREAKING ON UPGRADE
    • !The GGML_HEXAGON_EXPERIMENTAL environment variable is removed; use GGML_HEXAGON_OPFILTER instead to disable specific ops.
  295. b8752 Apr 11, 2026 · issue -130

    llama.cpp b8752 adds a callback interface for download progress in the common library.

    └──▷ GET THIS VERSION
    $ git clone --branch b8752 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8752
    • ›Adds a callback interface for download progress in the common library, enabling programmatic monitoring of model download state.
  296. b8742 Apr 10, 2026 · issue -131

    llama.cpp b8742 adds Vulkan backend support for the Q1_0 quantization format.

    └──▷ GET THIS VERSION
    $ git clone --branch b8742 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8742
    • ›Adds Q1_0 quantization format support to the Vulkan backend, enabling GPU-accelerated inference with Q1_0 models on Vulkan-capable hardware.
  297. b8739 Apr 9, 2026 · issue -132

    llama.cpp b8739 adds AMD Instinct MI350X/MI355X (gfx950, CDNA4) GPU support via HIP backend

    └──▷ GET THIS VERSION
    $ git clone --branch b8739 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8739
    • ›Adds GGML_CUDA_CC_CDNA4 and GGML_CUDA_CC_IS_CDNA4 macros in common.cuh to identify AMD CDNA4 (gfx950) architecture at build time.
    • ›Supports AMD Instinct MI350X/MI355X (gfx950, CDNA4) in the HIP backend, enabling GPU-accelerated inference on these accelerators when building with -DAMDGPU_TARGETS=gfx950.
    • ›Routes CDNA4 matrix operations to compatible MFMA instructions: mfma_f32_16x16x4f32 for f32, mfma_f32_16x16x16bf16_1k for bf16, and mfma_i32_16x16x32_i8/mfma_i32_32x32x16_i8 for int8.
    • ›Includes CDNA4 (gfx950) in stream-k kernel dispatch in mmq.cuh, enabling MMQ inference on MI355X.
  298. b8738 Apr 9, 2026 · issue -132

    llama.cpp b8738 adds experimental backend-agnostic tensor parallelism with NCCL/RCCL support across up to 8 GPUs.

    └──▷ GET THIS VERSION
    $ git clone --branch b8738 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8738
    • ›Introduces experimental backend-agnostic tensor parallelism via ggml-backend-meta.cpp and the new ggml-ext.h staging API header, supporting arbitrary numbers of GPUs and uneven tensor splits.
    • ›Adds NCCL support for CUDA multi-GPU tensor parallelism, enabling AllReduce operations across devices using pinned (vs. pageable) memory when all backends expose the same host buffer type.
    • ›Adds RCCL support for HIP/ROCm multi-GPU tensor parallelism, extending the same backend-agnostic tensor parallel path to AMD hardware.
    • ›Extends tensor parallelism support to GPT-OSS, Qwen 3 MoE, Qwen 3.5, Qwen 3.5 MoE, and Gemma 4 MoE model architectures.
    • ›Moves most meta backend public API into ggml-backend-impl.h, hiding previously exposed internals and introducing llama_device as the new abstraction for device selection.
    +1 moreshow less
    • ›Uses BF16 precision for AllReduce operations in the tensor parallel path.
    └──▷ BREAKING ON UPGRADE
    • !ggml_backend_dev_is_meta() has been removed; callers must migrate to the new llama_device abstraction.
  299. b8731 Apr 9, 2026 · issue -132

    llama.cpp b8731 adds multimodal support for the Dots.OCR model via the mtmd subsystem.

    └──▷ GET THIS VERSION
    $ git clone --branch b8731 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8731
    • ›Adds dots.ocr model support to the mtmd (multimodal) subsystem, including GGUF conversion and CLIP implementation.
  300. b8724 Apr 9, 2026 · issue -132

    SYCL Flash Attention now supports head size 512 (DKQ/DV), up from the previous 256 limit.

    └──▷ GET THIS VERSION
    $ git clone --branch b8724 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8724
    • ›Extends SYCL Flash Attention (both tile and vector kernels) to support head sizes (DKQ/DV) of 512, previously capped at 256, enabling larger-model attention workloads on SYCL backends.
  301. b8712 Apr 9, 2026 · issue -132

    llama.cpp b8712 adds Metal GPU backend support for Q1_0 quantization on Apple Silicon.

    └──▷ GET THIS VERSION
    $ git clone --branch b8712 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8712
    • ›Adds Metal GPU backend for Q1_0 quantization, enabling hardware-accelerated inference with 1-bit weights on Apple Silicon devices.
  302. b8705 Apr 8, 2026 · issue -133

    llama.cpp b8705 adds support for the Step3-VL-10B vision-language model.

    └──▷ GET THIS VERSION
    $ git clone --branch b8705 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8705
    • ›Adds support for the Step3-VL-10B multimodal vision-language model, including fused QKV, tensor mapping via tensor_mapping.py, and understand_projector_stride read from the global config.
  303. b8703 Apr 8, 2026 · issue -133

    llama.cpp b8703 adds a KleidiAI-enabled arm64 macOS release artifact for Apple Silicon.

    └──▷ GET THIS VERSION
    $ git clone --branch b8703 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8703
    • ›Adds a KleidiAI-enabled arm64 macOS release artifact (llama-b8703-bin-macos-arm64-kleidiai.tar.gz) for Apple Silicon, providing ARM-optimized compute kernels via the KleidiAI library.
  304. b8685 Apr 7, 2026 · issue -134

    SYCL backend gains Q8_0 reorder optimization, delivering ~3x token-generation speedup on Intel Arc GPUs.

    └──▷ GET THIS VERSION
    $ git clone --branch b8685 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8685
    • ›Extends the SYCL reorder optimization (scale-factor/weight separation for coalesced memory access) to Q8_0, achieving ~3.1x token-generation throughput on Intel Arc Pro B70 (Xe2) — e.g. 4.88 → 15.24 t/s on Qwen3.5-27B, with bandwidth utilization rising from 21% to 66%.
  305. b8683 Apr 6, 2026 · issue -135

    llama.cpp b8683 adds MUL_MAT_ID support to the WebGPU backend.

    └──▷ GET THIS VERSION
    $ git clone --branch b8683 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8683
    • ›Adds MUL_MAT_ID operation support to the WebGPU (ggml-webgpu) backend, enabling mixture-of-experts model inference via WebGPU.
  306. b8682 Apr 6, 2026 · issue -135

    llama.cpp b8682 adds Q1_0 1-bit quantization support on CPU, enabling extreme model compression.

    └──▷ GET THIS VERSION
    $ git clone --branch b8682 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8682
    • ›Adds Q1_0 1-bit quantization format (group size 128) with CPU support, including a generic fallback for x86 and other backends.
  307. b8679 Apr 6, 2026 · issue -135

    llama-bench gains -fitc and -fitt flags for finer benchmark filtering control.

    └──▷ GET THIS VERSION
    $ git clone --branch b8679 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8679
    └──▷ TRY IT
    Filter llama-bench runs to specific test cases and test types to isolate prompt-ingestion or token-generation performance.
    $ llama-bench -fitc <case-pattern> -fitt <type-pattern>
    • ›Adds -fitc and -fitt arguments to llama-bench for filtering benchmark test cases and test types.
  308. b8670 Apr 6, 2026 · issue -135

    llama.cpp b8670 adds HunyuanOCR vision model support with a perceiver-based projector and dedicated chat template.

    └──▷ GET THIS VERSION
    $ git clone --branch b8670 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8670
    • ›Adds HunyuanOCR vision projector (perceiver-based, with Conv2d merge) and tensor mappings for mm.before_rms, mm.after_rms, and related perceiver projector tensors.
    • ›Adds a dedicated HUNYUAN_OCR chat template (content-before-role format) for HunyuanOCR models.
    • ›Registers HunYuanVLForConditionalGeneration in convert_hf_to_gguf.py for both text and mmproj conversion.
    • ›Supports the xdrope RoPE scaling type introduced by HunyuanOCR.
  309. b8665 Apr 4, 2026 · issue -137

    llama.cpp b8665 adds a dedicated Gemma 4 chat-template parser with tool-call and interleaved-thinking support.

    └──▷ GET THIS VERSION
    $ git clone --branch b8665 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8665
    • ›Adds a specialized Gemma 4 Jinja chat-template parser (gemma4.jinja) that emits JSON from Gemma 4 tool-call ASTs and registers <|tool_response> as an end-of-generation token.
    • ›Adds a custom Gemma 4 template to support interleaved thinking, preserving reasoning content in model turns.
  310. b8658 Apr 3, 2026 · issue -138

    llama.cpp server gains --clear-idle to reclaim VRAM from idle KV cache slots on new task arrival.

    └──▷ GET THIS VERSION
    $ git clone --branch b8658 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8658
    └──▷ TRY IT
    Run the inference server with idle-slot KV cache clearing enabled to reclaim VRAM between requests in a shared-GPU environment.
    $ llama-server --model <model.gguf> --clear-idle
    • ›Adds --clear-idle flag to llama-server to automatically free VRAM used by idle KV cache slots (via LLAMA_KV_KEEP_ONLY_ACTIVE) when a new task arrives, reducing GPU memory pressure in multi-user deployments. Opt out with --no-kv-clear-idle.
  311. b8648 Apr 3, 2026 · issue -138

    llama.cpp b8648 adds ZenDNN MUL_MAT_ID op acceleration for Mixture-of-Experts models.

    └──▷ GET THIS VERSION
    $ git clone --branch b8648 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8648
    • ›Adds MUL_MAT_ID op acceleration in the ggml-zendnn backend for Mixture-of-Experts (MoE) models, falling back to the CPU backend when total experts exceed 32.
    • ›Updates the ZenDNN library reference to ZenDNN-2026-WW13.
  312. b8639 Apr 2, 2026 · issue -139

    llama.cpp b8639 adds vectorized flash attention for the WebGPU backend, with q4/q8 quantization support.

    └──▷ GET THIS VERSION
    $ git clone --branch b8639 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8639
    • ›Adds vectorized flash attention to the WebGPU (ggml-webgpu) backend, enabled when Q->ne[1] < 20, Q->ne[0] % 32 == 0, V->ne[0] % 4 == 0, and K->type == f16; also supports q4 and q8 quantized key/value types.
    • ›Introduces a fast single-workgroup (nwg=1) path for vectorized flash attention that skips temporary/reduce staging buffers, reducing overhead for small batch sizes.
    • ›Uses packed f16 K loads in the flash_attn_vec_split.wgsl shader for improved memory throughput on the WebGPU backend.
  313. b8634 Apr 2, 2026 · issue -139

    llama.cpp b8634 adds Granite 4.0 chat template with correct tool_call role mapping for agentic workflows.

    └──▷ GET THIS VERSION
    $ git clone --branch b8634 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8634
    • ›Adds LLM_CHAT_TEMPLATE_GRANITE_4_0 enum, map entry, and handler to support Granite 4.0 chat template, using <tool_call> XML tags and mapping the assistant_tool_call role to <|start_of_role|>assistant<|end_of_role|><|tool_call|> — fixing broken tool calling when --jinja is not used.
    • ›Auto-detects Granite model generation: presence of <|start_of_role|> plus <tool_call> or <tools> selects the 4.0 template; otherwise falls back to the 3.x template.
    └──▷ BREAKING ON UPGRADE
    • !The existing LLM_CHAT_TEMPLATE_GRANITE enum value is renamed to LLM_CHAT_TEMPLATE_GRANITE_3_X; any code referencing LLM_CHAT_TEMPLATE_GRANITE by name must be updated.
  314. b8607 Apr 1, 2026 · issue -140

    WebGPU backend gains quantized buffer support via u32 packing and wider browser/device compatibility.

    └──▷ GET THIS VERSION
    $ git clone --branch b8607 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8607
    • ›Extends the WebGPU (ggml) backend to pack quantized buffers as u32 and use unpackf16 for broader browser and device compatibility, removing the previous bitcast approach.
  315. b8589 Mar 30, 2026 · issue -142

    llama.cpp b8589 adds Q4_K GEMM and GEMV OpenCL kernels for Qualcomm Adreno GPUs

    └──▷ GET THIS VERSION
    $ git clone --branch b8589 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8589
    • ›Adds q4_K GEMM and GEMV OpenCL kernels for Qualcomm Adreno GPUs, enabling accelerated quantized matrix operations on Adreno hardware.
  316. b8581 Mar 30, 2026 · issue -142

    llama.cpp server gains header wrapping for the MCP CORS proxy

    └──▷ GET THIS VERSION
    $ git clone --branch b8581 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8581
    • ›Adds header wrapping in server-cors-proxy.h for the MCP proxy, enabling proper CORS header forwarding when the llama.cpp server is used as an MCP proxy.
  317. b8570 Mar 28, 2026 · issue -144

    llama.cpp b8570 adds reasoning_format = none support to the gpt-oss common layer.

    └──▷ GET THIS VERSION
    $ git clone --branch b8570 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8570
    • ›Adds reasoning_format = none support to the gpt-oss common layer, allowing callers to suppress reasoning-format output entirely.
  318. b8562 Mar 28, 2026 · issue -144

    llama.cpp b8562 adds /glob command to the CLI for batch file selection by pattern.

    └──▷ GET THIS VERSION
    $ git clone --branch b8562 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8562
    • ›Adds /glob command to the CLI, enabling wildcard-based file selection including paths outside the current directory, with an error output when the maximum file limit is reached.
  319. b8560 Mar 28, 2026 · issue -144

    llama-server gains --reuse-port flag to opt into SO_REUSEPORT for multi-process socket sharing

    └──▷ GET THIS VERSION
    $ git clone --branch b8560 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8560
    └──▷ TRY IT
    Run multiple llama-server instances on the same port so the OS load-balances incoming inference requests across processes.
    $ llama-server --reuse-port -m model.gguf --port 8080
    • ›Adds --reuse-port flag to llama-server, enabling SO_REUSEPORT on the listening socket to allow multiple server processes to bind the same port for load-balancing setups.
  320. b8558 Mar 27, 2026 · issue -145

    llama.cpp b8558 adds LLAMA_BUILD_WEBUI CMake flag to optionally strip the embedded server web UI at build time.

    └──▷ GET THIS VERSION
    $ git clone --branch b8558 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8558
    └──▷ TRY IT
    Build llama-server without the embedded web UI to reduce the binary's exposed surface in a production or containerised inference environment.
    $ cmake -B build -DLLAMA_BUILD_WEBUI=OFF && cmake --build build --config Release -t llama-server
    • ›Adds LLAMA_BUILD_WEBUI CMake build flag (default: ON) to allow disabling the embedded web UI in the llama-server binary, reducing attack surface in headless deployments.
  321. b8557 Mar 27, 2026 · issue -145

    llama.cpp b8557 adds IQ4_NL and MXFP4 quantization support to the Hexagon backend via HMX matmul.

    └──▷ GET THIS VERSION
    $ git clone --branch b8557 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8557
    • ›Adds IQ4_NL quantization type support to the Hexagon backend, including buffer set/get tensor repack, mul_mat, and mul_mat_id dispatch.
    • ›Implements HVX IQ4_NL vec_dot kernels (1x1, 2x1, 2x2) with LUT-based 4-bit index to int8 dequantization on the Hexagon backend.
    • ›Adds MXFP4 HMX dequantization path with E8M0 scale conversion, including a batch-4 fast path and single-tile fallback on the Hexagon backend.
    • ›Unifies quantized row size and scale offset logic in the DMA fetch path to handle Q4_0, Q8_0, IQ4_NL, and MXFP4.
  322. b8553 Mar 27, 2026 · issue -145

    llama.cpp server gains built-in tools backend with --tools all flag for agentic workflows.

    └──▷ GET THIS VERSION
    $ git clone --branch b8553 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8553
    └──▷ TRY IT
    Start the llama.cpp server with all built-in tools enabled for agentic, tool-calling workflows.
    $ llama-server --tools all -m <model.gguf>
    • ›Adds --tools all argument to the llama-server built-in tools backend, enabling server-side tool/function-call orchestration without an external middleware layer.
  323. b8532 Mar 26, 2026 · issue -146

    llama.cpp b8532 adds F32 kernel type support for CONV_TRANSPOSE_2D on CUDA and CPU backends.

    └──▷ GET THIS VERSION
    $ git clone --branch b8532 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8532
    • ›Adds F32 kernel type support for CONV_TRANSPOSE_2D on CUDA and CPU backends, alongside the existing F16 type, enabling full-precision 2D transposed convolution operations.
  324. b8530 Mar 26, 2026 · issue -146

    llama.cpp b8530 adds DeepSeek-OCR multimodal support with dynamic resolution and flash-attn compatibility in llama-cli.

    └──▷ GET THIS VERSION
    $ git clone --branch b8530 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8530
    • ›Adds DeepSeek-OCR model support via llama-mtmd-cli, including CLIP-ViT vision encoder, SAM-based image encoding, and DeepSeek2 language model backend with standard attention.
    • ›Supports dynamic and native resolution preprocessing for DeepSeek-OCR, with configurable min/base/max resolution tiers (640/1024/1280) for image quality vs. speed trade-offs.
    • ›Enables --flash-attn option compatibility with DeepSeek-OCR inference in llama-cli.
    • ›Adds convert_hf_to_gguf.py conversion support for DeepSeek-OCR models, outputting FP16 GGUF weights with corrected tensor names including mm_fc_w and image newline/separator tensors.
    • ›Adds a default Jinja chat template for DeepSeek-OCR for use with the llama.cpp server.
  325. b8526 Mar 25, 2026 · issue -147

    llama.cpp b8526 adds support for the codefuse-ai/F2LLM-v2 model.

    └──▷ GET THIS VERSION
    $ git clone --branch b8526 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8526
    • ›Adds support for the codefuse-ai/F2LLM-v2 model.
  326. b8498 Mar 24, 2026 · issue -148

    llama.cpp b8498 adds standard Hugging Face cache support, migrating manifests and using the HF API to resolve cached model files.

    └──▷ GET THIS VERSION
    $ git clone --branch b8498 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8498
    • ›Adds standard Hugging Face cache support: uses the HF API to enumerate all files, migrates existing manifests to the Hugging Face cache layout at startup, and falls back to cached files when the HF API is unavailable.
  327. b8493 Mar 23, 2026 · issue -149

    llama.cpp b8493 adds OpenCL q6_K GEMM and GEMV kernels for Adreno GPUs.

    └──▷ GET THIS VERSION
    $ git clone --branch b8493 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8493
    • ›Adds OpenCL q6_K GEMM (gemm_noshuffle_q6_k_f32) and GEMV kernels for Adreno GPUs, enabling quantized 6-bit inference acceleration on mobile/embedded hardware via the OpenCL backend.
  328. b8477 Mar 23, 2026 · issue -149

    llama.cpp b8477 adds dynamic high-resolution image preprocessing support for InternVL models.

    └──▷ GET THIS VERSION
    $ git clone --branch b8477 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8477
    • ›Adds dynamic high-resolution image preprocessing for InternVL models (including Qianfan-OCR), storing min/max dynamic patch counts in GGUF metadata; default values are provided for older models.
  329. b8475 Mar 22, 2026 · issue -150

    llama.cpp server router can now report sleep status of child instances

    └──▷ GET THIS VERSION
    $ git clone --branch b8475 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8475
    • ›Enables the llama.cpp server router to expose the sleep/wake state of child server instances in its status reporting.
  330. b8470 Mar 22, 2026 · issue -150

    llama.cpp b8470 adds native BF16 flash attention for the CUDA vec kernel, improving GPU inference efficiency.

    └──▷ GET THIS VERSION
    $ git clone --branch b8470 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8470
    • ›Adds native BF16 flash attention support for the CUDA vec kernel, avoiding prior BF16-to-FP16 conversion overhead on supported GPU backends including HIP.
  331. b8416 Mar 18, 2026 · issue -154

    llama.cpp b8416 adds control vector support to Qwen3.5 and Qwen-next models

    └──▷ GET THIS VERSION
    $ git clone --branch b8416 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8416
    • ›Extends control vector (cvec) support to Qwen3.5 and Qwen-next models, plus fills in missing cvec compatibility across the remaining model architectures.
  332. b8400 Mar 17, 2026 · issue -155

    llama.cpp b8400 adds Hexagon HVX ops (neg, exp, sigmoid, softplus, CONT, REPEAT) enabling Qwen 3.5 DeltaNet linear attention on-device.

    └──▷ GET THIS VERSION
    $ git clone --branch b8400 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8400
    • ›Adds neg, exp, sigmoid, softplus, CONT, and REPEAT element-wise ops to the Hexagon HVX backend, enabling Qwen 3.5 DeltaNet linear attention layers on Qualcomm hardware.
    • ›Hexagon REPEAT op supports f32 and f16 types with multi-threaded execution via the worker pool, parallelizing across output rows using memcpy per tile.
    • ›Hexagon CONT op reuses existing CPY infrastructure to make tensors contiguous on-device without a separate kernel.
  333. b8391 Mar 17, 2026 · issue -155

    Vulkan backend gains GGML_VK_ALLOW_GRAPHICS_QUEUE env var to opt back into graphics-queue execution on AMD GPUs.

    └──▷ GET THIS VERSION
    $ git clone --branch b8391 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8391
    • ›Adds GGML_VK_ALLOW_GRAPHICS_QUEUE environment variable to override the new default of avoiding graphics queues on non-RADV AMD drivers and small GPUs, re-enabling the transfer queue when the graphics queue is not used.
    └──▷ BREAKING ON UPGRADE
    • !The Vulkan backend no longer uses the graphics queue by default on non-RADV AMD drivers and small GPUs; workloads that relied on graphics-queue execution will now use compute/transfer queues unless GGML_VK_ALLOW_GRAPHICS_QUEUE is set.
  334. b8389 Mar 17, 2026 · issue -155

    llama.cpp b8389 adds refusal content support to the Responses API in tools/server.

    └──▷ GET THIS VERSION
    $ git clone --branch b8389 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8389
    • ›Adds refusal content support to the Responses API in tools/server.
  335. b8388 Mar 17, 2026 · issue -155

    llama.cpp b8388 adds support for the Mistral Small 4 model.

    └──▷ GET THIS VERSION
    $ git clone --branch b8388 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8388
    • ›Adds Mistral Small 4 model support via convert_hf_to_gguf.py.
  336. b8340 Mar 14, 2026 · issue -157

    llama.cpp b8340 adds native AVX512-FP16 support for F16 operations, cutting instruction count by 2.7 billion on compatible CPUs.

    └──▷ GET THIS VERSION
    $ git clone --branch b8340 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8340
    • ›Adds native AVX512-FP16 support for F16 operations in ggml, reducing instruction count by ~2.7 billion on AVX512-FP16-capable CPUs; enabled only for native builds or with custom compiler flags.
  337. b8339 Mar 14, 2026 · issue -157

    llama.cpp b8339 adds env variables to override cuBLAS compute type and uses fp32 on V100 GPUs to prevent overflows.

    └──▷ GET THIS VERSION
    $ git clone --branch b8339 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8339
    • ›Adds environment variables to override the cuBLAS compute type, giving operators direct control over CUDA matrix-multiply precision without recompiling.
  338. b8338 Mar 14, 2026 · issue -157

    llama.cpp gains an OpenVINO backend with CPU, GPU, and NPU support including quantized model execution and stateful graph inference.

    └──▷ GET THIS VERSION
    $ git clone --branch b8338 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8338
    └──▷ TRY IT
    Disable SDPA fusion in the OpenVINO backend when debugging accuracy issues on certain GPU platforms.
    $ GGML_OPENVINO_DISABLE_SDPA_OPTIMIZATION=1 llama-bench -m <model>.gguf -p 512 -n 128
    • ›Adds GGML_OPENVINO_DISABLE_SDPA_OPTIMIZATION environment variable to toggle scaled dot-product attention fusion in the new OpenVINO backend.
    • ›Adds the OpenVINO backend (ggml-openvino) targeting Intel CPU, GPU (OpenCL), and NPU devices — enabling hardware-accelerated inference on Intel silicon without CUDA.
    • ›Supports quantized model execution on the OpenVINO backend with custom quant types q8_1_c, q4_0_128, and Q8_0_C; NPU always requantizes weights to q4_0_128.
    • ›Adds NPU prefill chunking for large-context inference on Intel NPU devices.
    • ›Supports stateful graph execution mode on the OpenVINO backend (GPU and NPU) for improved decode-phase performance.
    +5 moreshow less
    • ›Supports BF16 model inference on the OpenVINO backend.
    • ›Supports -ctk f32 cache type on the OpenVINO backend.
    • ›Adds KV-cache fusion support (kvcachefusion) in the OpenVINO backend for improved memory and compute efficiency.
    • ›Adds OpenVINO backend documentation at docs/backend/OPENVINO.md.
    • ›Pre-built Ubuntu x64 binaries are now available with OpenVINO 2026.0 support.
  339. b8334 Mar 14, 2026 · issue -157

    llama-perplexity now supports -kvu flag for hellaswag and winogrande benchmarks, fixing batch decode failures on coupled sequences.

    └──▷ GET THIS VERSION
    $ git clone --branch b8334 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8334
    └──▷ TRY IT
    Run winogrande benchmark evaluation on a quantized model without batch decode failures caused by coupled sequences.
    $ llama-perplexity -hf unsloth/Qwen3-0.6B-GGUF:Q4_K_M -f winogrande-debiased-eval.csv --winogrande -kvu
    • ›Enables the -kvu flag in llama-perplexity for --winogrande and hellaswag evaluation modes, allowing coupled-sequence batches that previously failed with 'failed to find a memory slot' errors.
  340. b8329 Mar 14, 2026 · issue -157

    Adds RISC-V Vector (RVV) accelerated kernels for quantization types iq4_nl, mxfp4, iq2_xxs, iq4_xs, iq2_xs, and iq3_xxs on RISC-V hardware.

    └──▷ GET THIS VERSION
    $ git clone --branch b8329 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8329
    • ›Adds RVV (RISC-V Vector) vec_dot inference kernels for quantization types iq4_nl, mxfp4, iq2_xxs, iq4_xs, iq2_xs, and iq3_xxs, accelerating CPU inference on RISC-V hardware.
    • ›Adds RVV quantize_row_q8_K kernel for accelerated quantization on RISC-V CPUs.
  341. b8320 Mar 13, 2026 · issue -158

    test-backend-ops gains file-based test loading and a graph operator extraction tool via llama_graph_reserve

    └──▷ GET THIS VERSION
    $ git clone --branch b8320 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8320
    • ›Adds llama_graph_reserve (moved to new llama-ext header) as the public API for reserving graph computations, replacing previous internal API calls.
    • ›Adds file-based test loading to test-backend-ops, allowing backend operation tests to be defined and loaded from a text file instead of being hardcoded.
    • ›Adds the export-graph-ops tool (located in tests/) to extract model graph operators into a file for use with the new file-based test runner.
    • ›Adds per-operator error thresholds to test-backend-ops for more precise correctness validation across different backend operations.
    • ›Adds nb parameter support in test-backend-ops for non-contiguous input tensors.
  342. b8317 Mar 13, 2026 · issue -158

    llama.cpp b8317 adds Vulkan GATED_DELTA_NET op support for AMD GPUs via fused compute shader.

    └──▷ GET THIS VERSION
    $ git clone --branch b8317 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8317
    • ›Adds GATED_DELTA_NET op to the Vulkan backend as a fused compute shader, supporting scalar gate, KDA vector gate, GQA broadcast, multi-token sequences, and permuted (non-contiguous) q/k inputs — enabling Gated Delta Network model inference on Vulkan GPUs.
    • ›Vulkan GATED_DELTA_NET shader uses specialization constants to select head size (32, 64, or 128) and KDA mode at pipeline creation time, with a pipeline array layout of [3][2].
    • ›Vulkan GATED_DELTA_NET KDA path caches exp(g) in shared memory and uses vec4 dot products on all inner loops, yielding +5.4% throughput on the KDA path on AMD Radeon 890M (RADV GFX1150).
  343. b8304 Mar 13, 2026 · issue -158

    llama.cpp b8304 adds chat template parser support for GigaChatV3 and GigaChatV3.1 models.

    └──▷ GET THIS VERSION
    $ git clone --branch b8304 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8304
    • ›Adds GigaChatV3 and GigaChatV3.1 model support to the common chat template parser.
  344. b8303 Mar 13, 2026 · issue -158

    llama.cpp b8303 adds support for the Phi4ForCausalLMV multimodal model architecture.

    └──▷ GET THIS VERSION
    $ git clone --branch b8303 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8303
    • ›Adds support for the Phi4ForCausalLMV model architecture, enabling Phi-4 vision inference with corrected SigLIP2 patch-kernel layout and NaFlex resize behavior.
  345. b8300 Mar 13, 2026 · issue -158

    llama.cpp b8300 adds GGML_OP_REPEAT (including i16) support to the WebGPU backend.

    └──▷ GET THIS VERSION
    $ git clone --branch b8300 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8300
    • ›Adds GGML_OP_REPEAT operation support to the WebGPU (ggml-webgpu) backend, including i16 data type handling.
  346. b8299 Mar 13, 2026 · issue -158

    llama.cpp b8299 adds GPU-accelerated DeltaNet inference via fused Metal and CUDA kernels for GDN, boosting Qwen3.5 throughput by up to 25%.

    └──▷ GET THIS VERSION
    $ git clone --branch b8299 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8299
    • ›Adds fused Metal kernel for GGML_OP_GATED_DELTA_NET on Apple Silicon, enabling GPU-accelerated inference for DeltaNet-based models (Qwen3.5, etc.) with both GDA (scalar gate) and KDA (per-row gate) modes; supports head_size 64 and 128, falling back to CPU for head_size 32 or non-contiguous tensors — yields +25% token generation throughput (170 → 213 t/s) for Qwen3.5-0.8B Q4_K_M on M4 Max.
    • ›Enables chunked fused GDN path in the llama inference engine, extending DeltaNet recurrence support to prefill in addition to token generation.
  347. b8297 Mar 13, 2026 · issue -158

    llama.cpp b8297 adds NVFP4 quantization type with ARM NEON optimization delivering up to 3.8x throughput gains.

    └──▷ GET THIS VERSION
    $ git clone --branch b8297 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8297
    • ›Adds NVFP4 as a new quantization type option in llama-quantize, with CPU fallback dot product, quantization, dequantization, and conversion support across all platforms (ARM, x86, PowerPC, s390x).
    • ›Adds optimized ggml_vec_dot_nvfp4_q8_0 ARM NEON implementation using vqtbl1q_s8 LUT lookup, vpaddq_s32 reduction, and vfmaq_f32 accumulators — achieving ~31.0 t/s (tg128) versus 8.1 t/s generic, a 3.8x speedup at 77% of Q4_1 speed.
    • ›Stores NVFP4 scales as native UE4M3 format, preserving original model bits when converting from HuggingFace via convert_hf_to_gguf.py.
    • ›NVFP4 CPU-only backend uses a 64-block super-block layout; PPL benchmarks at 15.25 versus F16 at 14.33 at 4.70 BPW, comparable to Q4_1 (15.81) at lower bit-width.
  348. b8295 Mar 12, 2026 · issue -159

    llama.cpp b8295 adds support for the Nemotron 3 Super (120B.A12B) model with GGUF conversion.

    └──▷ GET THIS VERSION
    $ git clone --branch b8295 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8295
    • ›Adds support for the Nemotron 3 Super model (120B.A12B), enabling conversion to GGUF format and local inference in llama.cpp.
  349. b8291 Mar 12, 2026 · issue -159

    llama.cpp b8291 adds an environment variable to trigger Metal graph capture on macOS.

    └──▷ GET THIS VERSION
    $ git clone --branch b8291 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8291
    • ›Adds an environment variable to trigger Metal graph capture for macOS/iOS GPU backends, enabling GPU frame debugging workflows.
  350. b8287 Mar 12, 2026 · issue -159

    llama.cpp b8287 adds a reasoning budget sampler with CLI handling for thinking-model token control.

    └──▷ GET THIS VERSION
    $ git clone --branch b8287 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8287
    • ›Adds a reasoning budget sampler to the common parser, enabling CLI-level control over reasoning token budgets for thinking models.
  351. b8269 Mar 11, 2026 · issue -160

    llama.cpp b8269 adds concurrent SME and NEON kernel execution via KleidiAI on ARM.

    └──▷ GET THIS VERSION
    $ git clone --branch b8269 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8269
    • ›Enables concurrent SME and NEON kernel execution in the KleidiAI backend, improving throughput on ARM hardware that supports both instruction sets simultaneously.
  352. b8268 Mar 11, 2026 · issue -160

    llama.cpp b8268 adds RISC-V Vector (RVV) repacked GEMM/GEMV for q8_0, iq4_nl, q4_0, q4_K, and q2_K quantization types.

    └──▷ GET THIS VERSION
    $ git clone --branch b8268 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8268
    • ›Adds RVV (RISC-V Vector) repacked GEMM and GEMV kernel implementations for quantization types q8_0, iq4_nl, q4_0, q4_K, and q2_K, enabling accelerated CPU inference on RISC-V hardware with vector extensions.
  353. b8262 Mar 10, 2026 · issue -161

    llama.cpp server CORS proxy now correctly parses port numbers from MCP server URLs, enabling non-standard port and SSL routing.

    └──▷ GET THIS VERSION
    $ git clone --branch b8262 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8262
    • ›The server CORS proxy now parses port numbers from MCP server URLs and passes the scheme to the HTTP proxy to determine whether to use SSL, fixing routing for non-standard ports.
  354. b8261 Mar 10, 2026 · issue -161

    llama.cpp b8261 extends Metal mul_mv_ext small-batch kernels to BF16, Q2_K, and Q3_K quantization types.

    └──▷ GET THIS VERSION
    $ git clone --branch b8261 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8261
    • ›Extends Metal mul_mv_ext small-batch kernels (batch sizes 2–8) to BF16, Q2_K, and Q3_K quantization types, which previously fell through to the slower single-row mul_mv path — enabling faster Apple Silicon inference for models using these formats.
  355. b8248 Mar 9, 2026 · issue -162

    llama.cpp b8248 displays total and free VRAM capacity during CUDA device initialization.

    └──▷ GET THIS VERSION
    $ git clone --branch b8248 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8248
    • ›Displays total and free VRAM capacity during CUDA device initialization, giving practitioners immediate visibility into GPU memory availability at startup.
  356. b8247 Mar 9, 2026 · issue -162

    llama-bench gains -hf and -hff flags for Hugging Face model sourcing, with --mmap 1 now default.

    └──▷ GET THIS VERSION
    $ git clone --branch b8247 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8247
    └──▷ TRY IT
    Benchmark a model pulled directly from Hugging Face without manually downloading it first.
    $ llama-bench -hf <org>/<model-repo>
    • ›Adds -hf and -hff flags to llama-bench for specifying models directly from Hugging Face.
    • ›llama-bench now defaults to --mmap 1 (memory-mapped file I/O enabled by default).
    └──▷ BREAKING ON UPGRADE
    • !llama-bench now uses --mmap 1 by default; existing benchmarks that relied on the previous default (mmap off) may see different memory or performance characteristics without explicitly setting --mmap 0.
  357. b8240 Mar 8, 2026 · issue -163

    llama.cpp b8240 adds ELU op support to the Vulkan backend.

    └──▷ GET THIS VERSION
    $ git clone --branch b8240 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8240
    • ›Adds ELU (Exponential Linear Unit) operator support to the ggml-Vulkan backend, expanding the set of model ops that can run GPU-accelerated on Vulkan devices.
  358. b8234 Mar 8, 2026 · issue -163

    SYCL backend gains Flash Attention support for fp32, fp16, Q4, Q5, and Q8 precision levels.

    └──▷ GET THIS VERSION
    $ git clone --branch b8234 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8234
    • ›Adds Flash Attention support to the SYCL backend for fp32, fp16, Q4, Q5, and Q8 quantization formats, including JIT compilation support.
  359. b8233 Mar 7, 2026 · issue -164

    llama.cpp b8233 adds the GATED_DELTA_NET op to ggml with fused backend support, enabling Qwen3.5 dense model inference.

    └──▷ GET THIS VERSION
    $ git clone --branch b8233 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8233
    • ›Adds GATED_DELTA_NET op to ggml, enabling support for Gated DeltaNet-based architectures including Qwen3.5 dense models.
    • ›Adds fused gated delta net backend support with a corresponding check in the llama model loader.
  360. b8227 Mar 7, 2026 · issue -164

    llama.cpp b8227 introduces a fully refactored Autoparser architecture with Kimi 2.5 model support.

    └──▷ GET THIS VERSION
    $ git clone --branch b8227 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8227
    • ›Adds a Kimi 2.5 model parser as part of the new Autoparser architecture.
    • ›Complete refactoring of the parser architecture into a new Autoparser system.
  361. b8225 Mar 7, 2026 · issue -164

    llama.cpp server now preserves Anthropic thinking blocks during message conversion

    └──▷ GET THIS VERSION
    $ git clone --branch b8225 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8225
    • ›Server preserves Anthropic thinking blocks when converting messages, maintaining chain-of-thought content across API interactions.
  362. b8210 Mar 5, 2026 · issue -166

    CUDA backend gains async CPU-to-GPU tensor copies with relaxed sync requirements, improving multi-token throughput.

    └──▷ GET THIS VERSION
    $ git clone --branch b8210 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8210
    • ›Adds CPU-to-CUDA async copy capability to ggml_backend_cuda_cpy_tensor_async(), replacing synchronous copies to reduce GPU pipeline stalls.
    • ›Introduces ggml_backend_sync_mode opt-in mechanism to relax explicit synchronization between host-to-device copies and graph execution, extensible to backends beyond CUDA (e.g., Vulkan).
  363. b8191 Mar 3, 2026 · issue -168

    llama.cpp b8191 adds an optimized Q4_1 matrix-multiplication OpenCL kernel for Adreno GPUs.

    └──▷ GET THIS VERSION
    $ git clone --branch b8191 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8191
    • ›Adds an optimized q4_1 matrix-multiplication (mm) OpenCL kernel targeting Qualcomm Adreno GPUs, accelerating Q4_1 quantized inference on mobile/edge hardware via ggml_cl_mul_mat_q4_1_f32_adreno.
  364. b8179 Feb 27, 2026 · issue -171

    llama.cpp b8179 adds CDNA3 MFMA tensor core flash attention for AMD MI300X, yielding up to +39% prompt-processing throughput.

    └──▷ GET THIS VERSION
    $ git clone --branch b8179 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8179
    • ›Adds CDNA3 MFMA tensor core support for the flash attention MMA kernel on AMD MI300X (gfx942), using v_mfma_f32_16x16x16_f16 (FP16 input, FP32 accumulate), with head sizes 64, 80, 96, 112, and 128 supported — delivering pp512 +7%, pp1024 +13%, pp2048 +23%, pp4096 +39% on Qwen2.5-1.5B Q4_K_M.
    • ›Routes MI300X flash attention automatically: MMA kernel used for prompt processing (effective batch eff_nq >= 128), tile kernel for small batches — no manual dispatch required.
  365. b8177 Feb 27, 2026 · issue -171

    llama.cpp server now mirrors /v1/responses to /responses for API consistency.

    └──▷ GET THIS VERSION
    $ git clone --branch b8177 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8177
    └──▷ TRY IT
    Call the responses endpoint without the /v1 prefix, consistent with how clients may already hit /responses directly.
    $ curl http://localhost:8080/responses -H 'Content-Type: application/json' -d '{"model": "<model>", "input": "Hello"}'
    • ›Adds /responses as a mirror endpoint for /v1/responses in the server, matching the existing /v1/chat/completions pattern.
  366. b8173 Feb 27, 2026 · issue -171

    llama.cpp server gains comma-separated --alias and new --tags flag for multi-alias model routing via /v1/models

    └──▷ GET THIS VERSION
    $ git clone --branch b8173 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8173
    └──▷ TRY IT
    Serve a model under multiple aliases so clients using any of the names (e.g. 'gpt-4', 'llama3', 'default') are routed to the same backend model.
    $ llama-server --model /models/llama-3-8b.gguf --alias gpt-4,llama3,default --tags experimental,local
    • ›Adds comma-separated value support to --alias in the llama.cpp server, allowing a single model to be registered under multiple aliases simultaneously (stored as a deduplicated std::set).
    • ›Adds new --tags flag to the llama.cpp server for attaching informational metadata to a model (not used for routing).
    • ›The /v1/models endpoint now exposes aliases and tags fields, reflecting all registered aliases and tags for each loaded model.
  367. b8164 Feb 27, 2026 · issue -171

    llama.cpp b8164 adds option to merge gate and up (exp) weights for MoE models during GGUF conversion.

    └──▷ GET THIS VERSION
    $ git clone --branch b8164 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8164
    • ›Adds create_tensor_gate_up helper and gate_up merged-weight support to convert_hf_to_gguf.py for all MoE models, enabling fused gate/exp weight tensors in converted GGUF files.
  368. b8153 Feb 25, 2026 · issue -173

    llama.cpp server gains multi-modal prompt caching support

    └──▷ GET THIS VERSION
    $ git clone --branch b8153 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8153
    • ›Enables multi-modal prompt caching in the server, allowing image and other non-text prompt prefixes to be cached and reused across requests.
  369. b8152 Feb 25, 2026 · issue -173

    llama.cpp server gains multi-modal context checkpoint support for hybrid and recurrent memory architectures.

    └──▷ GET THIS VERSION
    $ git clone --branch b8152 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8152
    • ›Adds multi-modal context checkpoint support to the server, enabling checkpoint/restore of KV cache state across hybrid-ISWA and recurrent memory backends.
  370. b8144 Feb 24, 2026 · issue -174

    llama.cpp server adds max_completion_tokens request property as the replacement for deprecated max_tokens.

    └──▷ GET THIS VERSION
    $ git clone --branch b8144 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8144
    • ›Adds max_completion_tokens request property to the server API, setting the upper bound for reasoning+output tokens and superseding the now-deprecated max_tokens field.
  371. b8132 Feb 23, 2026 · issue -175

    llama.cpp b8132: CLI now passes the model's text filename to the model context.

    └──▷ GET THIS VERSION
    $ git clone --branch b8132 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8132
    • ›The CLI provides the model with its text filename, giving the model awareness of the file it was loaded from.
  372. b8128 Feb 22, 2026 · issue -176

    llama.cpp b8128 adds support for the Kanana-2 model architecture.

    └──▷ GET THIS VERSION
    $ git clone --branch b8128 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8128
    • ›Adds support for the Kanana-2 model architecture.
  373. b8123 Feb 21, 2026 · issue -177

    llama.cpp b8123 adds a ROCm 7.2 build target covering ten GPU architectures including gfx942 and gfx1201.

    └──▷ GET THIS VERSION
    $ git clone --branch b8123 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8123
    • ›Adds a build target to generate ROCm 7.2 artifacts supporting GPU architectures gfx1151, gfx1150, gfx1200, gfx1201, gfx1100, gfx1101, gfx1030, gfx908, gfx90a, and gfx942.
  374. b8117 Feb 20, 2026 · issue -178

    llama.cpp b8117 adds RISC-V Vector (RVV) vec dot kernels for iq1_s, iq1_m, iq2_s, iq3_s, tq1_0, and tq2_0 quantization types.

    └──▷ GET THIS VERSION
    $ git clone --branch b8117 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8117
    • ›Adds RVV (RISC-V Vector) vectorized dot-product kernels for quantization types iq2_s, iq3_s, tq1_0, tq2_0, iq1_s, and iq1_m, accelerating inference on RISC-V hardware with vector extensions.
    • ›Adds VLEN-aware switching logic for RVV vec dot kernels, enabling correct dispatch across RISC-V implementations with different vector register lengths.
  375. b8116 Feb 20, 2026 · issue -178

    llama-quantize gains --dry-run to preview quantization plans and imatrix warnings before committing.

    └──▷ GET THIS VERSION
    $ git clone --branch b8116 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8116
    └──▷ TRY IT
    Verify a quantization plan and imatrix requirements before a long quant run, without writing any output.
    $ llama-quantize --dry-run model.gguf output.gguf Q4_K_M
    • ›Adds --dry-run flag to llama-quantize, letting practitioners preview tensor layout, bits-per-weight, and imatrix requirements without writing output files.
    • ›Adds a courtesy warning when a selected quantization type requires an importance matrix (imatrix) but none is provided, including an edge-case guard for Q2_K.
    • ›Displays model and quantization BPW (bits per weight) summary upon quantization completion.
  376. b8110 Feb 20, 2026 · issue -178

    llama.cpp b8110 adds support for the PaddleOCR-VL multimodal model.

    └──▷ GET THIS VERSION
    $ git clone --branch b8110 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8110
    • ›Adds support for the PaddleOCR-VL vision-language model, including updated CLIP loader parameters and multimodal rope (mtmd_decode_use_mrope) handling via convert_hf_to_gguf.py.
  377. b8107 Feb 20, 2026 · issue -178

    llama.cpp b8107 adds flash attention toggle via ctx_params in the multimodal (mtmd) attention builder.

    └──▷ GET THIS VERSION
    $ git clone --branch b8107 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8107
    • ›Adds flash attention on/off control via ctx_params in the mtmd build_attn path, enabling runtime toggling of flash attention for multimodal contexts.
  378. b8106 Feb 20, 2026 · issue -178

    llama.cpp b8106 adds JAIS-2 Arabic-English bilingual model architecture support with quantization down to Q2_K.

    └──▷ GET THIS VERSION
    $ git clone --branch b8106 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8106
    • ›Adds JAIS-2 architecture support for Inception AI's Arabic-English bilingual model family (Jais-2-8B and Jais-2-70B), including LLAMA_VOCAB_PRE_TYPE_JAIS2 pre-tokenizer type and control vector support via build_cvec.
    • ›Supports JAIS-2 quantizations: BF16, Q8_0, Q6_K, Q5_K_M, Q5_0, Q4_K_M, Q4_0, Q3_K_M, and Q2_K.
  379. b8102 Feb 19, 2026 · issue -179

    llama.cpp b8102 adds tokenizer support for LFM2.5-Audio-1.5B, enabling lightweight audio model conversion via convert_hf_to_gguf.py.

    └──▷ GET THIS VERSION
    $ git clone --branch b8102 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8102
    └──▷ TRY IT
    Convert the LFM2.5-Audio-1.5B audio tokenizer to GGUF format for local inference.
    $ python3 convert_hf_to_gguf.py /path/to/LFM2.5-Audio-1.5B/audio_detokenizer
    • ›Adds tokenizer support for the LFM2.5-Audio-1.5B model architecture, convertible via python3 convert_hf_to_gguf.py /path/to/LFM2.5-Audio-1.5B/audio_detokenizer.
    • ›Adds LFM2 SWA (Sliding Window Attention) model support alongside the new tokenizer.
  380. b8100 Feb 19, 2026 · issue -179

    llama.cpp b8100 adds full Modern BERT support including mean and rank pooling with GELU op.

    └──▷ GET THIS VERSION
    $ git clone --branch b8100 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8100
    • ›Adds full Modern BERT model support, including mean pooling, rank pooling with GELU op, and a dense first layer as per the Modern BERT research paper, convertible via convert_hf_to_gguf.py.
  381. b8099 Feb 19, 2026 · issue -179

    llama.cpp b8099 adds FP16 MMA path on PowerPC for Q4/Q8 matmul, delivering 1.5–2x prompt-processing speedup.

    └──▷ GET THIS VERSION
    $ git clone --branch b8099 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8099
    • ›Adds FP16×FP16→FP32 MMA acceleration path on PowerPC for Q4 and Q8 matrix multiplication, achieving 1.5–2x improvement in prompt-processing speed for Q4_0 and Q8_0 models on CPU.
  382. b8094 Feb 18, 2026 · issue -180

    llama.cpp server now saves generated text in /slots endpoint when LLAMA_SERVER_SLOTS_DEBUG=1

    └──▷ GET THIS VERSION
    $ git clone --branch b8094 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8094
    • ›Sets LLAMA_SERVER_SLOTS_DEBUG=1 to enable saving generated text in the /slots endpoint for server-side slot inspection and debugging.
  383. b8091 Feb 18, 2026 · issue -180

    llama.cpp b8091 adds JIT shader compilation for WebGPU, covering mul_mat, get_rows, and scale operations.

    └──▷ GET THIS VERSION
    $ git clone --branch b8091 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8091
    • ›Adds JIT (just-in-time) compilation for WebGPU shaders covering mul_mat, get_rows, and scale operations, enabling runtime-optimized GPU kernel dispatch.
    • ›Reorganizes the WebGPU backend into an all-encompassing shader library, splitting large shaders into multiple static strings and refactoring argmax, set_rows, flash attention, and matrix multiplication into a unified format.
  384. b8093 Feb 18, 2026 · issue -180

    llama.cpp b8093 adds support for the GLM-OCR model.

    └──▷ GET THIS VERSION
    $ git clone --branch b8093 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8093
    • ›Adds support for the GLM-OCR model, including updates to convert_hf_to_gguf.py for conversion.
  385. b8073 Feb 17, 2026 · issue -181

    llama.cpp b8073 adds support for Tiny Aya models.

    └──▷ GET THIS VERSION
    $ git clone --branch b8073 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8073
    • ›Adds support for Tiny Aya models, including updated vocabulary and tokenizer regex handling.
  386. b8054 Feb 14, 2026 · issue -184

    llama.cpp b8054 adds multimodal support for Nemotron Nano 12B v2 VL.

    └──▷ GET THIS VERSION
    $ git clone --branch b8054 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8054
    • ›Adds vision-language (VL) support for the Nemotron Nano 12B v2 model, including pre-downsampled position embeddings during GGUF conversion for fixed input sizes.
  387. b8051 Feb 14, 2026 · issue -184

    llama.cpp b8051 adds NetBSD build support.

    └──▷ GET THIS VERSION
    $ git clone --branch b8051 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8051
    • ›Adds NetBSD build support, expanding platform coverage to include NetBSD targets.
  388. b8036 Feb 14, 2026 · issue -184

    llama.cpp b8036 adds support for the GLM MoE DSA model architecture.

    └──▷ GET THIS VERSION
    $ git clone --branch b8036 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8036
    • ›Adds support for the GLM MoE DSA model architecture (note: the indexer component is not yet supported in this release).
  389. b8033 Feb 14, 2026 · issue -184

    llama.cpp b8033 adds --verbose-prompt flag support.

    └──▷ GET THIS VERSION
    $ git clone --branch b8033 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8033
    └──▷ TRY IT
    Inspect exactly how your prompt is tokenized before a model run, useful when diagnosing unexpected model behavior.
    $ llama-cli --verbose-prompt -m model.gguf -p "Tell me about buffer overflows"
    • ›Adds --verbose-prompt flag to print the full tokenized prompt before inference, aiding prompt debugging and inspection.
  390. b8021 Feb 13, 2026 · issue -185

    llama.cpp b8021 adds OpenCL support for Q4_1 quantization in matrix-vector and matrix-matrix operations.

    └──▷ GET THIS VERSION
    $ git clone --branch b8021 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8021
    • ›Adds OpenCL backend support for q4_1 quantized matrix-vector (mv) and matrix-matrix (mm) operations, enabling GPU-accelerated inference with Q4_1 models on OpenCL devices.
    • ›Adds general q4_0 matrix-matrix (mm) support in the OpenCL backend.
  391. b8020 Feb 13, 2026 · issue -185

    llama-parallel gains the -kvu flag in b8020

    └──▷ GET THIS VERSION
    $ git clone --branch b8020 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8020
    • ›Adds -kvu argument to llama-parallel for controlling key-value cache usage during parallel inference.
  392. b8003 Feb 12, 2026 · issue -186

    llama.cpp b8003 adds Kimi-K2.5 multimodal model support including image and vision capabilities.

    └──▷ GET THIS VERSION
    $ git clone --branch b8003 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b8003
    • ›Adds support for the Kimi-K2.5 model, including image and vision inference via updated convert_hf_to_gguf.py with new kimi-k2.5 keys and V_MMPROJ / V_M_IMP_NORM tensor mappings.
  393. b7990 Feb 10, 2026 · issue -188

    llama.cpp b7990 adds support for the Qwen3.5 model series with FULL_ATTENTION_INTERVAL metadata.

    └──▷ GET THIS VERSION
    $ git clone --branch b7990 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7990
    • ›Adds FULL_ATTENTION_INTERVAL metadata key to support the Qwen3.5 series hybrid attention architecture.
    • ›Supports loading and running Qwen3.5 series models.
  394. b7983 Feb 10, 2026 · issue -188

    llama.cpp b7983 adds quantized MoE matrix multiplication for the CANN backend, supporting Q4_0 and Q8_0 weight formats.

    └──▷ GET THIS VERSION
    $ git clone --branch b7983 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7983
    • ›Adds ggml_cann_mul_mat_id_quant to the CANN backend, enabling quantized matrix multiplication for Mixture of Experts (MoE) models with Q4_0 and Q8_0 weight formats, automatic F16 type conversion, and both per-expert and broadcast input modes.
  395. b7973 Feb 9, 2026 · issue -189

    llama.cpp b7973 adds support for Qwen3.5 dense and MoE model architectures.

    └──▷ GET THIS VERSION
    $ git clone --branch b7973 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7973
    • ›Adds inference support for Qwen3.5 dense and Mixture-of-Experts (MoE) model architectures (vision not included).
  396. b7964 Feb 7, 2026 · issue -191

    llama.cpp b7964 adds support for the Step3.5-Flash model architecture.

    └──▷ GET THIS VERSION
    $ git clone --branch b7964 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7964
    • ›Adds support for the Step3.5-Flash model architecture (LLM_ARCH_STEP35), including GGUF conversion, swiglu clamp KV entries, and KV shifting disabled for this architecture.
  397. b7957 Feb 6, 2026 · issue -192

    llama.cpp b7957 adds Kimi-Linear model support with backend-agnostic inference and MLA KV cache.

    └──▷ GET THIS VERSION
    $ git clone --branch b7957 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7957
    • ›Adds support for the Kimi-Linear (Kimi-K2) model architecture, including convert_hf_to_gguf.py conversion, backend-agnostic inference, and MLA KV cache via the llm_arch_is_hybrid path.
    • ›Introduces llm_graph_input_mem_hybrid_k class to support MLA-style KV cache for hybrid architectures like Kimi-Linear.
    • ›Adds Kimi-K2-specific end-of-generation (EOG) tokens to the vocabulary recognition layer.
    • ›Adds causal_conv1d static function abstracting convolution logic for query, key, and value projections in Kimi-Linear layers.
    • ›Supports Kimi-Linear MoE parameters including n_experts, n_experts_used, and score_func via the shared TextModel conversion path in convert_hf_to_gguf.py.
    +1 moreshow less
    • ›Skips quantization of conv1d weights for Kimi-Linear models during GGUF conversion.
  398. b7951 Feb 5, 2026 · issue -193

    llama.cpp b7951 adds adaptive CPU/GPU interleave on Metal for improved Apple Silicon inference.

    └──▷ GET THIS VERSION
    $ git clone --branch b7951 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7951
    • ›Adds adaptive CPU/GPU interleave scheduling on Metal, dynamically balancing compute across CPU and GPU based on the number of graph nodes for Apple Silicon and iOS deployments.
  399. b7919 Feb 3, 2026 · issue -195

    llama.cpp b7919 adds Falcon-H1-Tiny-Coder FIM token support to the vocabulary.

    └──▷ GET THIS VERSION
    $ git clone --branch b7919 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7919
    • ›Adds Falcon-H1-Tiny-Coder Fill-in-the-Middle (FIM) tokens to the vocabulary, enabling FIM-mode code completion with that model.
  400. b7897 Jan 31, 2026 · issue -198

    llama-quantize gains --tensor-type-file to specify per-tensor quantization types via a file.

    └──▷ GET THIS VERSION
    $ git clone --branch b7897 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7897
    └──▷ TRY IT
    Apply custom per-tensor quantization types from a file instead of specifying them inline, useful for fine-grained control over large model quantization.
    $ llama-quantize --tensor-type-file tensor_types.txt input_model.gguf output_model.gguf Q4_K_M
    • ›Adds --tensor-type-file option to llama-quantize, allowing per-tensor quantization type assignments to be loaded from a file.
  401. b7896 Jan 31, 2026 · issue -198

    llama.cpp b7896 adds vision-only support for MiniCPM-o 4.5 via the multimodal (mtmd) subsystem.

    └──▷ GET THIS VERSION
    $ git clone --branch b7896 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7896
    • ›Adds MiniCPM-o 4.5 (vision-only) support to the mtmd multimodal subsystem.
  402. b7876 Jan 29, 2026 · issue -200

    llama.cpp b7876 enables Hexagon HTP offloading on Windows on Snapdragon (WoS) devices.

    └──▷ GET THIS VERSION
    $ git clone --branch b7876 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7876
    • ›Enables GPU offloading to the Qualcomm Hexagon HTP (via the ggml-backend-dl dynamic backend) on Windows on Snapdragon (WoS) devices, alongside existing Android support.
    • ›Adds run-bench.ps1 and run-tool.ps1 PowerShell scripts to simplify benchmarking and tool execution in the WoS/Hexagon build environment.
    • ›Renames the backend documentation directory from docs/backend/hexagon to docs/backends/snapdragon, consolidating Android and Windows build guides into a unified README.
    └──▷ BREAKING ON UPGRADE
    • !The backend documentation directory is renamed from docs/backend/hexagon to docs/backends/snapdragon; any scripts or CI referencing the old path will break.
  403. b7870 Jan 29, 2026 · issue -200

    llama-batched-bench gains the -kvu flag for additional benchmarking control.

    └──▷ GET THIS VERSION
    $ git clone --branch b7870 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7870
    • ›Adds -kvu argument to llama-batched-bench for expanded benchmark configuration.
  404. b7864 Jan 28, 2026 · issue -201

    llama.cpp b7864 adds draftless self-speculative decoding via --spec-type, no separate draft model required.

    └──▷ GET THIS VERSION
    $ git clone --branch b7864 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7864
    └──▷ TRY IT
    Run the llama.cpp server with self-speculative decoding enabled — no draft model binary needed, speeding up throughput on a single model.
    $ llama-server --model ./mistral-7b-q4_k_m.gguf --spec-type ngram
    • ›Adds --spec-type CLI flag to enable self-speculative decoding without a separate draft model, using an n-gram map to generate draft tokens from the primary model itself.
    • ›Introduces common_speculative_type enum and a vector of speculative states to support multiple speculation modes within a single server session.
    • ›Adds common_speculative_begin() and common_speculative_state::accept() to the speculative decoding API, enabling structured draft/accept lifecycle management in code integrating llama.cpp.
    └──▷ BREAKING ON UPGRADE
    • !The speculative.draftless_type config field is renamed to speculative.type.
    • !The --spec-draftless flag is replaced by --spec-type; any existing invocation using --spec-draftless will break.
    • !The --spec-config flag is removed entirely.
    • !common_speculative_params is removed from the API.
  405. b7858 Jan 28, 2026 · issue -201

    llama.cpp b7858 adds a new ggml backend for Virglrenderer API Remoting GPU acceleration.

    └──▷ GET THIS VERSION
    $ git clone --branch b7858 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7858
    • ›Adds a new ggml backend for Virglrenderer API Remoting acceleration, enabling GPU-accelerated inference over virtualized graphics stacks.
  406. b7851 Jan 28, 2026 · issue -201

    WebGPU backend gains new unary operators (SOFTPLUS, EXPM1, FLOOR, CEIL, ROUND, TRUNC) and per-thread state isolation.

    └──▷ GET THIS VERSION
    $ git clone --branch b7851 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7851
    • ›Adds SOFTPLUS, EXPM1, FLOOR, CEIL, ROUND, and TRUNC unary operators to the WebGPU backend, each with f16/f32 and inplace/non-inplace shader variants.
    • ›Splits webgpu_context shared state into global state (Instance, Device, Adapter, capabilities) and per-thread state (Queue, pipelines, parameter buffers), enabling safe multi-threaded WebGPU execution.
    • ›Re-enables XIELU unary operator support in the WebGPU backend after correcting IEEE 754 bit-pattern preservation in parameter passing.
  407. b7842 Jan 27, 2026 · issue -202

    llama.cpp b7842 enables FP16 MMA kernels on PowerPC CPUs for faster CPU inference.

    └──▷ GET THIS VERSION
    $ git clone --branch b7842 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7842
    • ›Enables FP16 MMA (matrix multiply-accumulate) kernels for PowerPC CPUs via ggml-cpu, accelerating on-CPU inference on PPC hardware.
  408. b7833 Jan 25, 2026 · issue -204

    llama.cpp b7833 adds tiled flash attention for CPU prompt processing, improving long-context performance.

    └──▷ GET THIS VERSION
    $ git clone --branch b7833 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7833
    • ›Adds tiled flash attention (FA) kernel for CPU prompt-processing, replacing the prior vector kernel that degraded performance on long contexts — tile sizes tuned on AMD EPYC 64-core hardware.
  409. b7832 Jan 25, 2026 · issue -204

    llama.cpp b7832 adds V-less KV-cache support, reducing memory overhead for compatible model architectures.

    └──▷ GET THIS VERSION
    $ git clone --branch b7832 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7832
    • ›Adds V-less KV-cache mode, allowing the value cache to be omitted for model architectures that share V with K, reducing memory usage during inference.
  410. b7821 Jan 24, 2026 · issue -205

    llama.cpp b7821 enables CUDA graphs for n-cpu-moe workloads, improving GPU inference performance for MoE models.

    └──▷ GET THIS VERSION
    $ git clone --branch b7821 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7821
    • ›Enables CUDA graphs support for n-cpu-moe (CPU-offloaded mixture-of-experts) inference, reducing GPU launch overhead and improving throughput on CUDA backends.
  411. b7813 Jan 22, 2026 · issue -207

    llama.cpp server now preserves reasoning_content in assistant messages, enabling round-trip chain-of-thought workflows.

    └──▷ GET THIS VERSION
    $ git clone --branch b7813 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7813
    • ›Adds reasoning_content input support to the server, allowing assistant messages that include reasoning content to be passed back in subsequent requests.
  412. b7807 Jan 22, 2026 · issue -207

    llama.cpp b7807 lets manual tensor-type overrides take precedence during quantization

    └──▷ GET THIS VERSION
    $ git clone --branch b7807 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7807
    • ›Manual overrides of tensor types now take precedence over automatic selection during quantization (#18952).
  413. b7793 Jan 22, 2026 · issue -207

    llama.cpp b7793 adds partial OpenAI Responses API support at POST /v1/responses including tool calls and image/file inputs.

    └──▷ GET THIS VERSION
    $ git clone --branch b7793 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7793
    • ›Adds POST /v1/responses endpoint to the llama.cpp server with partial OpenAI Responses API compatibility, supporting text, image, and file input message types.
    • ›Adds tool call support (function calling) to the /v1/responses endpoint for both streaming and non-streaming modes.
    • ›Feeds reasoning/thinking model outputs through the chat template pipeline in the /v1/responses endpoint.
    • ›Rejects input_file input type on /v1/responses with an explicit error since it is not supported by the underlying chat completion backend.
  414. b7789 Jan 21, 2026 · issue -208

    llama.cpp b7789 adds hybrid ISWA memory support via llama_memory_hybrid_iswa

    └──▷ GET THIS VERSION
    $ git clone --branch b7789 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7789
    • ›Adds llama_memory_hybrid_iswa hybrid sliding-window attention memory implementation in src/llama-memory-hybrid-iswa.cpp.
  415. b7761 Jan 17, 2026 · issue -212

    llama.cpp b7761 extends the WebGPU backend with sampling-critical operators including top-k, argsort, argmax, cumsum, PAD, LOG, and more.

    └──▷ GET THIS VERSION
    $ git clone --branch b7761 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7761
    • ›Adds SOFTPLUS, EXPM1, FLOOR, CEIL, ROUND, and TRUNC unary operators to the WebGPU backend, each with f16/f32 and inplace/non-inplace shader variants.
    • ›Adds argmax, cumsum, sum, and sum_rows operators to the WebGPU backend.
    • ›Adds argsort to the WebGPU backend using a multi-pass strategy.
    • ›Adds top-k support to the WebGPU backend, enabling GPU-accelerated sampling.
    • ›Adds PAD, LOG, CPY, GET_ROWS, and SET_ROWS (with i32 indices) operators to the WebGPU backend.
    +3 moreshow less
    • ›Adds FILL operator support to the WebGPU backend via pre-WGSL porting.
    • ›Adds get_memory reporting for the WebGPU backend.
    • ›Ports unary operators to the pre-WGSL pipeline path in the WebGPU backend.
  416. b7756 Jan 16, 2026 · issue -213

    llama.cpp b7756 replaces minja with a new built-from-scratch Jinja template engine for chat template rendering.

    └──▷ GET THIS VERSION
    $ git clone --branch b7756 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7756
    • ›Introduces a new built-in Jinja template engine (lexer, parser, and runtime) in common, replacing the prior minja dependency — supports filters (selectattr, rejectattr, split/rsplit with maxsplit, trim with args, tojson with separators and indent), keyword arguments, array slicing, macros, and lstrip_block behavior matched to Hugging Face JS semantics.
    • ›Adds support for rendering chat templates for additional model families including Gemma, MiniMax, GLM-4.5, Kimi-K2, Nemotron, IBM Granite, and Apertus through the new engine.
    • ›Extends test-chat-template to consolidate Jinja rendering tests (absorbing the former test-chat-jinja target) with an --output flag for inspecting rendered results.
    • ›Adds a fuzz test target for the Jinja engine covering both template rendering and function argument handling.
    └──▷ BREAKING ON UPGRADE
    • !minja is removed and its call sites are replaced by the new Jinja runtime; any build or code that depended on minja headers or the test-chat-jinja CMake target will break.
    • !The test-chat-jinja CMake target is removed; tests previously run through it now run under test-chat-template.
  417. b7752 Jan 16, 2026 · issue -213

    llama.cpp b7752 adds CANN backend support for the GGML_OP_GATED_LINEAR_ATTN operator.

    └──▷ GET THIS VERSION
    $ git clone --branch b7752 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7752
    • ›Adds GGML_OP_GATED_LINEAR_ATTN operator support to the CANN backend, enabling gated linear attention models to run on Ascend NPUs.
  418. b7748 Jan 15, 2026 · issue -214

    llama.cpp b7748 adds the adaptive-p sampler with EMA-based probability targeting and configurable decay.

    └──▷ GET THIS VERSION
    $ git clone --branch b7748 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7748
    • ›Adds adaptive-p sampler to common_params_sampling and the samplers vector, with parameters including decay (default 0.9), min_target, max_target (clamped to [0.0, 1.0]), and a use_power_law flag for power-law mode.
    • ›Exposes adaptive-p via common_sampler_types_from_chars so it can be selected alongside existing samplers using single-character sampler chain strings.
  419. b7740 Jan 15, 2026 · issue -214

    llama.cpp b7740 adds Hexagon HVX OP_CPY support and makes host buffers optional on Snapdragon backends.

    └──▷ GET THIS VERSION
    $ git clone --branch b7740 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7740
    • ›Adds GGML_HEXAGON_EXPERIMENTAL environment variable support, now documented in the Hexagon readme.
    • ›Adds OP_CPY support for fp16/fp32 → fp16/fp32 copy operations on the Hexagon HVX backend, enabling broader model layer offload to Snapdragon DSPs.
    • ›Makes host buffers optional on the Hexagon backend, with repack buffers automatically disabled when host buffers are not available.
  420. b7727 Jan 14, 2026 · issue -215

    llama.cpp b7727 adds support for the EXAONE Mixture-of-Experts (MoE) model architecture.

    └──▷ GET THIS VERSION
    $ git clone --branch b7727 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7727
    • ›Adds support for the EXAONE MoE model architecture, including Multi-Token Prediction (MTP) inference.
  421. b7723 Jan 13, 2026 · issue -216

    llama.cpp b7723 adds fattn-mma-f16 flash attention kernel for AMD RDNA4 GPUs via HIP.

    └──▷ GET THIS VERSION
    $ git clone --branch b7723 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7723
    • ›Adds fattn-mma-f16 flash attention kernel for RDNA4 GPUs via the HIP backend, enabling hardware-accelerated matrix multiply-accumulate attention on AMD RDNA4 hardware.
  422. b7718 Jan 13, 2026 · issue -216

    llama-bench gains a direct_io parameter for benchmarking with direct I/O enabled.

    └──▷ GET THIS VERSION
    $ git clone --branch b7718 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7718
    • ›Adds direct_io parameter to llama-bench to control direct I/O during benchmarking runs.
  423. b7716 Jan 13, 2026 · issue -216

    llama.cpp b7716 adds a server argument to disable prompt caching for deterministic OpenAI-compat responses.

    └──▷ GET THIS VERSION
    $ git clone --branch b7716 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7716
    • ›Adds a server argument to disable prompt caching, enabling deterministic responses for clients restricted to OpenAI-compatible requests.
  424. b7713 Jan 12, 2026 · issue -217

    llama.cpp b7713 adds --kv-unified flag to the batched example CLI.

    └──▷ GET THIS VERSION
    $ git clone --branch b7713 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7713
    • ›Adds --kv-unified flag to the batched example CLI, enabling unified KV cache mode required for batches containing coupled sequences.
  425. b7712 Jan 12, 2026 · issue -217

    Vulkan memory logger now togglable via environment variable

    └──▷ GET THIS VERSION
    $ git clone --branch b7712 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7712
    • ›Vulkan memory_logger is now controlled by an environment variable, allowing runtime enable/disable without recompilation.
  426. b7694 Jan 10, 2026 · issue -219

    llama.cpp b7694 adds support for named remote presets

    └──▷ GET THIS VERSION
    $ git clone --branch b7694 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7694
    • ›Supports named remote presets, allowing preset configurations to be referenced by name from remote sources
  427. b7691 Jan 10, 2026 · issue -219

    llama.cpp b7691 embeds all licenses into compiled binaries, accessible via new --license flag.

    └──▷ GET THIS VERSION
    $ git clone --branch b7691 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7691
    • ›Adds --license flag to display all embedded licenses directly from the binary, eliminating the need to distribute separate LICENSE files alongside executables.
  428. b7689 Jan 10, 2026 · issue -219

    llama.cpp b7689 adds Gemma3n multimodal support with MobileNetV5 vision encoder for image processing.

    └──▷ GET THIS VERSION
    $ git clone --branch b7689 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7689
    • ›Adds Gemma3nVisionModel with MobileNetV5 vision encoder to convert_hf_to_gguf.py, enabling conversion and inference of Gemma3n multimodal models.
    • ›Adds gemma3n vision projector entry to gguf-py/gguf/constants.py and supporting tensor mappings in tensor_mapping.py for the new architecture.
    • ›Implements mobilenetv5.cpp with clip_graph_mobilenetv5 struct to support the MobileNetV5 vision encoding pipeline within llama.cpp.
  429. b7681 Jan 9, 2026 · issue -220

    llama.cpp b7681 adds remote preset support, letting a single HF repo reference multiple GGUF sources.

    └──▷ GET THIS VERSION
    $ git clone --branch b7681 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7681
    • ›Adds remote preset support via --offline option awareness, allowing argument configurations to be fetched from a remote Hugging Face repo — including one HF repo pointing to multiple HF repos for multi-GGUF use cases.
  430. b7678 Jan 8, 2026 · issue -221

    llama.cpp b7678 adds initial FlashAttention implementation for the WebGPU backend.

    └──▷ GET THIS VERSION
    $ git clone --branch b7678 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7678
    • ›Adds initial FlashAttention implementation to the WebGPU (ggml-webgpu) backend, enabling attention computation on GPU via the WebGPU API.
    • ›Adds fast matrix and matrix-vector multiplication kernels to the WebGPU backend, improving throughput for inference on WebGPU-capable devices.
    • ›Adds Q4_0 quantized matrix multiplication support to the WebGPU backend.
    • ›Adds subgroup matrix (cooperative matrix) shader support to the WebGPU backend for improved GPU utilization where the hardware capability is present.
    • ›Adds F16 accumulation with shared-memory staging to WebGPU attention and matmul kernels.
    +2 moreshow less
    • ›Adds Emscripten/WASM build support for the WebGPU backend, including memory64 and pthread integration.
    • ›Adds a CI workflow (ggml-ci) for WebGPU backend testing.
  431. b7672 Jan 8, 2026 · issue -221

    llama.cpp b7672 adds per-device free memory targeting to llama-fit-params

    └──▷ GET THIS VERSION
    $ git clone --branch b7672 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7672
    • ›Adds free memory target per device to llama-fit-params, enabling more precise multi-GPU memory allocation control.
  432. b7668 Jan 8, 2026 · issue -221

    llama.cpp b7668 adds --direct-io flag to bypass OS page cache during model loading

    └──▷ GET THIS VERSION
    $ git clone --branch b7668 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7668
    • ›Adds --direct-io flag to enable O_DIRECT model loading, bypassing the OS page cache to reduce memory pressure on large model loads (automatically disabled when --mmap is explicitly enabled).
  433. b7655 Jan 7, 2026 · issue -222

    llama.cpp b7655 adds llama-debug, a new CLI utility for inspecting and validating converted GGUF models.

    └──▷ GET THIS VERSION
    $ git clone --branch b7655 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7655
    └──▷ TRY IT
    Capture logits and token IDs from a converted GGUF model to compare against a reference PyTorch run.
    $ ./build/bin/llama-debug -m models/Qwen2.5-0.5B-Instruct.gguf --prompt "Hello, my name is" --save-logits
    Verify that the original and converted model tokenizers produce identical token IDs before comparing logits.
    $ ./scripts/utils/compare_tokens.py pytorch-gemma-3-270m-it llamacpp-gemma-3-270m-it-bf16 -v
    • ›Adds llama-debug binary with --save-logits, --prompt / -p, --tensor-filter, --verbose, and --embedding flags for saving and inspecting logits, embeddings, and tensor outputs from a converted model.
    • ›Adds compare_tokens.py script (in scripts/utils/) with an optional -v flag to diff tokenizer output — token IDs and prompts — between an original and a converted model.
    • ›Integrates token-ID comparison into existing compare-logits.py and semantic_check.py verification scripts so tokenizer mismatches are caught before logit/embedding checks run.
    • ›Uses llama_model_n_embd_out in the debug example to correctly handle late-interaction retriever models (e.g. LFM2-ColBert-350M) whose output embeddings are down-projected to a lower dimension.
    └──▷ BREAKING ON UPGRADE
    • !logits.cpp and the examples/model-conversion directory have been removed; use llama-debug --save-logits instead.
  434. b7652 Jan 7, 2026 · issue -222

    llama.cpp b7652 adds f16/f32 flash attention, new matmul kernels, and GET_ROWS/SET_ROWS ops for Hexagon DSP backends.

    └──▷ GET THIS VERSION
    $ git clone --branch b7652 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7652
    • ›Adds f16 and f32 flash attention (flash_attn_ext) support for the Hexagon backend, with double-buffered DMA prefetching for K, V, Mask, and Q tensors using FLASH_ATTN_BLOCK_SIZE of 128.
    • ›Adds GET_ROWS f32→f32 and SET_ROWS fp32→f16 (with i32/i64 row index) operations to the Hexagon backend.
    • ›Introduces vec_dot_f16_f16 and vec_dot_f16_f16_rx2 HVX kernel variants for fp16×fp16 matrix multiplication in the Hexagon backend, with automatic fallback for complex broadcasting.
    • ›Adds quantize_fp32_f16 helper to copy and convert weights from DDR to VTCM on Hexagon, enabling the optimized matmul path when VTCM capacity allows.
    • ›Improves fp16 and fp32 matmul throughput on the Hexagon backend via aligned HVX operations, with optimized threading for SET_ROWS when processing few rows.
  435. b7644 Jan 6, 2026 · issue -223

    llama.cpp server adds thinking/reasoning content blocks to the Anthropic Messages API for reasoning models

    └──▷ GET THIS VERSION
    $ git clone --branch b7644 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7644
    • ›Adds thinking content blocks to the Anthropic Messages API when using --reasoning-format deepseek with the thinking parameter enabled — both non-streaming (thinking block before text in content array) and streaming (thinking_delta events with correct block indices) are supported.
    • ›Tracks reasoning state across streaming chunks via the anthropic_has_reasoning member variable to correctly handle partial streaming of thinking content.
  436. b7639 Jan 5, 2026 · issue -224

    llama.cpp b7639 adds support for the LFM2-ColBERT-350M embedding model.

    └──▷ GET THIS VERSION
    $ git clone --branch b7639 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7639
    • ›Adds llama_model_n_embd_out() API function that returns hparams.n_embd_out when set, falling back to hparams.n_embd.
    • ›Adds support for the LFM2-ColBERT-350M model architecture.
  437. b7630 Jan 5, 2026 · issue -224

    CANN backend gains ADD + RMS_NORM operator fusion, toggled via GGML_CANN_OPERATOR_FUSION env var.

    └──▷ GET THIS VERSION
    $ git clone --branch b7630 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7630
    • ›Adds GGML_CANN_OPERATOR_FUSION environment variable (default: false) to enable fused ADD + RMS_NORM kernel execution on the CANN backend, reducing memory access overhead via a single ACLNN AddRmsNorm call.
  438. b7614 Jan 2, 2026 · issue -227

    llama.cpp b7614 adds support for the Maincoder-1B model architecture.

    └──▷ GET THIS VERSION
    $ git clone --branch b7614 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7614
    • ›Adds support for the Maincoder-1B model architecture via a new maincoder.cpp backend.
  439. b7607 Jan 1, 2026 · issue -228

    llama.cpp b7607 adds support for the Youtu-VL vision-language model.

    └──▷ GET THIS VERSION
    $ git clone --branch b7607 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7607
    • ›Adds support for the Youtu-VL vision-language model.
  440. b7605 Jan 1, 2026 · issue -228

    llama.cpp b7605 adds support for JinaBertModel with non-gated feed-forward networks, including German tokenizer variant.

    └──▷ GET THIS VERSION
    $ git clone --branch b7605 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7605
    • ›Adds support for JinaBertModel with non-gated (standard GELU) feed-forward network variant, detected automatically from tensor dimensions.
    • ›Adds jina-v2-de tokenizer variant in the converter to support the German Semantic V3 model.
  441. b7603 Jan 1, 2026 · issue -228

    llama.cpp b7603 adds support for the Solar-Open model with chat template and reasoning tags.

    └──▷ GET THIS VERSION
    $ git clone --branch b7603 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7603
    • ›Adds support for the Solar-Open model, including its LLM type, vocabulary end-of-generation handling, and a solar-open chat template in llama-chat.
  442. b7574 Dec 29, 2025 · issue -231

    llama.cpp b7574 adds -to flag for configurable HTTP read timeouts on the inference server.

    └──▷ GET THIS VERSION
    $ git clone --branch b7574 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7574
    └──▷ TRY IT
    Run the llama.cpp server with a longer HTTP read timeout to handle slow first-token responses without the client disconnecting.
    $ llama-server -m model.gguf -to 3600
    • ›Adds -to command-line argument to server to set a configurable HTTP read timeout (previously fixed at 600 seconds), with internal TTFT crash protection raised to 90 days.
    • ›Enables per-child-model HTTP timeout configuration when running server with child models.
  443. b7567 Dec 29, 2025 · issue -231

    llama.cpp b7567 adds CUDA Blackwell feature support for non-native builds.

    └──▷ GET THIS VERSION
    $ git clone --branch b7567 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7567
    • ›Enables CUDA Blackwell GPU features for non-native (cross-compiled) builds via PR #18436.
  444. b7563 Dec 28, 2025 · issue -232

    llama.cpp b7563 adds support for the Plamo3 model architecture.

    └──▷ GET THIS VERSION
    $ git clone --branch b7563 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7563
    • ›Adds support for the Plamo3 model architecture, including chat template integration.
  445. b7557 Dec 28, 2025 · issue -232

    llama.cpp b7557 adds six new x86-64 CPU architecture targets when building with GGML_CPU_ALL_VARIANTS=On

    └──▷ GET THIS VERSION
    $ git clone --branch b7557 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7557
    └──▷ TRY IT
    Build llama.cpp with all x86-64 CPU variants — including the newly added zen4, cascadelake, and cooperlake targets — to produce optimized binaries for each micro-architecture.
    $ cmake -B build -DGGML_CPU_ALL_VARIANTS=On && cmake --build build --config Release
    • ›Adds ivybridge, piledriver, cannonlake, cascadelake, cooperlake, and zen4 as supported x86-64 CPU backends when building with GGML_CPU_ALL_VARIANTS=On
  446. b7531 Dec 24, 2025 · issue -236

    llama.cpp b7531 adds support for the LlamaBidirectionalModel architecture, enabling llama-embed-nemotron embedding models.

    └──▷ GET THIS VERSION
    $ git clone --branch b7531 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7531
    • ›Adds support for the LlamaBidirectionalModel architecture (llama-embed arch), enabling use of bidirectional embedding models such as llama-embed-nemotron.
  447. b7525 Dec 24, 2025 · issue -236

    llama.cpp b7525 adds LLAMA_ARG_OVERRIDE_TENSOR env var to expose the -ot tensor-override flag to environment-based config.

    └──▷ GET THIS VERSION
    $ git clone --branch b7525 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7525
    └──▷ TRY IT
    Override tensor placement via environment variable in a containerised or scripted deployment where modifying the CLI command is inconvenient.
    $ export LLAMA_ARG_OVERRIDE_TENSOR="blk\.([0-9]|1[0-5])\.attn=CPU"
llama-cli -m model.gguf -p "Hello"
    • ›Adds LLAMA_ARG_OVERRIDE_TENSOR environment variable as an alternative way to supply the -ot (override tensor) argument, enabling tensor-routing control without modifying CLI invocations.
  448. b7524 Dec 23, 2025 · issue -237

    llama.cpp server now reports 0% processing state via return_progress

    └──▷ GET THIS VERSION
    $ git clone --branch b7524 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7524
    • ›Extends return_progress in the llama.cpp server to also report the 0% processing state, giving clients visibility into the earliest stage of request handling.
  449. b7515 Dec 23, 2025 · issue -237

    llama.cpp b7515 adds Granite Embedding and ModernBERT model support with HuggingFace-to-GGUF conversion.

    └──▷ GET THIS VERSION
    $ git clone --branch b7515 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7515
    • ›Adds support for Granite Embedding models, including HuggingFace-to-GGUF conversion via convert_hf_to_gguf.py and inference via llama-embedding.
    • ›Adds ModernBERT architecture support (encoder-only, without head.norm) with alternating local/global sliding window attention, GEGLU activation, and alternating RoPE, implemented in src/models/modern-bert.cpp.
    • ›Adds swa_period (sliding window attention period) GGUF hyperparameter, written and read via gguf-py/gguf/gguf_writer.py and src/llama-hparams.h, enabling dense-every-N-layers SWA patterns for ModernBERT-family models.
  450. b7502 Dec 21, 2025 · issue -239

    Vulkan backend gains async tensor loading with GGML_VK_DISABLE_HOST_VISIBLE_VIDMEM for faster model load times

    └──▷ GET THIS VERSION
    $ git clone --branch b7502 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7502
    • ›Implements set_tensor_async and event interfaces for the Vulkan backend, enabling async load code paths in llama_model_loader::load_all_data.
  451. b7492 Dec 21, 2025 · issue -239

    llama.cpp server gains auto-sleep after configurable idle timeout to free resources automatically

    └──▷ GET THIS VERSION
    $ git clone --branch b7492 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7492
    • ›Adds auto-sleep to the llama-server that suspends the server context and frees model resources after a configurable number of idle seconds, reducing memory and GPU pressure when the server is not actively serving requests.
  452. b7487 Dec 20, 2025 · issue -240

    llama.cpp server gains autoload-on-startup and preset-only option support

    └──▷ GET THIS VERSION
    $ git clone --branch b7487 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7487
    • ›Adds load-on-startup support to llama-server, allowing a model to be loaded automatically when the server starts rather than requiring a separate API call.
    • ›Adds preset-only options to llama-server, enabling configuration presets to be applied without requiring a full model specification.
  453. b7484 Dec 19, 2025 · issue -241

    llama.cpp b7484 adds true Q8_0 quantization on the Hexagon NPU for more accurate mixed-precision matmul.

    └──▷ GET THIS VERSION
    $ git clone --branch b7484 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7484
    • ›Adds a CMake option for configuring FP32 quantize group size on the Hexagon NPU backend (ggml-hexagon).
    • ›Implements true Q8_0 quantization on the Hexagon NPU (ggml-hexagon) for more accurate mixed-precision matrix multiply operations, replacing the previous approximate path.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will migrate from .zip archives to .tar.gz archives — deployment scripts that fetch or unpack Linux release artifacts will need to be updated.
  454. b7480 Dec 19, 2025 · issue -241

    llama.cpp b7480 adds cascading presets with a global section and multi-source preset merging.

    └──▷ GET THIS VERSION
    $ git clone --branch b7480 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7480
    • ›Adds a global section to preset configuration, enabling cascade presets sourced from different origins to merge in a defined order.
    • ›Filters server-controlled args before to_ini() serialization, preventing server-managed values from being written back into preset files.
    • ›Linux releases will switch from .zip to .tar.gz archives in an upcoming release.
    └──▷ BREAKING ON UPGRADE
    • !Linux release archives will soon change from .zip to .tar.gz format, requiring updates to any deployment scripts that reference or extract the current archive format.
  455. b7476 Dec 19, 2025 · issue -241

    llama.cpp b7476 adds Vulkan concurrent perf logger mode and promotes sync logging to a runtime env var.

    └──▷ GET THIS VERSION
    $ git clone --branch b7476 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7476
    └──▷ TRY IT
    Profile overlapping Vulkan GPU workloads at runtime to identify whether individual ops or whole dispatch groups are the bottleneck.
    $ GGML_VK_PERF_LOGGER=1 GGML_VK_PERF_LOGGER_CONCURRENT=1 ./llama-cli -m model.gguf -p 'Hello world'
    Enable Vulkan sync logging at runtime without recompiling — replaces the old ENABLE_SYNC_LOGGING compile-time flag.
    $ GGML_VK_SYNC_LOGGER=1 ./llama-cli -m model.gguf -p 'Hello world'
    • ›Adds GGML_VK_PERF_LOGGER_CONCURRENT=1 env var to enable concurrent Vulkan perf logger mode, which times groups of overlapping GPU work rather than individual operations with barriers — useful for understanding whether operation-level or group-level optimization is needed (requires GGML_VK_PERF_LOGGER also set).
    • ›Adds GGML_VK_SYNC_LOGGER=1 env var, replacing the previous ENABLE_SYNC_LOGGING compile-time switch and making sync logging configurable at runtime without recompilation.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will switch from .zip archives to .tar.gz archives — deployment scripts that unzip Linux builds will need to be updated.
  456. b7475 Dec 19, 2025 · issue -241

    llama.cpp b7475 adds ASR support for the LFM2-Audio-1.5B conformer model.

    └──▷ GET THIS VERSION
    $ git clone --branch b7475 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7475
    • ›Adds automatic speech recognition (ASR) support for the LFM2-Audio-1.5B (conformer) audio model.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will soon switch from .zip to .tar.gz archives — update any deployment scripts that reference .zip download URLs.
  457. b7445 Dec 17, 2025 · issue -243

    llama.cpp b7445 enables -kvu flag in llama-perplexity for hellaswag and winogrande benchmarks with coupled sequences.

    └──▷ GET THIS VERSION
    $ git clone --branch b7445 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7445
    └──▷ TRY IT
    Run hellaswag or winogrande perplexity benchmarks that require a unified KV cache for coupled sequences.
    $ llama-perplexity -kvu -m <model.gguf> --hellaswag
    • ›Adds --kv-unified (-kvu) flag support to llama-perplexity, enabling hellaswag and winogrande benchmarks that use coupled sequences.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will switch from .zip archives to .tar.gz archives — update any deployment scripts that unzip Linux builds.
  458. b7429 Dec 16, 2025 · issue -244

    llama.cpp b7429 adds GLM4V vision encoder support including Metal acceleration and MoE variant.

    └──▷ GET THIS VERSION
    $ git clone --branch b7429 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7429
    • ›Adds support for the GLM4V vision encoder model, including Metal (Apple GPU) acceleration and MoE variant support.
    • ›Introduces GGML_ROPE_TYPE_MRNORM rope type to support the GLM4V text model's multi-resolution positional encoding.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will change from .zip archives to .tar.gz archives — update any deployment scripts that unzip Linux release artifacts.
  459. b7426 Dec 16, 2025 · issue -244

    llama.cpp b7426 adds Nemotron Nano 3 model parsing support

    └──▷ GET THIS VERSION
    $ git clone --branch b7426 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7426
    • ›Adds Nemotron Nano 3 chat template parsing and support in the common layer.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will switch from .zip archives to .tar.gz archives — deployment scripts that fetch or unpack Linux binaries must be updated.
  460. b7418 Dec 16, 2025 · issue -244

    llama.cpp b7418 adds support for the NVIDIA Nemotron Nano 3 model.

    └──▷ GET THIS VERSION
    $ git clone --branch b7418 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7418
    • ›Adds support for running and converting the NVIDIA Nemotron Nano 3 model.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will switch from .zip archives to .tar.gz archives — update deployment scripts accordingly.
  461. b7414 Dec 16, 2025 · issue -244

    llama.cpp b7414 adds support for the KORMo causal language model architecture.

    └──▷ GET THIS VERSION
    $ git clone --branch b7414 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7414
    • ›Adds support for the KORMoForCausalLM model architecture, enabling inference on KORMo causal language models.
    • ›Adds a KORMo-specific tokenizer (KORMo Tokenizer) using the Qwen2 pretokenizer under the hood.
    └──▷ BREAKING ON UPGRADE
    • !Linux release archives will switch from .zip to .tar.gz format — deployment scripts that unzip Linux builds will need to be updated.
  462. b7406 Dec 15, 2025 · issue -245

    SYCL backend gains GPU support for gpt-oss models via mxfp4 matrix multiply, add-id, and swiglu_oai ops

    └──▷ GET THIS VERSION
    $ git clone --branch b7406 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7406
    • ›Adds SYCL GPU support for gpt-oss models via the add-id, mul_mat (mxfp4), and swiglu_oai operations, enabling hardware-accelerated inference for this model family on Intel SYCL devices.
    └──▷ BREAKING ON UPGRADE
    • !Linux release archives will change from .zip to .tar.gz format — update any deployment scripts that download or unpack Linux builds.
  463. b7405 Dec 15, 2025 · issue -245

    llama.cpp b7405 adds GLM-ASR model support and announces Linux archive format change.

    └──▷ GET THIS VERSION
    $ git clone --branch b7405 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7405
    • ›Adds support for the GLM-ASR model architecture, including a new conversion script for HuggingFace checkpoints.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will switch from .zip archives to .tar.gz archives — update any deployment scripts that unzip Linux builds.
  464. b7378 Dec 13, 2025 · issue -247

    llama.cpp b7378 adds llama-completion to bash completion executables and warns of upcoming Linux archive format change.

    └──▷ GET THIS VERSION
    $ git clone --branch b7378 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7378
    • ›Adds llama-completion to the completion-bash executables, enabling tab-completion support for the llama-completion command in Bash.
    • ›Linux releases will switch from .zip to .tar.gz archives in an upcoming release — deployment scripts should be updated accordingly.
  465. b7376 Dec 13, 2025 · issue -247

    llama.cpp b7376 adds support for negated CLI arguments across common args.

    └──▷ GET THIS VERSION
    $ git clone --branch b7376 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7376
    • ›Adds negated argument support to common args, allowing boolean flags to be explicitly disabled at the command line (e.g. --no-* style negation), including LLAMA_ARG_NO_HOST and related options.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will change from .zip archives to .tar.gz archives — deployment scripts that unzip Linux builds will need to be updated.
  466. b7366 Dec 13, 2025 · issue -247

    llama.cpp b7366 adds -mm and -mmu as short forms for --mmproj and --mmproj-url.

    └──▷ GET THIS VERSION
    $ git clone --branch b7366 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7366
    └──▷ TRY IT
    Load a multimodal projection file using the new short flag instead of the full --mmproj argument.
    $ llama-cli -mm ./mmproj-model.gguf -m ./model.gguf
    • ›Adds -mm and -mmu as short-form aliases for --mmproj and --mmproj-url respectively.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will change from .zip archives to .tar.gz archives — deployment scripts that unzip Linux builds will need to be updated.
  467. b7364 Dec 12, 2025 · issue -248

    llama.cpp b7364 adds a minimalist multi-thread progress bar and upcoming Linux archive format change.

    └──▷ GET THIS VERSION
    $ git clone --branch b7364 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7364
    • ›Adds a minimalist multi-thread progress bar to common tooling for tracking parallel operation progress.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will soon switch from .zip archives to .tar.gz archives — deployment scripts that unzip Linux builds will need to be updated.
  468. b7353 Dec 11, 2025 · issue -249

    llama.cpp b7353 enables Jinja templating by default in the CLI.

    └──▷ GET THIS VERSION
    $ git clone --branch b7353 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7353
    • ›Jinja templating is now enabled by default in the CLI (previously required explicit opt-in via common/arg.cpp).
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will move from .zip to .tar.gz archives — deployment scripts that unzip Linux builds will need to be updated.
  469. b7352 Dec 11, 2025 · issue -249

    llama-server gains INI-based per-model presets and recursive GGUF directory loading for multi-model deployments.

    └──▷ GET THIS VERSION
    $ git clone --branch b7352 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7352
    └──▷ USE IT
    Define per-model settings in the auto-generated config.ini using LLAMA_ARG_* keys under a section named after each model.
    ini 
    [my-model]
LLAMA_ARG_N_GPU_LAYERS=35
LLAMA_ARG_FLASH_ATTN=1
    • ›Adds INI-based per-model configuration presets to llama-server via config.ini, with auto-creation and template generation; keys must follow the LLAMA_ARG_* naming convention and section names starting with LLAMA_ARG_* are rejected.
    • ›Adds LLAMA_ARG_* environment variable passthrough so global settings reach child model processes without CLI duplication — children receive only minimal args (executable, model, port, alias).
    • ›Adds LLAMA_ARG_HOST and CHILD_ADDR support for controlling child process network binding in multi-model router mode.
    • ›Replaces flat --models-dir directory scan with recursive traversal using nested vendor/model layouts (e.g. vendor/model/*.gguf); model names now reflect the relative path within --models-dir rather than just the filename.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will move from .zip archives to .tar.gz archives — deployment scripts that fetch or unpack Linux release artifacts will need to be updated.
  470. b7348 Dec 10, 2025 · issue -250

    llama.cpp b7348 ships a new interactive CLI with --show-timings, --image, --audio, /read, and speculative-decoding args.

    └──▷ GET THIS VERSION
    $ git clone --branch b7348 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7348
    └──▷ TRY IT
    Run an interactive session with a multimodal model, passing an image and displaying per-completion timings — useful for auditing inference latency during model evaluation.
    $ llama-cli --model model.gguf --image screenshot.png --show-timings
    Feed a large prompt from a file inside an interactive session without pasting it manually — handy for repeatable red-team prompt injection tests.
    $ /read prompt.txt
    • ›Adds --show-timings flag to the new CLI (enabled by default) to display inference timing information after each completion.
    • ›Adds --image and --audio flags to the new CLI for passing media inputs directly on the command line.
    • ›Adds /read command in the interactive CLI session to read input from a file.
    • ›Adds arguments for speculative decoding in the new CLI experience.
    • ›Introduces a new interactive CLI experience (the old CLI is moved to llama-completion), with a loading animation, colored output (default auto), and reasoning_content support.
    +1 moreshow less
    • ›Linux releases now ship as .tar.gz archives instead of .zip.
    └──▷ BREAKING ON UPGRADE
    • !The old llama.cpp CLI binary is replaced by the new CLI; the previous CLI is now available as llama-completion — any scripts invoking the old CLI entry point will need to be updated.
    • !Linux release archives are changing from .zip to .tar.gz format — deployment scripts that unzip Linux builds will break and must be updated.
  471. b7347 Dec 10, 2025 · issue -250

    llama.cpp b7347 adds support for the Qwen3-Next-80B-A3B model architecture with correct 48-layer configuration.

    └──▷ GET THIS VERSION
    $ git clone --branch b7347 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7347
    • ›Adds support for the Qwen3-Next-80B-A3B model architecture, correctly configured with 48 layers and a new 80B-A3B type name.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will switch from .zip archives to .tar.gz archives — deployment scripts that fetch or unpack Linux builds will need to be updated.
  472. b7342 Dec 10, 2025 · issue -250

    llama.cpp b7342 adds tokenizer/parser support for Ministral, Mistral Large 3, and Devstral 2 models.

    └──▷ GET THIS VERSION
    $ git clone --branch b7342 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7342
    • ›Adds parser support for Ministral, Mistral Large 3, and Devstral 2 model families, enabling correct tokenization and prompt formatting for these models.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will switch from .zip archives to .tar.gz archives — deployment scripts that fetch or unpack Linux builds will need to be updated.
  473. b7337 Dec 9, 2025 · issue -251

    llama.cpp b7337 adds GGML_BACKTRACE_LLDB env var for lldb-based backtraces on macOS.

    └──▷ GET THIS VERSION
    $ git clone --branch b7337 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7337
    • ›Adds GGML_BACKTRACE_LLDB environment variable to enable lldb-based backtrace printing on macOS, replacing the default that could cause terminal crashes.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will switch from .zip to .tar.gz archives — deployment scripts that reference .zip paths will need to be updated.
  474. b7332 Dec 9, 2025 · issue -251

    llama.cpp b7332 adds full readline-style console editing with history navigation, word movement, and Delete key support.

    └──▷ GET THIS VERSION
    $ git clone --branch b7332 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7332
    • ›Adds KEY_CTRL_ARROW_LEFT and KEY_CTRL_ARROW_RIGHT word-jump navigation to the interactive console, parsing ANSI sequences 1;5D/1;5C on Linux and dwControlKeyState CTRL modifier on Windows.
    • ›Adds Delete key support to the console — VK_DELETE on Windows and ESC[3~ sequence on Linux — enabling forward character deletion with full UTF-8 support.
    • ›Adds arrow-left/right in-line editing to the console with UTF-8 support, plus Home/End key support on both Windows and Linux.
    • ›Adds bash-style history navigation: Up/Down arrows browse history, edits to any history line persist per-session, and pressing Enter appends the edited version as a new history entry.
    └──▷ BREAKING ON UPGRADE
    • !Linux release archives will change from .zip to .tar.gz format — deployment scripts that unzip Linux releases will need to be updated.
  475. b7331 Dec 9, 2025 · issue -251

    llama.cpp b7331 adds CANN backend support for partial RoPE and Vision mode rotation in vision-language models.

    └──▷ GET THIS VERSION
    $ git clone --branch b7331 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7331
    • ›Adds CANN backend support for partial RoPE (rope_dims < ne0), splitting tensors into rotated head and unrotated tail portions to handle modern model architectures.
    • ›Adds CANN backend support for GGML_ROPE_TYPE_VISION mode, enabling full-tensor rotation with Vision-style dimension pairing (dimension i paired with i+n_dims) for vision-language models.
    • ›CANN RoPE implementation now supports both F32 and F16 data types with intermediate F32 conversion, and handles both contiguous and non-contiguous tensor layouts.
    • ›Improves CANN cache invalidation logic to include rope_dims and indep_sects parameters.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will switch from .zip archives to .tar.gz archives — deployment scripts that unzip Linux release artifacts will need to be updated.
  476. b7329 Dec 9, 2025 · issue -251

    llama.cpp b7329 adds token matching support to llama-grammar, enabling token-level trigger patterns.

    └──▷ GET THIS VERSION
    $ git clone --branch b7329 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7329
    • ›Adds token matching support to llama-grammar, allowing grammar rules to match on individual tokens in addition to string patterns.
    • ›Refactors trigger_patterns in llama-grammar to replay tokens instead of replaying the entire string, improving pattern-matching fidelity.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will switch from .zip archives to .tar.gz archives — deployment scripts that unzip Linux builds will need to be updated.
  477. b7328 Dec 9, 2025 · issue -251

    llama.cpp b7328 adds support for the Rnj-1 model architecture.

    └──▷ GET THIS VERSION
    $ git clone --branch b7328 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7328
    • ›Adds support for the Rnj-1 model architecture (built on a refactored Gemma3 backend).
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will switch from .zip to .tar.gz archives — update any deployment scripts that reference .zip paths.
  478. b7318 Dec 8, 2025 · issue -252

    llama.cpp b7318 lets you configure cache_reuse per request on the server.

    └──▷ GET THIS VERSION
    $ git clone --branch b7318 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7318
    • ›Adds cache_reuse as a per-request configurable field on the llama.cpp server, allowing fine-grained control over KV-cache reuse on a request-by-request basis.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will move from .zip archives to .tar.gz archives; deployment scripts that fetch or unpack Linux builds must be updated.
  479. b7315 Dec 7, 2025 · issue -253

    llama.cpp b7315 adds Llama 4 scaling support for Mistral-Large (DeepSeek architecture).

    └──▷ GET THIS VERSION
    $ git clone --branch b7315 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7315
    • ›Adds Llama 4 scaling support for Mistral-Large models using the DeepSeek architecture.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will switch from .zip archives to .tar.gz archives — update any deployment scripts that reference the .zip format.
  480. b7312 Dec 7, 2025 · issue -253

    llama.cpp b7312: --color gains on/off/auto modes, defaulting to auto; Linux releases switching to .tar.gz.

    └──▷ GET THIS VERSION
    $ git clone --branch b7312 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7312
    └──▷ TRY IT
    Force colorized output on even when piping llama.cpp output to a file or downstream tool.
    $ llama-cli --color on -m model.gguf -p "Explain buffer overflows"
    • ›Changes --color flag to accept explicit values on, off, or auto (default: auto), replacing the previous boolean toggle.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will use .tar.gz archives instead of .zip — deployment scripts that unzip Linux builds must be updated.
  481. b7310 Dec 6, 2025 · issue -254

    llama.cpp b7310 adds a Vulkan perf-logger env var to control stats-dump frequency and improves fusion-op tracking.

    └──▷ GET THIS VERSION
    $ git clone --branch b7310 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7310
    • ›Adds an environment variable to control how frequently the Vulkan performance logger dumps stats (set a very large value to defer output until context destruction).
    • ›Adds fusion info strings to Vulkan perf-logger tracking, logging only one item per fused op for cleaner output.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will soon switch from .zip to .tar.gz archives — update any deployment scripts that unzip Linux builds.
  482. b7276 Dec 5, 2025 · issue -255

    llama.cpp b7276 adds CUDA/HIP GPU acceleration for CUMSUM and TRI tensor operations.

    └──▷ GET THIS VERSION
    $ git clone --branch b7276 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7276
    • ›Adds GPU-accelerated CUMSUM and TRI tensor operations for CUDA and HIP (ROCm) backends, enabling models that use these ops to run on-device without CPU fallback.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will switch from .zip archives to .tar.gz archives — deployment scripts that unzip Linux builds will break when the format change takes effect.
  483. b7275 Dec 4, 2025 · issue -256

    llama.cpp b7275 adds Metal GPU kernels for TRI, FILL, EXPM1, and SOFTPLUS ops on Apple Silicon.

    └──▷ GET THIS VERSION
    $ git clone --branch b7275 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7275
    • ›Adds Metal GPU kernel implementations for the TRI, FILL, EXPM1, and SOFTPLUS operations, enabling these ops to run natively on Apple Silicon GPUs.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will switch from .zip archives to .tar.gz archives in an upcoming release — deployment scripts that unzip Linux builds will need to be updated.
  484. b7256 Dec 3, 2025 · issue -257

    llama.cpp b7256 extends CUDA Flash Attention to Volta GPUs via generalized MMA kernel support.

    └──▷ GET THIS VERSION
    $ git clone --branch b7256 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7256
    • ›Adds Volta GPU architecture support to the CUDA Flash Attention (FA) implementation via a generalized matrix multiply-accumulate (MMA) kernel, expanding hardware compatibility for CUDA-accelerated inference.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will soon switch from .zip archives to .tar.gz archives — deployment scripts that fetch or unpack Linux release artifacts will need to be updated.
  485. b7247 Dec 3, 2025 · issue -257

    llama.cpp b7247 adds WebGPU backend support for Emscripten/Wasm builds with fast matrix multiply and Q4_0 quantization.

    └──▷ GET THIS VERSION
    $ git clone --branch b7247 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7247
    • ›Adds WebGPU backend support for Emscripten (Wasm) builds, enabling browser-side inference with fast f16/f32 matrix and matrix-vector multiplication via subgroup matrix shaders.
    • ›Supports Q4_0 quantized matrix multiply in the WebGPU backend, bringing quantized inference to Wasm targets.
    • ›Adds memory64 support via Emscripten's memory64 option to enable get_memory in Wasm builds.
    • ›Enables subgroup matrix (cooperative matrix) acceleration in the WebGPU backend, with automatic detection of available subgroup matrix configurations at runtime.
    • ›Adds toggles for subgroup matrix and f16 support specifically on Nvidia+Vulkan targets.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will switch from .zip archives to .tar.gz archives — deployment scripts that unzip Linux release artifacts will need to be updated.
  486. b7243 Dec 3, 2025 · issue -257

    llama.cpp b7243 adds --media-path for serving local media files from the built-in server.

    └──▷ GET THIS VERSION
    $ git clone --branch b7243 https://github.com/ggml-org/llama.cpp.git
# already have the repo? check out this version:
$ git checkout b7243
    └──▷ TRY IT
    Serve a local model with media files (e.g. images for multimodal prompts) available to the server without a remote fetch.
    $ llama-server --model <model.gguf> --media-path /path/to/media
    • ›Adds --media-path flag to the server subcommand to specify a local directory from which the server will serve media files.
    └──▷ BREAKING ON UPGRADE
    • !Linux releases will switch from .zip archives to .tar.gz archives — deployment scripts that unzip Linux builds will need to be updated.
my-toolchain — 0 tools
paste an install list to detect your tools

A brew list, a Brewfile, requirements.txt, a Dockerfile — or just the product names, free-form. Nothing leaves your browser.

    browse all tools →