KoboldCpp — The AI Toolchain

KoboldCpp v1.120

Sat, 29 Aug 2026 03:22:33 GMT

KoboldCpp v1.120 adds DirectIO model loading, new model support for Qwen3.8-Flash-Next and Ling-3.0-flash, and custom JS tools in Kobold Lite. • Adds `--usedirectio` flag to enable DirectIO model load mode, and allows `mlock` and `mmap` to be combined simultaneously. • Adds support for Qwen3.8-Flash-Next and Ling-3.0-flash model architectures. • Adds custom user-configurable JavaScript tools in Kobold Lite, compatible with standard tool calling.

KoboldCpp v1.119

Sun, 16 Aug 2026 13:23:10 GMT

KoboldCpp v1.119 adds video generation with Minimax H3, I2V, image editing API, and expands LoRA and attachment limits. • Adds support for the OpenAI Image editing API at `/v1/images/edits`, enabling image-to-image editing workflows through the OpenAI-compatible endpoint. • Adds video generation and image-to-video (I2V) support via Minimax H3 (requires 4 model files; a `.kcppt` quick-setup template is available on HuggingFace). • Adds support for DSpark and Dflash speculative decoding backends. • Adds support for Mistral reasoning budget control. • Adds image generation progress previews so users can monitor generation in real time. • Adds support for Muse Glimmer models, including jinja template and tool-calling support; also adds full support for Qwen 3.8. • Increases maximum runtime image LoRAs from 4 to 10. • Increases maximum images and audio attachments per request from the previous limit to 64. • Adds Autoswap support for the OpenAI image endpoint.

KoboldCpp v1.119

Sun, 16 Aug 2026 13:23:10 GMT

KoboldCpp v1.119 adds Minimax H3 video/I2V generation, Muse Glimmer support, DSpark/Dflash speculative decoding, and the OpenAI image edits endpoint. • Adds support for the OpenAI Image Editing API at `/v1/images/edits`, plus Autoswap support for the OpenAI image endpoint. • Adds video generation and image-to-video (I2V) support via Minimax H3, using a 4-file model setup (quickstart `.kcppt` template available on HuggingFace). • Adds support for DSpark and Dflash speculative decoding modes. • Adds support for Mistral reasoning budget (Mistral3 arch `[THINK]/[/THINK]` thinking format). • Increases maximum runtime image LoRAs from 4 to 10. • Increases maximum images and audio attachments supported per request to 64. • Adds image generation progress and preview support during generation. • Adds support for Muse Glimmer, including jinja templates and tool calling. • Adds full support for Qwen 3.8.

KoboldCpp v1.118.1

Sat, 01 Aug 2026 11:02:24 GMT

KoboldCpp v1.118.1 adds image-gen abort support, multi-language Qwen3TTS, `--jinjathink` flag, and int8 convrot image model loading. • New `--jinjathink` flag automatically enables Jinja templating when specified. • Image generation can now be aborted mid-generation by closing the connection; disable this behavior per-request with the `keep_image_gen_on_disconnect` request field. • Adds support for loading int8 convrot image models (z-image models are excluded from image reference support). • Adds multiple language support for Qwen3TTS voices, available via the XTTS endpoint. • Adds support for Qwen3TTS aliases for all voices. • RPC behavior updated to match upstream llama.cpp, making llama.cpp RPC server and clients interchangeable with KoboldCpp ones. Breaking changes: • Row Split has been removed; selecting it will now default to tensor split. • Rocwmma has been removed, matching upstream behavior. • RPC protocol updated to match upstream llama.cpp behavior — existing RPC setups using prior KoboldCpp RPC wire format will need to upgrade both server and client.

KoboldCpp v1.118.1

Sat, 01 Aug 2026 11:02:24 GMT

KoboldCpp v1.118.1 adds image-gen abort support, multi-language Qwen3TTS, `--jinjathink` auto-Jinja, and int8 convrot image model loading. • Adds `--jinjathink` flag that automatically enables Jinja templating when set. • Adds `keep_image_gen_on_disconnect` request field to control whether in-progress image generation is aborted when the client connection closes (abort-on-disconnect is now the default; set this field to retain the old behavior). • Adds support for loading int8 convrot image models; excludes z-image from models that support image references. • Adds multiple language support for Qwen3TTS, usable via the XTTS endpoint. • Adds qwen3tts aliases for all voices. • Increases default server thread counts, raising capacity for parallel HTTP requests. • Adds more diagnostic information when running in batched (parallel requests) mode. Breaking changes: • RPC tensor name size limit is reduced from 128 to 64 characters to match upstream llama.cpp behavior; existing RPC setups using longer tensor names will break, but llama.cpp RPC servers and clients are now interchangeable with KoboldCpp ones. • Row Split option is removed; selecting it now silently falls back to tensor split.

KoboldCpp v1.117.1

Thu, 09 Jul 2026 14:25:12 GMT

KoboldCpp v1.117.1 adds full Ollama API compatibility, inline multimodal image insertion, and a 32k default generation limit. • Adds Ollama-compatible embeddings endpoint, enabling drop-in use with any Ollama-compatible client by pointing it at KoboldCpp's port. • Adds Ollama-compatible streaming and tool-calling support — setting the port to `11434` makes KoboldCpp fully compatible with all Ollama-using software out of the box. • Multimodal images are now inserted inline in context rather than at the top, improving vision coherence and quality on most models. • Adds ability to save and display Image LoRA information when generating an image with a pre-loaded LoRA. • Merges stable-diffusion.cpp image generation updates. • Adds Group Chat rework in Kobold Lite with multi-character avatar and memory management. • Disables llama.cpp UI mic input by default (now toggled in settings). Breaking changes: • The `--splitmode row` option for CUDA is removed in the next release; users must migrate to tensor split or layer split before upgrading beyond v1.117.1.

KoboldCpp v1.116.1

Sat, 27 Jun 2026 13:56:25 GMT

KoboldCpp v1.116.1 adds Krea 2 Turbo, Ideogram 4, Boogu Edit image models, MCP server support, and new CLI flags. • Adds `--allow-config-onready` CLI flag, which re-enables `--onready` commands from config files when swapping `.kcpps` configs via API or Web UI; must be triggered directly from the terminal and cannot be saved. • Adds `--failsafe` CLI flag for macOS builds to use Accelerate only, bypassing Metal. • Adds `response_format: mp3` support to the TTS API endpoint for saving TTS outputs as MP3, following the OpenAI API spec. • Adds MCP server support to the built-in llama.cpp Web UI. • Adds Krea 2 Turbo image generation model support (recommended: 8 steps at 1.0 guidance CFG); template available as `Krea2Turbo.kcppt`. • Adds Ideogram 4 image generation model support with JSON-format prompting; template available as `Ideogram4.kcppt`. • Adds Boogu Edit image editing model support; template available as `BooguEdit.kcppt`. • Adds experimental reference audio upload support for LTX2.3 Audio-To-Video generation. • Adds Jinja toggle to the Quick Tab in the Web UI. • Increases image generation prompt length limit to 3000 characters. • Switches to llama.cpp Jinja tool call response parser as the default, improving out-of-box support for new tool calling models.

KoboldCpp v1.115.2

Fri, 12 Jun 2026 13:44:51 GMT

KoboldCpp v1.115.2 adds Anthropic `/v1/messages` API, `--sdvramlimit` for image/video VRAM capping, and MTP/Gemma4 model support. • Adds full support for the Anthropic `/v1/messages` API, including multimodal and tool calling. • New `--sdvramlimit` flag limits VRAM usage for image and video generation by splitting the compute graph, enabling any image gen model to load on constrained hardware. • New `--usemtp` flag (and GUI toggle) enables Multi-Token Prediction drafting when built into the model; alternatively, pass a MTP drafter via `--draftmodel`. • New `--reasoningeffort` flag (and GUI toggle) sets the default reasoning effort, using `--gendefaults` under the hood to apply API defaults. • Use `--debugmode` to inspect MTP drafting hitrate during inference. • Adds support for Gemma4 UV (12B) models, their vision/audio mmprojs, and Gemma assistant models. • Music generation now falls back to a loaded text LLM when no `--musicllm` is specified. • Video generation in SDUI overhauled: supports selecting a specific end frame in Img2Img, specifying both start and end frames via two reference images in txt2img, up to 32 FPS with extended limits enabled, and a lower default VAE tiling threshold of 640. • Adds new chat templates for LFM2.5 and Cohere2Moe (hotfix 1.115.2). • Launcher context size default increased to 16k; max prompt length for image generation increased. • Accepts additional SSL file extensions for HTTPS configuration.

KoboldCpp v1.114.1

Sat, 30 May 2026 09:33:33 GMT

KoboldCpp v1.114.1 adds parallel text generation, distributed RPC GPU sharing, LTX2.3 video gen, and new image models. • Adds `--parallelrequests X` flag to enable experimental continuous batching, allowing X simultaneous text generation requests instead of queuing — useful for multi-user hosting or Horde deployments. • Adds `--rpcmode` (`connect` or `host`) and `--rpctargets` flags plus `--rpcport` (default `5551`) for RPC backend support, enabling distributed inference across multiple GPUs over the network. • Adds `--jinjathink [default/true/false]` flag (and GUI dropdown) to toggle model thinking mode, which internally sets the `thinking_enabled` jinja kwarg. • Adds `chat_template_kwargs` field to the API for passing jinja kwargs objects directly via API calls. • Adds `--sdtiledvae 1` flag (hotfix) to enable VAE tiling for video generation. • Adds `--noswa` flag to disable Sliding Window Attention, which is now on by default for all models that support it. • Adds support for LTX2.3 video generation (txt2vid and img2vid) via the SDUI at `http://localhost:5001/sdui`, including AVI download with audio. • Adds support for Microsoft Lens, HiDream o1, LongCat, and Ernie image generation models. • Adds support for setting individual devices for CLIP and VAE in image generation. • Adds support for loading custom TAE (Tiny Auto Encoder) image VAEs via the VAE file selector. • Adds multithreaded MP3 generation for AceStep with a new default bitrate of 192kbps. • Adds FPS controls, increased max frame limit, and option to continue video generation from the final frame. • Adds Sampler and Scheduler settings for ComfyUI remote endpoints and A1111-compatible endpoints in Kobold Lite. • Adds BotBooru support to the Import Character from Website URL feature in Kobold Lite. Breaking changes: • The GPU ID list in the GUI launcher is now 0-based index (0,1,2,3) instead of 1-based (1,2,3,4); existing GUI configurations referencing GPU IDs by 1-based index will target the wrong device. • SWA (Sliding Window Attention) is now enabled by default on all models that support it (was off by default); use `--noswa` or uncheck the SWA checkbox in the GUI launcher to restore previous behavior.

KoboldCpp v1.113.2

Sat, 16 May 2026 13:49:11 GMT

KoboldCpp v1.113.2 adds `--splitmode`, `--reqtimeout`, image LoRA directory selection, and customizable multiuser queue limits. • Adds --splitmode (layer/row/tensor) flag to control GPU split mode independently, including experimental tensor split support (replaces the old combined `--usecuda rowsplit` syntax). • Adds `--reqtimeout` flag to specify request timeouts for router mode and similar requests. • Adds `--musiclowvram` flag to allow Ace Step VAE decoding to fall back gracefully when the larger chunk size does not fit in VRAM. • Adds runtime image LoRA directory selection, enabling all image-gen LoRAs in a chosen directory to be loaded via the `` prompt syntax. • Allows customizing the multiuser queue limit via the GUI. • Supports loading MTP models (speculative decoding pass not yet enabled). • Speeds up Ace Step VAE decoding by increasing chunk size. • Switches wav output from ulaw encoding to pcm16. • Delivers Qwen3TTS inference speedups by switching back to original precision. Breaking changes: • Split mode is no longer set as an argument to `--usecuda`; it must now be specified separately with --splitmode (layer/row/tensor). The old syntax `--usecuda rowsplit` will no longer work.

KoboldCpp v1.112.2

Mon, 20 Apr 2026 12:04:24 GMT

KoboldCpp v1.112.2 adds AceStep XL music models, reasoning effort control, SWA padding, q5_1 KV cache, and base config preloading. • Adds `reasoning_effort` API field for thinking models, accepting values `high`, `medium`, `low`, `minimal`, or `none`; also settable via `--gendefaults '{"reasoning_effort":"minimal"}'` or in KoboldAI Lite under Settings > Tokens > Thinking > Reasoning Effort. • Adds `--swapadding` parameter to extend the SWA (Sliding Window Attention) context window by a specified number of tokens while keeping KV memory footprint small. • Adds `--quantkv` flag replacing the old single-digit values for KV cache quantization; supports `f16`, `bf16`, `q8_0`, `q5_1`, and `q4_0` as named values (e.g. `--quantkv q5_1`). • Adds `--baseconfig` flag to pre-load a base config file on every model swap, merged with the target config; overridable via `baseconfig` parameter on the `/api/admin/reload_config` API endpoint. • Adds `--jinjatemplate` / `--chat-template-file` flags to replace a model's built-in Jinja template with a custom one. • Adds `--image-min-tokens` and `--image-max-tokens` flags for controlling min/max vision token counts, similar to llama.cpp behavior. • Adds support for AceStep XL music generation models (same AceStep LM, Embedder, and VAE as AceStep 1.5). • Enables streaming alongside Jinja tool calling when using `--jinjatools`. • Gemma4 E4B and E2B models now support audio inputs. • Extends `/api/extra/tokencount` to accept OpenAI-style messages as input (not only raw prompts), returning the compiled prompt. • Increases the default multiuser connection limit from 7 to 10. Breaking changes: • The old single-digit `--quantkv` values are deprecated; `--quantkv` now requires a named cache type string (`f16`, `bf16`, `q8_0`, `q5_1`, `q4_0`) — configs or scripts using numeric values must be updated.

KoboldCpp v1.111.2

Fri, 03 Apr 2026 18:13:25 GMT

KoboldCpp v1.111.2 adds Gemma 4 and Qwen3 TTS voice design, `/v1/responses` API, BF16 KV, `--autoswap`, and env-var credentials. • Adds `--useswa` flag to optionally enable Sliding Window Attention for Gemma 4, reducing VRAM usage significantly. • Adds `--jinja` flag support for Gemma 4 to enforce correct chat completions format and avoid bad outputs from wrong templates. • Adds `--jinjatools` flag to enable Jinja-based tool calling for broader model compatibility; falls back to universal tool calling when not set. • Adds `--jinja-kwargs` / `--chat-template-kwargs` flags (matching llama.cpp syntax) to pass Jinja chat template kwargs, e.g. `--chat-template-kwargs '{"enable_thinking":false}'`. • Adds `--quantkv 3` option (also selectable in the GUI launcher) to enable BF16 KV cache type. • Adds `--autoswap` flag that, in router mode, swaps loaded features (Text/Images/Music) on and off per request type to save VRAM when running multi-feature configs. • Adds `--sdmaingpu` flag allowing image generation models to be independently placed on any GPU. • Adds support for credentials supplied via environment variables `KCPP_ADMINPASSWORD` and `KCPP_PASSWORD` at launch. • Adds basic `/v1/responses` and `/v1/messages` compatibility API endpoints. • Adds `encapsulate_thinking` request field (set to `false` to disable) controlling whether detected thinking content is sent via `reasoning_content` in chat completions. • Adds Qwen3 TTS CustomVoice and VoiceDesign support, enabling narration with voice instructions in square brackets at the start of a TTS prompt (e.g. `[A depressed woman is crying] I want to go home!`). • Adds config overwriting for admin mode: two config files (base and target) can now be specified on admin API reload and KoboldCpp will merge them before switching. • Adds planner mode in Music Gen that uses the main LLM to generate better lyrics, toggled in the MusicUI advanced settings. • Adds API key support for Music Gen. • Adds ESRGAN passthrough for image gen, enabling upscale-only mode via img2img with denoise 0.0 and 1 step. • Image gen now returns metadata alongside generated images. • Doubles the logical batch size (while physical batch size is unchanged) when using pipeline parallel, improving throughput on multi-GPU setups. • Adds a popular community models section in the help button menu, driven by `.kcppt` template files. • Supports Gemma 4 models including vision, with AutoGuess non-thinking template applied by default. • TTS embedded Music UI now supports both music and TTS generation across two tabs. • Increases the max vision image limit and the GUI launcher max context size slider limit. Breaking changes: • Detected thinking content is now sent via `reasoning_content` instead of `content` in chat completions API responses. Set `encapsulate_thinking` to `false` in your request to restore the previous behavior.

KoboldCpp v1.110

Thu, 19 Mar 2026 08:02:07 GMT

KoboldCpp v1.110 adds OpenAI-compatible router mode, auto-unload timeout, Qwen3TTS 1.7B, and runtime multi-LoRA loading. • Adds `--routermode` flag to enable OpenAI-compatible Router Mode, allowing full config-reloads and automatic model hotswapping via reverse proxy within existing requests and responses; requires admin mode enabled, streaming supported. • Adds `initial_model` option to model swapping to reference the originally loaded model when using router mode. • Adds Auto Unload Timeout feature to unload all models after a specified number of seconds, designed to pair with `--routermode` for automatic reload workflows. • Adds `--ttsgpu` toggle for Qwen3TTS GPU acceleration, with Vulkan speed improvements. • Extends `--sdlora` to accept directories, enabling all image LoRAs in the directory to be loaded at runtime via `` syntax in image generation prompts. • Supports the Qwen3TTS 1.7B model for text-to-speech with improved voice quality and voice cloning. • AceStep 1.5 music generation now supports Reference Audio uploads, MP3 output, and defaults to stereo output. • Improved Ollama emulation now buffers responses for endpoints that only support streaming. • MCP now supports notifications and can handle simultaneous STDIO requests and requests with multiple parts. • OpenAI-compatible endpoints now return dynamic IDs and accurate token usage in responses. • Adds Nemotron model support and Qwen3.5 improvements via upstream merge.

KoboldCpp v1.109.2

Tue, 03 Mar 2026 11:50:34 GMT

KoboldCpp v1.109.2 adds native music generation via Ace Step and Qwen3-TTS voice cloning with new CLI flags. • Adds music generation via Ace Step with CLI flags `--musicllm`, `--musicdiffusion`, `--musicembeddings`, `--musicvae`, and `--musiclowvram`; runs in as little as 4 GB of VRAM. • Adds a music generation UI at `http://localhost:5001/musicui` when music model files are loaded. • Adds Qwen3-TTS support for high-quality voice cloning; specify a directory of `.mp3` or `.wav` voice samples with `--ttsdir` to enable voice selection during TTS narration. • Adds support for importing SillyTavern JSONL exports. • Adds ComfyUI auth token support. • `--admindir` now indexes subdirectories up to 1 level deep. • SmartCache is now automatically enabled for RNN/hybrid models (e.g., Qwen 3.5) unless fast forwarding is disabled.

KoboldCpp v1.108.2

Wed, 18 Feb 2026 09:06:03 GMT

KoboldCpp v1.108.2 adds Ollama `/api/show` emulation, image gen up to 2048x2048, multiple image LoRA support, and autofit improvements. • Adds Ollama `/api/show` endpoint emulation, extending API compatibility for tools that expect an Ollama-compatible server. • Autofit is now triggered when `-1` gpulayers (the default) is selected and no tensor splits or tensor overrides are set; setting explicit GPU layers overrides this behavior. • `--autofit` mode now hides the GUI layers selector to reduce confusion when autofit is active. • Autofit toggle is now accessible from the Quick Launch menu. • Allows the Image Gen soft limit to be overridden up to 2048x2048 resolution when the user explicitly chooses to do so. • Adds support for loading multiple image LoRAs. • Improved MCP behavior with multipart content. • Adds a toggle in SDUI to send img2img as a reference image.

KoboldCpp v1.107.3

Sat, 31 Jan 2026 04:53:16 GMT

KoboldCpp v1.107.3 adds SDXS image gen, ESRGAN upscaling, Flux Klein, and Step 3.5 Flash/Kimi Linear model support. • Adds `--nopipelineparallel` flag to disable pipeline parallel, which is now on by default in CLI. • Adds `--noflashattention` flag to disable flash attention, which is now on by default in CLI. • Adds `--downloaddir` option to specify where model downloads are stored when using URL references. • Adds `--sdvaeauto` flag to enable TAE SD support for Flux2. • Adds Vulkan (Older PC) GPU support in `oldpc` builds, providing Vulkan acceleration without AVX2 or AVX CPU intrinsics. • Adds support for SDXS, a fast Stable Diffusion image generation model capable of generating images on CPU in under 10 seconds on a Raspberry Pi, or under half a second on GPU. • Adds support for ESRGAN 4x upscaler (RealESRGAN), loadable as an upscaler model to upscale generated images. • Adds Flux Klein image model support (requires separate image model, VAE, and text encoder). • Increases image generation hard total resolution limit from 1 megapixel to 1.6 megapixels. • Adds upstream model support for Step 3.5 Flash and Kimi Linear. • Adds GLM4.7-NoThink adapter template for GLM 4.7 Flash. • Adds experimental TTS chunked streaming mode in Kobold Lite, compatible with all TTS APIs. • Adds customizable sampler presets in Kobold Lite. • Adds `error` as a possible stop reason (e.g. when the backend fails to generate). • Adds Vulkan speedup via occam's coopmat1 optimization, benefiting AMD GPUs with matrixcores such as 7000 and 9000 series. • Reworks the newbie help menu with simple starter configs. Breaking changes: • Pipeline parallel is now enabled by default in CLI; existing setups relying on it being off must explicitly add `--nopipelineparallel`. • Flash attention is now enabled by default in CLI; existing setups relying on it being off must explicitly add `--noflashattention`. • The CLBlast backend has been fully removed; users requiring CLBlast must use an earlier version.

KoboldCpp v1.106.2

Sat, 17 Jan 2026 04:33:19 GMT

KoboldCpp v1.106.2 adds MCP server/client bridge, a Kobold Lite UI overhaul, and new GPU device override controls. • Adds `--device` flag (e.g. `--device Vulkan0,Vulkan1`) to directly override selected GPU devices from the command line. • Adds `GGML_VK_VISIBLE_DEVICES` environment variable support that always overrides any Vulkan device settings configured inside KoboldCpp. • Adds MCP (Model Context Protocol) bridge: KoboldCpp can now act as a drop-in replacement for Claude Desktop, connecting to any HTTP or STDIO MCP server using a `mcp.json` config format compatible with Claude Desktop. • Supports multiple simultaneous MCP servers, automatically combining their tools and dispatching requests appropriately. • Enables the `continue_assistant_turn` API field by default in all chat completions (assistant prefill). • Increases logprobs limit from 5 to 10. • Flash Attention is now enabled by default when launching via the GUI launcher. • Increases the maximum length for interrogate image inputs. • New full settings UI overhaul in Kobold Lite (by @Rose22), with a cleaner and more organized settings menu. • Adds 4 new OLED themes to Kobold Lite. • Adds browser-side MCP client implementation to Kobold Lite.

KoboldCpp v1.105.4

Fri, 02 Jan 2026 05:21:25 GMT

KoboldCpp v1.105.4 adds `--gendefaults` for API payload overrides, a new Adaptive-P sampler, and expanded image/video model support. • Adds `--gendefaults` flag, accepting a JSON dictionary to append or overwrite any API fields (e.g. temperature, top_k, step count) on all incoming API payloads — useful for correcting misbehaving frontends; supersedes and absorbs the now-deprecated `--sdgendefaults`. • Adds support for the 'Adaptive-P' sampler, which allows selecting lower-probability tokens; configure via adaptive target and adaptive decay parameters, recommended for use alongside min-P. • Adds support for Ovis Image and Qwen Image Edit model types. • Adds TAEHV support for WAN VAE (compatible with Wan2.2 videos, Qwen Image, and Qwen Image Edit), enabling large memory savings; activate via the 'TAE SD' checkbox or `--sdvaeauto`. • Adds support for using remote HTTP MCP servers for tool calling in Kobold Lite. • Extends `--autofit` to allocate extra space when multiple models are loaded simultaneously (e.g. image generation, embeddings). • Allows configuring the number of `--smartcache` slots (added in hotfix 1.105.1) and improves snapshotting logic for RNN models. • StableUI SDUI adds support for requesting AVI-formatted video output (must be enabled in settings first). Breaking changes: • `--sdgendefaults` has been deprecated and merged into `--gendefaults`; existing configurations using `--sdgendefaults` should be migrated to `--gendefaults`.

KoboldCpp v1.104

Sat, 20 Dec 2025 09:07:05 GMT

KoboldCpp v1.104 adds `--smartcache` dynamic KV snapshotting, `--autofit` GPU layer fitting, and pipeline-parallel opt-in via `--pipelineparallel`. • Adds `--smartcache` flag: a dynamic KV state snapshot system that automatically saves and reuses KV states, accelerating context-swap scenarios (e.g. AI Horde, shared instances) and supporting RNN/Hybrid models such as Qwen3Next and RWKV; max KV states increased to 6. • Adds `--autofit` flag: enables upstream llama.cpp automatic GPU layer fitting (`-fit`), overwriting manual layer configs and tensor overrides to find a better GPU fit automatically. • Adds `--pipelineparallel` flag to opt into pipeline parallelism on multi-GPU setups (previously the default); trades higher memory usage for faster speed. • Adds `remove_limits` payload/`--sdgendefaults` override to bypass the new `cfg_scale` cap of 3.0 for Z-Image. • Adds support for GLM4.6V and GLM4.6V Flash multimodal models. • Adds support for GLM ASR Nano audio model (works best with short audio clips). • Increases default generation token amount from 768 to 896. • Adds indentation to `.kcpps` config files for improved readability. Breaking changes: • Pipeline parallelism is no longer enabled by default on multi-GPU setups; existing users relying on it must add `--pipelineparallel` to restore the previous behaviour. • `--forceversion` flag is deprecated and may no longer function as expected. • cc7.0 is removed as a CUDA build target; Volta (V100) GPUs now fall back to PTX from cc6.1, which may affect performance.

KoboldCpp v1.103

Thu, 04 Dec 2025 12:29:29 GMT

KoboldCpp v1.103 adds Flux2 and Z-Image Turbo image generation, Mistral 3 model support, and a Linux auxiliary terminal UI. • Adds prefill support to chat completions via the `/lcpp/v1/chat/completions` endpoint (distinct from the OpenAI-compatible `/v1/chat/completions`), enabling 'Assistant Continue' partial-turn continuation for frontends that support it. • Adds support for the **Flux2** image generation model, loadable in GGUF format with a separate VAE (`flux2vae.safetensors`) and a Clip 1 text encoder. • Adds support for **Z-Image Turbo** image generation, a lightweight alternative to Flux2, using `z_image_turbo-Q4_0.gguf` with the Flux1 VAE and a Qwen3-4B text encoder as Clip 1; a `.kcppt` template is available to auto-download all three components. • Adds Mistral and Ministral 3 model support, merged from upstream llama.cpp. • Adds a secondary terminal UI on Linux to view console logging, launchable from the Extras tab even when not started from CLI.

KoboldCpp v1.102.3

Wed, 26 Nov 2025 13:03:32 GMT

KoboldCpp v1.102.3 adds llama.cpp UI, jinja2 template modes, Qwen3Next support, and universal tool-calling improvements. • Adds `--jinja` flag to use jinja2 templates from GGUF in `/v1/chat/completions` for normal messages while retaining the KoboldCpp universal tool-calling module. • Adds `--jinja_tools` flag to use jinja2 templates from GGUF for all messages and tool calls in `/v1/chat/completions`, giving the model and frontend full responsibility for compatibility. • Adds `--testmemory` debug flag to inspect values retrieved by automatic GPU detection. • Adds sampler support for smoothing curve. • Bundles the llama.cpp UI as an extra interface accessible at `http://localhost:5001/lcpp`, supporting document imports (`.pdf`) alongside the existing KoboldAI Lite UI. • Massively improves universal tool calling, enabling sequential multi-tool use automatically with all tool-calling-capable frontends (OpenWebUI, SillyTavern, etc.) in chat completions mode, including on models that do not natively support tool calling. • Merges Qwen3Next model support (batch size 512 or less required). • Adds Vulkan support for Whisper audio transcription. • Syncs Image Generation to latest stable-diffusion.cpp. • Serves KoboldAI Lite UI gzip-compressed to supporting browsers for faster loading. • Adds case-insensitive filename search when selecting chat completion adapters. • Adds a Scenario Library in Kobold Lite for saving and loading custom scenarios (stories without most settings). • Enables single-turn deletion and editing in Kobold Lite classic theme instruct mode. Breaking changes: • The `mirostat` sampler had `tau` and `eta` parameters swapped due to an old bug now fixed; to reproduce previous behavior, swap your configured values for `tau` and `eta`.

KoboldCpp v1.101.1

Sat, 01 Nov 2025 09:13:15 GMT

KoboldCpp v1.101.1 adds Qwen3-VL vision support, Qwen Image/Image-Edit generation, logprobs streaming, VITS API endpoint, and Kobold Lite branching. • Adds support for multiple `--overridekv` values split by commas in a single invocation. • Renames `--blasbatchsize` to `--batchsize` (old name remains functional as an alias). • Adds aliases for OpenAI-compatible endpoints without the `/v1/` prefix. • Adds logprobs streaming for the OpenAI chat completions API (values sent at end of stream). • Adds a VITS API server compatibility endpoint. • Adds an image generation scheduler selection option in SDUI. • Supports Qwen3-VL multimodal vision-language models (load model + mmproj as usual). • Adds Qwen Image and Qwen Image Edit generation support: load the Qwen Image Edit model as the image gen model, the Qwen Image VAE as VAE, Qwen2.5-VL-7B-Instruct as Clip-1, and its mmproj as Clip-2; image editing available via `http://localhost:5001/sdui` by uploading a reference image. • Adds Horde worker job stats. • Adds branching support in Kobold Lite, allowing multiple story branches (create, delete, and swap between branches) within a single save file. • Adds ccv3 tavern card support in Kobold Lite. • Upgrades bundled ROCm library in the ROCm Linux binary to 7.1, enabling AMD Radeon 9000-series GPU support on Linux without compiling from source. • Adds experimental fractional scaling support in the GUI launcher for Wayland on GNOME. • Adds right-click copy on the SDUI image preview. • Makes GPU layer count preview in the GUI more accurate (removes the previous +2 layer offset).

KoboldCpp v1.100.1

Sun, 12 Oct 2025 02:32:02 GMT

KoboldCpp v1.100.1 adds WAN video generation, new GPU/CPU offload flags, and GLM4.6 & Granite 4 model support. • Adds `--sdoffloadcpu` flag to dynamically load/unload image generation weights to RAM when not in use (e.g. during VAE decoding). • Adds `--sdvaecpu` flag to perform VAE decoding on CPU using RAM instead of VRAM. • Adds `--sdclipcpu` flag to perform CLIP/T5 decoding on CPU (new default behavior). • Adds `--sdgendefaults` flag to set default image generation parameters. • Adds `--lowvram` flag (equivalent to `-nkvo`) to avoid offloading the KV cache, now usable in both CUDA and Vulkan (replaces the former `lowvram` CUDA-only option). • Renames `--sdclipl` to `--sdclip1` and `--sdclipg` to `--sdclip2`; these flags now apply to any vision encoder (e.g. WAN clip_vision). • Adds WAN video generation: generate short videos in KoboldCpp using WAN models, loaded via `--sdvae`, `--sdmodel`, and `--sdt5xxl`, accessible through the SDUI at `http://localhost:5001/sdui`. • Updates StableUI to support animations and video output, including Image-To-Video (I2V) via the txt2img panel. • Adds `dry_penalty_last_n` parameter to Kobold Lite, decoupling DRY range from repetition penalty range. • Adds support for GLM4.6 and Granite 4 models, including a new Granite 4 chat template. • Moves all `.embd` resource files into a separate directory and extracts image generation vocabs into their own files. • Enables building for CUDA13 in CMake (untested, no binaries provided). Breaking changes: • `--sdclipl` is renamed to `--sdclip1` and `--sdclipg` is renamed to `--sdclip2`; any launch scripts or configs using the old flag names will break. • The `lowvram` CUDA option is moved to a new `--lowvram` flag; existing setups relying on the old CUDA-specific `lowvram` option must switch to the flag.

KoboldCpp v1.99.4

Sun, 21 Sep 2025 06:20:00 GMT

KoboldCpp v1.99.4 adds llama.cpp flag aliases, per-IP rate limiting, a new audio voices endpoint, and a UI design overhaul. • Adds aliases for llama.cpp command-line flags so llama.cpp users can pass familiar arguments directly: `-m`, `-t`, `--ctx-size`, `-c`, `--gpu-layers`, `--n-gpu-layers`, `-ngl`, `--tensor-split`, `-ts`, `--main-gpu`, `-mg`, `--batch-size`, `-b`, `--threads-batch`, `--no-context-shift`, `--mlock`, `-p`, `--no-mmproj-offload`, `--model-draft`, `-md`, `--draft-max`, `--draft-n`, `--gpu-layers-draft`, `--n-gpu-layers-draft`, `-ngld`, `--flash-attn`, `-fa`, `--n-cpu-moe`, `-ncmoe`, `--override-kv`, `--override-tensor`, `-ot`, `--no-mmap`. • Adds new `--ratelimit` flag for per-IP rate limiting to help prevent abuse of public instances. • Adds `/v1/audio/voices` API endpoint for listing available TTS voices. • Renames `--promptlimit` to `--genlimit`; the renamed flag now also applies to API requests and is configurable in the UI launcher. • Adds a KoboldAI Lite UI design overhaul with a new default color scheme; the previous scheme remains available as 'Nostalgia' in advanced settings, and several new custom color schemes have been added. • Adds a popup in Kobold Lite to enable TTS or image generation when those features are available but currently disabled. • Adds Think/NoThink instruct tags to the best-practices recommendation in Kobold Lite, providing per-instruct-sequence reasoning control instead of relying on `` prefill injection. • Adds new 'Aletheia' scenario to Kobold Lite. • Smarter automatic 'Enter Sends' default detection based on platform in Kobold Lite; the toggle is now located in advanced settings. Breaking changes: • The `--promptlimit` flag has been renamed to `--genlimit`; any scripts or configs using `--promptlimit` must be updated.

KoboldCpp v1.98.1

Sun, 24 Aug 2025 07:25:36 GMT

KoboldCpp v1.98.1 adds TTS.cpp-powered text-to-speech (Kokoro, Parler, Dia), new SD flags, and `--overridenativecontext`. • Adds `--sdflashattention` and `--sdconvdirect` flags to toggle flash attention and conv2d independently for image generation. • Extends `--sdquant` to accept a parameter `[0/1/2]` for three quantization levels (adds q8 support alongside existing q4) for image generation models. • Adds `--overridenativecontext` flag to override a model's expected trained context when computing automatic RoPE scaling. • Adds `--embeddingsmaxctx` flag to override the default embedding model max context (which now defaults to the lower of current model max context and trained context). • Adds `4096` as a valid value for `--blasbatchsize`. • Adds an alias for `--tensorsplit`. • Integrates TTS.cpp into KoboldCpp, supporting three new Text-To-Speech models — Kokoro (Q4 GGUF, ~2x realtime), Parler (Q5 GGUF, ~0.5x realtime), and Dia (Q5 GGUF, ~0.1x realtime) — loaded via the 'Audio' tab; WavTokenizer is not required. • Kokoro TTS supports 28 named speaker voices including `af_alloy`, `af_bella`, `af_heart`, `am_adam`, `am_michael`, `bf_emma`, `bm_george`, and others for English speech. • Adds Seed-OSS model support with instruct templates for thinking and non-thinking modes. • Termux `android_install.sh` script can now launch existing downloaded models. • KoboldAI Lite scenario picker can now load any adventure or chat scenario in Instruct mode. • KoboldAI Lite adds a setting to embed generated TTS audio into the context as part of the AI's turn. • KoboldAI Lite adds Vision and Auditory support indicators for inline multimodal media content. • KoboldAI Lite adds Seed-OSS instruct templates (thinking regex must be set manually by changing the think tag). • KoboldAI Lite overhauls the narration and media adding system, allowing TTS to be manually added with 'Add File'.

KoboldCpp v1.97.4

Wed, 06 Aug 2025 14:10:44 GMT

KoboldCpp v1.97.4 adds GLM4.5 and GPT-OSS model support, a `--moecpu` flag for CPU-bound MoE layers, and a `/ping` endpoint for Runpod serverless. • Adds --moecpu (layercount) flag to keep MoE layers on CPU; omitting the count keeps all MoE layers on CPU, enabling large MoE models on memory-constrained GPU setups. • Adds `/ping` stub endpoint to enable KoboldCpp as a Runpod serverless worker. • Adds support for GLM4.5 family of models. • Adds support for GPT-OSS models, including a GPT-OSS Harmony instruct template in Kobold Lite. • Adds support for Voxtral audio models (Voxtral Small 24B and Voxtral Mini 3B). • Allows multiple tool calls to be chained and triggered by any role. • Adds two additional save slots in Kobold Lite. • Adds a (+/-) modifier field for Adventure mode rolls in Kobold Lite. • Adds a button to insert a textDB separator in Kobold Lite. • Adds clearer per-modality indication of Vision/Audio multimodal support. • Increases max length of terminal prints in debug mode. • Disables context shifting for all mrope models.

KoboldCpp v1.96.2

Thu, 17 Jul 2025 16:58:33 GMT

KoboldCpp v1.96.2 adds audio input support, OpenAI image-generation endpoint emulation, and raises default context to 8k. • Renames `--usecublas` flag to `--usecuda` (old name still accepted for backwards compatibility). • Emulates the OpenAI `/v1/images/generations` endpoint, enabling image-generation API calls against KoboldCpp. • Adds audio input support for multimodal models, enabling `.wav`, `.mp3`, and `.flac` files on all audio endpoints (Whisper transcribe and multimodal audio) via the miniaudio library. • Adds support for Qwen 2.5 Omni 3B as the first audio-capable multimodal model (load base model + mmproj, same workflow as vision models). • Raises default context size to 8k (up from 4k); existing `.kcpps` config files are unaffected. • Adds AutoGuess prompt templates for Kimi K2, Jamba, and Dots models. • Adds ExaOne 4 model support (via hotfix 1.96.1). • Automatically resumes incomplete model downloads when aria2c is used. • Applies nsigma masking to sampling. • Allows flash attention to be used with image generation. • Prints system information to terminal on startup to aid debugging. • Adds microphone audio capture in Kobold Lite UI for embedding audio directly into stories. • Adds lamejs MP3 encoder to Kobold Lite for audio compression and allows uploading audio files embedded into saved stories. • Adds experimental flags in Kobold Lite to control audio compression, autoguess tags, and unsaved file warnings. • Allows connecting to OpenAI endpoints without an API key in Kobold Lite. • Adds a shortcut in Kobold Lite for inserting instructions into memory. Breaking changes: • Attached image and audio data in Kobold Lite save files is no longer stored inline in the story but as metadata — saves created in v1.96.2 that contain new media will not have that media accessible when re-opened in older versions of the UI.

KoboldCpp v1.95.1

Sun, 29 Jun 2025 13:59:50 GMT

KoboldCpp v1.95.1 adds Flux Kontext image editing, multi-reference Photomaker, and Gemma3n/ERNIE model support. • Adds support for Flux Kontext, a natural-language image editing model (background replacement, text editing, object insertion) loadable via a `.kcppt` template; accessible through StableUI at `http://localhost:5001/sdui` with prompt and reference image inputs. • Photomaker now accepts up to 4 reference images, matching the multi-image input capability added for Flux Kontext. • Adds AutoGuess template support for Gemma3n (text-only) and ERNIE models. • Further grammar sampling speedups via caching.

KoboldCpp v1.94.2

Sat, 21 Jun 2025 09:45:45 GMT

KoboldCpp v1.94.2 adds Chroma image generation, PhotoMaker face cloning, and several new CLI flags for GPU and embedding control. • Adds `--sdphotomaker` flag to load PhotoMaker for face cloning alongside any SDXL-based model, available in text2img, img2img, and inpaint modes via the SDUI at `http://localhost:5001/sdui`. • Adds `--sdclampedsoft` flag for soft total resolution clamping (e.g. a value of 640 allows 640x640, 512x768, and 768x512 images), combinable with the existing `--sdclamped` hard-clamp flag. • Adds `--sdtiledvae` flag to specify a resolution threshold beyond which VAE tiling is applied, replacing the previous `--sdnotile`. • Adds `--embeddingsmaxctx` flag to cap the max context length for embedding models, reducing memory usage. • Adds `--embeddingsgpu` flag to offload embedding model layers to GPU. • Adds `--maingpu` flag to select which GPU is treated as the main GPU in multi-GPU setups. • Adds Chroma image generation support — a new architecture based on Flux Schnell requiring a T5-XXL encoder and Flux VAE loaded alongside the Chroma GGUF model. • ComfyUI emulation now covers the `/upload/image` endpoint, enabling img2img ComfyUI workflows with files stored temporarily in memory. • Adds a mini 5 MB PyInstaller launcher generated alongside an unpacked KoboldCpp directory, allowing launch without Python or other dependencies installed. • Kobold Lite gains Word Frequency Search, webcam image import, WebSearch for corpo mode, ComfyUI img2img support, custom OpenAI endpoint for TextDB embedding model, and the ability to load usermods and CSS from file. • Improves GNBF grammar performance by attempting culled grammar search first. • Displays available RAM on startup and shows the version number in the terminal window title. • Adds more performance stats for token speeds and timings. Breaking changes: • `--sdnotile` is replaced by `--sdtiledvae`; setups using `--sdnotile` must switch to the new flag.

KoboldCpp v1.93.2

Sat, 07 Jun 2025 03:36:47 GMT

KoboldCpp v1.93.2 adds Windows Shell integration for .gguf files, in-memory save/load session states, model unloading via admin API, and a new --embeddingsmaxctx flag. • Adds `--embeddingsmaxctx` CLI option to control the maximum context size for the embeddings endpoint. • Adds `--singleinstance` flag awareness for Windows Shell integration: only instances started with this flag are replaced when a `.gguf` file is double-clicked to open in KoboldCpp. • Adds `add_sd_step_limit` adapter flag to cap the maximum number of steps in image generation. • Adds `/api/extra/shutdown` API endpoint (localhost-only) to power Windows Shell integration, allowing a running instance to be replaced when a new `.gguf` is launched. • Adds Windows Shell integration allowing `.gguf` files to be associated with KoboldCpp; double-clicking a `.gguf` opens it directly, replacing any existing local instance on the same port. Installable/uninstallable from the 'Extras' tab. • Adds Save and Load States (session snapshots) via the admin API, storing context snapshots entirely in memory across 3 available slots (4 including the current session), enabling seamless swap between chats with no reprocessing. • Adds model unload option to the admin API, freeing memory while keeping the server running so a different model can be loaded via the admin panel in Lite. • Raises the maximum allowed temperature for Function/Tool calling to 1.0. • Adds more Ollama compatibility endpoints. • Adds support for embeddings models in Kobold Lite's TextDB. • Adds 'Smart' Image Autogeneration mode in Kobold Lite, letting the AI decide when to generate images and create image prompts automatically. • Adds support for saving and loading world info files independently in Kobold Lite. • Adds support for importing character cards from character-tavern.com in Kobold Lite. • Adds a toggle to make a usermod permanent in Kobold Lite. • Adds support for welcome messages in corpo mode in Kobold Lite. • Adds a text LoRA scale option (the previously available text LoRA base option is removed; if provided it will be silently ignored). Breaking changes: • Linux binary `koboldcpp-linux-x64-cuda1210` is renamed to `koboldcpp-linux-x64`; automated scripts using the old name will break when the old filename is removed. • Linux binary `koboldcpp-linux-x64-cuda1150` is renamed to `koboldcpp-linux-x64-oldpc`; automated scripts using the old name will break when the old filename is removed. • Windows binary `koboldcpp_cu12.exe` is renamed to `koboldcpp.exe`; automated scripts using the old name will break when the old filename is removed. • Windows binary `koboldcpp_oldcpu.exe` is renamed to `koboldcpp-oldpc.exe`; automated scripts using the old name will break when the old filename is removed. • Windows binary `koboldcpp_nocuda.exe` is renamed to `koboldcpp-nocuda.exe`; automated scripts using the old name will break when the old filename is removed. • The text LoRA base option has been removed; any scripts or configs passing it will have the value silently ignored rather than applied.

KoboldCpp v1.92.1

Sat, 24 May 2025 08:41:24 GMT

KoboldCpp v1.92.1 adds SWA KV-cache mode, DDIM image sampler, Llama4 vision, and broad Kobold Lite enhancements. • Adds `--useswa` flag to enable Sliding Window Attention (SWA) mode, which significantly reduces KV cache memory usage — note: incompatible with ContextShifting and may degrade output with FastForwarding. • Disables `--showgui` automatically when `--skiplauncher` is used. • Adds DDIM sampler for image generation. • Merges Vision support for Llama4 models. • Adds integrated dPaste.org (open-source pastebin) support in Kobold Lite for sharing save files as a single URL via `Save/Load > Share > Export Share as Web URL`; self-hosted instances supported by changing the endpoint URL. • Adds support for RisuAI V3 character cards (`.charx` archive format) in Kobold Lite. • Adds TTS option via Pollinations API (routing through OpenAI TTS models) in Kobold Lite. • Adds ST-based randomizer macros such as `{{roll:3d6}}` in Kobold Lite. • SSE streaming is now the default for all cases in Kobold Lite, with an opt-out in Advanced Settings. • Enables threadpools, delivering a speedup for Qwen3MoE inference. • Greatly improved tool calling by enforcing grammar on output field names and performing automatic tool selection as a separate pass. • Adds model size information display in the HF Huggingface Search and download menu. • Adds a simple optional Python requirements install script in `launch.cmd` for launching from unpacked directories. • Adds an option for horizontal stacking of multiple images in one row in Kobold Lite. • Adds a new Immortal sampler preset in Kobold Lite. • Adds a debug option to change the connected API at runtime in Kobold Lite. • In polled streaming mode, Kobold Lite can now fetch the last generated text if a request fails halfway. • `clip_skip` value is now stored inside image metadata; actual random seed number is also displayed. • AI Horde default advertised context now matches the main max context by default.

KoboldCpp v1.91

Sat, 10 May 2025 15:15:02 GMT

KoboldCpp v1.91 adds a Hugging Face model browser, embedded aria2c downloader, CFG support via `--enableguidance`, and expanded Corpo mode. • Adds `--enableguidance` flag (or `Enable Guidance` checkbox in UI) to enable Classifier-Free Guidance (CFG); configure a negative prompt and CFG scale via the lite tokens menu — note CFG doubles KV usage and halves generation speed. • New Hugging Face Model Search Tool lets users search, browse, and download GGUF models directly from Hugging Face within KoboldCpp before launch. • Embeds aria2c downloader in Windows builds for high-speed model downloads when using provided URLs. • Adds CUDA compute capability 3.5 target, potentially enabling GPU acceleration on K6000, GTX 780, and K80 hardware. • Removes flash attention limits and warnings for Vulkan backends. • Improved ComfyUI emulation now adapts to any workflow containing a KSampler node connected to a text prompt. • Corpo mode in Kobold Lite now supports Text mode and Adventure mode in addition to existing modes, making it available across all 4 modes. • Adds quick save and delete buttons for Corpo mode in Kobold Lite. • Adds Pollinations.ai as an optional online service for TTS and Image Generation in Kobold Lite. • Adds a new built-in scenario: Nemesis, in Kobold Lite. Breaking changes: • `KoboldCppAuto` replaces the previous default instruct preset in Kobold Lite; existing users relying on the old default instruct format may see different instruct tags applied automatically — switch manually to a preferred format (e.g. Alpaca) if needed. • Chat 'match any name' is no longer enabled by default in Kobold Lite.

KoboldCpp v1.90.2

Tue, 29 Apr 2025 15:28:20 GMT

KoboldCpp v1.90.2 adds Android Termux auto-install, HuggingFace model search, Pixtral vision support, and OpenAI Structured Outputs in the chat completions API. • Adds `--mmprojcpu` flag to load and run the multimodal projector on CPU while keeping the main model on GPU. • Adds `--blasbatchsize -1` mode that exclusively uses a batch size of 1 when processing prompts; also formally permits `--blasbatchsize 16` to replicate the old non-GEMM batch-of-16 behavior. • Adds OpenAI Structured Outputs support in the chat completions API, including accepting a JSON schema sent as a stringified JSON object in the `grammar` field, enabling enforced structured JSON outputs. • Adds Android Termux auto-installer: a single command installs, downloads, compiles, and configures KoboldCpp with a Gemma3-1B model on Android via Termux (from F-Droid). • Adds a HuggingFace model search tool allowing users to find, browse, and download models directly from within KoboldCpp. • Adds Qwen3 model support, including automatic `--nobostoken` triggering when model metadata explicitly indicates no BOS token. • Adds functioning Pixtral vision model support; note Pixtral is token-heavy (~4000 tokens per 1024px image); `--contextsize` or `--visionmaxres` can be tuned accordingly. • Merged overhaul to Qwen2.5VL projector, supporting both HimariO and ngxson multimodal projector versions with backwards compatibility. • Adds automatic handling of multipart GGUF file downloading, supporting up to 9 parts. • Adds ComfyUI compatibility improvements via rudimentary WebSocket spoof. • Improved auto GPU layer assignment when loading multi-part GGUF models on a single GPU, with tightened memory estimation and quantized KV cache accounting. • Kobold Lite adds a toggle to disable LaTeX rendering while keeping Markdown enabled, and adds ChatGLM-4 and Qwen3 (ChatML think/nothinking) presets. Breaking changes: • `--onready` shell commands can no longer be embedded into a `.kcppt` or `.kcpps` file; they remain available only as a CLI parameter. • ChatML (No Thinking) preset is removed from Kobold Lite; thinking control is now handled globally via Settings > Tokens > CoT.

KoboldCpp v1.89

Mon, 21 Apr 2025 03:15:23 GMT

KoboldCpp v1.89 adds NoScript chat/image gen, new `--overridekv`/`--overridetensors` flags, and Vulkan coopmat2 support. • Adds `--overridekv` launcher flag to overwrite a single model metadata property at runtime; input format is `keyname=type:value`. • Adds `--overridetensors` launcher flag to place tensors matching a pattern onto a specific backend; input format is `tensornamepattern=buffertype`. • Enables Vulkan coopmat2 (CM2) support for Nvidia GPUs with Game Ready Driver 576.02 or later, adding flash attention and overall speed improvements; OldCPU Vulkan binaries now exclude coopmat, coopmat2, and DP4A. • Improved NoScript mode at `/noscript` now supports chat mode and image generation without JavaScript, compatible with browsers as old as Internet Explorer 5 and Netscape Navigator 4. • Enables Image Generation LoRAs for use with quantized diffusion models (LoRA itself should remain unquantized). • Displays available GPU memory when estimating layer counts. • Makes YAD the default filepicker, replacing Zenity; Legacy TK filepicker remains available in the extras page. • Increases Kobold Lite save slots to 10 local and 10 remote. • Relocates Tokens Tab and WebSearch Tab into the Settings Panel; regex and token sequence configs now persist in settings rather than per-story. • Adds Retain History toggle in Kobold Lite WebSearch to carry over prior search results into subsequent queries. • Adds an editable Template for the Kobold Lite character creator. • Reworks thinking-tag handling in Kobold Lite, separating display and submit regex behaviors across three modes each. Breaking changes: • Kobold Lite regex and token sequence configs are now stored in settings rather than in the story, so existing per-story values will not automatically migrate.

KoboldCpp v1.88

Sun, 13 Apr 2025 14:42:01 GMT

KoboldCpp v1.88 adds image inpainting, a JSON-to-GBNF grammar endpoint, `--maxrequestsize` flag, and Llama 4 model support. • Adds `--maxrequestsize` flag to configure the server's maximum HTTP request payload size before dropping a request (default: 32 MB). • Adds new API endpoint `POST /api/extra/json_to_grammar` to convert a JSON schema into GBNF grammar. • Adds Image Inpainting support to StableUI, including a masking UI for Img2Img editing (similar to A1111) with updated API docs. • Adds a clip-skip slider to StableUI. • Adds Zenity and YAD support for native file picker dialogs on Linux; falls back to the previous TKinter picker via 'Use Classic FilePicker' in the extras tab. • Adds GPU memory estimation via `vulkaninfo` when `nvidia-smi` is unavailable. • Merges Llama 4 support from upstream llama.cpp, with Qwen3 also included. • Adds Llama 4 prompt format to Kobold Lite. • Adds warnings in GUI and terminal when FlashAttention is used with the Vulkan backend due to performance concerns.

KoboldCpp v1.87.4

Tue, 01 Apr 2025 14:36:16 GMT

KoboldCpp v1.87.4 adds embeddings endpoint, voice cloning, Qwen2.5VL, auto tool calling, and a CLI chat mode. • Adds `--embeddingsmodel` flag to load GGUF embedding models, exposed via `/v1/embeddings` and `/api/extra/embeddings` for text encoding into vector databases. • Adds `--cli` flag to run KoboldCpp in a fully headless terminal chat mode, with no GUI required. • Adds `--quantkv` support without flash attention — when used without it, only quantized-K is applied (quantized-V is skipped). • Adds OuteTTS voice cloning support: Speaker JSON files representing a cloned voice can now be uploaded via the TTS API. • Adds automatic (auto) mode for function/tool calling, allowing the model to decide whether and which tool to invoke; the detection template is customizable via `custom_tools_prompt` in the chat completions adapter. • Merges Qwen2.5VL vision-language model support, with GGUF weights and mmproj projectors available for 7B and 32B variants. • Adds World Info Groups in Kobold Lite UI: categorize world info entries by group, toggle groups on/off with a single click, and import/export each group as JSON. • Adds a menu in Kobold Lite to upload a cloned speaker JSON for OuteTTS voice cloning. • Adds a toggle in Kobold Lite to allow uploading images as a new turn in a conversation. • Increases maximum resolution of uploaded images used with vision models in Kobold Lite. • Adds localtunnel as a fallback tunneling option in the Colab environment when Cloudflare tunnels are blocked.

KoboldCpp v1.86.2

Fri, 14 Mar 2025 13:58:30 GMT

KoboldCpp v1.86.2 adds Gemma3 vision support, quantized KV with context shift, and new CLI flags `--defaultgenamt` and `--nobostoken`. • Adds `--defaultgenamt` option to control the maximum number of tokens generated by default when a third-party client (e.g. via chat completions API) does not specify a value. • Adds `--nobostoken` option to prevent BOS tokens from being automatically prepended at generation start. • Allows `--quantkv` (quantized KV cache) to be used together with context shift — the only remaining requirement is flash attention enabled. • Exposes speculative decoding success rate information in the `/api/extra/perf/` endpoint. • Adds instruct preset `KoboldCppAutomatic` in Kobold Lite, which automatically retrieves the instruct template from KoboldCpp. • Integrates Gemma3 support including vision (mmproj) — load a Gemma3 GGUF alongside its mmproj file; inline images work via the Chat Completions API (e.g. in SillyTavern). • Allows admin mode to runtime-swap between GGUF model files in addition to swapping between `.kcpps` configs, with automatic GPU layer selection. • Adds support for downloading Image Generation LoRAs from URL via launch arguments. • Embeds Image Generation parameter metadata into generated images. • Re-enables CUDA compute capability 3.7 (K80) support. • Adds option to save stories to Google Drive when running in Colab.

KoboldCpp v1.85.1

Sat, 01 Mar 2025 09:46:36 GMT

KoboldCpp v1.85.1 adds networked save slots, Top-N Sigma sampling, and CLI config export flags. • Adds `--savedatafile ` flag to enable server-sided networked save slots, allowing persistent story save/load over the network from any browser or device; combine with `--password` to require an API key for save/load access. • Adds `--exportconfig` and `--exporttemplate` CLI flags to export any set of launch arguments as a `.kcpps` or `.kcppt` config file from the command line, usable for model switching in admin mode. • Adds Top-N Sigma sampler (combinable with Top-K, Temperature, and XTC only) with UI support in Kobold Lite. • Adds Side Panel Mode for KoboldAI Lite. • Adds improved thinking support in Kobold Lite, including display, forced injection of `` tokens in AI replies, and filtering of old thoughts in subsequent generations. • Adds customization options for assistant jailbreak prompt in Kobold Lite. • Reworks load/save UI in Kobold Lite with 2 extra local slots and 8 extra remote save slots. • Adds Granite model support and Vulkan/CUDA enhancements via upstream merge.

KoboldCpp v1.84.2

Sat, 15 Feb 2025 16:03:48 GMT

KoboldCpp v1.84.2 adds aria2c/wget model downloading and multi-URL multipart model loading via CLI. • Supports specifying multiple download URLs for multipart models via `--model [url1] [url2]...` (CLI only), enabling KoboldCpp to download multi-file models directly. • Adds support for using `aria2c` and `wget` as model download backends when detected on the system. • Adds automatic config rollback in admin mode when switching to a faulty config fails, restoring the last known-good state.

KoboldCpp v1.83.1

Sat, 08 Feb 2025 02:12:00 GMT

KoboldCpp v1.83.1 adds runtime model/config swapping via `--admin`, new vision and TTS limits, and multi-pass web search in Lite. • Adds `--admin` flag to enable runtime model, settings, and config switching (including remote model swapping) via a new Admin panel in Kobold Lite. • Adds `--admindir` flag pointing to a directory of `.kcpps` launch configs, allowing hot-swap between different models, layers, and backends without restarting manually. • Adds `--adminpassword` flag to password-protect the admin panel and remote model-swap functions. • Adds `--visionmaxres` flag to cap the maximum resolution accepted by vision mmprojs; images exceeding the limit are automatically downscaled before processing. • Adds `--ttsmaxlen` flag to set a token limit (range 512–4096) on TTS generation; approximately 75 tokens per second of audio. • Adds new CLBlast backend options for avx2, avx, and noavx (Regular, OldCPU, OlderCPU) to provide GPU-accelerated alternatives across a wider range of CPU generations. • Kobold Lite now supports individual start and end instruct tags independently, toggled via Settings > Toggle End Tags. • Kobold Lite adds a deepseek instruct template and reasoning/thinking template tag support, configurable under Context > Tokens > Thinking. • Kobold Lite adds multi-pass web search with a configurable query-generation template. • CLIP vision embeddings are now reused across multiple requests when images have not changed, reducing redundant reprocessing. • TTS audio output can now be downloaded as a file from the Lite test interface instead of only playing back in-browser. • Adds cloudflared tunnel download support for aarch64, and allows SSL combined with remote tunnels. Breaking changes: • Chat completions adapter now defaults to AutoGuess instead of Alpaca; existing setups relying on the implicit Alpaca default must explicitly set the adapter to 'Alpaca'.

KoboldCpp v1.82.4

Sat, 18 Jan 2025 08:20:45 GMT

KoboldCpp v1.82.4 adds OuteTTS text-to-speech with OpenAI Speech/XTTS API compatibility, a GGUF file analyzer, and multilingual Whisper support. • Adds OuteTTS text-to-speech via `--ttsmodel` (OuteTTS GGUF) and `--ttswavtokenizer` (WavTokenizer GGUF) flags; `--ttsgpu` offloads models to GPU and `--ttsthreads` sets a custom thread count. • Adds `--analyze` CLI flag (also in the GUI Extras tab) to inspect any GGUF file, displaying metadata, tensor names, dimensions, and types. • Adds `--sdnotile` flag to disable VAE tiling for image generation, eliminating bleeding graphical artifacts on some GPUs. • Adds `--usemmap` flag to opt in to memory-mapped loading; `mmap` is no longer enabled by default. • OuteTTS integration exposes OpenAI Speech API (`OpenAiSpeechApi`) and XTTS API (`XttsApi`) compatibility endpoints for hooking KoboldCpp TTS into existing TTS frontends. • OuteTTS New Speaker Synthesis lets you generate unique voices by entering a random name; supports v0.2 and v0.3 models (500M and 1B). • Extends TAESD (compressed to fp8, ~3 MB) to SD3 and Flux via `--sdvaeauto` or 'AutoFix VAE' in the GUI. • Increases max supported images per API request for Multimodal Vision from the previous limit to 8. • Enables multilingual Whisper (Voice Recognition) support with specific language codes, selectable via a 2-character code (e.g. `ja`, `fr`) in Kobold Lite. • KoboldCpp now displays all enabled capabilities and API endpoints on launch, listing modules such as `TextToSpeech`, `VoiceRecognition`, `MultimodalVision` and APIs such as `OpenAiSpeechApi`, `XttsApi`, `WhisperTranscribeApi`. • CLBlast (Older CPU) build no longer requires AVX, extending GPU-assisted inference to very old or low-cost systems. Breaking changes: • `mmap` is no longer enabled by default; existing setups relying on memory-mapped model loading must now explicitly pass `--usemmap` or enable it in the GUI.

KoboldCpp v1.81.1

Sat, 04 Jan 2025 14:19:53 GMT

KoboldCpp v1.81.1 adds local WebSearch augmentation via DuckDuckGo, a heuristic chat-template guesser, and browser-based RAG via TextDB. • Adds `--websearch` CLI flag (and GUI toggle) to enable KoboldCpp as a local WebSearch proxy, powered by DuckDuckGo, augmenting model queries with live web results. • Adds `/api/extra/websearch` endpoint for submitting web-search-augmented queries directly to the running KoboldCpp instance. • Adds TextDB Document Lookup in Kobold Lite (Context > TextDB tab), a browser-based RAG layer using lunr/minisearch to chunk, store, and retrieve relevant snippets from a pasted text document at inference time — no embedding model required. • Switches Kobold Lite autosaves and save slots from localStorage to indexedDb, significantly increasing the maximum supported browser save size; existing localStorage data is auto-migrated on first launch. • Expands supported resolutions and aspect ratios for generated and uploaded images, and improves multimodal image handling quality for larger, more detailed images. Breaking changes: • Windows builds now require explicit target flags (e.g. `make LLAMA_PORTABLE=1 LLAMA_VULKAN=1 LLAMA_CLBLAST=1`) where previously a bare `make` was sufficient — existing Windows build scripts will break without updating. • Kobold Lite save data written to indexedDb in v1.81.1 cannot be read by older versions of KoboldAI Lite.

KoboldCpp v1.80.3

Fri, 20 Dec 2024 05:21:01 GMT

KoboldCpp v1.80.3 adds Qwen2-VL multimodal support, Vulkan coopmat1 acceleration, new CLI flags, and LaTeX rendering in Kobold Lite. • Adds `--moeexperts` flag to overwrite the number of experts used in MoE models. • Adds `--failsafe` flag to explicitly disable all CPU intrinsics and GPU usage. • Adds `--draftgpulayers` flag to set the number of layers offloaded for a speculative decoding draft model. • Adds `--draftgpusplit` flag to set GPU layer distribution ratio for the draft model across multiple GPUs (defaults to same as main model). • Adds `suppress_non_speech` field to the Whisper API, enabling suppression of noise annotation logits (e.g. barking, doorbell, chime, muzak). • Adds `q` as a URL query alias for `query` in Kobold Lite (e.g. `http://localhost:5001?q=what+is+love`). • MMQ is now the default for CLI; use the `nommq` flag to disable (e.g. `--usecublas all nommq`). • Adds multimodal image support for Qwen2-VL (2B and 7B) via mmproj; works on CUDA and CPU backends. • Vulkan backend gains coopmat1 support, delivering significantly faster inference on modern Nvidia cards. • Restores support for ARM quants (e.g. `Q4_0_4_4`) in KoboldCpp. • Adds LaTeX rendering in Kobold Lite using standard `\[...\]`, $...$, and `$$...$$` syntax alongside markdown. • Adds support for AllTalk v2 API in Kobold Lite (AllTalk v1 remains supported automatically). • Adds support for Mantella XTTS (XTTS fork) in Kobold Lite. • Kobold Lite gains manual audio file upload for transcription via settings. • Handles additional generation parameters in Ollama API emulation. • Default image generation sampler changed to Euler. Breaking changes: • `trim_stop` now defaults to `true`; setups that expected stop sequences to appear in output will no longer receive them without explicitly setting `trim_stop` to `false`. • MMQ is now the default for CLI; existing CLI invocations relying on non-MMQ behavior must add `nommq` (e.g. `--usecublas all nommq`) to preserve prior behavior.

KoboldCpp v1.79.1

Sat, 30 Nov 2024 12:59:49 GMT

KoboldCpp v1.79.1 adds multiplayer sessions, Ollama/ComfyUI API emulation, speculative decoding, and new tokenizer endpoints. • Adds `--multiplayer` flag (also available in the GUI launcher Network tab) to enable real-time multiplayer sessions, exposing `/api/extra/multiplayer/status`, `/api/extra/multiplayer/getstory`, and `/api/extra/multiplayer/setstory` endpoints for live story sync across multiple users and devices. • Adds Ollama-compatible endpoints `/api/chat` and `/api/generate` for basic Ollama API emulation, letting third-party tools that only speak Ollama point to KoboldCpp at `http://localhost:5001` (or port `11434` for stricter tools); streaming is not supported. • Adds ComfyUI emulation via a new `/prompt` endpoint, supporting `txt2img` for tools that require ComfyUI API but lack A1111 API support. • Adds `/api/extra/detokenize` endpoint, which converts an array of token IDs into a detokenized string. • Adds `/props` endpoint exposing instruction and chat template data from the loaded model. • Adds speculative decoding (drafting) support via a second lightweight draft model with the same vocabulary, offering inference speedups at low temperatures; not well supported on Vulkan. • Adds a 'Dice Action' toggle in Kobold Lite Adventure mode, allowing the AI to roll a dice to determine action outcomes. • Adds ability to disable sentence trimming in all Kobold Lite modes. • Adds GGUF format text encoder support (clip/t5) for Flux and SD3.5 image models. • Adds chunked encoding support.

KoboldCpp v1.78

Sat, 16 Nov 2024 02:15:25 GMT

KoboldCpp v1.78 adds Flux and SD 3.5 image generation, new `--nofastforward` flag, and a `/.well-known/serviceinfo` identity provider endpoint. • Adds `--sdt5xxl`, `--sdclipl`, and `--sdclipg` CLI flags to load individual T5XXL, CLIP-L, and CLIP-G components for Flux and SD 3.5 image generation (supports URL resources). • Adds `--nofastforward` flag to force full prompt reprocessing on every request, enabling more repeatable and consistent inference results. • Adds `/.well-known/serviceinfo` common identity provider endpoint, aligning with AI-Horde, Aphrodite, and TabbyAPI interoperability standards. • Supports Flux and Stable Diffusion 3.5 model architectures for image generation, accepting fp16 or fp8 safetensor models and GGUF models, including all-in-one bundled models. • Adds VAE tiling for more consistent large image handling, enabling stable 1024x1024 generation with SDXL and Flux models. • Adds support for custom CSS styles in Kobold Lite UI. • Adds 'BigSquare' option in Kobold Lite image gen settings to generate larger images. • Adds Clip Skip support for local image generation in Kobold Lite. • Debug mode now prints XTC penalties for inspection during inference. Breaking changes: • CLBlast GPU offload now requires a duplicate copy of model layers in RAM (piggybacks off the CPU backend), increasing memory usage for existing CLBlast setups.

KoboldCpp v1.77

Fri, 01 Nov 2024 16:32:57 GMT

KoboldCpp v1.77 adds token probability (logprobs) API endpoints, ComfyUI image-gen backend support, and in-chat image generation from instruct mode. • Adds logprobs support over the sync API and a new dedicated `/api/extra/last_logprobs` endpoint for retrieving token probabilities after both streaming and non-streaming responses. • Response fields `prompt_tokens`, `completion_tokens`, and `total_tokens` now return accurate counts instead of placeholder values. • Adds experimental ComfyUI image generation backend support in KoboldAI Lite — requires ComfyUI launched with `--listen --enable-cors-header '*'`; no workflow customization needed. • Enables image generation from instruct mode via natural language (e.g. 'Please generate an image of a bag of sand'); on by default when an image model is loaded, regex-driven, and toggleable in settings. • Adds a Token Probability Viewer in KoboldAI Lite (disabled by default, enable in advanced settings) showing a table of alternative token probabilities per response. • Adds a new 'Interactive Storywriter' scenario in KoboldAI Lite — story-writing mode with support for hidden instruction prompts to steer the narrative. • Adds support for Tavern 'V3' (Risu-augmented V2) character cards in KoboldAI Lite; metadata chunks are ignored but base V2 card functionality is fully operational. • Enables CUDA graphs for the `koboldcpp_cu12.exe` (CUDA 12) build, improving performance on supported Nvidia GPUs. • Clarifies the A1111/Forge/KoboldCpp image generation backend selector in KoboldAI Lite to reflect compatibility with all three backends.

KoboldCpp v1.76

Fri, 11 Oct 2024 13:06:29 GMT

KoboldCpp v1.76 adds Anti-Slop phrase banning via `banned_strings` and a new Roleplay Character Creator scenario in Kobold Lite. • Adds Anti-Slop Sampling (Phrase Banning) via the `banned_strings` field: up to 48 case-insensitive words or phrases can be blocked by backtracking and regenerating when they appear in output. • The `/api/extra/perf/` endpoint now returns whether the instance was launched in quiet mode (terminal output suppression). • Adds timestamp information to each request start event. • Adds a new Roleplay Character Creator scenario in Kobold Lite: a wizard-driven UI for building roleplay bots with the Aesthetic UI, with support for loading pre-existing Tavern Character Cards.

KoboldCpp v1.75.2

Sat, 21 Sep 2024 08:01:29 GMT

KoboldCpp v1.75.2 auto-selects GPU backends by default, adds RWKV v6 support, new `--showgui` flag, and OpenAI-compatible TTS in Kobold Lite. • GPU backend is now auto-selected by default when no `--use...` flag is specified; override with `--usecpu`, `--usevulkan`, or `--usecublas`. Dragging a GGUF model onto `koboldcpp.exe` now launches with GPU and gpulayers auto-configured. • New `--showgui` flag allows the GUI to be displayed even when command-line flags are used, importing those flags into the GUI for interactive modification; also works with `.kcpps` config files. • OpenBLAS backend removed and unified with NoBLAS into a single CPU mode; use `--usecpu` to enable CPU mode (the `--noblas` flag is now deprecated). • Adds support for RWKV v6 models (context shifting not supported). • Adds support for OpenAI-compatible TTS endpoints in Kobold Lite. • Adds display-only regex replacement in Kobold Lite, hiding or replacing displayed text while keeping the original text in AI context. • Adds a first-start welcome screen in Kobold Lite for selecting a starting UI theme. • Adds a preview option for alternate greetings within V2 Tavern character cards in Kobold Lite. • Adds a new Instruct scenario in Kobold Lite to mimic CoT Reflection (Thinking). • Adds a toggle in Kobold Lite for chat name format matching, allowing matching any name or only predefined names. • Adds a warning display when loading legacy GGML models. • Embedded Horde workers now work with password-protected instances. • Adds system role support for the Metharme instruct format in Kobold Lite. Breaking changes: • The OpenBLAS backend has been removed; `--noblas` is deprecated. CPU mode must now be specified with `--usecpu`. • When no `--use...` backend flag is provided, a GPU backend is now selected automatically if available, replacing the previous default of CPU/NoBLAS.

KoboldCpp v1.74

Sat, 31 Aug 2024 03:41:31 GMT

KoboldCpp v1.74 adds XTC sampler, a `--nomodel` flag for external API providers, and SSL support for Horde workers. • Adds `--nomodel` flag to launch the Lite WebUI without loading a local model, enabling selection of external API providers such as Horde, Gemini, or OpenAI. • Adds `xtc_probability` and `xtc_threshold` sampling parameters for the new XTC (Exclude Top Choices) creative-writing sampler; recommended starting values are `xtc_threshold=0.15` and `xtc_probability=0.5`. • Horde Worker now supports connecting to SSL-secured KoboldCpp instances (use `--nocertify` for self-signed certificates) and enforces a 5-minute per-request timeout to prevent stalls. • Adds automatic image resizing and letterboxing for LLaVA/MiniCPM multimodal inputs to improve handling of non-standard image dimensions. • MacOS now defaults to full GPU offload when `-1` is passed to `gpulayers`. • Adds support for the Llama-3.1-Minitron-4B-Width model architecture.

KoboldCpp v1.73.1

Mon, 19 Aug 2024 08:45:31 GMT

KoboldCpp v1.73.1 adds IPv6 dual-stack support, macOS ARM binaries, and a new `--prompt` pure CLI mode. • Adds `--prompt` flag for pure CLI mode: runs a single prompt entirely from the command line, suppressing all other console output and piping the response to stdout. • Adds `--promptlimit` flag to control output length when using `--prompt` CLI mode. • Combines `--prompt` and `--promptlimit` with `--benchmark` to enable benchmarking with a custom prompt and capturing the response. • `--multiuser` now defaults to true; set `--multiuser 0` to disable multi-user mode. • Adds dual-stack IPv6 network support, allowing a single KoboldCpp instance to serve both IPv4 and IPv6 addresses automatically on the same port. • Adds official macOS PyInstaller binary builds (`koboldcpp-mac-arm64`) for M1, M2, and M3 Macs, eliminating the need to self-compile. • Adds support for Minitron and MiniCPM multimodal models (including image input via mmproj file). • Adds letterboxing mode for images in Kobold Lite. • Pre-filters to the top 5,000 token candidates before sampling, improving sampling speed on models with large vocabularies. • Moved chat completions adapter selection to the Model Files tab in the GUI. Breaking changes: • `--multiuser` now defaults to true; existing setups that relied on multi-user mode being disabled by default must explicitly set `--multiuser 0`.

KoboldCpp v1.72

Fri, 02 Aug 2024 10:29:44 GMT

KoboldCpp v1.72 adds Vulkan-accelerated Stable Diffusion image generation and Gemma2 2B support. • Adds GPU-accelerated Stable Diffusion image generation on Vulkan backends. • GPU layers now defaults to `-1` in GUI mode, preserving existing layer counts instead of overwriting them, with predicted layer count shown as an overlay label. • Auto GPU layer estimation now accounts for loaded image and Whisper models when calculating VRAM headroom. • Adds Gemma2 2B support via upstream merge. • Kobold Lite now supports SSE streaming over the OpenAI API, enabling use with alternative backends.

KoboldCpp v1.71.1

Thu, 25 Jul 2024 06:09:03 GMT

KoboldCpp v1.71.1 adds shareable `.kcppt` launch templates, automatic GPU layer guessing via `--gpulayers -1`, and Vulkan `iq4_nl` format support. • Adds `--gpulayers -1` to let KoboldCpp automatically estimate how many GPU layers to offload (experimental). • Adds KoboldCpp Launch Templates: shareable `.kcppt` files that bundle model URLs, a preloaded story, a chat-completions adapter, and full backend/thread/GPU-layer configuration; load one via `--config `. • Adds `iq4_nl` quantization format support in the Vulkan backend. • Adds TTS narration speed setting in Kobold Lite. • Adds support for selecting the greeting message in Character Cards that contain multiple greetings in Kobold Lite. • Adds Corpo UI theme availability for chat mode in Kobold Lite. • Enables inject-chatnames in Corpo UI to replace the AI's displayed name when toggled on.

KoboldCpp v1.70.1

Mon, 15 Jul 2024 02:15:08 GMT

KoboldCpp v1.70.1 adds Corpo Mode UI, DRY sampler, `--unpack` self-extraction, OpenAI function calling, and raises default context to 4096. • Adds `--unpack` flag enabling self-extraction of KoboldCpp PyInstaller binaries into an empty directory for easy file access and modification; also usable from the GUI launcher. • Adds `--chatcompletionsadapter` selection by bundled template filename (e.g. `Llama-3.json`), allowing pre-bundled chat completion templates to be referenced without a full path. • Raises default `--contextsize` from 2048 to 4096 and increases the default Chat Completions API output length. • Adds DRY dynamic N-gram anti-repetition sampler support. • Adds experimental support for OpenAI tools and function calling API. • Ships official Windows binaries with AVX1 CUDA support as `koboldcpp_oldcpu.exe` for older CPUs. • Introduces Corpo Mode, a new beginner-friendly ChatGPT-style UI theme for instruct mode in Kobold Lite. • Adds support for importing character cards from aicharactercards.com in Kobold Lite. • Adds dedicated System Tag and System Prompt support in Kobold Lite. • Completely reworks the Kobold Lite settings menu into 4 distinct panels.

KoboldCpp v1.69.1

Mon, 01 Jul 2024 06:44:38 GMT

KoboldCpp v1.69.1 adds premade Chat Completions adapters, SD prompt injection fields, Vulkan tensor-split UI, and multi-image LLaVA separator tokens. • Adds `add_sd_prompt` and `add_sd_negative_prompt` fields in a loaded adapter to inject forced positive or negative prompts for Stable Diffusion generation. • Adds premade Chat Completions adapters selectable in the GUI launcher (via `--chatcompletionsadapter`) to apply instruct templates for various models and formats when using third-party OpenAI-compatible frontends. • Adds a tensor split input field in the GUI launcher for Vulkan backends. • Adds separator tokens between multiple images in LLaVA multi-image requests. • Kobold Lite can now fall back to an alternative API or endpoint URL on connection failure, including retrying via the OpenAI API. • Merges upstream improvements including enhanced MMQ with int8 tensor core support and Gemma 2 features (including IQ MMVQ fixes). • KoboldCpp Colab notebook switched to precompiled Linux binaries for faster startup.

KoboldCpp v1.68

Wed, 19 Jun 2024 08:47:10 GMT

KoboldCpp v1.68 adds GradientAI RoPE scaling, Vulkan multi-GPU 'All' option, BOS-skip tokenization, and a plaintext export in Kobold Lite. • Adds `special: false` parameter to the `/api/extra/tokencount` tokenize endpoint to skip BOS tokens when counting. • Adds an 'All' GPU option in the Vulkan GUI launcher for easy multi-GPU configuration, mirroring the existing CUDA behavior; `--usevulkan` now defaults to the first dedicated GPU rather than the first GPU on the list. • Adds GradientAI automatic RoPE calculation for improved RoPE scaling at large context sizes. • Preserves CLBlast support despite its removal upstream. • Allows the SD StableUI to be launched even when no Stable Diffusion model is loaded. • Adds a plaintext export option to Kobold Lite. • Increases Kobold Lite retry history stack to 3. • Running a KCPP horde worker now automatically sets Whisper and SD to quiet mode. • Allows PTT in Kobold Lite to trigger 'Generate More' when tapped, while still functioning as push-to-talk when held.

KoboldCpp v1.67

Tue, 04 Jun 2024 10:50:52 GMT

KoboldCpp v1.67 adds Whisper.cpp speech-to-text with Push-To-Talk/VAD and quantized KV cache via `--quantkv`. • Adds `--quantkv [level]` flag to enable Quantized KV Cache, where `level` is `0`=f16, `1`=q8, or `2`=q4; requires `--flashattention` and disables Context Shifting automatically. • Adds `/api/extra/transcribe` endpoint for speech-to-text transcription, accepting payloads as `.wav` files (max 32 MB) or base64-encoded WAV data. • Adds `/v1/audio/transcriptions` as an OpenAI-compatible drop-in transcription endpoint, also accepting `.wav` or base64-encoded WAV payloads. • Integrates Whisper.cpp for local speech-to-text; whisper models can be sourced from the whisper.cpp repo or the mirrored HuggingFace collection at `koboldcpp/whisper`. • Kobold Lite now supports Push-To-Talk (PTT) and automatic Voice Activity Detection (VAD) / Hands-Free Mode for speech-to-text, with all resampling and WAV conversion running locally in the browser. • Kobold Lite's Logit Bias editor now includes a built-in tokenizer for strings when connected to KoboldCpp. • Kobold Lite now supports importing Tavern Cards in Instruct mode via the 'Show Advanced Load' option. • New MOE (Mixture of Experts) support for Vulkan backend.

KoboldCpp v1.66

Fri, 24 May 2024 10:33:18 GMT

KoboldCpp v1.66 adds SD LoRA/VAE support, URL-based model loading, TAE SD, and rep pen slope. • Adds `--sdlora` flag to specify a custom Stable Diffusion LoRA file, and `--sdloramult` to set its multiplier (requires 16-bit model; incompatible with `--sdquant`). • Adds `--sdvae [vae_file.safetensors]` flag (and Image Gen tab GUI option) to specify a custom SD VAE file. • Adds built-in TAE SD support for SD1.5 and SDXL as a fast VAE replacement — enabled via the 'Fix Bad VAE' checkbox or the `--sdvaeauto` flag. • Supports passing an http/https URL to a GGUF file via the `--model` parameter or model selector UI — KoboldCpp downloads the file to the current working directory and loads it automatically. • Adds experimental Rep Pen Slope support, applying a scaled reduction in repetition penalty for older tokens within the rep pen range (slope defaults to 1 for backward compatibility). • Adds viewport width controls in Kobold Lite settings, including horizontal fullscreen. • Kobold Lite now attempts to function correctly when hosted on a subdirectory URL path (e.g. behind a reverse proxy), falling back to root URL on failure.

KoboldCpp v1.65

Sat, 11 May 2024 02:44:23 GMT

KoboldCpp v1.65 adds a standalone Stable Diffusion UI, CUDA 12 binaries, a new `bypass_eos` API field, and replaces three deprecated flags with granular replacements. • Adds `--sdmodel`, `--sdthreads`, `--sdquant`, and `--sdclamped` flags to replace the deprecated `--sdconfig`, enabling per-parameter validation and easier extension of Stable Diffusion options. • Adds `--hordemodelname`, `--hordeworkername`, `--hordekey`, `--hordemaxctx`, and `--hordegenlen` flags to replace the deprecated `--hordeconfig`, giving each AI Horde setting its own named flag. • Adds `bypass_eos` field to the API, allowing EOS tokens to be skipped during generation while still permitting them to appear in output. • Adds official CUDA 12 binary (`koboldcpp_cuda12.exe` / `koboldcpp_cu12.exe`) for newer NVIDIA GPUs, providing increased inference speeds at the cost of a larger download. • Adds a standalone browser-based image generation UI (StableUI port, A1111-compatible) accessible at `http://localhost:5001/sdui/` when a Stable Diffusion model is loaded. • Increases interrogate mode token limit by 30% and default chat completions token limit by 250%. • Adds option to insert an Instruct System Prompt in Kobold Lite. • Adds toggle to return special tokens in Kobold Lite. • Adds Chat Names insertion for instruct mode in Kobold Lite. • Adds button in Kobold Lite to launch the StableUI image generation interface. • Adds option in Kobold Lite to bypass (skip) EOS tokens. Breaking changes: • The `--smartcontext`, `--hordeconfig`, and `--sdconfig` flags are deprecated and scheduled for removal; existing setups using them should migrate to the new named replacement flags (`--hordemodelname`, `--hordeworkername`, `--hordekey`, `--hordemaxctx`, `--hordegenlen`, `--sdmodel`, `--sdthreads`, `--sdquant`, `--sdclamped`).

KoboldCpp v1.64.1

Wed, 01 May 2024 12:55:24 GMT

KoboldCpp v1.64.1 adds `--flashattention`, dynamic `banned_tokens`, `render_special`, and removes `--bantokens`. • Adds `--flashattention` experimental flag to enable Flash Attention for compatible models. • Adds `banned_tokens` field to the generate API, allowing per-generation token banning dynamically (replaces the removed `--bantokens` flag). • Adds `render_special` to the generate API, enabling rendering of special tokens such as `<|start_header_id|>` or `<|eot_id|>`. • Adds `trim_stop` support in SSE streaming modes so stop sequences are hidden during streaming when enabled; Chat Completions endpoint automatically applies `trim_stop` to instruct tag format for better compatibility with third-party clients like LibreChat. • Adds `finish_reason` communication in both sync and SSE streamed mode responses when generation stops due to EOS/EOT tokens. • Automatically detects and applies both EOS and EOT tokens, with EOT tokens correctly biased when EOS is banned. • Adds additional debug information output when running with `--debugmode`. • Adds a benchmark button in the GUI launcher; `--benchmark` now includes version and clearer exit instructions in console output. • Supports resizing the GUI launcher with auto-scaling GUI elements, useful for high-DPI screens. • Kobold Lite adds token filter feature, enhanced regex replacement (including for submitted text), custom `{{placeholder}}` tag support, inverted world info secondary keys (triggers when key is NOT present), and language customization for XTTS. • Improves speed of the repetition penalty sampler. Breaking changes: • The `--bantokens` flag has been removed; token banning must now be submitted dynamically via the `banned_tokens` field in the generate API.

KoboldCpp v1.63

Sat, 20 Apr 2024 03:00:44 GMT

KoboldCpp v1.63 adds special-token stop sequences, Llama3 RoPE scaling, and Kobold Lite regex replacer and background color settings. • Adds support for special tokens in `stop_sequences`: setting a token such as `<|eot_id|>` as a stop sequence now works as an EOS-like token when it maps to a single token, enabling multiple EOS-like stop tokens. • Reworks automatic RoPE scaling to support Llama3 — specifying `--contextsize` is now sufficient to trigger the correct scaling automatically. • Adds a Llama3 prompt template to Kobold Lite. • Adds a regex replacer feature to Kobold Lite for transforming output text. • Adds aesthetic background color settings to Kobold Lite. • Adds more save slots and usermod saving to Kobold Lite. • Adds a console warning when the desired port is already in use by another program.

KoboldCpp v1.62.2

Tue, 09 Apr 2024 07:01:48 GMT

KoboldCpp v1.62.2 adds Img2Img via `/sdapi/v1/img2img`, a `--chatcompletionsadapter` flag, and an embedded Horde Worker priority system. • Adds `--chatcompletionsadapter` CLI flag to specify OpenAI Chat Completions adapter files at load time, enabling any instruct tag format via the Chat Completions API. • Emulates the A1111-compatible `/sdapi/v1/img2img` endpoint, enabling image-to-image generation directly from KoboldCpp. • Expands `/api/extra/perf/` with additional usage statistics. • Adds `/docs` endpoint as an alias for `/api`, serving built-in API documentation. • Embedded Horde Workers now prioritise the local user, automatically pausing and resuming when a local client is active so local and horde workloads can coexist. • Kobold Lite UI gains Img2Img support — click an existing generated image to use it as the basis for a new generation. • Kobold Lite adds API support for Cohere, Claude Haiku, and Gemini 1.5 as external provider targets.

KoboldCpp v1.61.2

Wed, 13 Mar 2024 02:19:05 GMT

KoboldCpp v1.61.2 adds multimodal vision via `--mmproj`, API key auth via `--password`, mamba model support, and a new `--ignoremissing` flag. • Adds `--mmproj` flag to load a multimodal projector file, enabling vision capabilities (LLaVA-style) for any compatible model of matching size and architecture — not limited to LLaVA models. • Adds `--password` flag to require Bearer token authentication on all text generation endpoints; image endpoints remain unsecured. • Adds `--ignoremissing` flag to silently skip optional missing files passed via flags like `--lora` or `--mmproj` instead of exiting. • Adds `clamped` option for `--sdconfig` to prevent generating overly large resolutions that could cause out-of-memory crashes. • Emulates a compatible OpenAI GPT-4V API endpoint, allowing GPT-4-Vision applications (e.g. SillyTavern in Chat Completions mode) to work out of the box; Kobold API and OpenAI Text-Completions API accept an array of base64-encoded `images` in the submit payload. • Emulates an A1111-compatible `/sdapi/v1/interrogate` endpoint for image captioning and interrogation frontends. • Supports passing up to 4 images per request (each consuming ~600 tokens of context for LLaVA 1.5), with token fast-forwarding and context-shifting working seamlessly so each image is processed only once. • Adds CPU-only support for **mamba** models (note: mamba does not support context shifting). • Increases `--benchmark` context limit to 16k. • Adds support for generating non-square images with correct aspect ratio scaling, including portrait and landscape in Kobold Lite. • Adds aliases for image sampler names in txt2img generation. • Kobold Lite gains ability to upload images for LLaVA usage with 4 selectable 'AI Vision' modes, paste images from clipboard, choose image samplers, and use API keys over the KoboldAI API.

KoboldCpp v1.60.1

Mon, 04 Mar 2024 12:15:36 GMT

KoboldCpp v1.60.1 adds native local image generation via stable-diffusion.cpp with A1111-compatible txt2img and flexible context sizing. • Adds `--sdconfig` CLI flag to load a compatible SD1.5 or SDXL `.safetensors` fp16 image model, enabling native local image generation via stable-diffusion.cpp with an A1111-compatible `txt2img` endpoint usable from Kobold Lite, SillyTavern, and other compatible frontends. • Adds `--remotetunnel` flag for Cloudflare tunnel support on macOS (amd64 only; M1 not supported). • Enables LCM sampler support and allows loading GGUF-format SD models. • Supports running KoboldCpp in text-gen-only, image-gen-only, or hybrid modes via launcher config, with a 'Compress Weights (quant)' option to reduce image model VRAM usage. • Integrates AllTalk TTS support in Kobold Lite. • Adds 'Auto Jailbreak' option for instruct mode in Kobold Lite to assist with stubborn or censored models. • Adds import and export of story via clipboard in Kobold Lite. • Adds personal notes/comments field for stories in Kobold Lite. • Auto-enables the image generation button in Kobold Lite when an image model is loaded. • Adds new GGUF quantization types pulled from upstream.

KoboldCpp v1.59.1

Sun, 25 Feb 2024 05:31:01 GMT

KoboldCpp v1.59.1 adds `--nocertify` mode for Horde workers, Old CPU Vulkan build target, and IQ3_S quantization support. • Adds `--nocertify` flag to disable SSL certificate checking on the embedded Horde worker, bypassing SSL certificate errors. • Adds a build target for Old CPU (NoAVX2) Vulkan support, enabling GPU acceleration on hardware without AVX2 instructions. • Adds IQ3_S quantization format support (merged in the 1.59.1 fix build). • Adds Google Gemma model support via upstream merges.

KoboldCpp v1.58

Sat, 17 Feb 2024 09:51:49 GMT

KoboldCpp v1.58 adds row-split CUDA multi-GPU mode, AI Vision image interrogation in Lite, and Pygmalion.Chat character imports. • Adds `rowsplit` option to `--usecublas` for row-split mode with CUDA multi-GPU setups; layer split is now the default. • Integrates 'AI Vision' image interrogation in Kobold Lite, using AI Horde or a local A1111 endpoint to let the AI recognize and interpret uploaded or generated images as a multimodal alternative to LLaVA. • Supports importing characters from Pygmalion.Chat in Kobold Lite via the scenarios selector. • Adds an option to run Kobold Lite in the background using a dynamically generated silent audio sound to prevent browser tab hibernation. • Adds an 'Auto' option for idle responses in Kobold Lite. • Allows importing images into the story from local disk in Kobold Lite. • Upgrades CLBlast to the latest version for a modest prompt-processing speedup when using OpenCL. Breaking changes: • The default CUDA multi-GPU split mode has changed from row split to layer split; users relying on row split must now explicitly add `rowsplit` to `--usecublas`.

KoboldCpp v1.57.1

Thu, 08 Feb 2024 15:49:12 GMT

KoboldCpp v1.57.1 adds `--benchmark` CSV output, Vulkan multi-GPU via `--usevulkan`, and `smoothing_factor` sampling. • Adds `--benchmark` flag to run an automated benchmark with current settings, reporting run parameters, timing, and coherence; supply a filename (e.g. `--benchmark result.csv`) to append results in CSV format. • Adds Vulkan multi-GPU support: specify multiple device IDs with `--usevulkan 0 2 3`; allocation is controlled by the existing `--tensor_split` flag (CLI only — GUI launcher does not support multiple Vulkan devices). • Adds temperature Quad-Sampling via the `smoothing_factor` API parameter, also configurable in the Kobold Lite dynamic temperature panel. • Adds `--debugmode` timing output in llama.cpp style, measuring only specific eval functions (excluding overhead) alongside display of the seed used. • Adds 'AI Impersonate' feature for Instruct mode in Kobold Lite. • Adds a toggle in Kobold Lite to inject timestamps, enabling the AI to be aware of time passing. • Adds a toggle in Kobold Lite for a printable view (unlocks vertical scrolling). • Extends the Aesthetic UI in Kobold Lite to Story and Adventure modes. • Adds option in Kobold Lite to request the A1111 backend to save generated images to disk. • Persists API info for A1111 and XTTS in Kobold Lite; supports custom negative prompts for image generation and custom Horde keys in KCPP mode. • Adds a 'narrate dialogue only' option for XTTS in Kobold Lite.

KoboldCpp v1.56

Sat, 27 Jan 2024 09:51:56 GMT

KoboldCpp v1.56 adds Vulkan GPU backend, XTTS text-to-speech, and `dynatemp_exponent` API support. • New `--usevulkan ` flag enables an early Vulkan GPU backend, now included in Windows and Linux prebuilt binaries (note: Mixtral on Vulkan not fully supported). • Adds `dynatemp_exponent` parameter (previously hard-coded to 1.0), now configurable via API and in Kobold Lite. • Adds XTTS API Server support in Kobold Lite for local AI-powered text-to-speech. • Adds Old CPU fallback build targets (NoAVX2 and Failsafe modes) to the Linux prebuilt binary and `koboldcpp.sh`. • Adds HD image generation options in Kobold Lite. • Adds popup-on-complete browser notification options in Kobold Lite. • Adds option in Kobold Lite to let the AI impersonate the user for a turn in chat. • Merges the new GGML backend rework from upstream, preserving support for earlier non-GGUF models via a fossilized earlier version of the library.

KoboldCpp v1.55.1

Wed, 10 Jan 2024 12:55:15 GMT

KoboldCpp v1.55.1 adds Dynamic Temperature sampling with `dynatemp_range` control and exposes latest seed in the perf endpoint. • Adds `dynatemp_range` parameter enabling Dynamic Temperature (DynaTemp) sampling, where the actual temperature is automatically adjusted between `temperature ± dynatemp_range` at inference time (e.g., `temperature=0.4` and `dynatemp_range=0.1` yields a 0.3–0.5 range). • Exposes the most recently used seed in the `/perf` endpoint, making it easier to reproduce or audit generation runs. • Adds a min/max temperature UI in Lite for configuring DynaTemp directly, with both input styles auto-syncing to each other.

KoboldCpp v1.54

Mon, 01 Jan 2024 08:52:17 GMT

KoboldCpp v1.54 adds `logit_bias` support for OpenAI and Kobold APIs, custom background images in Lite, and GUI launcher tooltips. • Adds `logit_bias` parameter to both the OpenAI and Kobold APIs, accepting a dictionary of token ID (int) to logit bias (float) pairs in the same object format as the official OpenAI implementation. • Adds support for custom background images in KoboldCpp Lite. • Adds customizable `stepcount` and `cfgscale` settings for Horde/A1111 image generation in Lite. • Adds mouseover tooltips for all labels in the GUI launcher. • Includes a prebuilt no-CUDA Linux binary.

KoboldCpp v1.53

Sat, 23 Dec 2023 03:18:55 GMT

KoboldCpp v1.53 adds HTTPS/SSL support, presence penalty, and Phi model support with Mixtral speedups. • Adds `--ssl [cert.pem] [key.pem]` flag to serve KoboldCpp over HTTPS using your own unencrypted PEM certificate and key files (also configurable via the GUI). • Adds presence penalty support over the KAI API and in Lite as an alternative repetition penalty mechanism. • When presence penalty is set via the OpenAI API and `rep_pen` is not explicitly set, `rep_pen` now defaults to 1.0 instead of 1.1. • Adds upstream support for Phi models. • Adds upstream performance speedups for Mixtral. • Increases the default non-blas batch size for GGUF models from 8 to 32, improving throughput on CPU inference. Breaking changes: • When presence penalty is set over the OpenAI API and `rep_pen` is not set, `rep_pen` now defaults to 1.0 instead of 1.1 — any workflow relying on the old implicit 1.1 default will produce different outputs.

KoboldCpp v1.52.2

Wed, 13 Dec 2023 14:21:39 GMT

KoboldCpp v1.52.2 adds a NoScript WebUI, per-layer KV offloading for CUDA, QWEN/Mixtral support, and richer API responses. • Adds a new NoScript WebUI at `/noscript` (e.g. `http://localhost:5001/noscript`) — W3C HTML-compliant, works in text-based browsers like Lynx over SSH, requires no JavaScript. • The `--usecublas lowvram` flag (equivalent to `-nkvo` in llama.cpp) disables per-layer KV offloading for CUDA when the reduced GPU layer budget is undesirable. • Merges partial per-layer KV offloading for CUDA, changing how GPU layer capacity is calculated — fully offloaded models are unaffected. • The `/api/extra/tokencount` endpoint now returns an array of token IDs in the response body alongside the token count. • Adds a new Horde Worker error-count field to the `/api/extra/perf` endpoint for monitoring embedded Horde Worker health. • Merges support for QWEN and Mixtral model architectures from upstream; `--noblas` or `--blasbatchsize -1` can mitigate slow batch prompt processing on Mixtral. • Selecting a `.kcpps` config file in the GUI now loads the model specified inside that file directly. • Adds the Mamba Multitool shell script for Linux to set up all dependencies needed to build and run KoboldCpp. • Improves embedded Horde Worker fault tolerance with exponential backoff and automatic recovery from up to 24 hours of Horde downtime. • Adds Linux PyInstaller binaries. Breaking changes: • Per-layer KV offloading is now applied for CUDA, meaning the number of layers that can be offloaded to GPU may be reduced compared to previous versions. Use `--usecublas lowvram` to restore the old behaviour.

KoboldCpp v1.51.1

Fri, 01 Dec 2023 16:30:18 GMT

KoboldCpp v1.51.1 adds `--quiet` console suppression and extends `--multiuser` with a concurrency queue limit. • Adds `--quiet` flag to suppress all input and output from appearing in the console. • Extends `--multiuser` with an optional integer parameter to set how many concurrent requests may queue (defaults to 5 when unset or set to 1). • API URL printouts on startup now include the tunnel URL when `--remotetunnel` is in use. • Allocates ~80 tokens of reserved KV cache space on top of the specified `--contextsize` (e.g. `--contextsize 2048` allocates 2128 internally) to reduce 'Failed to predict' errors caused by KV cache fragmentation during context shift. • Displays available KAI and OAI endpoint URLs in the terminal on startup. • Auto RoPE scaling is skipped when the model already overrides the RoPE freq scale with a value below 1.

KoboldCpp v1.50.1

Sat, 18 Nov 2023 08:45:16 GMT

KoboldCpp v1.50.1 adds SSE streaming and unofficial Aphrodite sampler params to OpenAI-compatible endpoints, plus custom DALL-E proxy support. • Adds SSE streaming support for the OpenAI-compatible `/v1/completions` endpoint (tested working with SillyTavern). • Extends `/v1/completions` with unofficial Aphrodite sampler parameters: Min-P, Top-A, and Mirostat. • Adds support for custom DALL-E endpoints to enable use with OpenAI-compatible proxies. • Improves automatic GPU layer selection in the CuBLAS GUI launcher to default to full GPU offload when sufficient VRAM is detected.

KoboldCpp v1.49

Sat, 11 Nov 2023 03:28:51 GMT

KoboldCpp v1.49 adds Split Memory, trim_stop, and --preloadstory API features for richer generation control. • Adds `memory` field to the `/v1/generate` API payload: forcefully prepends a string to any submitted prompt, and if the context limit is exceeded, overwrites from the beginning of the main `prompt` to make room — guaranteeing full memory insertion without needing exact token counts. • Adds `trim_stop` boolean field to the generate API payload: when `true`, strips detected stop sequences from the output and truncates everything after them (note: incompatible with SSE streaming). • Adds `--preloadstory` CLI flag to specify a JSON story savefile at server launch, hosting it at the `/api/extra/preloadstory` endpoint for frontends to consume over the API. • Adds `LLAMA_PORTABLE=1` makefile flag for building portable binaries targeting Colab or Docker environments. • Expands Kobold Lite with World Info inject position support, Split Memory, preloaded stories, and optional image generation via DALL-E 3 (OpenAI API). • Extends Colab prebuilt GPU support to A100 and V100 in addition to T4.

KoboldCpp v1.48.1

Sat, 04 Nov 2023 07:50:19 GMT

KoboldCpp v1.48.1 adds KV cache context shifting, Min-P sampler, remote tunneling, and GPU auto-configuration. • Adds `--noshift` flag to disable the new Context Shifting (EvenSmarterContext) feature, which uses KV cache shifting to remove old tokens and add new ones without reprocessing — enabled by default and overrides SmartContext when both are set. • Adds `--remotetunnel` flag, which downloads `Cloudflared` and creates a TryCloudFlare tunnel so KoboldCpp is reachable over the internet even behind a firewall. • Adds Min-P sampler, now available via the API and configurable in Kobold Lite under the Advanced settings tab. • Introduces a new build target `koboldcpp_clblast_noavx2` ('CLBlast NoAVX2 (Old CPU)') for Windows users without AVX2 intrinsics, enabling CLBlast GPU acceleration on older CPUs. • Changes MMQ/Tensor Core behavior: MMQ is always enabled until batch > 32, CuBLAS only activates for larger batches when the MMQ flag is explicitly disabled, and MMQ dimensions are set to 'FAVOR BIG' — diverging from upstream llama.cpp's approach. • Adds automatic GPU name display and GPU layer suggestion in the GUI using `clinfo` and `nvidia-smi` queries, based on available VRAM and model file size. • Adds Sampler Seeds support in Kobold Lite for deterministic generation. • Includes `Content-Length` header in HTTP responses. • Now accounts for `freq_base_train` when computing automatic RoPE scale. • Retains support for GGUFv1 (upstream has removed it). • Improved KoboldCpp Colab notebook now ships prebuilt CUDA binaries, reducing post-launch load time to under one minute (excluding model downloads), with additional default model options and support for custom GGUF model URLs.

KoboldCpp v1.47.2

Fri, 20 Oct 2023 08:11:20 GMT

KoboldCpp v1.47.2 adds an OpenAI Chat Completions adapter, SSE streaming, browser save slots, and AUTOMATIC1111 image generation. • Adds a GUI input field for `tensor_split` configuration, allowing multi-GPU tensor splitting to be configured without CLI editing. • Adds an unofficial OpenAI v1 Chat Completions endpoint adapter that allows customization of instruct tags over the API. • Adds SSE (Server-Sent Events) streaming support to Kobold Lite, toggled on in browser settings (requires a recent browser). • Adds Browser Storage Save Slots in Kobold Lite for temporarily saving and swapping between multiple stories within a browser session. • Adds AUTOMATIC1111 integration in Kobold Lite, enabling local image generation from an A1111 install as an alternative to Horde. • Adds World Info Search Depth setting in Kobold Lite. • Adds a Group Chat Management Panel in Kobold Lite for temporarily toggling participants in a group chat. • Adds auto-recovery pause timeout for the Embedded Horde Worker on repeated errors, instead of immediately halting; the worker still halts if total error count exceeds a high threshold. • Pulls upstream support for ChatML added token merges (requires a correctly converted GGUF model). • Improves server CORS and content-type handling.

KoboldCpp v1.46.1

Sun, 08 Oct 2023 07:03:16 GMT

KoboldCpp v1.46.1 adds OpenAI Chat Completions endpoint, 64k context support, parallel horde jobs, and full API docs. • Adds OpenAI Chat Completions-compatible endpoint at `/v1/chat/completions`. • Adds interactive API documentation accessible at `/api` (also hosted at https://lite.koboldai.net/koboldcpp_api). • Expands `--contextsize` support up to 64k tokens (GGUF models only; remains 16k for other formats). • Embedded horde worker now executes job pulls and job submits in parallel, yielding approximately 20% faster horde request handling with additional session stats. • `.kcpps` config files can now be drag-and-dropped or opened via 'Open With' on Windows. • Changes the default model name reported via API from `concedo/koboldcpp` to `koboldcpp/[model_filename]`. Breaking changes: • The `--psutil_set_threads` command-line argument has been removed. • The `--stream` command-line argument has been removed; streaming is now a toggle saved inside Kobold Lite's settings. • The `--unbantokens` command-line argument has been removed; EOS unbans must now be set via the generate API using the `use_default_badwordsids` JSON field. • The `--usemirostat` command-line argument has been removed; Mirostat values must now be set via the generate API using the `mirostat`, `mirostat_tau`, and `mirostat_eta` JSON fields. • The original tkinter GUI has been removed; only the customtkinter GUI remains. • The default model name is changed from `concedo/koboldcpp` to `koboldcpp/[model_filename]`, which breaks connections from old KoboldAI-Client users connecting via the Kobold API.

KoboldCpp v1.45.2

Sun, 01 Oct 2023 07:39:24 GMT

KoboldCpp v1.45.2 adds OpenAI-compatible `/v1/completions`, grammar state persistence, 32k context, and a new `--foreground` flag. • Adds `grammar_retain_state` parameter to the grammar sampler API, allowing grammar state to persist across multiple requests. • Adds `--foreground` flag for Windows users to bring the console terminal to the foreground on each prompt generation, avoiding idle slowdown. • Increases maximum supported context via `--contextsize` to 32k for GGUF models (still limited to 16k for older model formats). • Adds OpenAI-compatible completions API endpoint at `/v1/completions`. • Supports launching with a `.kcpps` settings file via the file selector GUI combined with `--skiplauncher` (the settings file must already have a model selected). • Increases `stop_sequence` limit to 16. • Upgrades Lite polled-streaming to work in multiuser mode by issuing a unique key per request. • Enhances **Remote-Link.cmd** to run on Linux, creating a Cloudflare tunnel for remote access. • Adds Session Stats (Total Kudos Earned, EarnRate, Timings) to the embedded horde worker. • Improves SSE streaming by batching pending tokens between events. Breaking changes: • The `--psutil_set_threads` argument is deprecated and will be removed in a future version. • The `--stream` argument is deprecated and will be removed; streaming is now a toggle saved inside Lite's settings. • The `--unbantokens` argument is deprecated and will be removed; EOS unbans must now be set via the generate API using the `use_default_badwordsids` JSON field. • The `--usemirostat` argument is deprecated and will be removed; Mirostat values must now be set via the generate API using the `mirostat`, `mirostat_tau`, and `mirostat_eta` JSON fields. • The OpenAI completions endpoint moved from `/api/extra/oai/v1/completions` to `/v1/completions`.

KoboldCpp v1.44.2

Wed, 20 Sep 2023 10:27:42 GMT

KoboldCpp v1.44.2 adds multi-user queuing, `--onready` hooks, Grammar Sampling, a new context-length API endpoint, and a major Kobold Lite UI overhaul. • Adds `--multiuser` flag enabling up to 5 concurrent incoming `/generate` requests to be queued and processed in sequence instead of being rejected while busy. • Adds `--onready` launcher argument to execute a terminal command (e.g. start a Python script or Cloudflare tunnel) as a subprocess after KoboldCpp finishes loading. • Adds `/api/extra/true_max_context_length` API endpoint to fetch the true maximum context limit separately from the horde-friendly value. • Adds Grammar Sampling for all architectures, including older models, accessible via the web API and Kobold Lite; a BNDF grammar string can be specified in Lite settings. • Extends GPU selection to a 4th GPU in both the UI and command line (previously capped at 3). • Adds a streaming toggle in the Kobold Lite settings panel to enable streaming without URL manipulation; `--stream` flag is retained for compatibility. • Adds Mirostat UI configuration controls to the Kobold Lite settings panel. • Adds Aesthetic UI for chat mode in Kobold Lite, automatically selected when importing Tavern cards, with easy switching between chat and instruct UIs from the settings panel. • Adds support for importing characters from Chub.AI in Kobold Lite. • Adds Instruct Tag Presets dropdown and instruct placeholder support (e.g. `{{[INPUT]}}` and `{{[OUTPUT]}}`) in Kobold Lite for easy format switching, with a toggle for 'Raw Instruct Tags' as an alternative. • Adds 'Newline After Memory' and 'Show Rename Save File' toggles in Kobold Lite settings. • Adds automatic expansion of the max context size slider limit in Kobold Lite when a larger context is detected. • Makes Idle Responses a global setting available in all Kobold Lite modes. • Adds smarter group chat behavior in Kobold Lite — mentioning a specific character name directs that character to respond instead of selecting randomly.

KoboldCpp v1.43

Thu, 07 Sep 2023 09:04:41 GMT

KoboldCpp v1.43 adds automatic RoPE scale calculation, older GGML support, and thread-safe SSE streaming improvements. • Adds automatic RoPE scale calculation based on a model's training context (`n_ctx_train`) when `--ropeconfig` is not explicitly set; `--ropeconfig` overrides this behavior. • Tweaks `--unbantokens` to push banned token logit values further down, reducing rare appearances while avoiding `-inf` to preserve sampling compatibility. • Adds support for older GGML format (`ggjt_v3`) for 34B LLaMA 2 models (note: may have issues when `n_gqa` is not 1; GGUF recommended in that case). • Adds mutex for thread-safe polled-streaming, enabling safer concurrent SSE stream consumption. • Integrates SSE streaming improvements for more reliable server-sent event delivery. • HIP and CUDA visible devices are now scoped to a single GPU when a GPU number is provided and tensor split is not specified. • Enables SSE3 flags for failsafe mode, restoring and improving Windows 7 compatibility in noavx2 and failsafe modes. • Updates Kobold Lite to use placeholders for instruct tags that are swapped during generation.

KoboldCpp v1.42.1

Wed, 30 Aug 2023 15:23:24 GMT

KoboldCpp v1.42.1 adds GGUFv2 model support, AMD HIPBLAS GPU builds, GGUF Falcon models, and API-level EOS token unbanning. • Supports unbanning EOS tokens via API, now also triggerable from Lite UI settings; the `--unbantokens` CLI flag forces an override of API-set values. • Adds support for LLAMA GGUFv2 models, handled automatically with backward compatibility for older models. • Adds AMD GPU build support via HIPBLAS (ROCm), offering faster inference than CLBlast for AMD hardware. • Adds support for GGUF Falcon models; GPU layer offload works with CUDA but is unavailable with `--useclblast`. • Adds tavern-style portraits in Kobold Lite's Aesthetic Instruct mode.

KoboldCpp v1.41

Thu, 24 Aug 2023 13:51:46 GMT

KoboldCpp v1.41 adds GGUF model support, a `--config` flag for settings files, and a new `/api/extra/tokencount` endpoint. • Adds `--config` flag to load a `.kcpps` settings file at launch from the command line; `.kcpps` files can also be imported and exported from the GUI. • Adds new `/api/extra/tokencount` endpoint to tokenize any string and accurately measure its token count. • Supports **LLAMA GGUF** model format, detected and handled automatically alongside all existing model formats.

KoboldCpp v1.40.1a

Wed, 09 Aug 2023 04:55:25 GMT

KoboldCpp v1.40.1a adds an `idle` field to `/api/extra/perf` and an aesthetic instruct UI option in Lite. • Adds `idle` field to the `GET /api/extra/perf` endpoint, allowing callers to check whether a generation is currently in progress without submitting a generation request. • Adds an aesthetic instruct UI option (early beta) in the Lite interface. • Converts the embedded horde worker to daemon mode to improve stability.

KoboldCpp v1.39.1

Mon, 07 Aug 2023 07:36:27 GMT

KoboldCpp v1.39.1 adds `--blasbatchsize 2048`, 12k/16k context sizes, and Tavern V2 card import with world info. • Adds experimental `--blasbatchsize 2048` support (note: buffers are doubled, consuming significantly more memory). • Adds `--contextsize` options for 12k and 16k context windows. • Adds Tavern V2 card format import in KoboldCpp Lite, including world info (character book) and improved settings edit boxes. • Pulls upstream CUDA speedups for MMQ mode across all quantization types.

KoboldCpp v1.38

Wed, 02 Aug 2023 15:14:48 GMT

KoboldCpp v1.38 adds Quantized MatMul (MMQ) prompt processing for CUDA via `--usecublas mmq`. • Adds `--usecublas mmq` flag (also togglable in GUI) to enable Quantized MatMul (MMQ) prompt processing on CUDA, reducing memory usage and improving speed for Q4_0 models. • Displays a terminal warning when received context exceeds the max launcher-allocated context. • Adds a Help button in the GUI. Breaking changes: • `--noavx2` mode no longer uses OpenBLAS, now behaving the same as Failsafe mode.

KoboldCpp v1.37.1a

Mon, 24 Jul 2023 15:22:05 GMT

KoboldCpp v1.37.1 adds an embedded Horde Worker, LLAMA2 70B support, and exposes `--tensor_split` for CUDA multi-GPU use. • Extends `--hordeconfig` to accept 5 parameters — `[hordemodelname] [hordegenlength] [hordemaxctx] [hordeapikey] [hordeworkername]` — which starts an embedded AI Horde worker that serves requests automatically in the background, eliminating the need for separate tools like HaidraScribe/KAIHordeBridge. • Exposes `--tensor_split` for splitting model layers across multiple CUDA GPUs, matching upstream behavior. • Retains `--blasbatchsize 1024` support after it was removed upstream (note: scratch and KV buffer sizes will be larger when used). • Adds LLAMA2 70B model support, with GQA automatically set to 8 upon detection. • Adds additional fields to `/api/extra/perf` response, including stopping reason and generated token counts for the last generation. • Adds experimental Kepler architecture (e.g. K80) as a CUDA build target.

KoboldCpp v1.36

Thu, 20 Jul 2023 10:15:50 GMT

KoboldCpp v1.36 replaces `--linearrope` with `--ropeconfig` for unified RoPE tuning and exposes new perf data via `/api/extra/perf`. • Adds `--ropeconfig ` CLI argument to control both RoPE frequency scale (Linear) and RoPE frequency base (NTK-Aware) in a single flag, replacing the removed `--linearrope` flag — e.g. `--ropeconfig 0.5 10000` for 2x linear scale or `--ropeconfig 1.0 10000` for native LLAMA2 4K tuning. • Exposes additional token counter data through the API endpoint `/api/extra/perf`. • Automatically configures long-context `NTK-Aware RoPE` based on the `--contextsize` parameter by default, with no manual tuning required. • Adds `--ropeconfig` support in the GUI for ease of use. • Updates Kobold Lite with improved whitespace trim support and a new toggle for partial chat responses. Breaking changes: • The `--linearrope` flag has been removed; replace it with `--ropeconfig ` (e.g. `--ropeconfig 0.5 10000` for equivalent 2x linear scaling).

KoboldCpp v1.35

Wed, 12 Jul 2023 06:28:48 GMT

KoboldCpp v1.35 adds CUDA 8-bit MMV acceleration, a new `/api/extra/perf` endpoint, and `--linearrope` context scaling. • Adds `--linearrope` launch parameter to enable linear RoPE scaling (using 2048 as base) as an alternative to NTK-Aware RoPE; combine with `--contextsize 8192` for a `0.25` linear scale suited to SuperHOT models. • Exposes prompt-processing and generation timing via new API endpoint `GET /api/extra/perf`. • Enables CUDA 8-bit MMV mode (quantized dot products) for formats `q4_0`, `q4_1`, `q5_0`, and `q5_1`, delivering significant GPU throughput gains when full GPU offload is used; K-quants and CL are unaffected. • Adds Save and Load settings options to the GUI launcher. • Adds 'All Devices' selection in the GUI for CUDA multi-GPU configurations. • Displays a warning when poor sampler orders are detected, nudging users toward the default configuration.

KoboldCpp v1.34.2

Fri, 07 Jul 2023 11:41:53 GMT

KoboldCpp v1.34.2 adds NTK-aware RoPE scaling to 8K context, `--bantokens`, per-generation sampler_order/mirostat, and a new customtkinter GUI. • Adds `--bantokens` CLI flag to block a list of token substrings from being generated — e.g. `--bantokens [ a ooo` bans all tokens matching those substrings. • Adds `--usecublas lowvram ` syntax for selecting a specific GPU by index when using CUDA multi-GPU setups. • Adds `sampler_order` and `mirostat`/`tau`/`eta` parameters to the `/generate` API, settable per-generation request. • Switches RoPE scaling to NTK-aware method driven by the existing `--contextsize` parameter, with support up to 8K context. • Extends NTK-aware scaled RoPE support to GPT-NeoX and GPT-J model architectures, enabling longer context (e.g. 4K) on older models. • Adds 3K and 6K as selectable context size options alongside existing sizes. • Adds a new `customtkinter`-based GUI with more configurable settings; requires the `customtkinter` Python module on Linux and macOS (bundled in Windows `.exe` builds). • Kobold Lite now displays submitted contexts after each generation and adds two new scenarios plus limited Tavern v2 card support.

KoboldCpp v1.33

Thu, 29 Jun 2023 13:44:33 GMT

KoboldCpp v1.33 adds unified CUDA+CL distribution, GPU acceleration for more model architectures, and long-context Scaled RoPE support. • Adds `--usecublas` flag to select CUDA inference mode, with `--usecublas lowvram` for low-VRAM operation, alongside `--useclblast` for CLBlast — all available in a single All-In-One distributable. • Adds `--contextsize` flag to enable long-context inference beyond 2048 tokens via Scaled RoPE for LLAMA models (tested up to 3k+). • Extends CUDA GPU layer offloading to MPT, GPT-2, GPT-J, and GPT-NeoX model architectures. • Back-ports CUDA support to all prior GGML file format versions (GGML, GGMF, GGJT v1/v2/v3), allowing legacy quants to load and run under CUDA mode. • Applies VRAM memory optimizations (previously OpenCL-only) to CUDA, enabling more layers to be offloaded than upstream llama.cpp. • Instruct mode now supports any number of newlines in start and end tags, configurable by the user.

KoboldCpp v1.32.3

Thu, 22 Jun 2023 09:18:38 GMT

KoboldCpp v1.32.3 brings OpenCL K-Quant CUDA kernel ports and GPU offloading for MPT, GPT-2, GPT-J, and GPT-NeoX architectures. • Ports optimized K-Quant CUDA kernels to OpenCL, delivering approximately 15% faster generation speed for K-Quant models on OpenCL backends. • Adds basic GPU offloading via OpenCL for MPT, GPT-2, GPT-J, and GPT-NeoX architectures, achieving roughly 50% speedup for GPT-J and 30% for WizardCoder (weights still copied to RAM). • Implements scratch buffers for MPT, GPT-2, NeoX, and GPT-J architectures, significantly reducing BLAS memory usage and enabling larger BLAS batch sizes on these models. • Extends GPT tokenizer with support for Starcoder special added tokens, improving coherence for non-llama models.

KoboldCpp v1.31.2

Sat, 17 Jun 2023 15:37:57 GMT

KoboldCpp v1.31.2 adds horde max context control, debug-mode-gated output, and console token generation rate display. • Adds `--hordeconfig` support for specifying a separate max context length allowed in horde, independent of the real context length used for memory allocation. • Displays token generation rate in the console during inference. • Gates server endpoint debug output behind `--debugmode`; also suppresses incoming horde prompt logs when `--hordeconfig` is set unless `--debugmode` is also enabled. • Adds new scenarios, italics chat text, and customizable idle message counts to Kobold Lite. Breaking changes: • RWKV sequence mode is disabled; setups relying on it for speed improvements will no longer benefit from those gains.

KoboldCpp v1.30.3

Tue, 13 Jun 2023 15:12:21 GMT

KoboldCpp v1.30.3 adds OpenCL K-Quant GPU offload, RWKV World Tokenizer, SSE streaming, abort endpoint, and LoRA base support. • Adds a true SSE-streaming endpoint at `/api/extra/generate/stream` (Agnaistic-compatible) that streams tokens in real time during generation. • Adds a polled-streaming endpoint at `/api/extra/generate/check` to fetch in-progress results without disrupting generation; the old 8-token chunked streaming remains available via `streamamount=8` in the URL. • Adds an abort endpoint at `/api/extra/abort` that halts in-progress generation without stopping the server, integrated into Kobold Lite via an 'abort' button. • Adds `--lora [lora_file] [base_model]` optional second parameter for specifying a LoRA base model. • Adds full OpenCL / CLBlast support for K-Quants, covering both prompt processing and GPU offloading across all K-quant formats. • Adds support for the RWKV World Tokenizer and RWKV-World models, with automatic detection and selection. • Adds CUDA support for older ggjtv2 models via backport, available through `koboldcpp_CUDA_only.exe`.

KoboldCpp v1.29

Wed, 07 Jun 2023 08:32:21 GMT

KoboldCpp v1.29 adds BLAS batch size control to the Easy Launcher GUI and merges upstream K-quantization support for OpenBLAS. • Adds BLAS batch size setting to the KoboldCpp Easy Launcher GUI, giving users direct control over batch sizing without CLI flags. • Merges upstream K-quantization (K-quants) implementations for OpenBLAS, expanding supported model quantization formats (note: K-quants remain unsupported in CLBlast). • Adds streaming support for aesthetic chat mode in KoboldCpp Lite.

KoboldCpp v1.28

Sun, 04 Jun 2023 14:45:57 GMT

KoboldCpp v1.28 adds MPT model support, a new `--hordeconfig` flag, and `--blasbatchsize -1` to maximize GPU layer offloading. • Renames `--renamemodel` to `--hordeconfig`, which now accepts two parameters: the horde display name and the advertised max generation length on horde. • Adds `--blasbatchsize -1` to disable BLAS while retaining CLBlast GPU layer offloading, enabling more layers to be offloaded and faster generation at the cost of slower prompt processing. • Adds support for MPT models; use `--contextsize` to set context length (up to ~5000 tokens tested). • Enhances the KoboldCpp Easy Launcher GUI with controls for the number of CLBlast GPU layers, thread count, and additional toggles. • Improves CLBlast memory allocation efficiency, allowing more layers to be offloaded to GPU than before. Breaking changes: • The `--renamemodel` flag has been renamed to `--hordeconfig`; any scripts or configs using `--renamemodel` will break on upgrade.

KoboldCpp v1.27

Thu, 01 Jun 2023 03:13:41 GMT

KoboldCpp v1.27 adds `--renamemodel`, RWKV v101 support, AutoGenerated Memory in Kobold Lite, and CL BLAST VRAM-only layer offloading. • Adds `--renamemodel` CLI flag to override the displayed model name (prefixed with `koboldcpp/`), useful for Horde operators. • Supports RWKV.cpp v101 model format (updated GGML library, smaller and faster), with automatic backwards-compatible detection of both old and new quantization formats. • Adds EOS token support for RWKV models. • Integrates CLBlast GPU offloading improvement that stores layers fully in VRAM without a duplicate RAM copy, reducing overall RAM usage when offloading GPU layers. • Adds AutoGenerated Memory feature in Kobold Lite, which generates a short story summary/memory with a single click (works best with instruct models). • Pulls upstream support for OpenLlama 3B models.

KoboldCpp v1.26b

Sat, 27 May 2023 10:06:31 GMT

KoboldCpp v1.26b adds token probability inspection via `--debugmode`, a new Top-A sampler, and Starcoder model support. • Adds token probability display to `--debugmode`: for every generated token, the console shows probabilities of up to 4 alternative tokens after all samplers are applied, enabling sampler configuration testing and model confidence analysis. • Adds `--debugmode` display of input/context contents and their token IDs (note: slight performance hit; off by default). • Adds `--unbantokens` flag to enable EOS stop tokens across all model types; the `[` token is also no longer banned by default. • Adds the Top-A sampler, a Kobold-exclusive implementation not present in upstream llama.cpp, which reduces randomness proportionally to the squared softmax probability of the most probable token (set to `0` to disable). • Adds support for Starcoder and Starcoder Chat models.

KoboldCpp v1.25.1

Wed, 24 May 2023 07:34:13 GMT

KoboldCpp v1.25.1 adds Failsafe mode for AVX-less hardware, sampler_seed API support, and streaming length control in Lite UI. • Adds Failsafe mode via `--noavx2 --noblas --nommap` flags, disabling all CPU intrinsics (AVX, SSE) to run on ancient hardware including Windows 7. • Adds `sampler_seed` parameter support to the `/generate` API endpoint for reproducible generation. • Adds `?streamamount=[value]` URL parameter to Kobold Lite UI for controlling variable streaming lengths (default: 8 tokens) when launched with `--stream`. • Expands `--debugmode` console logging to display context token contents. • Adds drag-and-drop file load functionality to Kobold Lite UI. • Greatly improved markdown rendering support in Kobold Lite UI. • Upgrades CLBlast to version 1.6.

KoboldCpp v1.24

Sun, 21 May 2023 12:17:53 GMT

KoboldCpp v1.24 adds GGJT v3 quantization support (q4_0, q4_1, q8_0) and new Kobold Lite UI toggles. • Supports the new GGJT v3 quantization format, including `q4_0`, `q4_1`, and `q8_0` quantization types, while retaining backwards compatibility with all historical GGML formats (GGML, GGHF, GGJT v1, v2, v3). • Adds a toggle in Kobold Lite to avoid inserting newlines in Instruct mode, useful for Pygmalion and OpenAssistant based instruct models. • Adds a toggle in Kobold Lite to enable basic markdown rendering in Instruct mode (off by default). • Provides an alternative CUDA build (via Henky) for this version, enabling access to the latest GGJT v3 quantizations for CUDA users on LLAMA-based models.

KoboldCpp v1.23.1

Wed, 17 May 2023 13:36:48 GMT

KoboldCpp v1.23.1 adds GPT-2/J/NeoX quantization support and experimental OpenCL GPU offloading via CLBlast. • Adds experimental OpenCL GPU offloading via CLBlast using `--useclblast` combined with `--gpulayers ` to select the number of layers to offload; works on all GPUs for new quantization formats of LLAMA models. • Extends GPU offloading support to q8 quantization formats after pulling the q8 dequant kernel fix. • Adds support for new quantization formats for GPT-2, GPT-J, and GPT-NeoX models.

KoboldCpp v1.20

Mon, 08 May 2023 13:19:54 GMT

KoboldCpp v1.20 adds `--contextsize` flag for large contexts and `--forceversion` for RedPajama/GPT-NeoX model support. • Adds `--contextsize` flag to allocate additional RAM and support context sizes beyond 2048 tokens. • Adds `--forceversion` flag to manually override model version detection when automatic RedPajama identification fails. • Adds experimental support for RedPajama variants of GPT-NeoX models.

KoboldCpp v1.19

Sat, 06 May 2023 04:12:57 GMT

KoboldCpp v1.19 adds mirostat sampling, force-version override, CLBlast thread control, and expanded RWKV quantization support. • Adds `--usemirostat [type] [tau] [eta]` flag to enable mirostat sampling on all model types, replacing normal stochastic samplers; e.g. `--usemirostat 2 5.0 0.1` for mirostat type 2. • Adds `--forceversion [ver]` flag to override automatic model file format detection when it fails, e.g. `401` for GPTNeoX-Type2. • Adds `--blasthreads` flag to set a separate thread count when CLBlast is active, defaulting to the value of `--threads` if not specified. • Expands RWKV support to include all new RWKV quantizations, with `q5_1` delivering significantly faster inference than `fp16` at similar quality. • Includes an experimental Windows 7-compatible `.exe` build for this release. Breaking changes: • RWKV `Q4_1_O` quantization is no longer supported following the upstream change.

KoboldCpp v1.18

Tue, 02 May 2023 14:52:29 GMT

KoboldCpp v1.18 adds group chat conversations, `--highpriority` CPU boost flag, and `--usemlock` RAM-pinning for Apple M1. • Adds `--highpriority` CLI flag to raise the process CPU priority, potentially reducing generation latency. • Adds `--usemlock` CLI parameter to pin the model in RAM, targeting Apple M1 users. • Adds Group Conversations to Kobold Lite chat mode: specify multiple chat opponents delimited with `||$||` (up to 10 custom stopping sequences) so the AI replies as different participants; works best with Multiline Replies disabled and on LLAMA-based models. • Adds a new built-in scenario `Class Reunion` in Kobold Lite to demonstrate group chat functionality.

KoboldCpp v1.17

Mon, 01 May 2023 16:57:34 GMT

KoboldCpp v1.17 adds `--unbantokens` flag to unban tokens including EOS, required for newer Pygmalion models. • Adds `--unbantokens` CLI flag to unban tokens during generation, including prevention of EOS token generation, enabling compatibility with newer Pygmalion models. • Exposes Token Unbanning in the UI, allowing it to be configured without CLI flags.

KoboldCpp v1.16

Sun, 30 Apr 2023 06:31:16 GMT

KoboldCpp v1.16 adds Tail Free Sampling and Typical Sampling, plus CLBlast support for q5_0 and q5_1 formats. • Adds CLBlast GPU acceleration support for the `q5_0` and `q5_1` quantization formats. • Adds two new token samplers: Tail Free Sampling (TFS) and Typical Sampling, available alongside the reworked Top-P, Top-K, and Rep Pen samplers. • Unifies sampling functions across all model architectures and types under a single overhauled sampling system. Breaking changes: • Upstream llama.cpp has completely removed support for the `q4_3` format; users are strongly advised to switch away from `q4_3` and reconvert any existing `q4_3` models.

KoboldCpp v1.15

Fri, 28 Apr 2023 07:23:50 GMT

KoboldCpp v1.15 adds an Easy Mode GUI, `--debugmode` flag, q5_0/q5_1 quantization, and multi-sequence stop strings. • Adds `--skiplauncher` flag to bypass the new Easy Mode GUI and proceed directly to CLI operation. • Adds `--debugmode` flag to print the tokenized prompt sent to the backend in the terminal window. • Setting `--stream` now automatically redirects the embedded Kobold Lite UI to streaming mode, removing the need to manually append `?streaming=1` to the URL. • Introduces a new Easy Mode GUI launcher that activates when no command-line arguments are provided, offering a guided setup for first-time users. • Adds q5_0 and q5_1 quantization format support for llama.cpp, GPT-2, GPT-J, and GPT-NeoX model formats (OpenBLAS supported; CLBlast not yet supported). • Kobold Lite UI now supports multiple custom stopping sequences, separated by the `||$||` delimiter, with sequences saved to save files and autosaved.

KoboldCpp v1.14

Wed, 26 Apr 2023 15:38:05 GMT

KoboldCpp v1.14 adds backwards compatibility for older NeoX quantizations and bundles non-AVX2 support in a single executable. • Adds backwards compatibility for an older version of NeoX with different quantizations. • Bundles non-AVX2 CPU support inside the same `.exe`, selectable with the `--noavx2` flag. • Supports GPU acceleration via CLBlast with the `--useclblast` flag. • Supports `--smartcontext` flag to reduce prompt processing frequency for large contexts.

KoboldCpp v1.13.1

Mon, 24 Apr 2023 13:55:33 GMT

KoboldCpp v1.13.1 adds `--unbantokens` flag and expands CLBlast GPU quantization support with up to 50% faster prompt processing. • Adds `--unbantokens` CLI flag to allow previously banned tokens such as EOS and Square Brackets to be generated. • Adds CLBlast dequantization support for `q4_2` and `q4_3` quantization formats, enabling GPU-accelerated inference for models using those formats. • Adds quantization handling for GPT-NeoX, GPT-2, and GPT-J model architectures. • Makes `mmap` automatic when a LoRA adapter is selected.

KoboldCpp v1.11

Sat, 22 Apr 2023 08:36:08 GMT

KoboldCpp v1.11 adds GPT-NeoX/Pythia/StableLM support, `--lora` for llama, and multi-backend build improvements. • Adds `--lora` parameter to enable LORA file support for llama models. • Adds GPT-NeoX, Pythia, and StableLM model architecture support. • Adds limited fast-forwarding for RWKV, allowing context reuse when the context is completely unmodified. • Kobold Lite UI now supports a custom stopping sequence, configurable in the Memory panel. • Improved OSX and Linux builds now compile multiple acceleration backends (e.g. `make LLAMA_OPENBLAS=1 LLAMA_CLBLAST=1`) and allow selecting between them at runtime via flags such as `--useclblast`.

KoboldCpp v1.10

Tue, 18 Apr 2023 13:52:26 GMT

KoboldCpp v1.10 adds RWKV model support, a browser-launch flag, and a new version endpoint. • Adds `--launch` CLI flag to automatically open the browser on startup (combinable with existing flags, e.g. `--stream --launch`). • New `/api/extra/version` endpoint returns the running KoboldCpp version number. • Adds native RWKV model support with no external dependencies — no PyTorch or tokenizers libraries required.

KoboldCpp v1.9

Sun, 16 Apr 2023 14:44:07 GMT

KoboldCpp v1.9 adds API stopping sequences support and BLAS mode for GPT-J and GPT2 models. • Adds stopping sequences support to the API, allowing generation to halt early when a stop sequence is matched and return the response immediately without consuming remaining tokens. • GPT-J and GPT2 models now support BLAS mode for faster inference, using a smaller batch size than LLaMA models.

KoboldCpp v1.8.1

Sun, 16 Apr 2023 08:01:08 GMT

KoboldCpp v1.8.1 brings ~20% CLBlast speed boost via GPU-side 4-bit dequantization. • CLBlast now performs 4-bit dequantization on the GPU (via `--useclblast [platform_id] [device_id]`), delivering approximately 20% faster inference for CLBlast users.

KoboldCpp v1.7

Fri, 14 Apr 2023 14:39:56 GMT

KoboldCpp v1.7 adds `--smartcontext` to avoid frequent prompt context recalculation. • Adds `--smartcontext` flag, a prompt context manipulation mode that avoids frequent context recalculation when the context window is full. • Adds `--noavx2` flag to enable a non-AVX2 execution path, now bundled in the same `.exe`.

KoboldCpp v1.6

Thu, 13 Apr 2023 06:44:22 GMT

KoboldCpp v1.6 adds GPU name display and bundles non-AVX2 support into a single executable. • Bundles the non-AVX2 build into the same `koboldcpp.exe`, enabled via the `--noavx2` flag — no separate download needed on older hardware.

KoboldCpp v1.5

Wed, 12 Apr 2023 16:03:02 GMT

KoboldCpp v1.5 adds AVX2/non-AVX2 unified binary and experimental CLBlast GPU acceleration for prompt processing. • Adds `--useclblast [platform_id] [device_id]` flag to enable experimental CLBlast GPU acceleration for faster prompt processing. • Adds `--noavx2` flag to switch to compatibility mode on CPUs without AVX2 support, now bundled into the same binary instead of a separate download. • Includes quantization tools in `tools.zip` for converting fp16 models to quantized format.

KoboldCpp v1.4

Mon, 10 Apr 2023 16:36:09 GMT

KoboldCpp v1.4 makes mmap the default and adds `--nommap` to opt out. • Adds `--nommap` flag to disable memory-mapped file loading, which is now enabled by default. Breaking changes: • mmap is now enabled by default; existing setups that relied on mmap being off will need to add `--nommap` to preserve previous behavior.

KoboldCpp v1.3

Mon, 10 Apr 2023 04:14:25 GMT

KoboldCpp v1.3 adds `--usemmap` flag, auto-detection of GPTJ/GPT2 quantized file versions, and startup version display. • Adds `--usemmap` flag to opt back in to memory-mapped file loading, which is now disabled by default following upstream enhancements. • Automatically distinguishes between older and newer GPTJ and GPT2 quantized model files, removing the need for manual format selection. • Displays version numbers at startup for easier identification of the running build. Breaking changes: • `mmap` (memory-mapped file loading) is now disabled by default; existing setups relying on mmap behavior must explicitly pass `--usemmap` to restore it.

KoboldCpp v1.2

Sat, 08 Apr 2023 17:33:31 GMT

KoboldCpp v1.2 adds support for newer GPT-2 model variants including Cerebras models from Hugging Face. • Supports new versions of GPT-2 models, including Cerebras models hosted on Hugging Face.

KoboldCpp v1.0.9beta

Wed, 05 Apr 2023 08:16:52 GMT

KoboldCpp v1.0.9beta adds GPT-2 model support and Alpaca Instruct Mode in the embedded Kobold Lite UI. • Adds GPT-2 model support (including theoretical compatibility with Cerebras models), enabling inference on very small ggml models at high token throughput on CPU. • Adds Stanford Alpaca-compatible Instruct Mode to the embedded Kobold Lite interface, enabling structured prompt/response formatting — configurable in Kobold Lite settings. • Adds repetition penalty (Rep Pen) support for GPT-J and GPT-2 models (and pyg.cpp), bringing penalty behavior in line with llama.cpp.

KoboldCpp v1.0.8beta

Mon, 03 Apr 2023 03:58:16 GMT

KoboldCpp v1.0.8beta adds GPT4ALL.CPP and GPT-J format support and boosts generation speed with `-Ofast`. • Adds support for the original GPT4ALL.CPP model format. • Adds support for GPT-J formats, including the original 16-bit legacy format and the 4-bit version from Pygmalion.cpp. • Switches compiler optimization flag from `-O3` to `-Ofast`, increasing token generation speed. • Changes default thread count to scale by physical core count rather than os.cpu_count(), with manual override available via `--threads`. Breaking changes: • Library file names and references are renamed as part of the rebranding from llamacpp-for-kobold to koboldcpp — any scripts or integrations referencing the old library names will break.

KoboldCpp v1.0.7

Sat, 01 Apr 2023 01:11:04 GMT

KoboldCpp v1.0.7 adds ggjt model format support, streaming opt-in via `--stream`, and richer console generation progress. • Adds `--stream` flag to opt into streaming mode; non-streaming is now the default. • Supports the new ggml llama.cpp model format (`magic=ggjt`, version 3) while retaining backward compatibility with all older versions. • Improved console debug output during generation now shows token progress and time taken in real time. Breaking changes: • Non-streaming mode is now the default; setups that relied on streaming behavior must now explicitly pass `--stream` to restore it.

KoboldCpp v1.0.6beta

Wed, 29 Mar 2023 17:13:57 GMT

KoboldCpp v1.0.6beta adds OpenBLAS acceleration and a `--noblas` flag, doubling initial prompt processing speed. • Adds `--noblas` flag to disable OpenBLAS acceleration at runtime. • Adds `--help` flag via switch to argparse, exposing all command-line options. • Integrates OpenBLAS to accelerate initial prompt processing by over 2x on compatible systems. • Updates Embedded Kobold Lite with pseudo token streaming for a more responsive generation UI.

KoboldCpp v1.0.5

Sat, 25 Mar 2023 03:29:17 GMT

KoboldCpp v1.0.5 adds a softprompts endpoint and selectable KV data types. • Adds `/api` softprompts endpoint, enabling softprompt injection via the KoboldAI-compatible API at `http://localhost:5001`. • Adds support for selecting KV cache data type, defaulting to `f32` instead of `f16`. • Caps maximum thread count at 4 to improve throughput on memory-bottlenecked inference workloads. Breaking changes: • The default KV cache data type changes from `f16` to `f32`, which increases memory usage for existing setups.

KoboldCpp v1.0.4

Fri, 24 Mar 2023 14:22:15 GMT

KoboldCpp v1.0.4 adds prompt token caching for faster generation and standalone PyInstaller executables for distribution. • Adds token caching for prompts, enabling fast-forward through partially duplicated prompt prefixes so edits near the end of a previous prompt regenerate significantly faster. • Introduces a standalone PyInstaller-built `llamacpp_for_kobold.exe` for all future releases, supporting drag-and-drop model loading or interactive model selection via a popup dialog.

KoboldCpp v1.0.3

Wed, 22 Mar 2023 14:50:46 GMT

KoboldCpp v1.0.3 adds dynamic context length support and reduces default batch sizes for better output quality. • Supports dynamic context lengths sent from the client, allowing context size to vary per request. • Reduces default batch sizes to lower memory usage and improve output quality.

KoboldCpp v1.0.2

Tue, 21 Mar 2023 13:19:15 GMT

KoboldCpp v1.0.2 embeds Kobold Lite UI and updates to the new ggml model format while retaining backward compatibility. • Embeds Kobold Lite directly in the server, accessible at `http://localhost:5001` after running `llama_for_kobold.py [ggml_quant_model.bin] [port]`. • Supports the new ggml model format while maintaining backward compatibility with the old ggml format and old tokenizer.