LocalAI — The AI Toolchain

LocalAI v4.9.0

Thu, 20 Aug 2026 15:59:49 GMT

LocalAI 4.9.0: deny-by-default auth, chat context compression, KNN routing, PII pseudonyms, parallel HF downloads, and MiniMax-H3 video+audio • Adds `--web-rtc-udp-port` CLI flag and `LOCALAI_WEBRTC_UDP_PORT` env var to reuse a single shared Pion ICE UDP mux across all Realtime WebRTC calls, simplifying container and firewall setup. • Adds `POST /api/router/{name}/corpus`, `GET /api/router/{name}/corpus/stats`, and `DELETE /api/router/{name}/corpus` endpoints to seed and manage the KNN router's labelled prompt corpus (admin-gated, also exposed as MCP tools). • Introduces `classifier: knn` router type: similarity-weighted voting over a persisted JSONL corpus under `/router-corpus`, with a configurable `knn.similarity_threshold`; prompts below threshold yield an undecidable fallback rather than a forced guess. • Adds opt-in `pii.reverse_in_response` config to turn masked PII values into request-scoped deterministic pseudonyms (`EMAIL_001`, `EMAIL_002`) that are restored when the backend echoes them, including across SSE tokens split over writes. • Adds opt-in per-model `compression` config for chat context compression: older complete turns are compressed through a configured LocalAI model before inference, preserving system prompts, newest messages, and whole tool-call/result units; ratio and duration are returned as response metadata and metrics. • Authentication is now deny-by-default: every HTTP route requires credentials unless listed in an explicit public registry, closing bypasses for unprefixed aliases such as `/moderations`, `/models`, `/backends`, and `/mcp/chat/completions`; narrow public paths can be added via `ApplicationConfig.PathWithoutAuth`. • Adds MiniMax-H3 video+audio generation to `vllm-cpp` (ABI v12): a second engine handle loads the H3 checkpoint set (`parameters.model` for the DiT; text encoder and two VAEs named in `options:`), and `GenerateVideo` renders jointly into an MP4 with a real AAC audio track. • Adds Qwen3-TTS to the `llama-cpp` backend across CUDA, ROCm, SYCL, Vulkan, Metal, and L4T, using upstream GGUF conversion; gallery entries `qwen3-tts-llamacpp` and `qwen3-tts-llamacpp-q4` provided. • Expands `vllm-cpp` CUDA architecture coverage on amd64 from `120a;121a` to `80;86;89;90a;100a;103a;120a;121a`, and on arm64 from `121a` to `87;90a;100a;110;121a`, adding support for A100, L4, 4090, H100/H200, B200, Jetson Orin, and Jetson Thor. • Adds global process-wide HTTP admission control bounding in-flight backend operations, with running backend traces now visible in the UI alongside direct log links. • Cold model loads now run as durable jobs rather than holding the per-model advisory lock across multi-GB transfers, preventing loads from appearing permanently broken during staging. • Unifies the model and backend UI: `/app/models` owns Explore and Installed views; `/app/backends` owns Catalog and Installed; `/app/manage` redirects with legacy query state preserved. • Adds Portuguese (Brazil) (`pt-BR`) UI translation with full 14-namespace key parity, and Indonesian translation covering admin, media, and navigation surfaces. • Adds 85 new model gallery entries including Qwen3 (9B, 27B), Gemma 4 Scotoma 2, DeepSeek V4 Pro 0813, Nemotron 3.5 Lightning 30B, MiniMax-H3 Ref2VA (`minimax-h3-fl2va-q4`, `minimax-h3-ref2va-q4`), Higgs Audio v3 TTS, and others. Breaking changes: • With database auth or legacy API keys configured, `/version` and generated audio, image, video, and 3D asset URLs now require credentials by default due to the deny-by-default authentication change. • Embedded deployments that previously relied on unauthenticated access to routes like `/moderations`, `/models`, `/backends`, or `/mcp/chat/completions` must explicitly add those paths to `ApplicationConfig.PathWithoutAuth`.

LocalAI v4.8.2

Fri, 07 Aug 2026 15:53:43 GMT

LocalAI v4.8.2 adds the NVIDIA NeMo-Speech.cpp backend and improves gallery index resilience with mirror fallback. • Adds the NVIDIA NeMo-Speech.cpp backend for speech processing. • Gallery now defaults to `index.localai.io` as the primary index source, with GitHub as a mirror fallback when the primary source fails.

LocalAI v4.8.2

Fri, 07 Aug 2026 15:53:43 GMT

LocalAI v4.8.2 adds the NVIDIA NeMo-Speech.cpp backend and gallery mirror/offline fallback support. • Adds `NEMO_SPEECH_ASR_LIBRARY`, `NEMO_SPEECH_TTS_LIBRARY`, and `NEMO_SPEECH_NMT_LIBRARY` environment variables to override shared-library paths for the new NVIDIA NeMo-Speech.cpp backend, which provides ASR, TTS, and NMT capabilities via a gRPC server. • Adds `mirrors` list to gallery configuration so LocalAI tries the primary URL, then each mirror in order, keeping failed sources out of rotation for 10 minutes; if all sources fail, the last successfully fetched index is served from disk. • New NVIDIA NeMo-Speech.cpp backend (`nemo-speech-cpp`) supports ASR, TTS, diarization, punctuation/capitalization, inverse text normalization, and neural machine translation, configured via `key:value` options including `vad_model`, `pnc_model`, `diar_model`, `itn_dir`, `language_code`, `codec_model`, `tokenizer_dir`, `tn_dir`, `source_language`, and `target_language`. • Gallery index now defaults to `index.localai.io` with GitHub as a mirror, improving resilience for air-gapped or rate-limited environments.

LocalAI v4.8.1

Thu, 06 Aug 2026 15:36:06 GMT

LocalAI v4.8.1 adds Qwen3.5 9B gallery models and exposes the full vllm-cpp engine config surface via `engine_args`. • Exposes the full vllm-cpp engine configuration surface through `engine_args`, allowing fine-grained control over backend engine parameters. • Adds Qwen3.5 9B HauhauCS variants to the model gallery. • Adds Qwen3.5 9B Defiant Fable variants to the model gallery.

LocalAI v4.8.1

Thu, 06 Aug 2026 15:36:06 GMT

LocalAI v4.8.1 adds full vLLM engine config via `engine_args` and two Qwen3.5 9B gallery variants. • Adds `engine_args` config key to the vLLM-cpp backend, accepting a JSON object whose keys mirror vLLM's own CLI flags verbatim — including `speculative_config` and `kv_transfer_config` — so existing vLLM configs work without translation; `engine_args` wins over the older `options:` list when both set the same key. • Adds Qwen3.5 9B HauhauCS and Qwen3.5 9B Defiant Fable variants to the model gallery.

LocalAI v4.8.0

Wed, 05 Aug 2026 10:24:01 GMT

LocalAI 4.8.0 adds vllm.cpp alpha backend, 3D generation, audio.cpp, VRAM budgets, two new TTS engines, and more • Adds `POST /v1/3d/generations` endpoint, `Generate3D` RPC, and `FLAG_3D` capability for 3D generation as a new modality, backed by the `trellis2cpp` image-to-3D backend and a UI page with a native GLB viewer and print remeshing. • Adds `LOCALAI_VRAM_BUDGET` environment variable and `--vram-budget` CLI flag (accepts `80%` or `12GB`) to cap GPU memory use per node, with admin API endpoints `PUT /api/nodes/:id/vram-budget` and `DELETE /api/nodes/:id/vram-budget`, and an `set_node_vram_budget` MCP tool. • Adds `--disable-http-compression` and `--http-compression-min-length` flags; `/api/traces` and `/api/backend-traces` now accept `limit`, `offset`, and `full` query parameters with per-record access via `GET /api/traces/{id}`. • Adds `GET /api/models?has_variants=true` to filter gallery models with variant ladders, with variant selection via `variant` on `POST /models/apply`, `local-ai models install --variant`, the `install_model` MCP tool, or the UI split-button. • Ships `vllm-cpp` as an alpha backend: a C++20 engine (no Python, PyTorch, or ggml at inference) implementing vLLM V1 architecture (paged KV cache, continuous batching, prefix caching), loading safetensors and GGUF, with in-engine structured output (JSON schema, regex, GBNF), running on CPU, CUDA 12/13, Metal, and Vulkan. • Ships `audio-cpp` backend serving six endpoints — `/v1/audio/speech`, `/v1/audio/transcriptions`, `/v1/audio/vad`, `/v1/audio/diarize`, `/audio/transform`, and `/v1/sound-generation` — with family auto-detected from the GGUF `audiocpp.model_spec.family` metadata key. • Ships `magpie-tts-cpp` backend for NVIDIA Magpie TTS Multilingual (5 voices, 9+ languages, 22.05 kHz mono, single self-contained GGUF). • Ships `moss-tts-cpp` backend for MOSS-TTS-Local v1.5 (48 kHz stereo, optional reference-audio voice cloning, no Python at inference). • Ships `bonsai` backend for Q1_0 and Q2_0 quant formats, covering Bonsai 8B/27B and Ternary-Bonsai 8B/27B models via eight gallery entries. • Ships `valkey-store` backend adding Valkey Search as a vector store option. • Adds a new admin `/app/activity` page tracking install, failure, and cancellation events with a 50-entry bounded history, per-node in-progress breakdown, and a 'needs attention' lane with Cancel and Retry actions. • Gallery entries can now declare `variants:` so LocalAI automatically selects the largest build that fits the host's VRAM and RAM at install time. • Adds Hugging Face artifact materialization with immutable snapshot resolution, authenticated downloads with progress reporting, and staged artifact binding for remote workers. • Adds systemd socket activation support. • Inline GRPO reward code in `POST /api/fine-tuning/jobs` is now blocked by default; operators must set `LOCALAI_TRL_ALLOW_INLINE_REWARD=true` to re-enable it.

LocalAI v4.8.0

Wed, 05 Aug 2026 10:24:01 GMT

LocalAI 4.8.0 adds vllm.cpp, 3D generation, audio.cpp, VRAM budgets, gallery variants, two new TTS engines, bonsai sub-2-bit models, and a Valkey vector store. • Adds `POST /v1/3d/generations` endpoint, `Generate3D` RPC, and `FLAG_3D` capability for a new 3D generation modality, backed by the `trellis2cpp` image-to-3D backend and a UI page with a native GLB viewer and print remeshing. • Adds `LOCALAI_VRAM_BUDGET` environment variable (e.g. `80%` or `12GB`) and `--vram-budget` CLI flag on `local-ai` and `local-ai worker` to cap GPU memory use per node; overridable via `PUT`/`DELETE /api/nodes/:id/vram-budget` and the `set_node_vram_budget` MCP tool. • Adds `GET /api/traces/{id}` endpoint and `limit`/`offset`/`full` query parameters on `GET /api/traces` and `GET /api/backend-traces` for paginated, summarized trace access. • Adds `--disable-http-compression` and `--http-compression-min-length` flags to control gzip response compression; streaming paths are skipped automatically. React bundle shrinks 3.48x (2.8 MB to 808 KB); backend-trace poll drops from 21 MB to 7 KB. • Adds `variants:` key to gallery entries so LocalAI selects the largest build that fits the host's hardware and VRAM at install time; override with `variant` on `POST /models/apply`, `local-ai models install --variant`, the `install_model` MCP tool, or the UI split-button. • Adds `GET /api/models?has_variants=true` query parameter to filter the model list to entries that declare variants. • Adds the `audio-cpp` backend serving six audio endpoints from one process — `/v1/audio/speech`, `/v1/audio/transcriptions`, `/v1/audio/vad`, `/v1/audio/diarize`, `/audio/transform`, and `/v1/sound-generation` — with model family selected from the GGUF's own `audiocpp.model_spec.family` metadata key. • Adds the `vllm-cpp` backend (alpha development builds) — a C++20 engine with paged KV cache, continuous batching, prefix caching, structured output, safetensors and GGUF loading — running on CPU, CUDA 12/13, Metal, and Vulkan with no Python or PyTorch at inference. • Adds the `magpie-tts-cpp` TTS backend for NVIDIA Magpie TTS Multilingual 357M: 5 voices, 9+ languages, 22.05 kHz mono, single self-contained GGUF. • Adds the `moss-tts-cpp` TTS backend for MOSS-TTS-Local v1.5: 48 kHz stereo output with optional reference-audio voice cloning, no Python at inference. • Adds the `bonsai` backend (via PrismML llama.cpp fork) as the only decoder for Q1_0 and Q2_0 quant formats, covering Bonsai 8B/27B and Ternary-Bonsai 8B/27B (~1.15 GB and up). • Adds the `valkey-store` backend, making Valkey Search available as a vector store option. • Adds an admin `/app/activity` page with per-node in-progress breakdown, a 'needs attention' lane with Cancel and Retry, and a bounded 50-entry history of completed, failed, and cancelled operations. • Adds Hugging Face artifact materialization with immutable snapshot resolution, authenticated downloads with progress reporting, staged artifact binding for remote workers, and Python backend reuse of the Go download path. • Adds `LOCALAI_TRL_ALLOW_INLINE_REWARD=true` environment variable required to permit inline GRPO reward code in `POST /api/fine-tuning/jobs`; inline reward code is refused by default. • Adds systemd socket activation support. • Adds gallery entries for MiniMax-M3, Gemma 4 llama.cpp MTP variants, Qwen3.5-4B DFlash, MOSS-TTS-Local v1.5, APEX families as variant ladders, and Bonsai families, growing the gallery from 1,221 to 1,515 entries. • Hardens distributed mode: reaper now queries workers directly over a `models.running` subject before terminating a backend, port-probe fallback requires three consecutive misses to declare a node gone, frontend model stubs are dropped only when no healthy replica remains, and model-load deadlines scale with checkpoint size.

LocalAI v4.7.0

Tue, 14 Jul 2026 13:24:24 GMT

LocalAI 4.7.0 adds voice cloning profiles, local video/avatar generation, diarized transcription, streaming TTS, and auto full-context resolution. • Adds `GET /api/voice-profiles`, `POST /api/voice-profiles`, `DELETE /api/voice-profiles/:id`, and `GET /api/voice-profiles/:id/audio` REST endpoints for admin-managed voice cloning profiles; profiles are referenceable via stable `localai://voice-profiles/` URIs from `/v1/audio/speech` or the TTS UI. • Adds MCP admin tools `list_voice_profiles`, `create_voice_profile`, and `delete_voice_profile` for voice profile management. • Adds typed `tts.voice_cloning` boolean config key to opt custom model names into or out of Voice Library compatibility; setting it to `false` rejects saved profile references with HTTP 400. • Adds `context_size: -1` sentinel (any negative value, or `LOCALAI_CONTEXT_SIZE=-1` / `--context-size -1` globally) to resolve a model's context window to its full trained `n_ctx_train` from GGUF metadata at load time, with a VRAM-fit warning and a safe clamp so no backend receives a negative value. • Adds `devices:` option to the llama.cpp `options:` array (e.g., `options: [devices:CUDA1,CUDA2]`) to restrict GPU offload to specific named devices from `llama-server --list-devices`. • Adds `known_input_modalities` and `known_output_modalities` config fields for declarative capability metadata, allowing generic code to discover what a model checkpoint accepts without branching on backend or checkpoint names. • Accepts `reasoning_content` as an inbound alias for the canonical `reasoning` field in OpenAI chat messages (vLLM/DeepSeek/cogito-style clients), enabling interleaved reasoning and tool calls in a single assistant turn. • Supports Anthropic Messages `thinking` blocks on the local path: inbound `thinking` blocks parse into message reasoning and are emitted before `tool_use` on streaming and non-streaming responses, gated on `thinking: {type: 'enabled'}` in the request. • New `longcat-video` Python backend supports text-to-video, image-to-video, and audio-driven talking-avatar generation; gallery entries `longcat-video` and `longcat-video-avatar-1.5` available for CUDA 12/13 x86_64 and CUDA 13 ARM64. • New `moss-transcribe-cpp` Go backend performs joint multi-speaker transcription, diarization, and timestamps in a single offline pass; gallery model `moss-transcribe-cpp-0.9b` (default `q5_k` GGUF), 1.6–2.2x faster than reference on CPU. • Links F5-TTS static runtime into the CrispASR build; new `f5-tts-crispasr` gallery model produces 24kHz mono audio with voice cloning via `options: [voice:/path/ref.wav, voice_text:Transcript...]`. • Replaces whole-clip-then-chunk synthesis in vibevoice-cpp with real incremental streaming via the new `vv_capi_tts_stream` callback ABI, reducing time-to-first-audio from 39.96s to 2.38s (~17x) on CPU for `VibeVoice-Realtime-0.5B`. • Adds model-load failure cooldown: failed loads return `503 + Retry-After` during a backoff window (default `10s`, geometric growth capped at 5 minutes) instead of respawning a crashing backend and leaking GPU memory. • Adds four Qwen DFlash speculative-decoding gallery entries (4B, 9B, 27B, and 35B-A3B), each bundling target and drafter models with `spec_type:draft-dflash`.

LocalAI v4.6.2

Mon, 06 Jul 2026 19:41:52 GMT

LocalAI v4.6.2 adds MiniCPM series models to the model gallery. • Adds MiniCPM series models to the model gallery.

LocalAI v4.6.1

Mon, 06 Jul 2026 10:57:02 GMT

LocalAI v4.6.1 adds a GET /v1/models/capabilities endpoint and native Prometheus metrics for agent chat runs. • Adds `GET /v1/models/capabilities` API endpoint to query per-model capability metadata. • Adds native Prometheus metrics for agent chat runs.

LocalAI v4.6.0

Sat, 04 Jul 2026 07:20:05 GMT

LocalAI v4.6.0 adds a load API, PII Prometheus counter, chat forking, and AMD ROCm GPU fixes • Adds `POST /backend/load` and `POST /v1/backend/load` endpoints to pre-warm any model (including full pipelines) on demand, complementing the existing `/backend/shutdown`. • Adds `disable_warmup: true` per-pipeline config key to opt out of eager realtime session warm-up. • Adds `LOCALAI_BACKEND_PARENT_WATCH` environment variable (default on, auto-off on Windows) to make spawned backend workers self-terminate when the parent LocalAI process dies. • Adds `LOCALAI_BACKEND_PARENT_WATCH_INTERVAL` environment variable (default `2s`) to control how frequently backend workers poll for parent-process death. • Adds `"force": true` field to `POST /backends/apply` to force re-download of an already-installed backend; installs are now idempotent by default. • Bundles `HIPBLASLT_TENSILE_LIBPATH` export in `llama-cpp` and `turboquant` `run.sh`, enabling full-speed hipBLASLt kernel-tuning data for AMD ROCm backends. • Adds a `load_model` MCP admin tool and 'Load into memory' UI button, both backed by the new `POST /backend/load` endpoint. • Adds `LOCALAI_EXTERNAL_BACKENDS` boot loop idempotency so backends are not re-pulled on every restart unless `force: true` is passed. • Adds conversation-forking to the built-in React chat UI: regenerate any assistant answer, branch a new chat from any turn, duplicate a chat, or copy the whole conversation as Markdown. • Realtime voice sessions now eagerly warm the full pipeline (VAD, ASR, LLM, TTS) at session start, with concurrent sub-model loading so warm-up time equals the slowest stage, not the sum. • AMD ROCm `rocm-vllm` backend now installs from the ROCm wheel index (`https://wheels.vllm.ai/rocm/`) pinned to CPython 3.12, replacing the CUDA-only PyPI wheel. • AMD ROCm ggml audio backends (`rocm-qwen3-tts-cpp`, `rocm-omnivoice-cpp`, `acestep-cpp`, `vibevoice-cpp`) now build with `-DGGML_HIP=ON` and link the HIP backend for real GPU offload.

LocalAI v4.5.6

Tue, 30 Jun 2026 15:59:18 GMT

LocalAI v4.5.6 adds native GGML voice/face-detect backends, semantic VAD EOU token for realtime, and distributed shared-model staging control via LOCALAI_DISTRIBUTED_SHARED_MODELS. • Adds `LOCALAI_DISTRIBUTED_SHARED_MODELS` environment variable to skip model staging on shared volumes in distributed deployments. • Adds native `voice-detect` and `face-detect` GGML backends, replacing the previous Python insightface/speaker-recognition dependencies. • Adds a Semantic VAD end-of-utterance (EOU) token to the realtime API. • Introduces a `SyncedMap` component in distributed mode and migrates finetune, quantization, and agent-task state to cross-replica synchronisation.

LocalAI v4.5.3

Fri, 26 Jun 2026 23:45:40 GMT

LocalAI v4.5.3 ships signed and notarized macOS binaries for the DMG, app, and server. • Signs and notarizes the macOS DMG, app bundle, and server binary, enabling Gatekeeper-approved distribution on macOS.

LocalAI v4.5.1

Fri, 26 Jun 2026 08:39:53 GMT

LocalAI v4.5.1 adds Metal/MPS support across multiple backends, new llama-cpp MoE options, UI enhancements, and a PII detector environment variable. • Adds `LOCALAI_PII_DEFAULT_DETECTORS` environment variable to load default PII detectors at startup. • Adds `cpu_moe` and `n_cpu_moe` options to llama-cpp configuration, plus generic upstream-flag passthrough for llama-cpp backends. • Adds darwin/Metal (MPS) build support for vllm via vllm-metal (MLX backend). • Adds darwin/Metal builds for liquid-audio and trl backends. • Adds darwin/Metal builds for vision C++/ggml backends: depth-anything, locate-anything, rfdetr-cpp, and sam3-cpp. • Adds darwin/Metal build for the localvqe acoustic echo cancellation backend. • Adds darwin/Metal build for the privacy-filter backend. • Adds single-build `ggml CPU_ALL_VARIANTS` for llama-cpp and turboquant covering x86, arm64, and Apple silicon. • Adds UI enhancements: last-used model tracking, polling, starter models, usage cost display, and accessibility improvements. • Adds data-driven hardware model recommendations and gallery surfacing to the UI. • Makes `PreferDevelopmentBackends` install the development image as the primary image.

LocalAI v4.5.0

Tue, 23 Jun 2026 21:31:26 GMT

LocalAI 4.5.0 adds four new backends (depth-anything, ced, supertonic, privacy-filter.cpp), speaker-aware realtime voice with compaction, and concurrent serving on by default. • New `depth-anything` backend exposes `POST /v1/depth` returning depth map, stats, and camera extrinsics (3×4) / intrinsics (3×3); also reachable via a typed `Depth` gRPC RPC and `GenerateImage`/`Predict` — no Python at inference. • New `ced` backend exposes `POST /v1/audio/classification` for 527-class AudioSet sound tagging (baby cry, glass breaking, alarms); opt-in `pipeline.sound_detection` emits `conversation.item.sound_detection` events in realtime, decoupled from VAD via `turn_detection: none`. • New `supertonic` ONNX TTS backend supports `voice`, `language`, `steps`, `speed`, and `silence` request knobs across 10 voice styles (F1–F5 / M1–M5) — no Python, no espeak-ng/G2P. • New `privacy-filter.cpp` backend adds NER token-classification PII filtering (encoder scans whole conversations as a single document) alongside a restricted-regex secret detector; detections labelled by source (`ner` vs `pattern`) with confidence and `analyze`/`redact` API; request filtering extended to completions, embeddings, edits, and Ollama. • Realtime voice sessions gain `pipeline.voice_recognition` config keys including `enforce: false` to surface speaker identity to the client via a new `conversation.item.speaker` event and feed it to the LLM via per-message `name` field. • Conversation compaction via `pipeline.compaction` config block — keys `enabled`, `trigger_items`, `summary_model`, and `max_summary_tokens` — rolls aged-out turns into a rolling summary instead of dropping them; `pipeline.max_history_items` controls the live verbatim window. • Adds OpenAI-parity realtime history events `conversation.item.delete`, `conversation.item.truncate`, and `input_audio_buffer.clear`. • Prefix caching (`n_cache_reuse`) now defaults to 256 (was 0), so system prompts, RAG context, and multi-turn chat prefixes are reused across requests without configuration. • VRAM-scaled `n_parallel` default enables continuous batching out of the box: ≥32 GiB → 8 slots, ≥8 GiB → 4, ≥4 GiB → 2, <4 GiB → 1 (unchanged); explicit config always wins. • Default `n_batch`/`n_ubatch` raised to 2048 on NVIDIA Blackwell consumer GPUs (sm_120/121, including GB10 / DGX Spark) when `batch:` is unset. • New `alias:` field in model config transparently routes all traffic for one model name to another configured model, swappable live without client reconfiguration; alias→alias chains are rejected at load. • Word-level ASR timestamps supported for NeMo and CrispASR, plumbed through the gRPC transcription path. • Eight `depth-anything` GGUFs ship at `mudler/depth-anything.cpp-gguf` (base/small/large/giant + mono-large, q4_k/q8_0/f16/f32); eight `ced` GGUFs at `mudler/ced-gguf` (6 MB–86 MB); Depth Anything V2 gallery models also added. • Redesigned React UI ships end-to-end: new design language, NER-centric PII editor, detector-models table, middleware default-policy controls, sortable/mobile tables, unsaved-changes guards, and restructured Cluster Nodes view. • Distributed staging detached from the request context so large models finish cold-loading; staging progress broadcast across replicas; resumable downloader added.

LocalAI v4.4.3

Sat, 13 Jun 2026 23:27:53 GMT

LocalAI v4.4.3 adds open-vocabulary object detection, OmniVoice TTS with voice cloning, Kokoro TTS, 60 new Piper voices, and declarative distributed scheduling. • Adds declarative per-model scheduling for distributed deployments, configurable via environment variables and CLI arguments (`feat(distributed): declarative per-model scheduling via env/args`). • Adds `locate-anything-cpp` backend for open-vocabulary object detection via ggml. • Adds `omnivoice-cpp` backend supporting OmniVoice TTS with file and streaming output, voice cloning, and voice design. • Adds Kokoro TTS and multilingual Piper voices to the `sherpa-onnx` backend. • Migrates `qwen3-tts-cpp` to `ServeurpersoCom/qwentts.cpp`, adding streaming, multiple speakers, and voice design support. • Bundles `espeak-ng` with `crispasr` and adds Piper TTS voices to the model gallery. • Expands the gallery with 60 Piper TTS voices across 42 languages (Phase 2). • Enables `GGML_CUDA_GRAPHS` in the CUBLAS build for `parakeet-cpp`, improving GPU inference performance. • Adds voice-recognition gating for realtime pipeline voice models. • Adds Indonesian language support to the React UI. • Adds Korean (`ko`) translation to the UI.

LocalAI v4.4.1

Thu, 11 Jun 2026 16:33:15 GMT

LocalAI v4.4.1 adds streaming across the LLM, TTS, and transcription pipeline stages in the realtime feature. • Streams the LLM, TTS, and transcription pipeline stages in the realtime feature, enabling lower-latency end-to-end audio/language workflows.

LocalAI v4.4.0

Wed, 10 Jun 2026 20:10:10 GMT

LocalAI v4.4.0 adds two ASR backends, object detection, video I/O, distributed prefix-cache routing, PII middleware, and a `local-ai chat` CLI. • Adds `LOCALAI_FILE_TRANSFER_BUDGET` environment variable (default 1h) to control the time budget for resumable large-model GGUF transfers over the distributed worker `PUT /v1/files/` endpoint, with `Content-Range` / 308/416 resume semantics and `X-Content-SHA256` binding. • Adds `--distributed-prefix-cache=false` flag to disable the new prefix-cache-aware routing in distributed mode (on by default), which biases requests toward the replica already holding the relevant KV/prefix cache. • Adds `LOCALAI_PREFETCH_MODELS` environment variable for boot-time gallery model prefetch in distributed deployments. • Plumbs `instructions` string and `params` map through the `/v1/audio/speech` request body end-to-end (proto, schema, `core/backend/tts.go`), enabling per-request voice/style control for Qwen3-TTS VoiceDesign, Qwen3-TTS CustomVoice, and Chatterbox. • New `crispasr` backend (purego, `CGO_ENABLED=0`) serving ASR or TTS from a single binary, auto-detecting architecture from GGUF; supports `backend:`, `codec:`, `speaker:`, and `voice:` model options, with 36 gallery entries (32 ASR + 4 TTS including vibevoice, chatterbox, qwen3-tts, orpheus). • New `rfdetr-cpp` native gRPC backend dlopening `librfdetr.so` for RF-DETR object detection and segmentation, returning bbox + class_name + confidence and per-detection PNG-encoded masks; 32 prebuilt GGUFs at F32/F16/Q8_0/Q4_K. • New `local-ai chat` interactive terminal CLI for chat against a running server, with `/models`, `/model`, and `/clear` commands. • Video input/understanding support in `llama-cpp` via mtmd: chat requests can now include video content parts (`video_url`), with `allow_video` auto-gated by whether the loaded mmproj supports it. • Video generation via LTX-2 in `stablediffusion-ggml`, wiring `audio_vae_path` and `embeddings_connectors_path`; new `gallery/ltx-ggml.yaml` template with T2V/I2V/FLF2V recipes and six LTX-2.3 22B GGUF gallery entries (dev + distilled, UD-Q4_K_M/Q4_K_M/Q8_0). • New intelligent middleware layer (#9802) for capability-based model routing, PII detection/redaction/rerouting on `/v1/chat/completions`, Anthropic `/v1/messages`, and `/v1/completions`, and cloud-model proxying including a MITM proxy for Claude Code/Codex OAuth subscriptions; emits `proxy_connect` and `proxy_traffic` audit events and restores listener state from `runtime_settings.json` on restart. • New `ds4-worker` binary for manual layer-split distributed inference via `local-ai worker ds4-distributed`; coordinator owns layers `0:K`, workers own higher ranges, controlled by `ds4_role` config key. • Adds gated `X-LocalAI-Node` response header for node attribution in distributed deployments. • Adds Ideogram4 support in `stablediffusion-ggml`. • RAG agent answers now append a `Sources:` citations block from the Knowledge Base. • LocalVQE updated to v1.3 with input/output spectrogram views in the Audio Transform UI.

LocalAI v4.3.6

Sat, 30 May 2026 21:23:53 GMT

LocalAI v4.3.6 adds NVIDIA NeMo Parakeet ASR backend and hardens outbound HTTP redirect handling. • Adds NVIDIA NeMo Parakeet ASR backend (`parakeet-cpp`) for automatic speech recognition via the `parakeet.cpp` integration. • Hardens outbound HTTP clients to refuse redirects, reducing server-side request forgery exposure via `pkg/httpclient`.

LocalAI v4.3.5

Fri, 29 May 2026 23:18:59 GMT

LocalAI v4.3.5 adds per-request `reasoning_effort` control on chat completions. • Supports `reasoning_effort` as a per-request parameter on chat completions, letting callers tune reasoning depth without changing model config.

LocalAI v4.3.2

Wed, 27 May 2026 22:06:52 GMT

LocalAI v4.3.2 adds model routing, PII filtering, cloud proxies, LTX-2 video generation, and a native object detection backend. • Adds `X-LocalAI-Node` response header (gated) in distributed/middleware mode to expose which node served a request. • Adds middleware-layer model routing, PII filtering, and cloud model proxy support for request pipelines. • Adds `rfdetr-cpp` native backend enabling object detection and segmentation without external dependencies. • Adds LTX-2 and LTX-2.3 GGUF support to the stablediffusion-ggml backend, including gallery entries for image-to-video and first-last-frame-to-video variants. • Adds 'Fits in my GPU' toggle filter on the Install Models UI screen to surface only models that fit available VRAM.

LocalAI v4.3.0

Sun, 24 May 2026 20:25:26 GMT

LocalAI v4.3.0 adds keyless cosign backend verification, prompt cache on by default, per-API-key usage tracking, and distributed mode v3. • Adds `--require-backend-integrity` flag (or `LOCALAI_REQUIRE_BACKEND_INTEGRITY=true`) to enforce strict mode: missing `verification:` policy or empty SHA256 in gallery YAML becomes a hard-fail instead of a warning. • Adds `verification:` block to gallery YAML supporting `issuer_regex`, `identity_regex`, and `not_before` fields for keyless cosign/sigstore-go signature verification of backend OCI images with digest pinning and TUF root caching. • Adds `LOCALAI_TRACING_MAX_BODY_BYTES` environment variable (default 64 KiB) to cap captured request/response body size in trace middleware, recording `body_truncated` and original `body_bytes` fields when truncation occurs. • Adds `LOCALAI_NATS_BACKEND_INSTALL_TIMEOUT` and `LOCALAI_NATS_BACKEND_UPGRADE_TIMEOUT` environment variables (default 15 minutes, previously hardcoded 3 minutes) to control distributed backend install timeouts. • Adds new API endpoints `GET /api/auth/usage/sources` (self) and `GET /api/auth/admin/usage/sources` (admin, with `user_id` / `api_key_id` filters and 200-key truncation) for per-API-key and per-user token attribution. • Changes `kv_unified` default to `true` in `grpc-server.cpp` and `prompt_cache_all` default to `true` at the YAML layer, enabling `llama-cpp` server-side prompt cache out of the box — collapsing repeated system-prompt prefill from minutes to seconds. • Adds `cache_idle_slots` and `checkpoint_every_nt` as new model-config option keys for tuning `llama-cpp` KV cache behavior. • Changes `POST /api/nodes/:id/backends/install` to return HTTP 202 plus a `jobID` immediately instead of blocking up to 3 minutes, with per-node scoped opcache and async streaming progress events on `nodes..backend.install..progress` NATS subjects. • Adds unified `/app/backend-logs/:modelId` route that is mode-aware: routes to local WebSocket view in standalone mode, or to a per-node picker in distributed mode, with `?from=` deep-link timestamp support. • Adds a Sources tab to the React Usage page with a source-mix ribbon, top-7 + Other time chart, and searchable/sortable table; admin view rolls up (source, user_id, user_name) and shows owning account per named key. • Adds `usage_records` fields `Source` (`apikey` / `web` / `legacy`), `APIKeyID`, and `APIKeyName` with idempotent backfill of pre-feature rows on `InitDB`; revoked keys remain visible in history as (revoked). • Adds `flake.nix` and `flake.lock` for a reproducible, dockerless LocalAI setup on NixOS plus a dev shell. • Adds a distributed test harness under `tests/distributed/` for regression testing of multi-node routing behavior. • Implements per-request replica routing in `SmartRouter.Route` using `PickBestReplica` with `in_flight ASC, last_used ASC, available_vram DESC` ordering, and memoizes `probeHealth` per (nodeID, addr) with a 30-second TTL and singleflight coalescing.

LocalAI v4.2.6

Sat, 16 May 2026 21:12:25 GMT

LocalAI v4.2.6 bumps llama.cpp to MTP-merge SHA and automatically configures MTP defaults. • Upgrades `llama-cpp` to the MTP-merge SHA and automatically sets Multi-Token Prediction (MTP) defaults, enabling MTP inference without manual configuration.

LocalAI v4.2.5

Sat, 16 May 2026 16:44:56 GMT

LocalAI v4.2.5 exposes 12 additional llama.cpp common_params and updates Swagger docs. • Exposes 12 previously missing `common_params` via `options[]` for the llama-cpp backend, giving practitioners finer-grained inference control. • Updates Swagger API documentation to reflect current endpoint definitions. • Validates archive member paths before extraction, closing a path-traversal exposure during model downloads. • Validates video image URLs before download, preventing unexpected outbound requests. • Updates bundled vllm cu130 wheel to version `0.21.0`.

LocalAI v4.2.4

Wed, 13 May 2026 22:32:06 GMT

LocalAI v4.2.4 adds Liquid Audio speech-to-speech model support and Vulkan VRAM budget parsing. • Adds Liquid Audio speech-to-speech (s2s) model and assistant mode on the talk page via the realtime interface. • Extends VRAM budget and usage detection to parse output from `vulkaninfo`, broadening GPU visibility for Vulkan-backed hardware.

LocalAI v4.2.2

Tue, 12 May 2026 15:39:30 GMT

LocalAI v4.2.2 bumps llama.cpp and exposes new speculative-decoding options via the gRPC server. • Bumps the bundled llama.cpp to commit `1ec7ba0c`, adapts the gRPC server, and exposes new speculative-decoding options.

LocalAI v4.2.1

Mon, 11 May 2026 22:47:50 GMT

LocalAI v4.2.1 adds a DeepSeek V4 Flash backend with tool calls and KV cache, plus richer Ollama API model metadata. • New `ds4` backend adds DeepSeek V4 Flash support with tool calls, thinking mode, and KV cache. • The `/api/tags` and `/api/show` Ollama-compatible endpoints now report model capabilities and details.

LocalAI v4.2.0

Mon, 11 May 2026 12:48:17 GMT

LocalAI 4.2.0 adds voice/face recognition, diarization, Ollama drop-in API, video generation, 11 new backends, and hardened distributed mode v2. • Adds `/v1/voice/*` endpoints for 1:1 speaker verification, 1:N speaker identification, voice embeddings, and per-segment age/gender/emotion analysis, powered by a SpeechBrain + ONNX backend. • Adds `/v1/audio/diarization` endpoint for 'who spoke when?' segmentation via sherpa-onnx + vibevoice.cpp. • Adds word-level timestamps to faster-whisper transcriptions, plus `segments`, `duration`, and `language` fields on the stream-done event from `/v1/audio/transcriptions`. • Adds client cancellation for Whisper via the ggml `abort_callback`, allowing a transcription to be stopped mid-flight and GPU freed. • Adds a complete face-biometrics pipeline (1:1 verify, 1:N identify, detection, analysis, embeddings) with antispoofing/liveness rejection, built on InsightFace + ONNX. • Adds Ollama drop-in API compatibility — point an existing Ollama client at LocalAI by setting `OLLAMA_HOST`. • Adds video generation (image-to-video, first-last-frame) to the `stable-diffusion.ggml` backend, with gallery entries for Wan 2.1 FLF2V 14B 720P and Wan i2v 720p, plus a `stablediffusion-ggml-development` meta backend. • Adds `engine_args` to vLLM, exposing the full `AsyncEngineArgs` via a generic YAML map, tensor-parallel distributed workers, and feature parity with llama.cpp. • Adds `split_mode` config for llama.cpp for explicit multi-GPU placement. • Adds speculative decoding support for llama.cpp and Gemma 4 thinking mode. • Adds concurrency groups — per-model exclusive groups to prevent heavy backends from trampling each other during loading. • Adds a universal model importer supporting most backends, with multi-shard GGUF handling and dedicated importers for vibevoice-cpp and whisper.cpp HuggingFace repos. • Adds an interactive model config editor in the UI with autocomplete over known fields, live validation, and automatic file-renaming on save. • Adds admin-configurable branding (instance name, tagline, logo, favicon) and i18n support for English, Italiano, Español, Deutsch, and 简体中文 in the React UI. • Adds 11 new backends: `sglang` (high-throughput LLM + speculative decoding with EAGLE/EAGLE3/DFLASH/MTP), `ik-llama.cpp`, `TurboQuant`, `sam.cpp` (Segment Anything), `Kokoros` (Rust-native Kokoro TTS), `qwen3tts.cpp`, `tinygrad-multimodal` (experimental), `vibevoice.cpp`, `LocalVQE` (audio FX), `insightface` (face antispoofing), and `voice-rec` (speaker recognition/embeddings). • Adds CUDA 13 builds for vLLM, vLLM-omni, and sglang, plus NVIDIA L4T arm64 (CUDA 13) variants for Jetson-class boards. • Adds `AMDGPU_TARGETS` as an exposed build argument and support for gfx1151 (Strix Halo / Ryzen AI MAX) GPUs. • Adds `model` as a query parameter to the backend monitor endpoint. • Adds round-robin routing across replicas of the same model in distributed mode v2. • Adds `backend.upgrade` as a separate NATS action, split from install, in distributed mode v2. • Adds a Settings API that strips env-supplied `ApiKeys` before persisting, preventing accidental secret leaks. • Adds backend versioning with automatic upgrade detection and a revamped backend management UI showing concrete versions. • Adds pin-model capability so models survive the backend reaper, and a per-model on-demand auto-load toggle. • Adds audio transformations UI (LocalVQE) for exploring audio FX directly from the React UI. • Config loader now ignores YAML backup files. • Adds arm64 images for cpu-whisperx and cpu-faster-whisper.

LocalAI v4.1.2

Mon, 06 Apr 2026 08:54:04 GMT

LocalAI v4.1.2 wires speculative decoding settings into the llama.cpp backend. • Enables speculative decoding settings in the `llama.cpp` backend, unlocking faster inference via draft-model speculation. • Adds Qwen3.5 model files to the model index.

LocalAI v4.1.1

Sun, 05 Apr 2026 00:06:56 GMT

LocalAI v4.1.1 adds Gemma 4 thinking support and an API for coding agents to discover LocalAI configuration. • Adds a new API endpoint allowing coding agents to interactively discover how to control and configure LocalAI at runtime. • Adds thinking support for Gemma 4 models via the `gemma4` backend. • Adds autoparser preference for chat deltas emitted directly from backends, improving streaming fidelity.

LocalAI v4.1.0

Thu, 02 Apr 2026 22:14:48 GMT

LocalAI 4.1 adds distributed clustering, built-in auth/quotas, `local-ai agent run`, fine-tuning, and `min_p` inference support. • New `local-ai agent run` CLI subcommand runs standalone agents from the terminal, with single-turn `--prompt` mode and pool-based configurations loaded from `pool.json`. • Adds `HF_ENDPOINT` environment variable support so the downloader rewrites HuggingFace URIs to a corporate mirror or custom endpoint. • Adds `min_p` sampling parameter support wired through all inference endpoints. • Adds distributed cluster mode with smart VRAM-aware routing, node groups for workload isolation, min/max autoscaling, and drain/resume lifecycle management via API call. • Adds built-in multi-user platform with OIDC/OAuth SSO, invite-only registration, per-user API keys, admin impersonation, and per-user quota enforcement. • Adds per-user usage quota system with predictive analytics and per-user breakdown dashboards. • Adds experimental fine-tuning support using Hugging Face TRL — trains LoRA adapters, auto-exports to GGUF, and imports results back into LocalAI, all from the UI. • Adds experimental quantization backend for on-the-fly model quantization. • Streaming tool calls now work in agent mode, delivering results in real-time. • Adds automatic tool parsing fallback: when native tool call parsing fails, an iterative fallback parser engages automatically. • Adds automatic inference defaults sourced from Unsloth, applied across all endpoints and gallery models. • New React UI model pipeline editor lets users visually wire up model pipelines without editing YAML. • New per-model backend log view in the React UI scopes log output to individual models. • Media history added to Studio pages, showing past generated images and audio. • Adds searchable model/backend selector with inline search and filtering in the React UI. • Structured error toasts in the UI now link directly to traces for one-click debugging. • Adds cluster status dashboard on the home page showing all node states at a glance. • Supports S3 and peer-to-peer model transfer in distributed mode. • Adds first-class NVIDIA Jetson/Tegra GPU platform detection. • Bumps HuggingFace Transformers to >5.0 with generic model loading support.

LocalAI v4.0.0

Sat, 14 Mar 2026 18:18:41 GMT

LocalAI 4.0 adds native agent orchestration, Agenthub, MCP client support, React UI, WebRTC, and new audio backends • Adds `--data-path` CLI flag and `LOCALAI_DATA_PATH` environment variable to separate persistent agent/skill data from configuration. • Adds `LOCALAI_DISABLE_MCP` environment variable to completely disable Model Context Protocol support. • Adds `HF_MIRROR` environment variable to redirect HuggingFace model downloads to a mirror. • Adds `sample_rate` support to the TTS API via post-processing resampling. • Adds shell completion scripts for `bash`, `zsh`, and `fish` via dynamic completion support. • Adds WebSocket mode support for the Responses API. • Introduces native agentic orchestration: full lifecycle management of agents (create, start, stop, connect to Slack, configure MCP servers and skills) via the new UI. • Launches Agenthub (https://agenthub.localai.io) — a community hub for sharing and importing agents directly into LocalAI instances. • Adds agent memory backends: hybrid search via PostgreSQL or embedded in-memory storage via Chromem. • Adds 'Events' column to the Agents list page for observability of agent status and activity. • Adds Canvas Mode to the chat UI: code blocks and artifacts generated by the LLM appear in a dedicated preview panel on the right. • Adds MCP Apps support: select which MCP servers to enable per chat session from the UI; tools are automatically injected into the standard chat interface. • Adds full client-side Model Context Protocol (MCP) support with tool streaming in chat. • Adds WebRTC support to the Realtime API and Talk page for low-latency audio conversations. • Adds experimental `mlx-distributed` backend for running distributed workloads using Apple's MLX framework. • Adds `fish-speech`, `ace-step.cpp`, and `faster-qwen3-tts` (CUDA-only) audio backends. • Adds multi-voice support for Qwen TTS. • Adds `vllm-omni` backend detection for video generation model selection. • Adds `FreeRPC` to `backend.proto` for explicit VRAM cleanup. • Adds model storage size display with visual RAM warnings when model size exceeds system RAM. • Migrates the web frontend to React for a modernized UI with tabbed System view separating Models and Backends. • Adds improved trace display using accordions in the UI. Breaking changes: • HuggingFace backend support has been removed; workloads relying on it will fail after upgrade. • AIO images have been dropped; users relying on AIO images must switch to main images.

LocalAI v3.12.1

Sat, 21 Feb 2026 13:49:24 GMT

LocalAI v3.12.1 adds backend tracing and updates llama.cpp for Qwen 3 Coder compatibility. • Adds backend traces via the new traces feature (feat(traces)), enabling observability into backend inference calls.

LocalAI v3.12.0

Fri, 20 Feb 2026 18:16:23 GMT

LocalAI v3.12.0 adds Voxtral TTS backend, multi-modal realtime conversations, multi-GPU Diffusers, and a new dark/light UI theme. • Adds `voxtral` backend for high-quality text-to-speech, available via the model gallery. • Adds experimental `sd_embed`-style prompt embedding support to the `diffusers` backend for richer image generation pipelines. • Enables multi-GPU support for the `diffusers` backend, improving image generation throughput. • Extends the realtime API to accept text, image, and audio conversation items in a single session. • Adds a left navbar with dark/light theme variants to the web UI. • Improves legacy CPU support for the `stablediffusion-ggml` backend, broadening compatibility with older processors. • Adds model gallery entries for Ministral 3 family, npc-llm-3-8b, nemo-asr, voxcpm, whisperx, moonshine-tiny, neutts, vllm-omni models, voxtral, and nanbeige4.1-3b.

LocalAI v3.11.0

Sat, 07 Feb 2026 21:31:43 GMT

LocalAI v3.11.0 adds realtime audio conversations, four new ASR backends, TTS streaming, music generation UI, and vLLM Omni support. • Adds `ace-step` backend for music generation (MusicGen/Ace-Step), with a new Web UI for text-to-music generation directly in the browser. • Adds `transcribe` response format request parameter to the STT API endpoint, letting callers control transcription output format. • Adds `whisperx` backend for transcription with native speaker diarization (identifying who is speaking in multi-speaker audio). • Adds `vllm-omni` backend for high-performance inference with vLLM Omni models. • Adds `VoxCPM` TTS backend with streaming mode support, reducing time-to-first-audio for speech synthesis. • Adds `qwen-asr` backend for Qwen speech recognition models. • Adds `nemo` backend for Nvidia NeMo ASR (speech-to-text). • Adds `vibevoice` ASR support alongside its existing TTS capability. • Adds `qwen-tts` (Qwen-TTS) model support for text-to-speech. • Introduces native realtime audio conversation support compatible with OpenAI's Realtime API, including STT/TTS pipelines, voice-to-voice models, and tool call support. • Extends MLX build support to CUDA 12, CUDA 13, L4T (Nvidia Jetson), SBSA, and macOS Metal for broader hardware coverage. • Adds most remaining Piper voices from Hugging Face to the model gallery. Breaking changes: • The `ExLlama` backend has been removed; migrate to ExLlamaV2 or llama.cpp loaders. • The `Bark` backend has been removed (upstream unmaintained); use the new TTS backends (VoxCPM, Qwen-TTS, Piper) instead.

LocalAI v3.10.1

Fri, 23 Jan 2026 14:21:45 GMT

LocalAI v3.10.1 adds Qwen3-TTS backend, reasoning block support in openresponses, and GGUF support for LTX-2 video generation. • Adds `qwen-tts` backend, bringing support for Qwen3-TTS text-to-speech models. • Supports reasoning blocks in the `openresponses` API, enabling structured reasoning output from compatible models. • Adds GGUF support for LTX-2 video generation via the videogen backend. • Automatically detects thinking/reasoning support from the backend when not explicitly configured, removing the need to manually set it per model.

LocalAI v3.10.0

Sun, 18 Jan 2026 21:00:06 GMT

LocalAI v3.10.0 adds Anthropic `/v1/messages` API, Open Responses API, video generation UI, Moonshine STT, and Pocket-TTS with voice cloning. • Adds `GET /api/v1/responses/{response_id}` and `GET /api/v1/trace` endpoints as part of new Open Responses API support, enabling stateful agent sessions with `response_id`, `background: true` async mode, and `stream: true` streaming of tool calls, images, and audio. • Adds `GET /api/v1/trace` endpoint for in-memory request/response tracing; logs are capped by max size and exportable to JSON for agent debugging and fine-tuning analysis. • Adds native Anthropic `messages` API support at `/v1/messages`, providing a drop-in replacement for Claude with full tool/function calling, streaming, and compatibility with `anthropic-sdk-go` and LangChain. • Adds new `/video` page in the web UI for text-to-video and image-to-video generation, supporting LTX-2 and `diffusers` with adjustable `fps`, `num_frames`, and `guidance_scale` parameters. • Adds Moonshine backend — an ONNX-based, ultra-fast speech-to-text engine optimised for low-end devices (Raspberry Pi, older laptops) with live transcription support. • Adds Pocket-TTS backend for lightweight, high-fidelity text-to-speech with voice cloning support (requires HF authentication and a registered voice model). • Adds streaming tool call support (`input_json_delta`) and XML-style tool call parsing (`...`) across all backends (llama.cpp, vLLM, diffusers, etc.). • Adds a new 'Reasoning' field that automatically detects and extracts thinking tags from model output, surfacing reasoning steps in a dedicated 'Thinking' tab in the chat UI in both SSE and non-SSE modes. • Unified GPU backend packaging: CUDA, ROCm, and Vulkan GPU libraries are now bundled inside backend containers, so any `latest`/`master` image works on Nvidia, AMD, and ARM64 (Vulkan, experimental) without manual driver configuration. • Adds Vulkan arm64 backend builds, extending GPU-accelerated inference to ARM64 platforms. • System-aware backend gallery now auto-detects CPU/GPU capabilities and hides incompatible backends (e.g., MLX on Linux, CUDA on AMD), showing only runnable options alongside detected capabilities.

LocalAI v3.9.0

Wed, 24 Dec 2025 14:31:28 GMT

LocalAI v3.9.0 adds Agent Jobs panel, LRU model eviction, smart memory reclaimer, VibeVoice TTS, and CUDA 13 support. • Adds `single_active_backend` mode now defaults to LRU eviction with a limit of 1, automatically unloading the least recently used model when a new one is loaded — configurable to any model count limit. • New Memory Reclaimer watchdog monitors GPU/VRAM consumption across all backends and evicts LRU models when usage exceeds a configured threshold, preventing out-of-memory crashes. • New Agent Jobs panel in the web UI and API lets you create, schedule (via cron syntax), and trigger background agentic tasks programmatically, with optional webhook delivery of results and dynamic variable-populated prompts. • New `vibevoice` TTS backend adds real-time text-to-speech with voice cloning support, installable from the model gallery. • Adds CUDA 13 support in Docker images and L4T builds. • MLX backend gains thread-safe LRU prompt cache and `min_p`/`top_k` sampling parameters. • Whisper backend adds a `prompt` parameter to condition transcription output. • StableDiffusion-GGML gains LoRA support for image generation. • Hugging Face API now returns files in nested directories, enabling model imports from URLs with subfolders (e.g., `huggingface://user/model/GGUF/model.gguf`). • New UI mask for installing custom backends from the web interface. • UI search results can now be ordered by user selection. • Agent Jobs support multimedia inputs. Breaking changes: • Data storage path changed from `/usr/share` to `/var/lib`; existing deployments must migrate mutable data to the new location. • x86_64 macOS (darwin-x86_64) support dropped; ARM64 (M1/M2/M3/M4) is now the only supported Mac architecture.

LocalAI v3.8.0

Wed, 26 Nov 2025 20:22:49 GMT

LocalAI v3.8.0 adds universal model import from HF/Ollama/OCI, MCP live streaming, hot-reloadable settings, logprobs/logitbias, and new llama.cpp YAML options. • Adds `context_shift`, `cache_ram`, `use_jinja`, `parallel`, and `grpc_servers` as tunable options under the `options:` key in the model YAML config for llama.cpp backends. • Adds a new LocalAI endpoint to stream live MCP agent results — tool calls, reasoning steps, and intermediate actions are now visible in real-time via the Model Context Protocol. • Adds full support for `logprobs` and `logitbias` per the OpenAI specification, enabling token-level probability data for agentic logic, Self-RAG, and hallucination evaluation. • Introduces universal model import from Hugging Face, Ollama, OCI registries, or local file paths via URL — auto-detects backends (e.g., `llama.cpp` vs `diffusers`) and applies native chat templates (e.g., `llama-3`, `mistral`) from model metadata. • Adds a hot-reloadable 'Settings > System' panel for toggling watchdogs, P2P, gallery availability, and API key management (generate, rotate, expire) without restarting — requires mounting `/configuration` in containerized deployments. • Adds request cancellation support for streaming and non-streaming requests across `llama.cpp`, `vLLM`, `transformers`, and `diffusers` backends — stopping generation immediately frees backend resources. • Persists chat history and parallel conversations in browser local storage, with support for renaming and exporting chats to JSON. • Adds onboarding wizard for first-time users to import or install a model, plus auto-selection of a default model on startup. • Allows `top_n` in the reranker to be omitted or set to `0` to return all results with no arbitrary limit. • Adds multimodal auto-detection: vision components (`mmproj`) are identified and configured automatically during model import. • Adds a file-safety guard to prevent deletion of model blobs shared by multiple model configurations. Breaking changes: • Persisting runtime settings (hot-reload) requires mounting the `/configuration` directory from the container image in existing LocalAI deployments — without this mount, settings changes will not survive restarts.

LocalAI v3.7.0

Fri, 31 Oct 2025 21:34:21 GMT

LocalAI v3.7.0 adds agentic MCP support with `POST /mcp/v1/chat/completions`, a neutts TTS backend, `/v1/videos` endpoint, fuzzy gallery search, and Qwen 3 VL. • Adds `POST /mcp/v1/chat/completions` endpoint (OpenAI-compatible) for agentic tool-using AI with MCP, configurable via `mcp` and `agent` YAML blocks including `max_attempts`, `max_iterations`, `enable_reasoning`, and `enable_re_evaluation` keys. • Adds `neutts` backend for TTS, configured with `backend: neutts` in the model YAML, supporting `parameters.model`, `tts.audio_path`, `tts.streaming`, and `options` (e.g. `ref_text`) for low-latency natural speech generation. • Adds `POST /v1/videos` OpenAI-compatible endpoint for text-to-video generation, accepting `model`, `prompt`, and `size` fields. • Adds `whisper.cpp` CPU-variant builds for `avx`, `avx2`, `avx512`, and `fallback` instruction sets to support a wider range of hardware without crashes. • Adds Qwen 3 VL model support via llama.cpp/gguf, with gallery entries including `qwen3-vl-2b-instruct`, `qwen3-vl-4b-instruct`, `qwen3-vl-32b-instruct`, `qwen3-vl-30b-a3b-instruct`, and thinking variants. • Adds fuzzy and case-insensitive gallery search so queries like `gema` or `gemaa` match `gemma`, `gemma-3`, `gemma2`, etc. • Adds long-form TTS text chunking for the chatterbox backend, splitting long inputs intelligently to preserve audio context. • Adds `strict_mode: true` enforcement in tool/function definitions, returning a structured JSON error instead of crashing when a required tool call is invalid. • Adds a WebUI MCP toggle in the chat interface for models with `mcp` settings enabled, plus simplified YAML model editing and Alpine.js-powered reactivity replacing HTMX. • Adds support for nullable JSON schema types (`["string", "null"]`) and empty `parameters` in function definitions without panics. Breaking changes: • Upgrading the llama.cpp backend is required for existing LocalAI installations to gain Qwen 3 VL support and the updated llama.cpp version.

LocalAI v3.6.0

Fri, 03 Oct 2025 13:08:57 GMT

LocalAI v3.6.0 adds L4T device support for Kokoro, multilingual Chatterbox TTS, and six new gallery models including IBM Granite 4.0 and Qwen image editing. • Adds Kokoro TTS backend support for L4T (Linux for Tegra) devices, enabling on-device speech synthesis on NVIDIA Jetson hardware. • Adds multilingual support to the Chatterbox TTS backend. • Adds `qwen-image-edit-2509` to the model gallery for image editing workflows. • Adds `whisper-turbo` via the whisper.cpp backend to the model gallery. • Adds `ibm-granite_granite-4.0-h-small`, `ibm-granite_granite-4.0-h-tiny`, `ibm-granite_granite-4.0-h-micro`, and `ibm-granite_granite-4.0-micro` to the model gallery.

LocalAI v3.5.1

Wed, 17 Sep 2025 17:03:06 GMT

LocalAI v3.5.1 adds whisper diarization, HuggingFace endpoint override, CUDA 12.8 GPU images, and a launcher welcome page. • Adds `HF_ENDPOINT` environment variable to override the HuggingFace endpoint used for model downloads. • Adds diarization (tinydiarize) support to the Whisper backend for speaker-separated transcription. • Bumps NVIDIA GPU container images to CUDA 12.8. • Adds MPS and CPU support to the Chatterbox backend. • Adds a welcome page to the launcher UI.

LocalAI v3.5.0

Wed, 03 Sep 2025 20:23:15 GMT

LocalAI v3.5.0 adds MLX, WAN video, CPU diffusers, Purego Whisper with VAD, P2P model sync, and a system backend path. • Adds `LOCALAI_BACKENDS_SYSTEM_PATH` environment variable (and equivalent command-line argument) to specify a read-only system backend directory, defaulting to `/usr/share/localai/backends`, enabling package-manager-friendly and system-wide installations. • New `mlx-audio` backend (`backend: mlx-audio`) enables text-to-speech on macOS using MLX-powered Kokoro voice models. • New `mlx-vlm` backend (`backend: mlx-vlm`) supports vision-language models (VLMs) on macOS via MLX. • New `mlx` backend brings MLX-accelerated LLM inference to macOS. • New `WanPipeline` pipeline type in the `diffusers` backend enables text-to-video and image-to-video generation with Wan2.1 and Wan2.2 models using config keys `pipeline_type: WanPipeline`, `num_frames`, and `guidance_scale`. • Diffusers backend now supports CPU and macOS MPS execution, removing the GPU requirement for image generation. • Whisper backend rewritten in Purego with integrated Voice Activity Detection (VAD) for speech detection, improving efficiency and macOS compatibility. • Stablediffusion backend converted to Purego, improving architecture and enabling macOS CI builds. • Adds automatic P2P model synchronization across federated LocalAI instances for models installed via the gallery. • WebUI gains a 'Get Config' button to download a model's YAML configuration file without installing the model. • WebUI adds a manual model refresh button to reload on-disk YAML configurations without restarting. • WebUI gains model import and edit screens for managing model configurations directly in the browser. • WebUI now displays the list of currently installed backends. • WebUI surfaces SSE streaming errors directly to the user for improved debugging visibility. • New LocalAI Launcher app (Alpha) for Linux and macOS simplifies installation, startup, shutdown, updates, and WebUI access. • Diffusers backend prioritizes `ref_images` over `src` for image source selection, enabling more robust image input handling.

LocalAI v3.4.0

Tue, 12 Aug 2025 07:13:55 GMT

LocalAI v3.4.0 adds KittenTTS, Kokoro, and Dia backends, reasoning effort support, and sideloadable backends. • Adds `--alias` and `--name` flags to the backend installation CLI command to override configurations during manual installation. • Adds KittenTTS, Kokoro, and Dia as new TTS/audio backends, installable directly from the model gallery. • Adds support for reasoning effort in the OpenAI chat completion API. • Enables the Diffusers backend for NVIDIA L4T images and devices. • Supports sideloading backends from the system by dropping them into the backends folder, with no additional configuration required. • Adds image size control to the WebUI image generation interface. • Adds qwen-image model support to the model gallery.

LocalAI v3.3.2

Mon, 04 Aug 2025 14:52:43 GMT

LocalAI v3.3.2 lets backends install from a local path and run without a metadata file. • Supports installing backends from a local filesystem path, enabling offline or custom backend deployments. • Allows backends to operate without a metadata file, removing a previously required dependency for backend registration.

LocalAI v3.3.1

Fri, 01 Aug 2025 13:02:00 GMT

LocalAI v3.3.1 adds Flux Kontext image editing via `ref_images` API field and LoRA loading for stable-diffusion-ggml. • Adds `ref_images` field to the `POST /v1/images/generations` API, enabling Flux Kontext-powered image editing by passing one or more reference image URLs alongside a prompt. • Adds support for loading LoRAs in the stable-diffusion-ggml backend. • Adds `flux.1-kontext-dev` model, launchable with `local-ai run flux.1-kontext-dev`, for in-context image editing. • Adds new models to the model gallery: `flux.1-krea-dev-ggml`, `flux.1-dev-ggml-q8_0`, `flux.1-dev-ggml-abliterated-v2-q8_0`, `qwen_qwen3-30b-a3b-instruct-2507`, `qwen_qwen3-30b-a3b-thinking-2507`, and `arcee-ai_afm-4.5b`. Breaking changes: • Intel GPU container images `latest-gpu-intel-f32` and `latest-gpu-intel-f16` are replaced by a single unified image `latest-gpu-intel`; any existing scripts or deployments referencing the old tags will fail to pull.

LocalAI v3.3.0

Mon, 28 Jul 2025 15:03:28 GMT

LocalAI 3.3.0 adds a new object detection API powered by the `rfdetr-base` model. • Adds a new object detection API backed by the `rfdetr-base` model for fast, local object detection. • Backends now have defined mirror sources for downloads, improving resilience when primary registries are unavailable. Breaking changes: • The assistants endpoint has been dropped.

LocalAI v3.2.2

Fri, 25 Jul 2025 21:05:04 GMT

LocalAI v3.2.2 adds mirror support to the backend gallery for more resilient model downloads. • Adds mirror support to the backend gallery, enabling fallback download sources for backends.

LocalAI v3.2.0

Thu, 24 Jul 2025 20:36:41 GMT

LocalAI 3.2.0 splits all backends into a modular gallery with new `local-ai backends` CLI, auto hardware detection, and Intel GPU Whisper support. • Adds `local-ai backends list`, `local-ai backends install `, and `local-ai backends uninstall ` CLI commands to browse and manage backends from the new Backend Gallery independently of LocalAI releases. • Adds `local-ai backends install 'ocifile://'` to install backends from a local OCI tar file, enabling offline and air-gapped deployments. • Adds a CLI command to create custom OCI images from directories, supporting custom backend packaging. • Introduces the `LOCALAI_FORCE_META_BACKEND_CAPABILITY` environment variable to override automatic hardware detection; accepted values are `default`, `nvidia`, `amd`, and `intel`. • Supports the `input_audio` field in the `/v1/chat/completions` endpoint for multimodal audio inputs, improving OpenAI API compatibility. • Adds `speech started` and `speech stopped` realtime audio events for more precise control over interactive voice streams. • Enables SYCL acceleration for the Whisper backend, adding Intel GPU hardware-accelerated transcription support. • All inference backends (llama.cpp, whisper.cpp, piper, stablediffusion-ggml) are now separated from the core binary into the Backend Gallery, significantly reducing binary and container image size. • Adds over 50 new models to the model gallery, including releases from Qwen3, Gemma, Mistral, Nemotron, devstral-small, and more. Breaking changes: • llama.cpp, whisper.cpp, piper, and stablediffusion-ggml are no longer bundled in the main LocalAI binary; existing models installed before v3.2.0 may have no backend assigned and will require manually running `local-ai backends install ` after upgrading.

LocalAI v3.1.1

Fri, 27 Jun 2025 21:48:46 GMT

LocalAI v3.1.1 automatically installs missing backends when a model from the gallery requires them. • Automatically installs missing backends alongside models when installing from the gallery, removing the need to manually pre-install backend dependencies.

LocalAI v3.1.0

Thu, 26 Jun 2025 20:14:10 GMT

LocalAI v3.1 adds Gemma 3n support, GPU-aware meta-packages in the backend gallery, and a leaner container image layout. • Adds `local-ai run gemma-3n-e2b-it` and `local-ai run gemma-3n-e4b-it` commands to run Google Gemma 3n models (text generation) locally. • Introduces meta-packages to the backend gallery so installing a backend (e.g. `vllm`) automatically selects the correct GPU variant without manual version picking. • Moves default model and backend paths in container images to `/models/` and `/backends/` respectively. Breaking changes: • Default model path in container images changed from `/build/models` to `/models/`; mounts or scripts referencing the old path will break. • Default backend path in container images changed from `/build/backends` to `/backends/`; mounts or scripts referencing the old path will break. • Container image tag `cublas-cuda11` is renamed to `gpu-nvidia-cuda11` and `cublas-cuda12` to `gpu-nvidia-cuda12`; pipelines pinned to the old tag names will stop pulling the intended image. • Container image tag `sycl-f16` is renamed to `gpu-intel-f16` and `sycl-f32` to `gpu-intel-f32`; pipelines pinned to the old tag names will stop pulling the intended image. • Sources are no longer bundled in the container images; workflows that relied on in-container source access must build from scratch instead.

LocalAI v3.0.0

Thu, 19 Jun 2025 15:55:10 GMT

LocalAI 3.0 ships a Backend Gallery (OCI-based, API-driven), Realtime WebSocket API, audio/PDF upload in the UI, llama.cpp reranking, and 50+ new models. • Adds a Backend Gallery that lets you install and remove backends at runtime or startup via API or the WebUI, backed by standard OCI images — replaces the old `-extra` image approach. • Adds Backend Gallery download progress display in the WebUI so you can track backend installations in real time. • Adds Realtime WebSocket API with OpenAI-compatible streaming, suitable for chat apps and agents. • Adds `llama.cpp` reranking support. • Adds enhanced multimodal support to `llama.cpp` via `libmtmd`. • Adds audio input support to `llama.cpp`, enabling audio understanding models such as Qwen Omni and Ultravox. • Adds an audio upload button in the chat UI, letting users send audio files directly to supported models. • Adds PDF and text file upload in the chat UI, with support for multiple input files simultaneously. • Adds visual 'thinking' tags in the chat UI for reasoning models, showing inference progress in real time. • Adds dynamic VRAM estimation and automatic layer offloading based on GPU capabilities, improving GPU utilization without manual tuning. • Adds an error page in the UI to surface backend errors more clearly. • Adds 50+ new models to the model gallery, including `skywork-or1-32b`, `rivermind-lux-12b`, `qwen3-embedding-*`, `llama3-24b-mullein`, and `ultravox-v0_5`. • Adds new GPU-specific Docker image tags: `latest-gpu-nvidia-cuda-12`, `latest-gpu-nvidia-cuda-11`, `latest-nvidia-l4t-arm64`, `latest-gpu-hipblas`, `latest-gpu-intel-f16`, `latest-gpu-intel-f32`, and `latest-gpu-vulkan`, replacing the former extras-image model. • Adds AIO (all-in-one, pre-downloaded models) image variants: `latest-aio-cpu`, `latest-aio-gpu-nvidia-cuda-12`, `latest-aio-gpu-nvidia-cuda-11`, `latest-aio-gpu-intel-f16`, and `latest-aio-gpu-hipblas`. Breaking changes: • The `-extra` Docker images containing Python backends (e.g. `localai/localai:*-extra`) are no longer published; switch to standard GPU-specific images (`latest-gpu-nvidia-cuda-12`, `latest-gpu-hipblas`, etc.) and install additional backends via the Backend Gallery. • `bark-cpp` is removed from the bundled image and moved to the Backend Gallery — installations relying on the built-in `bark-cpp` backend must reinstall it via the Backend Gallery after upgrading.

LocalAI v2.29.0

Mon, 12 May 2025 20:31:20 GMT

LocalAI v2.29.0 adds video generation endpoint, Qwen3 support, Whisper.cpp GPU acceleration, and auto GPU offload for llama.cpp • Adds `--uninstall` flag to `install.sh` for removing LocalAI installations. • Introduces experimental `/video/generations` endpoint for video generation. • Adds GPU auto-detection and automatic layer offloading for `llama.cpp` and `CLIP` backends. • Enables GPU acceleration for `whisper.cpp` via cuBLAS (NVIDIA) and Vulkan; SYCL and HIPblas support in progress. • Adds `-extras` suffix image tags (e.g., `latest-gpu-nvidia-cuda-12-extras`) for images carrying extra Python dependencies such as `diffusers`; default tags are now slim. • Bundles FFmpeg in all core images, replacing the former `-ffmpeg` tagged variants. • Adds new `latest-*` image tags: `latest-gpu-hipblas`, `latest-gpu-intel-f16`, `latest-gpu-intel-f32`, `latest-gpu-nvidia-cuda-12`, and `latest-gpu-vulkan`. • Adds official support for the Qwen3 model family. • Expands HIPblas AMD GPU architecture targets to include `gfx803`, `gfx900`, `gfx906`, `gfx908`, `gfx90a`, `gfx942`, `gfx1010`, `gfx1030`, `gfx1032`, `gfx1100`, `gfx1101`, and `gfx1102`. • Increases gRPC message size limits to 50 MB. • Adds VRAM usage estimation for `llama.cpp`. Breaking changes: • Images with extra Python dependencies (e.g., for `diffusers`) now require the `-extras` suffix (e.g., `latest-gpu-nvidia-cuda-12-extras`); the former default tags no longer include those libraries. • The separate `-ffmpeg` image tags have been removed; switch to the corresponding base tag (e.g., `latest-gpu-hipblas-ffmpeg` becomes `latest-gpu-hipblas`). • The AutoGPTQ backend has been dropped entirely.

LocalAI v2.28.0

Tue, 15 Apr 2025 20:41:03 GMT

LocalAI v2.28.0 adds SYCL support for stablediffusion.cpp, Lumina model family support, and relaunches LocalAGI v2 as a Go-based agent orchestration platform. • Adds SYCL support for `stablediffusion.cpp`, enabling Intel GPU-accelerated image generation. • Adds support for the Lumina model family (e.g., Lumina-Image-2.0) for local image generation. • Enhances the `LOCALAI_SINGLE_ACTIVE_BACKEND` loader to treat the backend as a true singleton, improving single-backend reliability. • Introduces LocalAGI v2, a fully rewritten Go-based AI agent orchestration platform with a no-code WebUI, compatible with the OpenAI Responses API and supporting built-in connectors for Slack, Telegram, Discord, GitHub Issues, and IRC. • Introduces LocalRecall, a standalone REST API for persistent agent memory, spun out of LocalAGI v2 as its own component.

LocalAI v2.27.0

Mon, 31 Mar 2025 10:32:22 GMT

LocalAI v2.27.0 ships a full WebUI redesign, vLLM config additions, GGUF auto context detection, and reply-prefix support. • Adds vLLM config options to disable logging, set `dtype`, and enforce per-prompt media limits. • Adds GGUF auto-detection of default context size from file, removing the need to set it manually. • Adds `reply prefix` support, allowing a custom prefix to be specified for model replies. • Supports new model architectures: Gemma 3, Mistral, and Deepseek. • Complete WebUI redesign with modernised navigation, chat interface, Talk, Audio Generation, Image Generation, Models Overview, API Overview, Login, and Swarm/P2P dashboard screens. • WebUI model gallery gains pagination and filtering, including separate filtering for text, TTS, and image models. • WebUI chat interface automatically detects model usage type and surfaces a link to relevant model documentation. • AIO (All-in-One) images updated: CPU AIO ships `llama3.1` (text), `granite-embeddings` (embeddings), `minicpm` (vision); GPU AIO ships `localai-functioncall-qwen2.5-7b-v0.5` (text), `granite-embeddings` (embeddings), `minicpm` (vision).

LocalAI v2.26.0

Sat, 15 Feb 2025 17:22:53 GMT

LocalAI v2.26.0 adds Kokoro/OuteTTS/Fast-Whisper backends, grammar triggers for llama.cpp, AVX512 support, and Nvidia L4T arm64 images. • Adds `function.grammar.triggers` config block (with `word` and `at_start` fields) to model YAML config files for llama.cpp, enabling grammar-triggered JSON/tool-call generation only when a specific token (e.g. ``) is seen. • Adds `hf.co` and `hf://` URI schemes to the model downloader, allowing models to be referenced directly by Hugging Face URLs. • New `Kokoro` TTS backend added for text-to-speech inference. • New `OuteTTS` backend added with voice cloning capabilities, available via the `transformers` backend. • New `Fast-Whisper` backend added for faster Whisper model inference. • Adds function argument parsing using named regular expressions, simplifying structured function call handling. • Adds tokenization support for llama.cpp. • Adds machine tag and inference timing tracking to monitor per-machine performance during inference. • Adds support for Sana pipelines in the `diffusers` backend. • Adds image generation option overrides to the `diffusers` backend. • Adds bundled AVX512 build support for CPUs with the AVX512 instruction set. • Adds Nvidia L4T arm64 container images for devices such as Nvidia AGX Orin, launchable via `--runtime nvidia --gpus all quay.io/go-skynet/local-ai:master-nvidia-l4t-arm64-core`. • Adds fine-tuned function-calling models (`LocalAI-functioncall-phi-4-v0.3`, `LocalAI-functioncall-llama3.2-1b-v0.4`, `LocalAI-functioncall-llama3.2-3b-v0.5`, `localai-functioncall-qwen2.5-7b-v0.5`) to the LocalAI gallery. • Adds new models to the gallery including DeepSeek-R1, Mistral-small-24b, nightwing3-10b, rombos-qwen2.5-writer, and negative_llama_70b. • Merges `Mamba`, `Transformers-Musicgen`, and `Sentencetransformers` backends into the unified `transformers` backend. Breaking changes: • The `vall-e-x` backend has been removed; use the CoquiTTS community fork, Kokoro, or OuteTTS instead. • The `openvoice` backend has been removed; use Kokoro or OuteTTS instead. • The `stablediffusion-NCN` (ONNX-based) backend has been removed and replaced by the `stablediffusion-ggml` backend. • The `llama-ggml` (pre-GGUF) backend has been removed; only GGUF models are supported going forward. • Mamba, Transformers-Musicgen, and Sentencetransformers backends now route through the `transformers` backend — existing config files referencing the old backend names may be incompatible. • Mirostat is no longer enabled by default in llama.cpp (previously it was on by default).

LocalAI v2.25.0

Fri, 10 Jan 2025 22:02:48 GMT

LocalAI v2.25.0 adds KV cache quantization, Jinja templates from GGUF, streaming token usage, and resumable downloads • Exposes `cache_type_k` and `cache_type_v` llama.cpp config keys to enable quantization of the KV cache, reducing VRAM usage for large models. • Supports reading Jinja templates directly from GGUF model files, eliminating the need for separate template configuration. • Streams token usage statistics alongside generated tokens during inference. • Adds UI path prefix support via HTTP header, enabling reverse-proxy deployments at non-root paths. • Allows skipping driver installation in the Dockerfile build process. • Enables resuming partial downloads when fetching models, avoiding full re-downloads on interrupted transfers. • Adds a large batch of new models to the model gallery, including phi-4, phi-3.5-moe-instruct, llama-3.3-70b-instruct, qwen2-vl-72b-instruct, qvq-72b-preview, falcon3 series (1b/3b/7b/10b), dolphin3.0 series, and many others.

LocalAI v2.24.0

Wed, 04 Dec 2024 20:56:15 GMT

LocalAI v2.24.0 adds bark.cpp and stablediffusion.cpp backends, silero-vad speech detection, WebUI API key auth, and new P2P config options. • Adds `bark.cpp` backend for transformer-based text-to-audio generation, installable via `local-ai models install bark-cpp-small` or runnable with `local-ai run bark-cpp-small`. • Adds `stablediffusion.cpp` backend for ggml-powered image generation from text descriptions. • Adds `silero-vad` backend for voice activity detection in audio streams, installable via `local-ai models install silero-vad` or configurable through the WebUI. • Adds P2P configuration keys `listen_maddrs`, `dht_announce_maddrs`, and `bootstrap_peers` for fine-grained control of peer-to-peer node addressing and connectivity. • WebUI now supports API key authentication, automatically prompting for a key when one or more API keys are configured. • Adds real-time token usage reporting during streamed outputs. Breaking changes: • The `rwkv.cpp` backend has been removed; RWKV model support now routes through `llama.cpp`. • The `bert.cpp` backend has been dropped; its functionality is now integrated into `llama.cpp`.

LocalAI v2.23.0

Sun, 10 Nov 2024 17:07:39 GMT

LocalAI v2.23.0 adds vLLM load_format exposure, TTS response_format, multi-LoRA diffusers, Flux single-file support, and a disableable /metrics endpoint. • Exposes `load_format` backend config key for the vLLM backend, allowing operators to control how model weights are loaded. • Implements `response_format` parameter on the TTS endpoint, letting callers specify the desired audio output format. • Allows disabling the `/metrics` endpoint for local stats collection via configuration. • Supports multiple LoRA adapters simultaneously in the diffusers backend. • Adds Flux single-file support to the diffusers backend. • Moves model detailed info to a modal in the UI, reducing clutter on the model list view. Breaking changes: • Multimodal message templates now use a single unified template; existing per-role multimodal templates will need to be migrated to the new single-template format.

LocalAI v2.22.1

Mon, 21 Oct 2024 12:50:31 GMT

LocalAI v2.22.1 adds Sprig templating and text extraction for multimodal requests, plus Phi-3 Vision and 30+ new gallery models. • Adds Sprig template functions to multimodal templates, enabling richer data manipulation in prompt construction. • Adds text extraction from multimodal requests in templates, allowing templates to pull plain text out of mixed-media inputs. • Adds `phi-3 vision` to the model gallery, enabling multimodal vision inference locally. • Adds 30+ new models to the gallery, including `llama-3.1-nemotron-70b-instruct-hf`, `supernova-medius`, `cursorcore-qw2.5-7b-i1`, `ml-ms-etheris-123b`, and multiple CursorCore, Hermes-3, and Llama-3.x variants.

LocalAI v2.22.0

Sat, 12 Oct 2024 13:09:59 GMT

LocalAI v2.22.0 adds vLLM video/image-to-text, a tokenization endpoint, loaded-model listing via `/system`, and configurable multimodal placeholders. • Adds `template.video`, `template.image`, and `template.audio` keys to model YAML config to override per-backend multimodal placeholder notation (e.g. `<|video_{{.ID}}|> {{.Text}}`). • Extends the chat completions API to accept `video_url` and `audio` content types alongside images, enabling video and audio understanding via the vLLM backend. • Lists all currently loaded models via the `GET /system` endpoint for real-time model management. • Adds a tokenization endpoint to the API. • Adds token metrics retrieval to the GRPC server. • Adds a `correlationID` field to chat request tracking via the API. • Adds force-shutdown capability for backends. • The Transformers backend now reuses an already-downloaded model instead of re-downloading it.

LocalAI v2.21.0

Tue, 24 Sep 2024 14:22:30 GMT

LocalAI v2.21 adds a /system info endpoint, video/audio API inputs, ElevenLabs sound generation, vLLM embeddings, and auto model warm-up. • Adds `GET /system` endpoint to list system information. • Adds ElevenLabs `sound-generation` API support via the elevenlabs backend. • Adds support for passing videos to backends via the API. • Adds support for passing audios to backends via the API. • Adds embedding support to the `vllm` backend. • Adds `trust_remote_code` configuration support for the `sentencetransformers` backend. • Adds regex-based output extraction from LLM responses. • Adds auto warm-up to load models into memory on startup. • AIO images now ship `gpt-4o` as the Vision API model alias. Breaking changes: • The `exllama` backend has been removed; any configuration referencing it will break on upgrade. • The AIO image model alias `gpt-4-vision-preview` has been renamed to `gpt-4o`; configurations or API calls using `gpt-4-vision-preview` will need to be updated.

LocalAI v2.20.1

Fri, 23 Aug 2024 08:29:52 GMT

LocalAI v2.20 adds FLUX-1 image gen, strict JSON mode, multi-cluster P2P, gallery disable, and Hugging Face integration. • Adds `LOCALAI_DISABLE_GALLERY_ENDPOINT` environment variable to disable the gallery endpoint and its UI routes. • Adds `LOCALAI_RANDOM_WORKER` environment variable to disable load balancing (now enabled by default in federated mode). • Adds `LOCALAI_TARGET_WORKER` environment variable to target a specific worker in federated mode. • Adds `LOCALAI_P2P_NETWORK_ID` environment variable (set to a shared UUID) to run multiple logically separated P2P clusters within the same network using the same token. • Adds `strict: true` field in API calls to enforce grammar-compliant JSON outputs via BNF grammar rules, matching OpenAI's Structured Outputs spec. • Adds `json_schema` as a supported `format_type`; when set, BNF grammars are automatically generated from the schema. • Adds FLUX-1 image generation support (`flux.1-dev` and `flux.1-schnell` in the model gallery), including federated mode. • Adds a network explorer and community pools UI at explorer.localai.io, enabling sharing and discovery of LocalAI instances. • Adds P2P load balancing enabled by default across federated clusters. • LocalAI is now available as a selectable option inside Hugging Face's local apps for one-click model deployment. • Adds Gemma 2 to the model guesser for automatic backend selection. Breaking changes: • The `gpt4all.cpp` backend is removed; `gpt4all` models in `ggml` format are no longer compatible. • The `petals` backend is removed; use LocalAI's P2P federation capabilities as a replacement.

LocalAI v2.20.0

Thu, 22 Aug 2024 20:48:59 GMT

LocalAI v2.20.0 adds FLUX-1 image generation, OpenAI strict mode, multi-cluster P2P networking, and a global community explorer. • Adds `strict: true` to API call payloads to enforce BNF-grammar-backed JSON outputs, matching OpenAI's Structured Outputs spec; when `format_type` is set to `json_schema`, BNF grammars are auto-generated from the schema. • Adds `LOCALAI_DISABLE_GALLERY_ENDPOINT` environment variable to disable the gallery endpoint and its UI routes. • Adds `LOCALAI_P2P_NETWORK_ID` environment variable (set to a shared UUID) to run multiple logically separated P2P clusters on the same network using the same token. • Adds `LOCALAI_TARGET_WORKER` environment variable to target a specific worker in federated mode. • Adds `LOCALAI_RANDOM_WORKER` environment variable to disable the new default load-balancing behavior across P2P workers. • Adds `json_schema` as a supported `format_type` value in API requests, enabling schema-driven structured output. • Adds FLUX-1 image generation support via the Diffusers backend, including `flux.1-dev` and `flux.1-schnell` gallery models, usable in federated mode. • Launches a network explorer at explorer.localai.io and a public demo instance at demo.localai.io, backed by community P2P pools. • LocalAI is now selectable as a local app within Hugging Face, enabling one-click model deployment. Breaking changes: • The `gpt4all.cpp` backend is removed; `gpt4all` models in `ggml` format are no longer compatible. • The `petals` backend is removed; workloads must migrate to LocalAI's P2P federation capabilities. • P2P load balancing is now enabled by default; existing setups that relied on random worker selection must set `LOCALAI_RANDOM_WORKER` to restore the previous behavior.

LocalAI v2.19.3

Sun, 28 Jul 2024 12:24:10 GMT

LocalAI v2.19.3 adds a Llama 3.1 grammar schema and expands the model gallery with 14 new models including Mistral Nemo and Llama 3.1 variants. • Adds Llama 3.1 grammar schema for structured output generation. • Adds function calling template for Llama 3.1 models to the gallery. • Adds Llama 3.1 with enforced function calling via grammars to the model gallery. • Adds 13 new models to the gallery: `llama3.1-claude`, `darkidol-llama3.1`, `gemmoy`, `mistral-nemo`, `llama3.1-8b-fireplace2`, `lumimaid-v0.2-12b`, `darkidol-llama-3.1-8b-instruct-1.1-uncensored`, `meta-llama-3.1-8b-instruct-abliterated`, `llama-3.1-70b-japanese-instruct-2407`, `llama-3.1-8b-instruct-fei-v1-uncensored`, `openbuddy-llama3.1-8b-v22.1-131k`, `lumimaid-8b`.

LocalAI v2.19.1

Sat, 20 Jul 2024 07:16:46 GMT

LocalAI v2.19.1 adds P2P federated AI swarms, llama.cpp embeddings, model pull by URL, and a HuggingFace /scan endpoint. • Adds `--p2p` and `--p2p --federated` CLI flags to launch federated LocalAI instances that auto-discover each other across networks using a shared token, with optional load balancing. • Adds `--p2p` flag support for distributing model weights across nodes as P2P workers, currently available on the llama.cpp backend. • Adds a HuggingFace `/scan` API endpoint for scanning HuggingFace resources. • Enables embedding support in the llama-cpp (gRPC) backend via the embeddings endpoints. • Adds model pulling directly from URLs via `feat(models): pull models from urls`. • Adds a 'Swarm' section to the WebUI providing a P2P dashboard with auto-discovery setup instructions and one-liner connection commands. • Adds TTS (Text-to-Speech) to binary releases. • Updates `install.sh` to support federated instance setup. • Adds the ability to remove a backend from the backend list in the WebUI. • Improves tool/function call handling by parsing broken JSON in raw results and using dynamic rules for grammar keys when grammars are disabled. • WebUI welcome page now shows models without a config file alongside configured models.

LocalAI v2.19.0

Fri, 19 Jul 2024 17:44:18 GMT

LocalAI v2.19.0 adds P2P federated instances, llama-cpp embeddings, HuggingFace /scan endpoint, and URL-based model pulling. • Adds `--p2p` flag to start LocalAI and generate a shared token for auto-discovering and clustering multiple instances into a load-balanced AI swarm without Docker or Kubernetes. • Adds `--p2p --federated` flags to launch fully federated LocalAI instances that share requests across the cluster, with auto-discovery across different networks. • Adds HuggingFace `/scan` endpoint for scanning HuggingFace model assets. • Adds embeddings endpoint support to the `llama-cpp` (grpc) backend, enabling local embedding generation via llama.cpp. • Adds load-balanced option to the P2P federation mode. • Supports pulling models directly from URLs via the model management interface. • Includes TTS (Text-to-Speech) in binary releases, removing the need for a separate install. • Adds federated instance setup support to the `install.sh` installer script. • Adds a P2P 'Swarm' dashboard in the WebUI with one-liner connection instructions and auto-discovery guidance. • Improves function/tool calling by parsing broken JSON in raw results and using dynamic rules for grammar keys when grammars are disabled. • Allows removing a backend from the backend list via the WebUI. • Shows models without a config file in the WebUI welcome/model list page.

LocalAI v2.18.1

Mon, 01 Jul 2024 20:53:22 GMT

LocalAI v2.18.1 adds backend autodetect fallback and six new models to the gallery. • Adds backend fallback with autodetection, allowing LocalAI to automatically select an available backend when the primary one is unavailable. • Adds `new-dawn-llama` to the model gallery. • Adds `l3-aethora-15b-v2` to the model gallery. • Adds `bungo-l3-8b-iq-imatrix` to the model gallery. • Adds `llama3-8b-darkidol-2.1-uncensored-1048k-iq-imatrix` to the model gallery. • Adds `llm-compiler` to the model gallery. • Adds `llama3-turbcat-instruct-8b` to the model gallery.

LocalAI v2.18.0

Fri, 28 Jun 2024 14:17:44 GMT

LocalAI v2.18.0 adds OCI/Ollama model support, Vulkan backend, transcription translation, and new YAML config options. • Adds `oci://` and `ollama://` URI prefixes for specifying models in YAML config files, enabling `local-ai run ollama://gemma:2b` and `local-ai models install ollama://gemma:2b` directly from the CLI. • Adds `translate: true` field to transcription requests, enabling the transcription endpoint to translate audio to English in a single call. • Adds `repeat_last_n` as a model YAML config parameter (specific to the llama.cpp backend) to control repetition penalty window. • Adds `properties_order` under `function.grammar` in model YAML config to force JSON responses to return properties in a specified order. • Introduces Vulkan support for the llama.cpp backend, available via new container image tags `latest-vulkan-ffmpeg-core` and `v2.18.0-vulkan-ffmpeg-core`. • Upgrades CUDA support to version 12.5 in container images. • Adds UI model selector allowing users to choose between all available models in the chat interface. • Adds gallery support for Gemma 2 models, runnable via `local-ai run gemma-2-27b-it` and `local-ai run gemma-2-9b-it`.

LocalAI v2.17.1

Wed, 19 Jun 2024 06:56:14 GMT

LocalAI v2.17.1 adds Stable Diffusion 3 support via a new `StableDiffusion3Pipeline` backend config. • Adds Stable Diffusion 3 image generation support via the `diffusers` backend with `pipeline_type: StableDiffusion3Pipeline`; install from the gallery as `stable-diffusion-3-medium` or configure manually with `backend`, `diffusers.cuda`, `diffusers.enable_parameters`, `f16`, `parameters.model`, and `step` keys in a model YAML file. • Improves the `talk` UI with an informative display box and better colors.

LocalAI v2.17.0

Mon, 17 Jun 2024 18:10:53 GMT

LocalAI v2.17.0 adds auto-GGUF detection, voice chat UI, gallery CLI installs, and single binaries for ROCm/NVIDIA/Intel/ARM64. • Adds `local-ai run ` to install and launch a model from the gallery directly via CLI. • Adds `local-ai models install` subcommand to install models from galleries via the CLI. • Adds `install.sh` quick-start installer configurable via environment variables `DOCKER_INSTALL`, `USE_AIO`, `API_KEY`, `CORE_IMAGES`, `PORT`, `THREADS`, `VERSION`, and `MODELS_PATH` (default port `8080`, default `MODELS_PATH` `/usr/share/local-ai/models`). • Adds `response_regex` support as a list in function-calling grammar config, enabling multiple regex patterns for free-string matching. • Supports `response_type` field in OpenAI-compatible image API requests. • Adds CSRF protection support via the `fiber CSRF` feature, enabling CSRF enforcement at LocalAI startup. • Adds `OpaqueErrors` mode to hide internal error details from API responses. • Automatic identification and configuration of GGUF models — drop a GGUF file into the models folder and LocalAI infers defaults without requiring a YAML config. • Adds a `util` command to print GGUF file metadata and model information. • Automatic template detection from GGUF files with built-in defaults for Qwen2, Phi, Mistral, ChatML, Command-R, and Gemma model families. • New WebUI page for voice-based LLM interaction using audio transcription and TTS, accessible with a couple of clicks. • Linux single binaries now bundle ROCm (AMD), NVIDIA, and Intel GPU support in one binary (system GPU libraries still required separately). • Single-binary builds now available for ARM64 (Linux and macOS/Apple Silicon), with bundled gRPC and supporting libraries. • ARM64 Docker images added (e.g., `localai/localai:v2.17.0-ffmpeg-core`). • Parallel function calls with mixed or no grammar constraints now supported. • New model gallery site at https://models.localai.io for browsing available models without running LocalAI locally. • Gallery expanded with new models including Mirai Nova, Mahou, MiniCPM, Codestral, Gemma-2b, Llama3-11b, all Whisper variants, and others.

LocalAI v2.16.0

Fri, 24 May 2024 17:35:17 GMT

LocalAI v2.16.0 adds distributed llama.cpp inferencing, peer-to-peer LLM clusters, mixed JSON grammar function calling, and a single binary release. • New `local-ai llamacpp-worker ` subcommand starts llama.cpp RPC workers to offload inferencing workload to remote nodes. • New `LLAMACPP_GRPC_SERVERS` environment variable accepts a comma-separated list of `address:port` pairs to distribute llama.cpp inference across multiple nodes when running `local-ai run`. • New `--p2p` flag on `local-ai run` enables fully decentralized peer-to-peer LLM inferencing over the libp2p protocol without manual IP configuration, using DHT and mDNS for discovery. • New `--p2ptoken` flag (and `P2P_TOKEN` environment variable) lets you supply a pre-shared token to rejoin an existing private p2p cluster on server restart. • New `p2p-llama-cpp-rpc` subcommand (with `TOKEN=XXX` env var or token argument) starts a worker node that joins the p2p cluster and contributes compute. • New `function.grammar.mixed_mode: true` config key in YAML model configuration enables mixed JSON BNF grammars, allowing models to output both structured JSON and free text in function-calling responses. • New `function.grammar.disable: true` config key turns off grammar enforcement entirely, letting users supply regex-based parsing instead. • New `function.json_regex_match` config key accepts a list of regex patterns to extract function-call results from raw LLM output (e.g. for Hermes-style `` tags). • New `function.replace_llm_results` and `function.replace_function_results` config keys accept key/value regex replacement lists to clean LLM output before OpenAI-spec compliance checks. • New `function.return_name_in_function_response` config key includes the function name in the response payload. • New `function.disable_no_action` config key suppresses injection of the default 'answer' tool in function-calling prompts. • Single binary release consolidates all variants (CUDA and non-CUDA) and dependencies into one portable executable, simplifying installation and upgrades. • Model gallery adds Aya-35b, Mistral-0.3, Hermes-Theta, Hermes-2-Pro-Mistral, Hermes-2-Theta-Llama-3, and a fine-tuned LocalAI-Llama3-8b-Function-Call-v0.2 model with enhanced out-of-the-box function-calling support. • Python backends migrated from Conda to UV, reducing setup time and dependency management complexity.

LocalAI v2.15.0

Thu, 09 May 2024 17:20:05 GMT

LocalAI v2.15.0 adds Vision API in Chat WebUI, single binary releases, `--debug` CPU/GPU info, and `trust_remote_code` UI flag. • Adds `--debug` flag output to display CPU/GPU information at startup. • Exposes `trust_remote_code` as a configurable flag in the WebUI for model loading. • Adds llama.cpp backend autoloading without requiring explicit backend specification, plus llama.cpp variant support. • Integrates Vision API into the Chat WebUI, enabling image processing model testing directly in the browser. • Adds system prompt configuration in the WebUI chat interface. • Introduces single binary releases for simplified deployment without AVX/SSE instruction sets (CUDA builds planned). • Adds model gallery filtering by tag and category in the WebUI. • Adds a background operations indicator to the WebUI to show when tasks are running. • Adds a revamped welcome/onboarding page in the WebUI to guide new users through model installation. • Expands the model gallery with new one-click-install models including 'moondream2', 'llama3-llava', 'llama3-instruct-coder', 'lumimaid', 'openbiollm', 'Soliloquy', 'tess', 'aurora', 'kunocchini', 'tiamat', and several OpenVINO models. • Updates ROCM support with a smaller base image.

LocalAI v2.14.0

Fri, 03 May 2024 07:29:36 GMT

LocalAI v2.14.0 adds OpenVINO acceleration, user-defined inference devices, model deletion, llama3 AIO, and a new WebUI with chat/TTS/image-gen pages. • Adds user-defined inference device selection for CUDA and OpenVINO backends, letting practitioners pin workloads to specific hardware. • Adds OpenVINO acceleration for embeddings in the transformer backend, enabling fast inference on Intel CPUs and GPUs. • Adds model deletion support to the gallery UI, allowing installed models to be removed directly from the interface. • Adds gallery job status display during navigation so model install progress is visible while browsing. • Adds Chat, TTS, and image-generation pages to the WebUI for quick interactive debugging and model assessment. • Switches the AIO image default LLM to a llama3-based model (Hermes-2-Pro-Llama-3-8B-GGUF), which supports function calling out of the box. • Adds numerous new models to the gallery including Einstein v6.1, SOVL, Hermes-2-Pro-Llama-3-8B, biomistral-7b, WizardLM2, llama3-32k, suzume-llama-3-8B-multilingual, and several OpenVINO-optimized models.

LocalAI v2.13.0

Thu, 25 Apr 2024 20:34:31 GMT

LocalAI v2.13.0 adds a model gallery UI, rerankers backend with Jina API compatibility, and a new parler-tts backend. • Adds `rerankers` backend implementing the Jina reranker API at `/v1/rerank`, compatible with existing Jina clients; configured via a model YAML with `backend: rerankers` and `parameters.model: cross-encoder`. • Adds `parler-tts` backend for text-to-speech, installable from the gallery or via model config. • Adds `tensor_parallel_size` setting to vLLM backend configuration. • Adds `use_tokenizer_template` and `stop_prompts` options to the Transformers backend. • Adds `ConfigURLs` support to the gallery, enabling custom hosted model index repositories. • Adds a Golang client for the LocalAI store backend. • Adds `trace` log level and zerolog-based fiber request logging. • Adds `flash-attn` support in NVIDIA and ROCm environments. • Adds tokenizer.apply_chat_template() support in the vLLM backend. • Adds function calling support for models with no grammar. • Adds consumed token count reporting in GRPC backend responses. • Adds a model gallery UI in the WebUI with browseable models including stablediffusion, llama3, phi-3, hermes, tts, and embeddings. • Adds llama3, hermes, phi-3, and other models to the gallery. • Adds Transformers backend adherence to OpenAI API `max_tokens` behavior.

LocalAI v2.12.3

Wed, 10 Apr 2024 09:15:24 GMT

LocalAI v2.12.3 adds Assistant API, OpenVINO runtime, Swagger UI, Vision support for AutoGPTQ, and an Intel AIO profile. • Adds Assistant and AssistantFiles API endpoints, enabling OpenAI-compatible assistant workflows. • Adds Swagger UI at the LocalAI front page for interactively exploring and testing API calls directly in the browser. • Adds OpenVINO runtime support for the transformer backend, enabling hardware-accelerated inference on Intel hardware. • Adds token streaming support for the transformer backend (including OpenVINO and CUDA paths). • Adds Intel GPU profile for AIO (all-in-one) images, enabling out-of-the-box Intel GPU acceleration. • Adds Vision (VL model) support to the AutoGPTQ backend. • Adds a landing welcome page when accessing the LocalAI front page, with a model list and quick-start guidance. • Web UI now shows which backends are associated with each model. • AIO CPU images now default to `NousResearch/Hermes-2-Pro-Mistral-7B-GGUF`, pre-configured for functions and tools API support. • Improves structured logging across the LocalAI server.

LocalAI v2.12.1

Tue, 09 Apr 2024 13:46:52 GMT

LocalAI v2.12.1 adds Assistant API, OpenVINO runtime, Swagger UI, vision support for AutoGPTQ, and an Intel AIO profile. • Adds Assistant and AssistantFiles API endpoints, enabling OpenAI-compatible assistant workflows. • Adds Swagger UI at the LocalAI front page for interactively exploring and testing API calls directly in the browser. • Adds a landing welcome page served at the LocalAI front page root. • Adds OpenVINO runtime support for the transformer backend, including token streaming for both OpenVINO and CUDA. • Adds token streaming support for the transformer backend. • Adds Vision/VL model support to the AutoGPTQ backend. • Adds an Intel GPU profile for AIO images, with `Hermes-2-Pro-Mistral-7B-GGUF` as the new default CPU AIO model, pre-configured for functions and tools API support. • Web UI now shows which backends are associated with each model.

LocalAI v2.12.0

Tue, 09 Apr 2024 07:03:40 GMT

LocalAI v2.12.0 adds Assistant API, OpenVINO runtime, Swagger UI, Vision for AutoGPTQ, and an Intel AIO GPU profile. • Adds Assistant and AssistantFiles API endpoints, enabling assistant-style workflows against a local inference server. • Adds Swagger UI for interactively exploring and testing LocalAI API calls directly from the browser. • Adds OpenVINO runtime support for the transformer backend, with token streaming for both OpenVINO and CUDA. • Adds Vision (VL model) support to the AutoGPTQ backend. • Adds an Intel GPU profile for AIO (all-in-one) images. • Adds a landing welcome page shown when accessing the LocalAI front page, with a model list and backend associations visible in the web UI. • AIO CPU images now default to `NousResearch/Hermes-2-Pro-Mistral-7B-GGUF`, pre-configured for functions and tools API support.

LocalAI v2.11.0

Tue, 26 Mar 2024 17:18:14 GMT

LocalAI v2.11.0 ships All-in-One Docker images and a new vector store backend for CPU and GPU environments. • Adds `localai/localai:latest-aio-cpu`, `localai/localai:latest-aio-gpu-cuda-11`, and `localai/localai:latest-aio-gpu-cuda-12` All-in-One images that bundle LLM, TTS, speech-to-text, image generation, and function calling out of the box — no manual model setup required. • Adds `localai/localai:latest-aio-gpu-hipblas` and Intel GPU AIO images for AMD and Intel GPU environments. • Adds a new 'vector stores' backend type, enabling LocalAI to operate as an in-memory vector database. • Adds pre-configured model definitions for `llava-1.6-vicuna`, `llava-1.6-mistral`, `hermes-2-pro-mistral`, `phi-2-chat`, `bakllava`, and `cerbero`. • Changes llama.cpp model defaults to automatically offload the maximum number of GPU layers when a GPU is available.

LocalAI v2.10.0

Sat, 16 Mar 2024 15:43:57 GMT

LocalAI v2.10.0 adds Intel GPU support, Elevenlabs/OpenAI TTS compatibility, Bitsandbytes quantization, and vLLM streaming. • Now honors `trust_remote_code` in model YAML config files for the AutoGPTQ and transformers backends, controlling whether remote code execution is allowed during model loading. • Adds Bitsandbytes quantization support for the transformer backend, enabling lower-memory model loading on CUDA hardware. • Adds compatibility layers for Elevenlabs and OpenAI TTS, allowing LocalAI to serve as a drop-in endpoint for both TTS APIs. • Adds diffusers/transformers support for Intel GPUs, enabling image generation and transformer-based inference on Intel GPU hardware. • Enables `stream: true` for the vLLM backend via AsyncLLMEngine, supporting true token streaming from vLLM-served models. • Adds initial implementation of the Assistants API, beginning OpenAI-compatible assistant endpoint support. • Embeddings endpoints no longer require explicit model configuration to function. Breaking changes: • The `trust_remote_code` setting in model YAML config files is now enforced for AutoGPTQ and transformers backends. Models that previously loaded remote code without this setting will now be blocked; set `trust_remote_code: true` in the model YAML config file to restore the old behavior.

LocalAI v2.9.0

Sat, 24 Feb 2024 10:47:43 GMT

LocalAI v2.9.0 adds Tools API with parallel function calling, ROCm/SYCL GPU images, file upload API, and Gemma model support. • Adds Tools API endpoint support, including parallel function calling and SSE with function calling; the now-deprecated functions API call remains supported. • Adds initial implementation of the upload files API (early Assistant API groundwork). • Publishes ROCm container images for AMD GPUs, tagged `hipblas` (e.g. `master-hipblas-ffmpeg-core`). • Publishes SYCL container images for Intel GPUs in two precision flavors: `sycl-f16` and `sycl-f32` (e.g. `master-sycl-f16-core`). • Adds support for Google Gemma models via a llama.cpp dependency update.

LocalAI v2.8.0

Sat, 10 Feb 2024 00:29:52 GMT

LocalAI v2.8.0 adds Intel GPU support via SYCL container images and drops legacy ggml-based backends. • Adds Intel GPU support via new container images tagged `sycl-f16` and `sycl-f32`, e.g. `quay.io/go-skynet/local-ai:master-sycl-f32-ffmpeg-core`, enabling f16 or f32 inference on Intel hardware. • Container images are now based on Ubuntu 22.04 LTS instead of Debian Bullseye. Breaking changes: • The old ggml-based falcon backend has been removed; use the llama.cpp backend instead. • The ggml-based gpt2 and starcoder backends have been removed; these architectures are now served by llama.cpp. • The ggml-transformers backends have been dropped from container images.

LocalAI v2.7.0

Mon, 29 Jan 2024 09:13:38 GMT

LocalAI v2.7.0 adds transformer backend support for LLM text generation and remote model definition fetching. • Enables fetching model definitions remotely at startup, so models can be referenced by name without pre-downloading config files.

LocalAI v2.6.1

Tue, 23 Jan 2024 17:22:07 GMT

LocalAI v2.6.1 adds pluggable gRPC backend SPI in embedding mode • Enables gRPC backend SPI pluggability in embedding mode, allowing custom embedding backends to be wired in via the gRPC interface.

LocalAI v2.6.0

Sat, 20 Jan 2024 17:34:14 GMT

LocalAI v2.6.0 adds Mamba model support and allows building without GRPC backends. • Adds Mamba architecture support as a new extra backend, enabling local inference with Mamba-based models. • Allows building LocalAI without `GRPC_BACKENDS`, reducing build complexity for minimal deployments.

LocalAI v2.5.0

Mon, 08 Jan 2024 13:55:02 GMT

LocalAI v2.5.0 adds phi-2 and more embedded models, plus URL-based YAML model loading at startup. • Supports passing a URL pointing to a valid YAML model config file (e.g. a GitHub Gist) directly as a startup argument to load models like `llava` without pre-bundling them. • Adds `phi-2` and additional embedded models launchable by name as a direct CLI argument (e.g. `phi-2`) when starting LocalAI. • Adds model usage and description metadata to embedded model definitions.

LocalAI v2.4.1

Sat, 06 Jan 2024 00:05:28 GMT

LocalAI v2.4.1 ships embedded model configurations with popular model examples ready to use out of the box. • Adds embedded model configurations bundled directly into LocalAI, including pre-built examples for popular models, removing the need to author config files from scratch.

LocalAI v2.4.0

Thu, 04 Jan 2024 16:30:39 GMT

LocalAI v2.4.0 adds gallery model preloading, URL-based model sharing, and additional file download support. • Supports specifying additional files to download alongside a model at prepare time. • Enables preloading and preparing models directly from galleries before first use. • Adds support for sharing models by URL, allowing remote model sources to be referenced directly. • Adds `trimsuffix` template function for model configuration templating.

LocalAI v2.3.0

Fri, 29 Dec 2023 19:01:43 GMT

LocalAI v2.3.0 adds Coqui TTS, tiny-dream Stable Diffusion backend, and partial model download support. • Adds Coqui TTS backend for text-to-speech generation. • Adds tiny-dream backend for Stable Diffusion image generation. • Adds partial download support, enabling resumable model downloads. • Switches the VALL-E-X conda environment to use the transformers library.

LocalAI v2.2.0

Thu, 21 Dec 2023 16:48:41 GMT

LocalAI v2.2.0 adds inline prompt templates and URL-based model loading in YAML config files. • Supports specifying `parameters.model` as a direct HTTPS URL or `huggingface://` URI in model YAML config files, enabling on-demand model downloads without pre-staging files. • Supports inline prompt templates under `template.completion` (and siblings) directly in model YAML config files, eliminating the need for separate template files. • Transformer-based backends now share a single conda environment, reducing container image size.

LocalAI v2.1.0

Sat, 16 Dec 2023 21:04:29 GMT

LocalAI v2.1.0 adds ExLlamaV2, music generation, image/video synthesis backends, and API key file validation • Adds API key file validation support to the API layer, enabling file-based key management. • Adds the `transformers-musicgen` backend for AI-powered music generation. • Adds `exllamav2` backend support alongside restored `exllamav1` compatibility. • Adds `autopipeline` and `controlnet` support to the diffusers backend. • Adds initial support for `img2vid` and `txt2vid` (image-to-video and text-to-video) generation. • Adds CUDA-accelerated transformers backend for GPU-accelerated inference. • Adds optional preparation of extra endpoints via the entrypoint configuration. Breaking changes: • The `llama` backend alias is renamed to `llama-cpp`; any configuration referencing the `llama` backend name will break on upgrade.

LocalAI v2.0.0

Mon, 04 Dec 2023 13:22:52 GMT

LocalAI v2.0.0 adds LLaVA/GPT Vision API, parallel requests, conda environments, watchdog, and GPU-accelerated Whisper. • Adds LLaVA support and the GPT Vision API, enabling multimodal image+text inference via llama.cpp. • Adds `.yml` as an accepted extension for YAML config files (previously only `.yaml` was recognized). • Enables parallel request handling for llama.cpp, with automatic request queuing when parallelism is disabled. • Adds support for custom CA certificates for outbound connections. • Adds conda environment support for Python backends. • Adds an initial watchdog implementation to detect and recover stuck backends. • Adds embeddings support via the Transformers `AutoModel` backend. • Adds `-core` Docker images that ship without Python dependencies for leaner deployments. • Adds CUBLAS, HIPBLAS, Metal, OpenBLAS, and CLBlast acceleration support to `whisper_cpp`. • Adds the Petals backend for distributed model inference. • Adds LoRA support with scale and YaRN rope scaling in the `llama.cpp` backend. Breaking changes: • The `llama-stable` backend is renamed to `llama-ggml`; any config or tooling referencing `llama-stable` will break.

LocalAI v1.40.0

Mon, 30 Oct 2023 18:34:30 GMT

LocalAI v1.40.0 adds a native llama-cpp C++ backend, AMD ROCm/HIPBLAS support, new CLI subcommands, and Prometheus metrics. • Adds `models` CLI subcommand with `list` and `install` sub-subcommands for managing the model gallery directly from the `local-ai` binary. • Adds `tts` CLI subcommand for running text-to-speech inference from the command line. • Adds `transcript` CLI subcommand for audio transcription from the command line. • Adds initial Prometheus metrics support for observability. • Introduces the `llama-cpp` backend — a native C++ gRPC backend that tracks recent llama.cpp versions more closely and reduces abstraction layers compared to the existing `llama` (Go/C++) backend. • Enables ROCm/HIPBLAS support, adding AMD GPU acceleration to LocalAI. • Drops the `bloomz` and `ggllm` backends, as their model formats are now covered by the `llama.cpp` backend. Breaking changes: • The `bloomz` and `ggllm` backends have been removed; configurations or workflows relying on them must migrate to the `llama-cpp` backend.

LocalAI v1.30.0

Mon, 25 Sep 2023 16:49:54 GMT

LocalAI v1.30.0 adds vllm and vall-e-x backends, speculative sampling, LoRA for diffusers, and new model/job API endpoints. • Adds `GET /models/jobs` endpoint to expose all background model jobs. • Adds `PYTHON_GRPC_MAX_WORKERS` environment variable to control the maximum number of workers for the Python gRPC backend. • Adds initial `vllm` backend implementation, including support for setting quantization. • Adds `vall-e-x` backend for audio generation. • Adds speculative sampling support via a draft model field in the model config. • Adds LoRA support to the diffusers backend. • Adds mutable/refactored Model Gallery endpoints. • Allows manually setting the seed for the Stable Diffusion pipeline.

LocalAI v1.25.0

Sat, 26 Aug 2023 14:43:20 GMT

LocalAI v1.25.0 adds GGUF support, LoRA adapters, img2img diffusion, a stable llama backend, and a backend monitor shutdown endpoint. • Adds `--single-active-backend` CLI flag to allow only one backend active at a time, limiting resource contention. • Adds `llama-stable` backend as an alternative to the llama.cpp backend for users who need a pinned, stable build. • Adds LoRA adapter loading support for llama.cpp backends via model configuration. • Adds backend monitor shutdown endpoint (process-based) for programmatic backend lifecycle management. • Adds GGUF model format support via bumped llama.cpp (see breaking changes). • Adds `img2img` and `depthimg2img` pipeline support to the diffusers backend, along with `clip_skip` and additional kernel schedulers. • Adds `DPMSolverMultistepScheduler++`, `DPMSolverMultistepSchedulerSDE++`, and `guidance_scale` options to the diffusers backend. • Overcomes the prompt length limit in the diffusers backend. • Adds customizable RWKV tokenizer support. • Adds usage tracking features (API-level usage collection). • Allows restricting which backends are built via Makefile targets. • Makes the gRPC backend initializer accept configurable delay times. Breaking changes: • llama.cpp has been bumped with GGUF support added; existing GGML-format models may no longer be compatible and will require conversion or replacement with GGUF variants.

LocalAI v.1.24.0

Sat, 12 Aug 2023 14:11:17 GMT

LocalAI v1.24.0 adds Bark, AutoGPTQ, Diffusers, ExLlama, API key list support, and pre-configured model galleries. • Adds `API_KEY` list support, enabling multiple API keys to be accepted by the LocalAI server for access control. • Adds Bark text-to-audio and AutoGPTQ quantized model backends as new inference options. • Adds Diffusers backend, enabling image generation via Hugging Face Diffusers models. • Adds ExLlama backend for fast, memory-efficient inference with GPTQ-quantized LLaMA models. • Adds pre-configured LocalAI galleries, allowing one-step model discovery and installation from curated sources.

LocalAI v1.23.2

Sat, 05 Aug 2023 13:11:39 GMT

LocalAI v1.23.2 adds system prompt config templating, new model parameters, and expanded grammar primitive support. • Adds `ngqa` and `RMSNormEps` parameters to model configuration for finer control over model inference behavior. • Adds a 'simple' system prompt templating mechanism for chat, edit, and completion endpoints via model config files. • Expands grammar rules to allow integers, numbers, and strings as root types, enabling primitives as top-level output constraints.

LocalAI v1.23.1

Wed, 02 Aug 2023 16:37:38 GMT

LocalAI v1.23.1 adds a filter query string parameter to the model list endpoint. • Adds a `filter` query string parameter to the model list API, enabling callers to narrow results without fetching the full list.

LocalAI v1.23.0

Sat, 29 Jul 2023 20:53:00 GMT

LocalAI v1.23.0 adds rope settings, negative prompt support, and stream cancellation when clients disconnect. • Adds rope settings and negative prompt support to model configuration, replacing the grammar backend. • Cancels stream generation automatically when a streaming client disconnects, freeing backend resources. Breaking changes: • The grammar backend has been dropped; configurations relying on it will break on upgrade.

LocalAI v1.22.0

Sun, 23 Jul 2023 13:04:21 GMT

LocalAI v1.22.0 adds a new llama-master backend, JSONSchema ref resolution for planners, and a Llama 2 chat message template. • Adds the `llama-master` backend for running llama.cpp models via the latest upstream llama.cpp code. • Adds a chat message template for Llama 2 models to correctly format multi-turn conversation prompts. • Improves backend internals with general backends improvements in this release. Breaking changes: • The backend formerly named `llama-master` is renamed to `llama`; the backend formerly named `llama` is renamed to `llama-grammar` — any model config or API calls referencing these backend names must be updated.

LocalAI v1.21.0

Sun, 16 Jul 2023 22:35:24 GMT

LocalAI v1.21.0 adds gRPC backends, LocalAI functions, Falcon model support, and verbose_json transcription. • Adds `gRPC`-based backends, enabling a new extensible backend architecture for model serving. • Adds support for Falcon 7B and 40B models via `ggllm.cpp`. • Adds LocalAI functions capability for function-calling workflows. • Adds partial support for `verbose_json` format in the transcribe endpoint (Whisper).

LocalAI v1.20.0

Wed, 28 Jun 2023 17:18:20 GMT

LocalAI v1.20.0 adds TTS via go-piper, a model gallery repository, a `/version` endpoint, and NUMA support for llama. • Adds `GET /version` endpoint and displays the version in the CLI. • Renames `/models/list` to `/models/available` for listing available models. • Adds text-to-speech (TTS) support via `go-piper` integration. • Introduces a gallery repository for browsing and managing models. • Enables NUMA support in the updated llama backend, improving multi-socket CPU performance. • Supports `$THREADS` environment variable in `entrypoint.sh` to control parallel compilation at startup. Breaking changes: • The `/models/list` endpoint is renamed to `/models/available`; any client or script calling `/models/list` will break.

LocalAI v1.19.2

Tue, 20 Jun 2023 19:36:11 GMT

LocalAI v1.19.2 adds a LowVRAM option parameter for constrained GPU environments. • Adds `LowVRAM` option parameter to enable inference on systems with limited GPU memory.

LocalAI v1.19.0

Mon, 19 Jun 2023 13:31:51 GMT

LocalAI v1.19.0 adds `typical_p` sampling, extended llama.cpp config, full GPU offloading, and model download progress display. • Adds `typical_p` parameter to model configuration for llama.cpp-based models. • Extends model configuration options for llama.cpp backends. • Enables full GPU offloading via updated go-llama bindings, with fixed CUDA images. • Displays download progress when installing models via the model gallery.

LocalAI v1.18.0

Tue, 06 Jun 2023 21:25:04 GMT

LocalAI v1.18.0 adds streaming completions, esrgan upscaling, falcon-7b, Metal GPU, k-quants, and a HuggingFace backend via LangChainGo. • Adds `Stream: true` support to the completion endpoint, enabling server-sent event streaming for inference responses. • Adds LangChainGo HuggingFace backend, allowing LocalAI to route completions through HuggingFace-hosted models via LangChainGo. • Adds experimental support for falcon-7b models. • Adds upscaled image generation using esrgan. • Adds Metal GPU support for gpt4all on Apple Silicon. • Updates llama.cpp to support k-quants model quantization formats. • Enables Stable Diffusion by default in container images. • Adds ffmpeg-enabled Docker image variants for audio processing use cases. • Enables static builds for Linux binaries, removing dynamic library dependencies. • Builds multiple Docker image variants via CI, expanding deployment options. Breaking changes: • gpt4all is updated with support for multiple implementations selectable at runtime; existing gpt4all configurations may need to be updated to specify the desired implementation.

LocalAI v1.17.1

Tue, 30 May 2023 14:15:48 GMT

LocalAI v1.17.1 adds CuBLAS support to Docker images for GPU-accelerated inference. • Adds CuBLAS support to Docker images, enabling GPU-accelerated model inference without a separate build.

LocalAI v1.17.0

Mon, 29 May 2023 17:10:00 GMT

LocalAI v1.17.0 adds CLBlast GPU acceleration support and bumps the RWKV backend. • Adds CLBlast support, enabling OpenCL-based GPU acceleration for inference. • Bumps the RWKV backend to a newer revision. Breaking changes: • The RWKV backend has been bumped to a new version, which may break existing RWKV model configurations or bindings.

LocalAI v1.16.0

Sat, 27 May 2023 20:53:03 GMT

LocalAI v1.16.0 adds Kubernetes health endpoints, static builds, model preloading via env var, and prompt cache path support. • Adds `/healthz` and `/readyz` HTTP endpoints for Kubernetes liveness and readiness probes. • Allows preloading models before startup via environment variable or config files. • Adds support for setting a prompt cache path and enabling state saving. • Adds static builds of LocalAI. • Adds Docker image `HEALTHCHECK` support.