LocalAI

Summary

LocalAI is an open-source AI engine that runs models for text, vision, voice, and image on any hardware without requiring a GPU. It is licensed under the MIT license and operates as a self-hosted service, exposing APIs compatible with OpenAI and Anthropic. The tool is designed for users who need to run diverse models locally and fits into the category of local-llm-runtime cyber tools. It has a clear, minimal structure that pulls only necessary model backends on demand.

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

Command line

Values are placeholders taken from each option’s declared default. Nothing is executed here — the output shown is a recording of a run that already happened.

Release history

1. v4.9.0 Aug 20, 2026 · issue 003

   LocalAI 4.9.0: deny-by-default auth, chat context compression, KNN routing, PII pseudonyms, parallel HF downloads, and MiniMax-H3 video+audio

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch v4.9.0 https://github.com/mudler/LocalAI.git
   # already have the repo? check out this version:
   $ git checkout v4.9.0

   
   └──▷ TRY IT
   Expose a single fixed UDP port for all Realtime WebRTC sessions so one firewall rule covers all calls.
   copy

   $ docker run -ti --name local-ai -p 8080:8080 -p 3478:3478/udp -e LOCALAI_WEBRTC_UDP_PORT=3478 localai/localai:latest

   Seed the KNN router corpus with a labelled example so the router can classify similar prompts by similarity-weighted voting without a classifier model.
   copy

   $ curl -X POST http://localhost:8080/api/router/my-router/corpus \
     -H 'Authorization: Bearer <token>' \
     -H 'Content-Type: application/json' \
     -d '{"text": "What is my account balance?", "label": "finance"}'

   • ›Adds --web-rtc-udp-port CLI flag and LOCALAI_WEBRTC_UDP_PORT env var to reuse a single shared Pion ICE UDP mux across all Realtime WebRTC calls, simplifying container and firewall setup.
   • ›Adds POST /api/router/{name}/corpus, GET /api/router/{name}/corpus/stats, and DELETE /api/router/{name}/corpus endpoints to seed and manage the KNN router's labelled prompt corpus (admin-gated, also exposed as MCP tools).
   • ›Introduces classifier: knn router type: similarity-weighted voting over a persisted JSONL corpus under <data path>/router-corpus, with a configurable knn.similarity_threshold; prompts below threshold yield an undecidable fallback rather than a forced guess.
   • ›Adds opt-in pii.reverse_in_response config to turn masked PII values into request-scoped deterministic pseudonyms (EMAIL_001, EMAIL_002) that are restored when the backend echoes them, including across SSE tokens split over writes.
   • ›Adds opt-in per-model compression config for chat context compression: older complete turns are compressed through a configured LocalAI model before inference, preserving system prompts, newest messages, and whole tool-call/result units; ratio and duration are returned as response metadata and metrics.
   +9 moreshow less

   • ›Authentication is now deny-by-default: every HTTP route requires credentials unless listed in an explicit public registry, closing bypasses for unprefixed aliases such as /moderations, /models, /backends, and /mcp/chat/completions; narrow public paths can be added via ApplicationConfig.PathWithoutAuth.
   • ›Adds MiniMax-H3 video+audio generation to vllm-cpp (ABI v12): a second engine handle loads the H3 checkpoint set (parameters.model for the DiT; text encoder and two VAEs named in options:), and GenerateVideo renders jointly into an MP4 with a real AAC audio track.
   • ›Adds Qwen3-TTS to the llama-cpp backend across CUDA, ROCm, SYCL, Vulkan, Metal, and L4T, using upstream GGUF conversion; gallery entries qwen3-tts-llamacpp and qwen3-tts-llamacpp-q4 provided.
   • ›Expands vllm-cpp CUDA architecture coverage on amd64 from 120a;121a to 80;86;89;90a;100a;103a;120a;121a, and on arm64 from 121a to 87;90a;100a;110;121a, adding support for A100, L4, 4090, H100/H200, B200, Jetson Orin, and Jetson Thor.
   • ›Adds global process-wide HTTP admission control bounding in-flight backend operations, with running backend traces now visible in the UI alongside direct log links.
   • ›Cold model loads now run as durable jobs rather than holding the per-model advisory lock across multi-GB transfers, preventing loads from appearing permanently broken during staging.
   • ›Unifies the model and backend UI: /app/models owns Explore and Installed views; /app/backends owns Catalog and Installed; /app/manage redirects with legacy query state preserved.
   • ›Adds Portuguese (Brazil) (pt-BR) UI translation with full 14-namespace key parity, and Indonesian translation covering admin, media, and navigation surfaces.
   • ›Adds 85 new model gallery entries including Qwen3 (9B, 27B), Gemma 4 Scotoma 2, DeepSeek V4 Pro 0813, Nemotron 3.5 Lightning 30B, MiniMax-H3 Ref2VA (minimax-h3-fl2va-q4, minimax-h3-ref2va-q4), Higgs Audio v3 TTS, and others.
   └──▷ BREAKING ON UPGRADE

   • !With database auth or legacy API keys configured, /version and generated audio, image, video, and 3D asset URLs now require credentials by default due to the deny-by-default authentication change.
   • !Embedded deployments that previously relied on unauthenticated access to routes like /moderations, /models, /backends, or /mcp/chat/completions must explicitly add those paths to ApplicationConfig.PathWithoutAuth.
2. v4.8.2 Aug 7, 2026 · issue -012

   LocalAI v4.8.2 adds the NVIDIA NeMo-Speech.cpp backend and improves gallery index resilience with mirror fallback.

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch v4.8.2 https://github.com/mudler/LocalAI.git
   # already have the repo? check out this version:
   $ git checkout v4.8.2

   • ›Adds the NVIDIA NeMo-Speech.cpp backend for speech processing.
   • ›Gallery now defaults to index.localai.io as the primary index source, with GitHub as a mirror fallback when the primary source fails.
3. v4.8.2 Aug 7, 2026 · issue 002

   LocalAI v4.8.2 adds the NVIDIA NeMo-Speech.cpp backend and gallery mirror/offline fallback support.

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch v4.8.2 https://github.com/mudler/LocalAI.git
   # already have the repo? check out this version:
   $ git checkout v4.8.2

   └──▷ USE IT
   Configure a gallery with an internal mirror so model listing works even when the primary index is unreachable.
   json copy

   [
     {
       "name": "localai",
       "url": "https://my-internal-mirror.corp/index.yaml",
       "mirrors": ["github:mudler/LocalAI/gallery/index.yaml@master"]
     }
   ]

   Override the NeMo-Speech.cpp ASR shared library path when the default library name does not match your installation.
   copy

   $ export NEMO_SPEECH_ASR_LIBRARY=/opt/nemo/libnemo_speech_asr_c.so
   export NEMO_SPEECH_TTS_LIBRARY=/opt/nemo/libnemo_speech_tts.so
   export NEMO_SPEECH_NMT_LIBRARY=/opt/nemo/libnemo_speech_nmt_c.so
   docker run -ti --name local-ai -p 8080:8080 --gpus all \
     -e NEMO_SPEECH_ASR_LIBRARY \
     -e NEMO_SPEECH_TTS_LIBRARY \
     -e NEMO_SPEECH_NMT_LIBRARY \
     localai/localai:latest-gpu-nvidia-cuda-12

   • ›Adds NEMO_SPEECH_ASR_LIBRARY, NEMO_SPEECH_TTS_LIBRARY, and NEMO_SPEECH_NMT_LIBRARY environment variables to override shared-library paths for the new NVIDIA NeMo-Speech.cpp backend, which provides ASR, TTS, and NMT capabilities via a gRPC server.
   • ›Adds mirrors list to gallery configuration so LocalAI tries the primary URL, then each mirror in order, keeping failed sources out of rotation for 10 minutes; if all sources fail, the last successfully fetched index is served from disk.
   • ›New NVIDIA NeMo-Speech.cpp backend (nemo-speech-cpp) supports ASR, TTS, diarization, punctuation/capitalization, inverse text normalization, and neural machine translation, configured via key:value options including vad_model, pnc_model, diar_model, itn_dir, language_code, codec_model, tokenizer_dir, tn_dir, source_language, and target_language.
   • ›Gallery index now defaults to index.localai.io with GitHub as a mirror, improving resilience for air-gapped or rate-limited environments.
4. v4.8.1 Aug 6, 2026 · issue -013

   LocalAI v4.8.1 adds Qwen3.5 9B gallery models and exposes the full vllm-cpp engine config surface via engine_args.

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch v4.8.1 https://github.com/mudler/LocalAI.git
   # already have the repo? check out this version:
   $ git checkout v4.8.1

   • ›Exposes the full vllm-cpp engine configuration surface through engine_args, allowing fine-grained control over backend engine parameters.
   • ›Adds Qwen3.5 9B HauhauCS variants to the model gallery.
   • ›Adds Qwen3.5 9B Defiant Fable variants to the model gallery.
5. v4.8.1 Aug 6, 2026 · issue 002

   LocalAI v4.8.1 adds full vLLM engine config via engine_args and two Qwen3.5 9B gallery variants.

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch v4.8.1 https://github.com/mudler/LocalAI.git
   # already have the repo? check out this version:
   $ git checkout v4.8.1

   • ›Adds engine_args config key to the vLLM-cpp backend, accepting a JSON object whose keys mirror vLLM's own CLI flags verbatim — including speculative_config and kv_transfer_config — so existing vLLM configs work without translation; engine_args wins over the older options: list when both set the same key.
   • ›Adds Qwen3.5 9B HauhauCS and Qwen3.5 9B Defiant Fable variants to the model gallery.
6. v4.8.0 Aug 5, 2026 · issue -014

   LocalAI 4.8.0 adds vllm.cpp alpha backend, 3D generation, audio.cpp, VRAM budgets, two new TTS engines, and more

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch v4.8.0 https://github.com/mudler/LocalAI.git
   # already have the repo? check out this version:
   $ git checkout v4.8.0

   └──▷ TRY IT
   Cap GPU memory to 12 GB on a shared node so LocalAI does not crowd out other workloads.
   copy

   $ local-ai --vram-budget 12GB

   • ›Adds POST /v1/3d/generations endpoint, Generate3D RPC, and FLAG_3D capability for 3D generation as a new modality, backed by the trellis2cpp image-to-3D backend and a UI page with a native GLB viewer and print remeshing.
   • ›Adds LOCALAI_VRAM_BUDGET environment variable and --vram-budget CLI flag (accepts 80% or 12GB) to cap GPU memory use per node, with admin API endpoints PUT /api/nodes/:id/vram-budget and DELETE /api/nodes/:id/vram-budget, and an set_node_vram_budget MCP tool.
   • ›Adds --disable-http-compression and --http-compression-min-length flags; /api/traces and /api/backend-traces now accept limit, offset, and full query parameters with per-record access via GET /api/traces/{id}.
   • ›Adds GET /api/models?has_variants=true to filter gallery models with variant ladders, with variant selection via variant on POST /models/apply, local-ai models install --variant, the install_model MCP tool, or the UI split-button.
   • ›Ships vllm-cpp as an alpha backend: a C++20 engine (no Python, PyTorch, or ggml at inference) implementing vLLM V1 architecture (paged KV cache, continuous batching, prefix caching), loading safetensors and GGUF, with in-engine structured output (JSON schema, regex, GBNF), running on CPU, CUDA 12/13, Metal, and Vulkan.
   +10 moreshow less

   • ›Ships audio-cpp backend serving six endpoints — /v1/audio/speech, /v1/audio/transcriptions, /v1/audio/vad, /v1/audio/diarize, /audio/transform, and /v1/sound-generation — with family auto-detected from the GGUF audiocpp.model_spec.family metadata key.
   • ›Ships magpie-tts-cpp backend for NVIDIA Magpie TTS Multilingual (5 voices, 9+ languages, 22.05 kHz mono, single self-contained GGUF).
   • ›Ships moss-tts-cpp backend for MOSS-TTS-Local v1.5 (48 kHz stereo, optional reference-audio voice cloning, no Python at inference).
   • ›Ships bonsai backend for Q1_0 and Q2_0 quant formats, covering Bonsai 8B/27B and Ternary-Bonsai 8B/27B models via eight gallery entries.
   • ›Ships valkey-store backend adding Valkey Search as a vector store option.
   • ›Adds a new admin /app/activity page tracking install, failure, and cancellation events with a 50-entry bounded history, per-node in-progress breakdown, and a 'needs attention' lane with Cancel and Retry actions.
   • ›Gallery entries can now declare variants: so LocalAI automatically selects the largest build that fits the host's VRAM and RAM at install time.
   • ›Adds Hugging Face artifact materialization with immutable snapshot resolution, authenticated downloads with progress reporting, and staged artifact binding for remote workers.
   • ›Adds systemd socket activation support.
   • ›Inline GRPO reward code in POST /api/fine-tuning/jobs is now blocked by default; operators must set LOCALAI_TRL_ALLOW_INLINE_REWARD=true to re-enable it.
7. v4.8.0 Aug 5, 2026 · issue 002

   LocalAI 4.8.0 adds vllm.cpp, 3D generation, audio.cpp, VRAM budgets, gallery variants, two new TTS engines, bonsai sub-2-bit models, and a Valkey vector store.

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch v4.8.0 https://github.com/mudler/LocalAI.git
   # already have the repo? check out this version:
   $ git checkout v4.8.0

   
   └──▷ TRY IT
   Cap GPU memory on a worker node to avoid OOM evictions when running multiple models concurrently.
   copy

   $ local-ai worker --vram-budget 80%

   Install the best-fit quantization of a model for your hardware without manually hunting for the right gallery entry.
   copy

   $ local-ai models install --variant q4_k_m apex-llama-3.1-8b

   Fetch a single large trace record by ID without pulling the full uncompressed payload from the summary endpoint.
   copy

   $ curl http://localhost:8080/api/traces/$(curl -s 'http://localhost:8080/api/traces?limit=1' | jq -r '.[0].id')

   • ›Adds POST /v1/3d/generations endpoint, Generate3D RPC, and FLAG_3D capability for a new 3D generation modality, backed by the trellis2cpp image-to-3D backend and a UI page with a native GLB viewer and print remeshing.
   • ›Adds LOCALAI_VRAM_BUDGET environment variable (e.g. 80% or 12GB) and --vram-budget CLI flag on local-ai and local-ai worker to cap GPU memory use per node; overridable via PUT/DELETE /api/nodes/:id/vram-budget and the set_node_vram_budget MCP tool.
   • ›Adds GET /api/traces/{id} endpoint and limit/offset/full query parameters on GET /api/traces and GET /api/backend-traces for paginated, summarized trace access.
   • ›Adds --disable-http-compression and --http-compression-min-length flags to control gzip response compression; streaming paths are skipped automatically. React bundle shrinks 3.48x (2.8 MB to 808 KB); backend-trace poll drops from 21 MB to 7 KB.
   • ›Adds variants: key to gallery entries so LocalAI selects the largest build that fits the host's hardware and VRAM at install time; override with variant on POST /models/apply, local-ai models install --variant, the install_model MCP tool, or the UI split-button.
   +13 moreshow less

   • ›Adds GET /api/models?has_variants=true query parameter to filter the model list to entries that declare variants.
   • ›Adds the audio-cpp backend serving six audio endpoints from one process — /v1/audio/speech, /v1/audio/transcriptions, /v1/audio/vad, /v1/audio/diarize, /audio/transform, and /v1/sound-generation — with model family selected from the GGUF's own audiocpp.model_spec.family metadata key.
   • ›Adds the vllm-cpp backend (alpha development builds) — a C++20 engine with paged KV cache, continuous batching, prefix caching, structured output, safetensors and GGUF loading — running on CPU, CUDA 12/13, Metal, and Vulkan with no Python or PyTorch at inference.
   • ›Adds the magpie-tts-cpp TTS backend for NVIDIA Magpie TTS Multilingual 357M: 5 voices, 9+ languages, 22.05 kHz mono, single self-contained GGUF.
   • ›Adds the moss-tts-cpp TTS backend for MOSS-TTS-Local v1.5: 48 kHz stereo output with optional reference-audio voice cloning, no Python at inference.
   • ›Adds the bonsai backend (via PrismML llama.cpp fork) as the only decoder for Q1_0 and Q2_0 quant formats, covering Bonsai 8B/27B and Ternary-Bonsai 8B/27B (~1.15 GB and up).
   • ›Adds the valkey-store backend, making Valkey Search available as a vector store option.
   • ›Adds an admin /app/activity page with per-node in-progress breakdown, a 'needs attention' lane with Cancel and Retry, and a bounded 50-entry history of completed, failed, and cancelled operations.
   • ›Adds Hugging Face artifact materialization with immutable snapshot resolution, authenticated downloads with progress reporting, staged artifact binding for remote workers, and Python backend reuse of the Go download path.
   • ›Adds LOCALAI_TRL_ALLOW_INLINE_REWARD=true environment variable required to permit inline GRPO reward code in POST /api/fine-tuning/jobs; inline reward code is refused by default.
   • ›Adds systemd socket activation support.
   • ›Adds gallery entries for MiniMax-M3, Gemma 4 llama.cpp MTP variants, Qwen3.5-4B DFlash, MOSS-TTS-Local v1.5, APEX families as variant ladders, and Bonsai families, growing the gallery from 1,221 to 1,515 entries.
   • ›Hardens distributed mode: reaper now queries workers directly over a models.running subject before terminating a backend, port-probe fallback requires three consecutive misses to declare a node gone, frontend model stubs are dropped only when no healthy replica remains, and model-load deadlines scale with checkpoint size.
8. v4.7.0 Jul 14, 2026 · issue -036

   LocalAI 4.7.0 adds voice cloning profiles, local video/avatar generation, diarized transcription, streaming TTS, and auto full-context resolution.

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch v4.7.0 https://github.com/mudler/LocalAI.git
   # already have the repo? check out this version:
   $ git checkout v4.7.0

   └──▷ USE IT
   Run a model at its full trained context window without hardcoding a size, useful when a gallery YAML pins a smaller value.
   yaml copy

   context_size: -1

   Restrict llama.cpp GPU offload to specific devices to exclude a display GPU or isolate inference to particular cards.
   yaml copy

   options:
     - devices:CUDA1,CUDA2

   Reference a saved voice cloning profile from the TTS endpoint to synthesize speech without hand-editing YAML or copying audio files.
   copy

   $ curl -X POST http://localhost:8080/v1/audio/speech \
     -H 'Content-Type: application/json' \
     -d '{"model": "qwen3-tts-cpp", "input": "Hello, world!", "voice": "localai://voice-profiles/my-profile"}'

   • ›Adds GET /api/voice-profiles, POST /api/voice-profiles, DELETE /api/voice-profiles/:id, and GET /api/voice-profiles/:id/audio REST endpoints for admin-managed voice cloning profiles; profiles are referenceable via stable localai://voice-profiles/<id> URIs from /v1/audio/speech or the TTS UI.
   • ›Adds MCP admin tools list_voice_profiles, create_voice_profile, and delete_voice_profile for voice profile management.
   • ›Adds typed tts.voice_cloning boolean config key to opt custom model names into or out of Voice Library compatibility; setting it to false rejects saved profile references with HTTP 400.
   • ›Adds context_size: -1 sentinel (any negative value, or LOCALAI_CONTEXT_SIZE=-1 / --context-size -1 globally) to resolve a model's context window to its full trained n_ctx_train from GGUF metadata at load time, with a VRAM-fit warning and a safe clamp so no backend receives a negative value.
   • ›Adds devices: option to the llama.cpp options: array (e.g., options: [devices:CUDA1,CUDA2]) to restrict GPU offload to specific named devices from llama-server --list-devices.
   +9 moreshow less

   • ›Adds known_input_modalities and known_output_modalities config fields for declarative capability metadata, allowing generic code to discover what a model checkpoint accepts without branching on backend or checkpoint names.
   • ›Accepts reasoning_content as an inbound alias for the canonical reasoning field in OpenAI chat messages (vLLM/DeepSeek/cogito-style clients), enabling interleaved reasoning and tool calls in a single assistant turn.
   • ›Supports Anthropic Messages thinking blocks on the local path: inbound thinking blocks parse into message reasoning and are emitted before tool_use on streaming and non-streaming responses, gated on thinking: {type: 'enabled'} in the request.
   • ›New longcat-video Python backend supports text-to-video, image-to-video, and audio-driven talking-avatar generation; gallery entries longcat-video and longcat-video-avatar-1.5 available for CUDA 12/13 x86_64 and CUDA 13 ARM64.
   • ›New moss-transcribe-cpp Go backend performs joint multi-speaker transcription, diarization, and timestamps in a single offline pass; gallery model moss-transcribe-cpp-0.9b (default q5_k GGUF), 1.6–2.2x faster than reference on CPU.
   • ›Links F5-TTS static runtime into the CrispASR build; new f5-tts-crispasr gallery model produces 24kHz mono audio with voice cloning via options: [voice:/path/ref.wav, voice_text:Transcript...].
   • ›Replaces whole-clip-then-chunk synthesis in vibevoice-cpp with real incremental streaming via the new vv_capi_tts_stream callback ABI, reducing time-to-first-audio from 39.96s to 2.38s (~17x) on CPU for VibeVoice-Realtime-0.5B.
   • ›Adds model-load failure cooldown: failed loads return 503 + Retry-After during a backoff window (default 10s, geometric growth capped at 5 minutes) instead of respawning a crashing backend and leaking GPU memory.
   • ›Adds four Qwen DFlash speculative-decoding gallery entries (4B, 9B, 27B, and 35B-A3B), each bundling target and drafter models with spec_type:draft-dflash.
9. v4.6.2 Jul 6, 2026 · issue -044

   LocalAI v4.6.2 adds MiniCPM series models to the model gallery.

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch v4.6.2 https://github.com/mudler/LocalAI.git
   # already have the repo? check out this version:
   $ git checkout v4.6.2

   • ›Adds MiniCPM series models to the model gallery.
10. v4.6.1 Jul 6, 2026 · issue -044

    LocalAI v4.6.1 adds a GET /v1/models/capabilities endpoint and native Prometheus metrics for agent chat runs.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.6.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.6.1

    └──▷ TRY IT
    Discover which capabilities (e.g. chat, embeddings, vision) a deployed model supports before routing requests to it.
    copy

    $ curl -s http://localhost:8080/v1/models/capabilities | jq .

    • ›Adds GET /v1/models/capabilities API endpoint to query per-model capability metadata.
    • ›Adds native Prometheus metrics for agent chat runs.
11. v4.6.0 Jul 4, 2026 · issue -046

    LocalAI v4.6.0 adds a load API, PII Prometheus counter, chat forking, and AMD ROCm GPU fixes

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.6.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.6.0

    └──▷ TRY IT
    Pre-warm a specific model into VRAM before the first request arrives, avoiding cold-start latency for users.
    copy

    $ curl -X POST http://localhost:8080/backend/load -H 'Content-Type: application/json' -d '{"model": "mistral-7b"}'

    Keep a realtime pipeline from blocking on warm-up for a low-latency session where cold-start is acceptable.
    yaml copy

    disable_warmup: true

    • ›Adds POST /backend/load and POST /v1/backend/load endpoints to pre-warm any model (including full pipelines) on demand, complementing the existing /backend/shutdown.
    • ›Adds disable_warmup: true per-pipeline config key to opt out of eager realtime session warm-up.
    • ›Adds LOCALAI_BACKEND_PARENT_WATCH environment variable (default on, auto-off on Windows) to make spawned backend workers self-terminate when the parent LocalAI process dies.
    • ›Adds LOCALAI_BACKEND_PARENT_WATCH_INTERVAL environment variable (default 2s) to control how frequently backend workers poll for parent-process death.
    • ›Adds "force": true field to POST /backends/apply to force re-download of an already-installed backend; installs are now idempotent by default.
    +7 moreshow less

    • ›Bundles HIPBLASLT_TENSILE_LIBPATH export in llama-cpp and turboquant run.sh, enabling full-speed hipBLASLt kernel-tuning data for AMD ROCm backends.
    • ›Adds a load_model MCP admin tool and 'Load into memory' UI button, both backed by the new POST /backend/load endpoint.
    • ›Adds LOCALAI_EXTERNAL_BACKENDS boot loop idempotency so backends are not re-pulled on every restart unless force: true is passed.
    • ›Adds conversation-forking to the built-in React chat UI: regenerate any assistant answer, branch a new chat from any turn, duplicate a chat, or copy the whole conversation as Markdown.
    • ›Realtime voice sessions now eagerly warm the full pipeline (VAD, ASR, LLM, TTS) at session start, with concurrent sub-model loading so warm-up time equals the slowest stage, not the sum.
    • ›AMD ROCm rocm-vllm backend now installs from the ROCm wheel index (https://wheels.vllm.ai/rocm/) pinned to CPython 3.12, replacing the CUDA-only PyPI wheel.
    • ›AMD ROCm ggml audio backends (rocm-qwen3-tts-cpp, rocm-omnivoice-cpp, acestep-cpp, vibevoice-cpp) now build with -DGGML_HIP=ON and link the HIP backend for real GPU offload.
12. v4.5.6 Jun 30, 2026 · issue -050

    LocalAI v4.5.6 adds native GGML voice/face-detect backends, semantic VAD EOU token for realtime, and distributed shared-model staging control via LOCALAI_DISTRIBUTED_SHARED_MODELS.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.5.6 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.5.6

    • ›Adds LOCALAI_DISTRIBUTED_SHARED_MODELS environment variable to skip model staging on shared volumes in distributed deployments.
    • ›Adds native voice-detect and face-detect GGML backends, replacing the previous Python insightface/speaker-recognition dependencies.
    • ›Adds a Semantic VAD end-of-utterance (EOU) token to the realtime API.
    • ›Introduces a SyncedMap component in distributed mode and migrates finetune, quantization, and agent-task state to cross-replica synchronisation.
13. v4.5.3 Jun 26, 2026 · issue -054

    LocalAI v4.5.3 ships signed and notarized macOS binaries for the DMG, app, and server.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.5.3 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.5.3

    • ›Signs and notarizes the macOS DMG, app bundle, and server binary, enabling Gatekeeper-approved distribution on macOS.
14. v4.5.1 Jun 26, 2026 · issue -054

    LocalAI v4.5.1 adds Metal/MPS support across multiple backends, new llama-cpp MoE options, UI enhancements, and a PII detector environment variable.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.5.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.5.1

    └──▷ USE IT
    Offload Mixture-of-Experts layers to CPU to reduce GPU VRAM pressure when running large MoE models via llama-cpp.
    yaml copy

    cpu_moe: true
    n_cpu_moe: 8

    • ›Adds LOCALAI_PII_DEFAULT_DETECTORS environment variable to load default PII detectors at startup.
    • ›Adds cpu_moe and n_cpu_moe options to llama-cpp configuration, plus generic upstream-flag passthrough for llama-cpp backends.
    • ›Adds darwin/Metal (MPS) build support for vllm via vllm-metal (MLX backend).
    • ›Adds darwin/Metal builds for liquid-audio and trl backends.
    • ›Adds darwin/Metal builds for vision C++/ggml backends: depth-anything, locate-anything, rfdetr-cpp, and sam3-cpp.
    +6 moreshow less

    • ›Adds darwin/Metal build for the localvqe acoustic echo cancellation backend.
    • ›Adds darwin/Metal build for the privacy-filter backend.
    • ›Adds single-build ggml CPU_ALL_VARIANTS for llama-cpp and turboquant covering x86, arm64, and Apple silicon.
    • ›Adds UI enhancements: last-used model tracking, polling, starter models, usage cost display, and accessibility improvements.
    • ›Adds data-driven hardware model recommendations and gallery surfacing to the UI.
    • ›Makes PreferDevelopmentBackends install the development image as the primary image.
15. v4.5.0 Jun 23, 2026 · issue -057

    LocalAI 4.5.0 adds four new backends (depth-anything, ced, supertonic, privacy-filter.cpp), speaker-aware realtime voice with compaction, and concurrent serving on by default.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.5.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.5.0

    └──▷ USE IT
    Enable conversation compaction on a long-running realtime session to keep LLM context costs bounded while preserving a rolling summary of older turns.
    yaml copy

    pipeline:
      max_history_items: 6
      compaction:
        enabled: true
        trigger_items: 12
        summary_model: ""
        max_summary_tokens: 512

    • ›New depth-anything backend exposes POST /v1/depth returning depth map, stats, and camera extrinsics (3×4) / intrinsics (3×3); also reachable via a typed Depth gRPC RPC and GenerateImage/Predict — no Python at inference.
    • ›New ced backend exposes POST /v1/audio/classification for 527-class AudioSet sound tagging (baby cry, glass breaking, alarms); opt-in pipeline.sound_detection emits conversation.item.sound_detection events in realtime, decoupled from VAD via turn_detection: none.
    • ›New supertonic ONNX TTS backend supports voice, language, steps, speed, and silence request knobs across 10 voice styles (F1–F5 / M1–M5) — no Python, no espeak-ng/G2P.
    • ›New privacy-filter.cpp backend adds NER token-classification PII filtering (encoder scans whole conversations as a single document) alongside a restricted-regex secret detector; detections labelled by source (ner vs pattern) with confidence and analyze/redact API; request filtering extended to completions, embeddings, edits, and Ollama.
    • ›Realtime voice sessions gain pipeline.voice_recognition config keys including enforce: false to surface speaker identity to the client via a new conversation.item.speaker event and feed it to the LLM via per-message name field.
    +10 moreshow less

    • ›Conversation compaction via pipeline.compaction config block — keys enabled, trigger_items, summary_model, and max_summary_tokens — rolls aged-out turns into a rolling summary instead of dropping them; pipeline.max_history_items controls the live verbatim window.
    • ›Adds OpenAI-parity realtime history events conversation.item.delete, conversation.item.truncate, and input_audio_buffer.clear.
    • ›Prefix caching (n_cache_reuse) now defaults to 256 (was 0), so system prompts, RAG context, and multi-turn chat prefixes are reused across requests without configuration.
    • ›VRAM-scaled n_parallel default enables continuous batching out of the box: ≥32 GiB → 8 slots, ≥8 GiB → 4, ≥4 GiB → 2, <4 GiB → 1 (unchanged); explicit config always wins.
    • ›Default n_batch/n_ubatch raised to 2048 on NVIDIA Blackwell consumer GPUs (sm_120/121, including GB10 / DGX Spark) when batch: is unset.
    • ›New alias: field in model config transparently routes all traffic for one model name to another configured model, swappable live without client reconfiguration; alias→alias chains are rejected at load.
    • ›Word-level ASR timestamps supported for NeMo and CrispASR, plumbed through the gRPC transcription path.
    • ›Eight depth-anything GGUFs ship at mudler/depth-anything.cpp-gguf (base/small/large/giant + mono-large, q4_k/q8_0/f16/f32); eight ced GGUFs at mudler/ced-gguf (6 MB–86 MB); Depth Anything V2 gallery models also added.
    • ›Redesigned React UI ships end-to-end: new design language, NER-centric PII editor, detector-models table, middleware default-policy controls, sortable/mobile tables, unsaved-changes guards, and restructured Cluster Nodes view.
    • ›Distributed staging detached from the request context so large models finish cold-loading; staging progress broadcast across replicas; resumable downloader added.
16. v4.4.3 Jun 13, 2026 · issue -067

    LocalAI v4.4.3 adds open-vocabulary object detection, OmniVoice TTS with voice cloning, Kokoro TTS, 60 new Piper voices, and declarative distributed scheduling.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.4.3 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.4.3

    • ›Adds declarative per-model scheduling for distributed deployments, configurable via environment variables and CLI arguments (feat(distributed): declarative per-model scheduling via env/args).
    • ›Adds locate-anything-cpp backend for open-vocabulary object detection via ggml.
    • ›Adds omnivoice-cpp backend supporting OmniVoice TTS with file and streaming output, voice cloning, and voice design.
    • ›Adds Kokoro TTS and multilingual Piper voices to the sherpa-onnx backend.
    • ›Migrates qwen3-tts-cpp to ServeurpersoCom/qwentts.cpp, adding streaming, multiple speakers, and voice design support.
    +6 moreshow less

    • ›Bundles espeak-ng with crispasr and adds Piper TTS voices to the model gallery.
    • ›Expands the gallery with 60 Piper TTS voices across 42 languages (Phase 2).
    • ›Enables GGML_CUDA_GRAPHS in the CUBLAS build for parakeet-cpp, improving GPU inference performance.
    • ›Adds voice-recognition gating for realtime pipeline voice models.
    • ›Adds Indonesian language support to the React UI.
    • ›Adds Korean (ko) translation to the UI.
17. v4.4.1 Jun 11, 2026 · issue -069

    LocalAI v4.4.1 adds streaming across the LLM, TTS, and transcription pipeline stages in the realtime feature.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.4.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.4.1

    • ›Streams the LLM, TTS, and transcription pipeline stages in the realtime feature, enabling lower-latency end-to-end audio/language workflows.
18. v4.4.0 Jun 10, 2026 · issue -070

    LocalAI v4.4.0 adds two ASR backends, object detection, video I/O, distributed prefix-cache routing, PII middleware, and a local-ai chat CLI.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.4.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.4.0

    └──▷ TRY IT
    Request TTS with a per-request voice style description using the new instructions field — useful for dynamic emotion/persona without changing model YAML config.
    copy

    $ curl http://localhost:8080/v1/audio/speech -H "Content-Type: application/json" -d '{"model": "qwen-tts-design", "input": "Hello world, this is a test.", "instructions": "A calm, low-pitched elderly storyteller with a warm tone."}'

    • ›Adds LOCALAI_FILE_TRANSFER_BUDGET environment variable (default 1h) to control the time budget for resumable large-model GGUF transfers over the distributed worker PUT /v1/files/<key> endpoint, with Content-Range / 308/416 resume semantics and X-Content-SHA256 binding.
    • ›Adds --distributed-prefix-cache=false flag to disable the new prefix-cache-aware routing in distributed mode (on by default), which biases requests toward the replica already holding the relevant KV/prefix cache.
    • ›Adds LOCALAI_PREFETCH_MODELS environment variable for boot-time gallery model prefetch in distributed deployments.
    • ›Plumbs instructions string and params map through the /v1/audio/speech request body end-to-end (proto, schema, core/backend/tts.go), enabling per-request voice/style control for Qwen3-TTS VoiceDesign, Qwen3-TTS CustomVoice, and Chatterbox.
    • ›New crispasr backend (purego, CGO_ENABLED=0) serving ASR or TTS from a single binary, auto-detecting architecture from GGUF; supports backend:, codec:, speaker:, and voice: model options, with 36 gallery entries (32 ASR + 4 TTS including vibevoice, chatterbox, qwen3-tts, orpheus).
    +10 moreshow less

    • ›New rfdetr-cpp native gRPC backend dlopening librfdetr.so for RF-DETR object detection and segmentation, returning bbox + class_name + confidence and per-detection PNG-encoded masks; 32 prebuilt GGUFs at F32/F16/Q8_0/Q4_K.
    • ›New local-ai chat interactive terminal CLI for chat against a running server, with /models, /model, and /clear commands.
    • ›Video input/understanding support in llama-cpp via mtmd: chat requests can now include video content parts (video_url), with allow_video auto-gated by whether the loaded mmproj supports it.
    • ›Video generation via LTX-2 in stablediffusion-ggml, wiring audio_vae_path and embeddings_connectors_path; new gallery/ltx-ggml.yaml template with T2V/I2V/FLF2V recipes and six LTX-2.3 22B GGUF gallery entries (dev + distilled, UD-Q4_K_M/Q4_K_M/Q8_0).
    • ›New intelligent middleware layer (#9802) for capability-based model routing, PII detection/redaction/rerouting on /v1/chat/completions, Anthropic /v1/messages, and /v1/completions, and cloud-model proxying including a MITM proxy for Claude Code/Codex OAuth subscriptions; emits proxy_connect and proxy_traffic audit events and restores listener state from runtime_settings.json on restart.
    • ›New ds4-worker binary for manual layer-split distributed inference via local-ai worker ds4-distributed; coordinator owns layers 0:K, workers own higher ranges, controlled by ds4_role config key.
    • ›Adds gated X-LocalAI-Node response header for node attribution in distributed deployments.
    • ›Adds Ideogram4 support in stablediffusion-ggml.
    • ›RAG agent answers now append a Sources: citations block from the Knowledge Base.
    • ›LocalVQE updated to v1.3 with input/output spectrogram views in the Audio Transform UI.
19. v4.3.6 May 30, 2026 · issue -081

    LocalAI v4.3.6 adds NVIDIA NeMo Parakeet ASR backend and hardens outbound HTTP redirect handling.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.3.6 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.3.6

    • ›Adds NVIDIA NeMo Parakeet ASR backend (parakeet-cpp) for automatic speech recognition via the parakeet.cpp integration.
    • ›Hardens outbound HTTP clients to refuse redirects, reducing server-side request forgery exposure via pkg/httpclient.
20. v4.3.5 May 29, 2026 · issue -082

    LocalAI v4.3.5 adds per-request reasoning_effort control on chat completions.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.3.5 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.3.5

    • ›Supports reasoning_effort as a per-request parameter on chat completions, letting callers tune reasoning depth without changing model config.
21. v4.3.2 May 27, 2026 · issue -084

    LocalAI v4.3.2 adds model routing, PII filtering, cloud proxies, LTX-2 video generation, and a native object detection backend.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.3.2 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.3.2

    • ›Adds X-LocalAI-Node response header (gated) in distributed/middleware mode to expose which node served a request.
    • ›Adds middleware-layer model routing, PII filtering, and cloud model proxy support for request pipelines.
    • ›Adds rfdetr-cpp native backend enabling object detection and segmentation without external dependencies.
    • ›Adds LTX-2 and LTX-2.3 GGUF support to the stablediffusion-ggml backend, including gallery entries for image-to-video and first-last-frame-to-video variants.
    • ›Adds 'Fits in my GPU' toggle filter on the Install Models UI screen to surface only models that fit available VRAM.
22. v4.3.0 May 24, 2026 · issue -087

    LocalAI v4.3.0 adds keyless cosign backend verification, prompt cache on by default, per-API-key usage tracking, and distributed mode v3.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.3.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.3.0

    └──▷ USE IT
    Pin a gallery's backend images to signatures issued after a specific date, revoking all earlier keyless Fulcio certs for that gallery.
    yaml copy

    verification:
      issuer_regex: "^https://token\.actions\.githubusercontent\.com$"
      identity_regex: "^https://github\.com/mudler/LocalAI/\.github/workflows/backend_merge\.yml@.*$"
      not_before: "2026-05-22T00:00:00Z"

    Identify which API keys and users are consuming the most GPU tokens — useful for chargeback or quota enforcement.
    copy

    $ curl -s -H 'Authorization: Bearer <admin-token>' 'http://localhost:8080/api/auth/admin/usage/sources?user_id=<uid>&api_key_id=<kid>'

    • ›Adds --require-backend-integrity flag (or LOCALAI_REQUIRE_BACKEND_INTEGRITY=true) to enforce strict mode: missing verification: policy or empty SHA256 in gallery YAML becomes a hard-fail instead of a warning.
    • ›Adds verification: block to gallery YAML supporting issuer_regex, identity_regex, and not_before fields for keyless cosign/sigstore-go signature verification of backend OCI images with digest pinning and TUF root caching.
    • ›Adds LOCALAI_TRACING_MAX_BODY_BYTES environment variable (default 64 KiB) to cap captured request/response body size in trace middleware, recording body_truncated and original body_bytes fields when truncation occurs.
    • ›Adds LOCALAI_NATS_BACKEND_INSTALL_TIMEOUT and LOCALAI_NATS_BACKEND_UPGRADE_TIMEOUT environment variables (default 15 minutes, previously hardcoded 3 minutes) to control distributed backend install timeouts.
    • ›Adds new API endpoints GET /api/auth/usage/sources (self) and GET /api/auth/admin/usage/sources (admin, with user_id / api_key_id filters and 200-key truncation) for per-API-key and per-user token attribution.
    +9 moreshow less

    • ›Changes kv_unified default to true in grpc-server.cpp and prompt_cache_all default to true at the YAML layer, enabling llama-cpp server-side prompt cache out of the box — collapsing repeated system-prompt prefill from minutes to seconds.
    • ›Adds cache_idle_slots and checkpoint_every_nt as new model-config option keys for tuning llama-cpp KV cache behavior.
    • ›Changes POST /api/nodes/:id/backends/install to return HTTP 202 plus a jobID immediately instead of blocking up to 3 minutes, with per-node scoped opcache and async streaming progress events on nodes.<nodeID>.backend.install.<opID>.progress NATS subjects.
    • ›Adds unified /app/backend-logs/:modelId route that is mode-aware: routes to local WebSocket view in standalone mode, or to a per-node picker in distributed mode, with ?from= deep-link timestamp support.
    • ›Adds a Sources tab to the React Usage page with a source-mix ribbon, top-7 + Other time chart, and searchable/sortable table; admin view rolls up (source, user_id, user_name) and shows owning account per named key.
    • ›Adds usage_records fields Source (apikey / web / legacy), APIKeyID, and APIKeyName with idempotent backfill of pre-feature rows on InitDB; revoked keys remain visible in history as (revoked).
    • ›Adds flake.nix and flake.lock for a reproducible, dockerless LocalAI setup on NixOS plus a dev shell.
    • ›Adds a distributed test harness under tests/distributed/ for regression testing of multi-node routing behavior.
    • ›Implements per-request replica routing in SmartRouter.Route using PickBestReplica with in_flight ASC, last_used ASC, available_vram DESC ordering, and memoizes probeHealth per (nodeID, addr) with a 30-second TTL and singleflight coalescing.
23. v4.2.6 May 16, 2026 · issue -095

    LocalAI v4.2.6 bumps llama.cpp to MTP-merge SHA and automatically configures MTP defaults.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.2.6 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.2.6

    • ›Upgrades llama-cpp to the MTP-merge SHA and automatically sets Multi-Token Prediction (MTP) defaults, enabling MTP inference without manual configuration.
24. v4.2.5 May 16, 2026 · issue -095

    LocalAI v4.2.5 exposes 12 additional llama.cpp common_params and updates Swagger docs.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.2.5 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.2.5

    • ›Exposes 12 previously missing common_params via options[] for the llama-cpp backend, giving practitioners finer-grained inference control.
    • ›Updates Swagger API documentation to reflect current endpoint definitions.
    • ›Validates archive member paths before extraction, closing a path-traversal exposure during model downloads.
    • ›Validates video image URLs before download, preventing unexpected outbound requests.
    • ›Updates bundled vllm cu130 wheel to version 0.21.0.
25. v4.2.4 May 13, 2026 · issue -098

    LocalAI v4.2.4 adds Liquid Audio speech-to-speech model support and Vulkan VRAM budget parsing.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.2.4 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.2.4

    • ›Adds Liquid Audio speech-to-speech (s2s) model and assistant mode on the talk page via the realtime interface.
    • ›Extends VRAM budget and usage detection to parse output from vulkaninfo, broadening GPU visibility for Vulkan-backed hardware.
26. v4.2.2 May 12, 2026 · issue -099

    LocalAI v4.2.2 bumps llama.cpp and exposes new speculative-decoding options via the gRPC server.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.2.2 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.2.2

    • ›Bumps the bundled llama.cpp to commit 1ec7ba0c, adapts the gRPC server, and exposes new speculative-decoding options.
27. v4.2.1 May 11, 2026 · issue -100

    LocalAI v4.2.1 adds a DeepSeek V4 Flash backend with tool calls and KV cache, plus richer Ollama API model metadata.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.2.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.2.1

    └──▷ TRY IT
    Query the Ollama-compatible endpoint to inspect model capabilities and details for a loaded model.
    copy

    $ curl http://localhost:8080/api/show -d '{"name": "<model-name>"}'

    • ›New ds4 backend adds DeepSeek V4 Flash support with tool calls, thinking mode, and KV cache.
    • ›The /api/tags and /api/show Ollama-compatible endpoints now report model capabilities and details.
28. v4.2.0 May 11, 2026 · issue -100

    LocalAI 4.2.0 adds voice/face recognition, diarization, Ollama drop-in API, video generation, 11 new backends, and hardened distributed mode v2.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.2.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.2.0

    └──▷ TRY IT
    Route an existing Ollama client at a local LocalAI instance without changing any other client configuration.
    copy

    $ OLLAMA_HOST=http://localhost:8080 ollama run qwen3

    • ›Adds /v1/voice/* endpoints for 1:1 speaker verification, 1:N speaker identification, voice embeddings, and per-segment age/gender/emotion analysis, powered by a SpeechBrain + ONNX backend.
    • ›Adds /v1/audio/diarization endpoint for 'who spoke when?' segmentation via sherpa-onnx + vibevoice.cpp.
    • ›Adds word-level timestamps to faster-whisper transcriptions, plus segments, duration, and language fields on the stream-done event from /v1/audio/transcriptions.
    • ›Adds client cancellation for Whisper via the ggml abort_callback, allowing a transcription to be stopped mid-flight and GPU freed.
    • ›Adds a complete face-biometrics pipeline (1:1 verify, 1:N identify, detection, analysis, embeddings) with antispoofing/liveness rejection, built on InsightFace + ONNX.
    +21 moreshow less

    • ›Adds Ollama drop-in API compatibility — point an existing Ollama client at LocalAI by setting OLLAMA_HOST.
    • ›Adds video generation (image-to-video, first-last-frame) to the stable-diffusion.ggml backend, with gallery entries for Wan 2.1 FLF2V 14B 720P and Wan i2v 720p, plus a stablediffusion-ggml-development meta backend.
    • ›Adds engine_args to vLLM, exposing the full AsyncEngineArgs via a generic YAML map, tensor-parallel distributed workers, and feature parity with llama.cpp.
    • ›Adds split_mode config for llama.cpp for explicit multi-GPU placement.
    • ›Adds speculative decoding support for llama.cpp and Gemma 4 thinking mode.
    • ›Adds concurrency groups — per-model exclusive groups to prevent heavy backends from trampling each other during loading.
    • ›Adds a universal model importer supporting most backends, with multi-shard GGUF handling and dedicated importers for vibevoice-cpp and whisper.cpp HuggingFace repos.
    • ›Adds an interactive model config editor in the UI with autocomplete over known fields, live validation, and automatic file-renaming on save.
    • ›Adds admin-configurable branding (instance name, tagline, logo, favicon) and i18n support for English, Italiano, Español, Deutsch, and 简体中文 in the React UI.
    • ›Adds 11 new backends: sglang (high-throughput LLM + speculative decoding with EAGLE/EAGLE3/DFLASH/MTP), ik-llama.cpp, TurboQuant, sam.cpp (Segment Anything), Kokoros (Rust-native Kokoro TTS), qwen3tts.cpp, tinygrad-multimodal (experimental), vibevoice.cpp, LocalVQE (audio FX), insightface (face antispoofing), and voice-rec (speaker recognition/embeddings).
    • ›Adds CUDA 13 builds for vLLM, vLLM-omni, and sglang, plus NVIDIA L4T arm64 (CUDA 13) variants for Jetson-class boards.
    • ›Adds AMDGPU_TARGETS as an exposed build argument and support for gfx1151 (Strix Halo / Ryzen AI MAX) GPUs.
    • ›Adds model as a query parameter to the backend monitor endpoint.
    • ›Adds round-robin routing across replicas of the same model in distributed mode v2.
    • ›Adds backend.upgrade as a separate NATS action, split from install, in distributed mode v2.
    • ›Adds a Settings API that strips env-supplied ApiKeys before persisting, preventing accidental secret leaks.
    • ›Adds backend versioning with automatic upgrade detection and a revamped backend management UI showing concrete versions.
    • ›Adds pin-model capability so models survive the backend reaper, and a per-model on-demand auto-load toggle.
    • ›Adds audio transformations UI (LocalVQE) for exploring audio FX directly from the React UI.
    • ›Config loader now ignores YAML backup files.
    • ›Adds arm64 images for cpu-whisperx and cpu-faster-whisper.
29. v4.1.2 Apr 6, 2026 · issue -135

    LocalAI v4.1.2 wires speculative decoding settings into the llama.cpp backend.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.1.2 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.1.2

    • ›Enables speculative decoding settings in the llama.cpp backend, unlocking faster inference via draft-model speculation.
    • ›Adds Qwen3.5 model files to the model index.
30. v4.1.1 Apr 5, 2026 · issue -136

    LocalAI v4.1.1 adds Gemma 4 thinking support and an API for coding agents to discover LocalAI configuration.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.1.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.1.1

    • ›Adds a new API endpoint allowing coding agents to interactively discover how to control and configure LocalAI at runtime.
    • ›Adds thinking support for Gemma 4 models via the gemma4 backend.
    • ›Adds autoparser preference for chat deltas emitted directly from backends, improving streaming fidelity.
31. v4.1.0 Apr 2, 2026 · issue -139

    LocalAI 4.1 adds distributed clustering, built-in auth/quotas, local-ai agent run, fine-tuning, and min_p inference support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.1.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.1.0

    └──▷ TRY IT
    Run a standalone agent from the terminal with a single prompt, useful for scripted or CI-driven AI tasks.
    copy

    $ local-ai agent run --prompt 'Summarize the top CVEs from this week'

    Run agents with a pool-based configuration file, enabling multi-agent coordination across a defined node pool.
    copy

    $ local-ai agent run pool.json

    • ›New local-ai agent run CLI subcommand runs standalone agents from the terminal, with single-turn --prompt mode and pool-based configurations loaded from pool.json.
    • ›Adds HF_ENDPOINT environment variable support so the downloader rewrites HuggingFace URIs to a corporate mirror or custom endpoint.
    • ›Adds min_p sampling parameter support wired through all inference endpoints.
    • ›Adds distributed cluster mode with smart VRAM-aware routing, node groups for workload isolation, min/max autoscaling, and drain/resume lifecycle management via API call.
    • ›Adds built-in multi-user platform with OIDC/OAuth SSO, invite-only registration, per-user API keys, admin impersonation, and per-user quota enforcement.
    +15 moreshow less

    • ›Adds per-user usage quota system with predictive analytics and per-user breakdown dashboards.
    • ›Adds experimental fine-tuning support using Hugging Face TRL — trains LoRA adapters, auto-exports to GGUF, and imports results back into LocalAI, all from the UI.
    • ›Adds experimental quantization backend for on-the-fly model quantization.
    • ›Streaming tool calls now work in agent mode, delivering results in real-time.
    • ›Adds automatic tool parsing fallback: when native tool call parsing fails, an iterative fallback parser engages automatically.
    • ›Adds automatic inference defaults sourced from Unsloth, applied across all endpoints and gallery models.
    • ›New React UI model pipeline editor lets users visually wire up model pipelines without editing YAML.
    • ›New per-model backend log view in the React UI scopes log output to individual models.
    • ›Media history added to Studio pages, showing past generated images and audio.
    • ›Adds searchable model/backend selector with inline search and filtering in the React UI.
    • ›Structured error toasts in the UI now link directly to traces for one-click debugging.
    • ›Adds cluster status dashboard on the home page showing all node states at a glance.
    • ›Supports S3 and peer-to-peer model transfer in distributed mode.
    • ›Adds first-class NVIDIA Jetson/Tegra GPU platform detection.
    • ›Bumps HuggingFace Transformers to >5.0 with generic model loading support.
32. v4.0.0 Mar 14, 2026 · issue -157

    LocalAI 4.0 adds native agent orchestration, Agenthub, MCP client support, React UI, WebRTC, and new audio backends

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v4.0.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v4.0.0

    └──▷ TRY IT
    Separate persistent agent and skill data from your config directory so upgrades don't clobber agent state.
    copy

    $ local-ai --data-path /var/lib/localai/data

    Disable MCP entirely on a deployment where outbound tool calls are not permitted by policy.
    copy

    $ export LOCALAI_DISABLE_MCP=true
    local-ai

    Mirror HuggingFace model downloads through an internal proxy to avoid direct internet access.
    copy

    $ export HF_MIRROR=https://hf-mirror.internal.example.com
    local-ai

    • ›Adds --data-path CLI flag and LOCALAI_DATA_PATH environment variable to separate persistent agent/skill data from configuration.
    • ›Adds LOCALAI_DISABLE_MCP environment variable to completely disable Model Context Protocol support.
    • ›Adds HF_MIRROR environment variable to redirect HuggingFace model downloads to a mirror.
    • ›Adds sample_rate support to the TTS API via post-processing resampling.
    • ›Adds shell completion scripts for bash, zsh, and fish via dynamic completion support.
    +17 moreshow less

    • ›Adds WebSocket mode support for the Responses API.
    • ›Introduces native agentic orchestration: full lifecycle management of agents (create, start, stop, connect to Slack, configure MCP servers and skills) via the new UI.
    • ›Launches Agenthub (https://agenthub.localai.io) — a community hub for sharing and importing agents directly into LocalAI instances.
    • ›Adds agent memory backends: hybrid search via PostgreSQL or embedded in-memory storage via Chromem.
    • ›Adds 'Events' column to the Agents list page for observability of agent status and activity.
    • ›Adds Canvas Mode to the chat UI: code blocks and artifacts generated by the LLM appear in a dedicated preview panel on the right.
    • ›Adds MCP Apps support: select which MCP servers to enable per chat session from the UI; tools are automatically injected into the standard chat interface.
    • ›Adds full client-side Model Context Protocol (MCP) support with tool streaming in chat.
    • ›Adds WebRTC support to the Realtime API and Talk page for low-latency audio conversations.
    • ›Adds experimental mlx-distributed backend for running distributed workloads using Apple's MLX framework.
    • ›Adds fish-speech, ace-step.cpp, and faster-qwen3-tts (CUDA-only) audio backends.
    • ›Adds multi-voice support for Qwen TTS.
    • ›Adds vllm-omni backend detection for video generation model selection.
    • ›Adds FreeRPC to backend.proto for explicit VRAM cleanup.
    • ›Adds model storage size display with visual RAM warnings when model size exceeds system RAM.
    • ›Migrates the web frontend to React for a modernized UI with tabbed System view separating Models and Backends.
    • ›Adds improved trace display using accordions in the UI.
    └──▷ BREAKING ON UPGRADE

    • !HuggingFace backend support has been removed; workloads relying on it will fail after upgrade.
    • !AIO images have been dropped; users relying on AIO images must switch to main images.
33. v3.12.1 Feb 21, 2026 · issue -177

    LocalAI v3.12.1 adds backend tracing and updates llama.cpp for Qwen 3 Coder compatibility.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.12.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.12.1

    • ›Adds backend traces via the new traces feature (feat(traces)), enabling observability into backend inference calls.
34. v3.12.0 Feb 20, 2026 · issue -178

    LocalAI v3.12.0 adds Voxtral TTS backend, multi-modal realtime conversations, multi-GPU Diffusers, and a new dark/light UI theme.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.12.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.12.0

    • ›Adds voxtral backend for high-quality text-to-speech, available via the model gallery.
    • ›Adds experimental sd_embed-style prompt embedding support to the diffusers backend for richer image generation pipelines.
    • ›Enables multi-GPU support for the diffusers backend, improving image generation throughput.
    • ›Extends the realtime API to accept text, image, and audio conversation items in a single session.
    • ›Adds a left navbar with dark/light theme variants to the web UI.
    +2 moreshow less

    • ›Improves legacy CPU support for the stablediffusion-ggml backend, broadening compatibility with older processors.
    • ›Adds model gallery entries for Ministral 3 family, npc-llm-3-8b, nemo-asr, voxcpm, whisperx, moonshine-tiny, neutts, vllm-omni models, voxtral, and nanbeige4.1-3b.
35. v3.11.0 Feb 7, 2026 · issue -191

    LocalAI v3.11.0 adds realtime audio conversations, four new ASR backends, TTS streaming, music generation UI, and vLLM Omni support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.11.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.11.0

    └──▷ TRY IT
    Transcribe a multi-speaker audio file and get speaker-labeled output using the new WhisperX backend with diarization.
    copy

    $ curl http://localhost:8080/v1/audio/transcriptions \
      -F [email protected] \
      -F model=whisperx \
      -F response_format=json

    • ›Adds ace-step backend for music generation (MusicGen/Ace-Step), with a new Web UI for text-to-music generation directly in the browser.
    • ›Adds transcribe response format request parameter to the STT API endpoint, letting callers control transcription output format.
    • ›Adds whisperx backend for transcription with native speaker diarization (identifying who is speaking in multi-speaker audio).
    • ›Adds vllm-omni backend for high-performance inference with vLLM Omni models.
    • ›Adds VoxCPM TTS backend with streaming mode support, reducing time-to-first-audio for speech synthesis.
    +7 moreshow less

    • ›Adds qwen-asr backend for Qwen speech recognition models.
    • ›Adds nemo backend for Nvidia NeMo ASR (speech-to-text).
    • ›Adds vibevoice ASR support alongside its existing TTS capability.
    • ›Adds qwen-tts (Qwen-TTS) model support for text-to-speech.
    • ›Introduces native realtime audio conversation support compatible with OpenAI's Realtime API, including STT/TTS pipelines, voice-to-voice models, and tool call support.
    • ›Extends MLX build support to CUDA 12, CUDA 13, L4T (Nvidia Jetson), SBSA, and macOS Metal for broader hardware coverage.
    • ›Adds most remaining Piper voices from Hugging Face to the model gallery.
    └──▷ BREAKING ON UPGRADE

    • !The ExLlama backend has been removed; migrate to ExLlamaV2 or llama.cpp loaders.
    • !The Bark backend has been removed (upstream unmaintained); use the new TTS backends (VoxCPM, Qwen-TTS, Piper) instead.
36. v3.10.1 Jan 23, 2026 · issue -206

    LocalAI v3.10.1 adds Qwen3-TTS backend, reasoning block support in openresponses, and GGUF support for LTX-2 video generation.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.10.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.10.1

    • ›Adds qwen-tts backend, bringing support for Qwen3-TTS text-to-speech models.
    • ›Supports reasoning blocks in the openresponses API, enabling structured reasoning output from compatible models.
    • ›Adds GGUF support for LTX-2 video generation via the videogen backend.
    • ›Automatically detects thinking/reasoning support from the backend when not explicitly configured, removing the need to manually set it per model.
37. v3.10.0 Jan 18, 2026 · issue -211

    LocalAI v3.10.0 adds Anthropic /v1/messages API, Open Responses API, video generation UI, Moonshine STT, and Pocket-TTS with voice cloning.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.10.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.10.0

    └──▷ TRY IT
    Fetch in-memory request/response traces after an agent run to diagnose unexpected tool calls or context drift.
    copy

    $ curl http://localhost:8080/api/v1/trace

    Use LocalAI as a drop-in Claude replacement by pointing the Anthropic SDK at the local /v1/messages endpoint.
    copy

    $ curl -X POST http://localhost:8080/v1/messages \
      -H 'Content-Type: application/json' \
      -d '{"model": "claude-3-opus-20240229", "max_tokens": 1024, "messages": [{"role": "user", "content": "Summarise this document."}]}'

    • ›Adds GET /api/v1/responses/{response_id} and GET /api/v1/trace endpoints as part of new Open Responses API support, enabling stateful agent sessions with response_id, background: true async mode, and stream: true streaming of tool calls, images, and audio.
    • ›Adds GET /api/v1/trace endpoint for in-memory request/response tracing; logs are capped by max size and exportable to JSON for agent debugging and fine-tuning analysis.
    • ›Adds native Anthropic messages API support at /v1/messages, providing a drop-in replacement for Claude with full tool/function calling, streaming, and compatibility with anthropic-sdk-go and LangChain.
    • ›Adds new /video page in the web UI for text-to-video and image-to-video generation, supporting LTX-2 and diffusers with adjustable fps, num_frames, and guidance_scale parameters.
    • ›Adds Moonshine backend — an ONNX-based, ultra-fast speech-to-text engine optimised for low-end devices (Raspberry Pi, older laptops) with live transcription support.
    +6 moreshow less

    • ›Adds Pocket-TTS backend for lightweight, high-fidelity text-to-speech with voice cloning support (requires HF authentication and a registered voice model).
    • ›Adds streaming tool call support (input_json_delta) and XML-style tool call parsing (<function>...</function>) across all backends (llama.cpp, vLLM, diffusers, etc.).
    • ›Adds a new 'Reasoning' field that automatically detects and extracts thinking tags from model output, surfacing reasoning steps in a dedicated 'Thinking' tab in the chat UI in both SSE and non-SSE modes.
    • ›Unified GPU backend packaging: CUDA, ROCm, and Vulkan GPU libraries are now bundled inside backend containers, so any latest/master image works on Nvidia, AMD, and ARM64 (Vulkan, experimental) without manual driver configuration.
    • ›Adds Vulkan arm64 backend builds, extending GPU-accelerated inference to ARM64 platforms.
    • ›System-aware backend gallery now auto-detects CPU/GPU capabilities and hides incompatible backends (e.g., MLX on Linux, CUDA on AMD), showing only runnable options alongside detected capabilities.
38. v3.9.0 Dec 24, 2025 · issue -236

    LocalAI v3.9.0 adds Agent Jobs panel, LRU model eviction, smart memory reclaimer, VibeVoice TTS, and CUDA 13 support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.9.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.9.0

    • ›Adds single_active_backend mode now defaults to LRU eviction with a limit of 1, automatically unloading the least recently used model when a new one is loaded — configurable to any model count limit.
    • ›New Memory Reclaimer watchdog monitors GPU/VRAM consumption across all backends and evicts LRU models when usage exceeds a configured threshold, preventing out-of-memory crashes.
    • ›New Agent Jobs panel in the web UI and API lets you create, schedule (via cron syntax), and trigger background agentic tasks programmatically, with optional webhook delivery of results and dynamic variable-populated prompts.
    • ›New vibevoice TTS backend adds real-time text-to-speech with voice cloning support, installable from the model gallery.
    • ›Adds CUDA 13 support in Docker images and L4T builds.
    +7 moreshow less

    • ›MLX backend gains thread-safe LRU prompt cache and min_p/top_k sampling parameters.
    • ›Whisper backend adds a prompt parameter to condition transcription output.
    • ›StableDiffusion-GGML gains LoRA support for image generation.
    • ›Hugging Face API now returns files in nested directories, enabling model imports from URLs with subfolders (e.g., huggingface://user/model/GGUF/model.gguf).
    • ›New UI mask for installing custom backends from the web interface.
    • ›UI search results can now be ordered by user selection.
    • ›Agent Jobs support multimedia inputs.
    └──▷ BREAKING ON UPGRADE

    • !Data storage path changed from /usr/share to /var/lib; existing deployments must migrate mutable data to the new location.
    • !x86_64 macOS (darwin-x86_64) support dropped; ARM64 (M1/M2/M3/M4) is now the only supported Mac architecture.
39. v3.8.0 Nov 26, 2025 · issue -264

    LocalAI v3.8.0 adds universal model import from HF/Ollama/OCI, MCP live streaming, hot-reloadable settings, logprobs/logitbias, and new llama.cpp YAML options.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.8.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.8.0

    └──▷ USE IT
    Tune llama.cpp context shifting, KV cache RAM, and parallel worker slots for a large-context deployment without rebuilding the container.
    yaml copy

    options:
    - context_shift:false
    - cache_ram:-1
    - use_jinja:true
    - parallel:2
    - grpc_servers:localhost:50051,localhost:50052

    • ›Adds context_shift, cache_ram, use_jinja, parallel, and grpc_servers as tunable options under the options: key in the model YAML config for llama.cpp backends.
    • ›Adds a new LocalAI endpoint to stream live MCP agent results — tool calls, reasoning steps, and intermediate actions are now visible in real-time via the Model Context Protocol.
    • ›Adds full support for logprobs and logitbias per the OpenAI specification, enabling token-level probability data for agentic logic, Self-RAG, and hallucination evaluation.
    • ›Introduces universal model import from Hugging Face, Ollama, OCI registries, or local file paths via URL — auto-detects backends (e.g., llama.cpp vs diffusers) and applies native chat templates (e.g., llama-3, mistral) from model metadata.
    • ›Adds a hot-reloadable 'Settings > System' panel for toggling watchdogs, P2P, gallery availability, and API key management (generate, rotate, expire) without restarting — requires mounting /configuration in containerized deployments.
    +6 moreshow less

    • ›Adds request cancellation support for streaming and non-streaming requests across llama.cpp, vLLM, transformers, and diffusers backends — stopping generation immediately frees backend resources.
    • ›Persists chat history and parallel conversations in browser local storage, with support for renaming and exporting chats to JSON.
    • ›Adds onboarding wizard for first-time users to import or install a model, plus auto-selection of a default model on startup.
    • ›Allows top_n in the reranker to be omitted or set to 0 to return all results with no arbitrary limit.
    • ›Adds multimodal auto-detection: vision components (mmproj) are identified and configured automatically during model import.
    • ›Adds a file-safety guard to prevent deletion of model blobs shared by multiple model configurations.
    └──▷ BREAKING ON UPGRADE

    • !Persisting runtime settings (hot-reload) requires mounting the /configuration directory from the container image in existing LocalAI deployments — without this mount, settings changes will not survive restarts.
40. v3.7.0 Oct 31, 2025 · issue -290

    LocalAI v3.7.0 adds agentic MCP support with POST /mcp/v1/chat/completions, a neutts TTS backend, /v1/videos endpoint, fuzzy gallery search, and Qwen 3 VL.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.7.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.7.0

    └──▷ USE IT
    Configure an MCP-powered agent with DuckDuckGo web search and bounded reasoning loops, then call it via the OpenAI-compatible endpoint.
    yaml copy

    # model config YAML
    name: my-mcp-agent
    backend: llama-cpp
    mcp:
      stdio: |
        {
          "mcpServers": {
            "duckduckgo": {
              "command": "docker",
              "args": ["run", "-i", "--rm", "ghcr.io/mudler/mcps/duckduckgo:master"]
            }
          }
        }
    agent:
      max_attempts: 3
      max_iterations: 5
      enable_reasoning: true
      enable_re_evaluation: true

    Generate a short video from a text prompt using the new OpenAI-compatible video endpoint.
    copy

    $ curl http://localhost:8080/v1/videos \
      -H 'Content-Type: application/json' \
      -H 'Authorization: Bearer sk-...' \
      -d '{"model": "sora", "prompt": "A cat walking through a forest at sunset", "size": "1024x576"}'

    Set up the neutts backend for streaming low-latency TTS with a reference audio transcript.
    yaml copy

    name: neutts-english
    backend: neutts
    parameters:
      model: neuphonic/neutts-air
    tts:
      audio_path: ./output.wav
      streaming: true
    options:
      - ref_text: 'So I am live on radio...'
    known_usecases:
      - tts

    • ›Adds POST /mcp/v1/chat/completions endpoint (OpenAI-compatible) for agentic tool-using AI with MCP, configurable via mcp and agent YAML blocks including max_attempts, max_iterations, enable_reasoning, and enable_re_evaluation keys.
    • ›Adds neutts backend for TTS, configured with backend: neutts in the model YAML, supporting parameters.model, tts.audio_path, tts.streaming, and options (e.g. ref_text) for low-latency natural speech generation.
    • ›Adds POST /v1/videos OpenAI-compatible endpoint for text-to-video generation, accepting model, prompt, and size fields.
    • ›Adds whisper.cpp CPU-variant builds for avx, avx2, avx512, and fallback instruction sets to support a wider range of hardware without crashes.
    • ›Adds Qwen 3 VL model support via llama.cpp/gguf, with gallery entries including qwen3-vl-2b-instruct, qwen3-vl-4b-instruct, qwen3-vl-32b-instruct, qwen3-vl-30b-a3b-instruct, and thinking variants.
    +5 moreshow less

    • ›Adds fuzzy and case-insensitive gallery search so queries like gema or gemaa match gemma, gemma-3, gemma2, etc.
    • ›Adds long-form TTS text chunking for the chatterbox backend, splitting long inputs intelligently to preserve audio context.
    • ›Adds strict_mode: true enforcement in tool/function definitions, returning a structured JSON error instead of crashing when a required tool call is invalid.
    • ›Adds a WebUI MCP toggle in the chat interface for models with mcp settings enabled, plus simplified YAML model editing and Alpine.js-powered reactivity replacing HTMX.
    • ›Adds support for nullable JSON schema types (["string", "null"]) and empty parameters in function definitions without panics.
    └──▷ BREAKING ON UPGRADE

    • !Upgrading the llama.cpp backend is required for existing LocalAI installations to gain Qwen 3 VL support and the updated llama.cpp version.
41. v3.6.0 Oct 3, 2025 · issue -317

    LocalAI v3.6.0 adds L4T device support for Kokoro, multilingual Chatterbox TTS, and six new gallery models including IBM Granite 4.0 and Qwen image editing.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.6.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.6.0

    • ›Adds Kokoro TTS backend support for L4T (Linux for Tegra) devices, enabling on-device speech synthesis on NVIDIA Jetson hardware.
    • ›Adds multilingual support to the Chatterbox TTS backend.
    • ›Adds qwen-image-edit-2509 to the model gallery for image editing workflows.
    • ›Adds whisper-turbo via the whisper.cpp backend to the model gallery.
    • ›Adds ibm-granite_granite-4.0-h-small, ibm-granite_granite-4.0-h-tiny, ibm-granite_granite-4.0-h-micro, and ibm-granite_granite-4.0-micro to the model gallery.
42. v3.5.1 Sep 17, 2025 · issue -333

    LocalAI v3.5.1 adds whisper diarization, HuggingFace endpoint override, CUDA 12.8 GPU images, and a launcher welcome page.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.5.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.5.1

    └──▷ TRY IT
    Route all HuggingFace model downloads through a private or regional mirror instead of the default hub.
    copy

    $ HF_ENDPOINT=https://hf-mirror.example.com ./local-ai

    • ›Adds HF_ENDPOINT environment variable to override the HuggingFace endpoint used for model downloads.
    • ›Adds diarization (tinydiarize) support to the Whisper backend for speaker-separated transcription.
    • ›Bumps NVIDIA GPU container images to CUDA 12.8.
    • ›Adds MPS and CPU support to the Chatterbox backend.
    • ›Adds a welcome page to the launcher UI.
43. v3.5.0 Sep 3, 2025 · issue -347

    LocalAI v3.5.0 adds MLX, WAN video, CPU diffusers, Purego Whisper with VAD, P2P model sync, and a system backend path.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.5.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.5.0

    └──▷ USE IT
    Run text-to-speech on macOS using Kokoro voice models via the new MLX audio backend.
    yaml copy

    backend: mlx-audio
    name: kokoro-mlx
    parameters:
      model: prince-canuma/Kokoro-82M
      voice: "af_heart"
      known_usecases:
        - tts

    Generate video on a GPU from a text prompt using the new WAN pipeline in the diffusers backend.
    yaml copy

    name: wan21
    f16: true
    backend: diffusers
    known_usecases:
      - video
    parameters:
      model: Wan-AI/Wan2.1-T2V-1.3B-Diffusers
    diffusers:
      cuda: true
      pipeline_type: WanPipeline
      step: 40
    options:
      - guidance_scale:5.0
      - num_frames:81
      - torch_dtype:bf16

    • ›Adds LOCALAI_BACKENDS_SYSTEM_PATH environment variable (and equivalent command-line argument) to specify a read-only system backend directory, defaulting to /usr/share/localai/backends, enabling package-manager-friendly and system-wide installations.
    • ›New mlx-audio backend (backend: mlx-audio) enables text-to-speech on macOS using MLX-powered Kokoro voice models.
    • ›New mlx-vlm backend (backend: mlx-vlm) supports vision-language models (VLMs) on macOS via MLX.
    • ›New mlx backend brings MLX-accelerated LLM inference to macOS.
    • ›New WanPipeline pipeline type in the diffusers backend enables text-to-video and image-to-video generation with Wan2.1 and Wan2.2 models using config keys pipeline_type: WanPipeline, num_frames, and guidance_scale.
    +11 moreshow less

    • ›Diffusers backend now supports CPU and macOS MPS execution, removing the GPU requirement for image generation.
    • ›Whisper backend rewritten in Purego with integrated Voice Activity Detection (VAD) for speech detection, improving efficiency and macOS compatibility.
    • ›Stablediffusion backend converted to Purego, improving architecture and enabling macOS CI builds.
    • ›Adds automatic P2P model synchronization across federated LocalAI instances for models installed via the gallery.
    • ›WebUI gains a 'Get Config' button to download a model's YAML configuration file without installing the model.
    • ›WebUI adds a manual model refresh button to reload on-disk YAML configurations without restarting.
    • ›WebUI gains model import and edit screens for managing model configurations directly in the browser.
    • ›WebUI now displays the list of currently installed backends.
    • ›WebUI surfaces SSE streaming errors directly to the user for improved debugging visibility.
    • ›New LocalAI Launcher app (Alpha) for Linux and macOS simplifies installation, startup, shutdown, updates, and WebUI access.
    • ›Diffusers backend prioritizes ref_images over src for image source selection, enabling more robust image input handling.
44. v3.4.0 Aug 12, 2025 · issue -363

    LocalAI v3.4.0 adds KittenTTS, Kokoro, and Dia backends, reasoning effort support, and sideloadable backends.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.4.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.4.0

    └──▷ TRY IT
    Install a backend from the CLI with a custom alias and name to override default configuration.
    copy

    $ local-ai backend install <backend> --alias my-kokoro --name kokoro-custom

    • ›Adds --alias and --name flags to the backend installation CLI command to override configurations during manual installation.
    • ›Adds KittenTTS, Kokoro, and Dia as new TTS/audio backends, installable directly from the model gallery.
    • ›Adds support for reasoning effort in the OpenAI chat completion API.
    • ›Enables the Diffusers backend for NVIDIA L4T images and devices.
    • ›Supports sideloading backends from the system by dropping them into the backends folder, with no additional configuration required.
    +2 moreshow less

    • ›Adds image size control to the WebUI image generation interface.
    • ›Adds qwen-image model support to the model gallery.
45. v3.3.2 Aug 4, 2025 · issue -363

    LocalAI v3.3.2 lets backends install from a local path and run without a metadata file.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.3.2 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.3.2

    • ›Supports installing backends from a local filesystem path, enabling offline or custom backend deployments.
    • ›Allows backends to operate without a metadata file, removing a previously required dependency for backend registration.
46. v3.3.1 Aug 1, 2025 · issue -363

    LocalAI v3.3.1 adds Flux Kontext image editing via ref_images API field and LoRA loading for stable-diffusion-ggml.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.3.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.3.1

    └──▷ TRY IT
    Pull and start the Flux Kontext model locally before making API calls against it.
    copy

    $ local-ai run flux.1-kontext-dev

    • ›Adds ref_images field to the POST /v1/images/generations API, enabling Flux Kontext-powered image editing by passing one or more reference image URLs alongside a prompt.
    • ›Adds support for loading LoRAs in the stable-diffusion-ggml backend.
    • ›Adds flux.1-kontext-dev model, launchable with local-ai run flux.1-kontext-dev, for in-context image editing.
    • ›Adds new models to the model gallery: flux.1-krea-dev-ggml, flux.1-dev-ggml-q8_0, flux.1-dev-ggml-abliterated-v2-q8_0, qwen_qwen3-30b-a3b-instruct-2507, qwen_qwen3-30b-a3b-thinking-2507, and arcee-ai_afm-4.5b.
    └──▷ BREAKING ON UPGRADE

    • !Intel GPU container images latest-gpu-intel-f32 and latest-gpu-intel-f16 are replaced by a single unified image latest-gpu-intel; any existing scripts or deployments referencing the old tags will fail to pull.
47. v3.3.0 Jul 28, 2025 · issue -364

    LocalAI 3.3.0 adds a new object detection API powered by the rfdetr-base model.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.3.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.3.0

    • ›Adds a new object detection API backed by the rfdetr-base model for fast, local object detection.
    • ›Backends now have defined mirror sources for downloads, improving resilience when primary registries are unavailable.
    └──▷ BREAKING ON UPGRADE

    • !The assistants endpoint has been dropped.
48. v3.2.2 Jul 25, 2025 · issue -364

    LocalAI v3.2.2 adds mirror support to the backend gallery for more resilient model downloads.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.2.2 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.2.2

    • ›Adds mirror support to the backend gallery, enabling fallback download sources for backends.
49. v3.2.0 Jul 24, 2025 · issue -364

    LocalAI 3.2.0 splits all backends into a modular gallery with new local-ai backends CLI, auto hardware detection, and Intel GPU Whisper support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.2.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.2.0

    └──▷ TRY IT
    Install a backend in an air-gapped environment from a pre-downloaded OCI tar file instead of pulling from the gallery.
    copy

    $ local-ai backends install "ocifile:///opt/localai-backends/llama-cpp.tar"

    List all available backends in the gallery to see what can be installed or updated.
    copy

    $ local-ai backends list

    • ›Adds local-ai backends list, local-ai backends install <backend>, and local-ai backends uninstall <backend> CLI commands to browse and manage backends from the new Backend Gallery independently of LocalAI releases.
    • ›Adds local-ai backends install 'ocifile://<PATH_TO_TAR_FILE>' to install backends from a local OCI tar file, enabling offline and air-gapped deployments.
    • ›Adds a CLI command to create custom OCI images from directories, supporting custom backend packaging.
    • ›Introduces the LOCALAI_FORCE_META_BACKEND_CAPABILITY environment variable to override automatic hardware detection; accepted values are default, nvidia, amd, and intel.
    • ›Supports the input_audio field in the /v1/chat/completions endpoint for multimodal audio inputs, improving OpenAI API compatibility.
    +4 moreshow less

    • ›Adds speech started and speech stopped realtime audio events for more precise control over interactive voice streams.
    • ›Enables SYCL acceleration for the Whisper backend, adding Intel GPU hardware-accelerated transcription support.
    • ›All inference backends (llama.cpp, whisper.cpp, piper, stablediffusion-ggml) are now separated from the core binary into the Backend Gallery, significantly reducing binary and container image size.
    • ›Adds over 50 new models to the model gallery, including releases from Qwen3, Gemma, Mistral, Nemotron, devstral-small, and more.
    └──▷ BREAKING ON UPGRADE

    • !llama.cpp, whisper.cpp, piper, and stablediffusion-ggml are no longer bundled in the main LocalAI binary; existing models installed before v3.2.0 may have no backend assigned and will require manually running local-ai backends install <backend_name> after upgrading.
50. v3.1.1 Jun 27, 2025 · issue -365

    LocalAI v3.1.1 automatically installs missing backends when a model from the gallery requires them.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.1.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.1.1

    • ›Automatically installs missing backends alongside models when installing from the gallery, removing the need to manually pre-install backend dependencies.
51. v3.1.0 Jun 26, 2025 · issue -365

    LocalAI v3.1 adds Gemma 3n support, GPU-aware meta-packages in the backend gallery, and a leaner container image layout.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.1.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.1.0

    └──▷ TRY IT
    Run the Gemma 3n E4B instruction-tuned model locally for text generation in a single command.
    copy

    $ local-ai run gemma-3n-e4b-it

    • ›Adds local-ai run gemma-3n-e2b-it and local-ai run gemma-3n-e4b-it commands to run Google Gemma 3n models (text generation) locally.
    • ›Introduces meta-packages to the backend gallery so installing a backend (e.g. vllm) automatically selects the correct GPU variant without manual version picking.
    • ›Moves default model and backend paths in container images to /models/ and /backends/ respectively.
    └──▷ BREAKING ON UPGRADE

    • !Default model path in container images changed from /build/models to /models/; mounts or scripts referencing the old path will break.
    • !Default backend path in container images changed from /build/backends to /backends/; mounts or scripts referencing the old path will break.
    • !Container image tag cublas-cuda11 is renamed to gpu-nvidia-cuda11 and cublas-cuda12 to gpu-nvidia-cuda12; pipelines pinned to the old tag names will stop pulling the intended image.
    • !Container image tag sycl-f16 is renamed to gpu-intel-f16 and sycl-f32 to gpu-intel-f32; pipelines pinned to the old tag names will stop pulling the intended image.
    • !Sources are no longer bundled in the container images; workflows that relied on in-container source access must build from scratch instead.
52. v3.0.0 Jun 19, 2025 · issue -365

    LocalAI 3.0 ships a Backend Gallery (OCI-based, API-driven), Realtime WebSocket API, audio/PDF upload in the UI, llama.cpp reranking, and 50+ new models.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v3.0.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v3.0.0

    └──▷ TRY IT
    Run LocalAI on an NVIDIA CUDA 12 GPU without needing extras images — additional backends can be added at runtime via the gallery.
    copy

    $ docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12

    Run the CPU-only AIO image with models pre-downloaded, ready to use immediately with no manual gallery steps.
    copy

    $ docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-cpu

    • ›Adds a Backend Gallery that lets you install and remove backends at runtime or startup via API or the WebUI, backed by standard OCI images — replaces the old -extra image approach.
    • ›Adds Backend Gallery download progress display in the WebUI so you can track backend installations in real time.
    • ›Adds Realtime WebSocket API with OpenAI-compatible streaming, suitable for chat apps and agents.
    • ›Adds llama.cpp reranking support.
    • ›Adds enhanced multimodal support to llama.cpp via libmtmd.
    +9 moreshow less

    • ›Adds audio input support to llama.cpp, enabling audio understanding models such as Qwen Omni and Ultravox.
    • ›Adds an audio upload button in the chat UI, letting users send audio files directly to supported models.
    • ›Adds PDF and text file upload in the chat UI, with support for multiple input files simultaneously.
    • ›Adds visual 'thinking' tags in the chat UI for reasoning models, showing inference progress in real time.
    • ›Adds dynamic VRAM estimation and automatic layer offloading based on GPU capabilities, improving GPU utilization without manual tuning.
    • ›Adds an error page in the UI to surface backend errors more clearly.
    • ›Adds 50+ new models to the model gallery, including skywork-or1-32b, rivermind-lux-12b, qwen3-embedding-*, llama3-24b-mullein, and ultravox-v0_5.
    • ›Adds new GPU-specific Docker image tags: latest-gpu-nvidia-cuda-12, latest-gpu-nvidia-cuda-11, latest-nvidia-l4t-arm64, latest-gpu-hipblas, latest-gpu-intel-f16, latest-gpu-intel-f32, and latest-gpu-vulkan, replacing the former extras-image model.
    • ›Adds AIO (all-in-one, pre-downloaded models) image variants: latest-aio-cpu, latest-aio-gpu-nvidia-cuda-12, latest-aio-gpu-nvidia-cuda-11, latest-aio-gpu-intel-f16, and latest-aio-gpu-hipblas.
    └──▷ BREAKING ON UPGRADE

    • !The -extra Docker images containing Python backends (e.g. localai/localai:*-extra) are no longer published; switch to standard GPU-specific images (latest-gpu-nvidia-cuda-12, latest-gpu-hipblas, etc.) and install additional backends via the Backend Gallery.
    • !bark-cpp is removed from the bundled image and moved to the Backend Gallery — installations relying on the built-in bark-cpp backend must reinstall it via the Backend Gallery after upgrading.
53. v2.29.0 May 12, 2025 · issue -366

    LocalAI v2.29.0 adds video generation endpoint, Qwen3 support, Whisper.cpp GPU acceleration, and auto GPU offload for llama.cpp

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.29.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.29.0

    └──▷ TRY IT
    Run LocalAI with NVIDIA CUDA 12 GPU support using the new slim image tag (without extra Python deps).
    copy

    $ docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12

    Run LocalAI with NVIDIA CUDA 12 and extra Python dependencies (e.g., diffusers) using the new -extras image tag.
    copy

    $ docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12-extras

    • ›Adds --uninstall flag to install.sh for removing LocalAI installations.
    • ›Introduces experimental /video/generations endpoint for video generation.
    • ›Adds GPU auto-detection and automatic layer offloading for llama.cpp and CLIP backends.
    • ›Enables GPU acceleration for whisper.cpp via cuBLAS (NVIDIA) and Vulkan; SYCL and HIPblas support in progress.
    • ›Adds -extras suffix image tags (e.g., latest-gpu-nvidia-cuda-12-extras) for images carrying extra Python dependencies such as diffusers; default tags are now slim.
    +6 moreshow less

    • ›Bundles FFmpeg in all core images, replacing the former -ffmpeg tagged variants.
    • ›Adds new latest-* image tags: latest-gpu-hipblas, latest-gpu-intel-f16, latest-gpu-intel-f32, latest-gpu-nvidia-cuda-12, and latest-gpu-vulkan.
    • ›Adds official support for the Qwen3 model family.
    • ›Expands HIPblas AMD GPU architecture targets to include gfx803, gfx900, gfx906, gfx908, gfx90a, gfx942, gfx1010, gfx1030, gfx1032, gfx1100, gfx1101, and gfx1102.
    • ›Increases gRPC message size limits to 50 MB.
    • ›Adds VRAM usage estimation for llama.cpp.
    └──▷ BREAKING ON UPGRADE

    • !Images with extra Python dependencies (e.g., for diffusers) now require the -extras suffix (e.g., latest-gpu-nvidia-cuda-12-extras); the former default tags no longer include those libraries.
    • !The separate -ffmpeg image tags have been removed; switch to the corresponding base tag (e.g., latest-gpu-hipblas-ffmpeg becomes latest-gpu-hipblas).
    • !The AutoGPTQ backend has been dropped entirely.
54. v2.28.0 Apr 15, 2025 · issue -367

    LocalAI v2.28.0 adds SYCL support for stablediffusion.cpp, Lumina model family support, and relaunches LocalAGI v2 as a Go-based agent orchestration platform.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.28.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.28.0

    • ›Adds SYCL support for stablediffusion.cpp, enabling Intel GPU-accelerated image generation.
    • ›Adds support for the Lumina model family (e.g., Lumina-Image-2.0) for local image generation.
    • ›Enhances the LOCALAI_SINGLE_ACTIVE_BACKEND loader to treat the backend as a true singleton, improving single-backend reliability.
    • ›Introduces LocalAGI v2, a fully rewritten Go-based AI agent orchestration platform with a no-code WebUI, compatible with the OpenAI Responses API and supporting built-in connectors for Slack, Telegram, Discord, GitHub Issues, and IRC.
    • ›Introduces LocalRecall, a standalone REST API for persistent agent memory, spun out of LocalAGI v2 as its own component.
55. v2.27.0 Mar 31, 2025 · issue -368

    LocalAI v2.27.0 ships a full WebUI redesign, vLLM config additions, GGUF auto context detection, and reply-prefix support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.27.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.27.0

    • ›Adds vLLM config options to disable logging, set dtype, and enforce per-prompt media limits.
    • ›Adds GGUF auto-detection of default context size from file, removing the need to set it manually.
    • ›Adds reply prefix support, allowing a custom prefix to be specified for model replies.
    • ›Supports new model architectures: Gemma 3, Mistral, and Deepseek.
    • ›Complete WebUI redesign with modernised navigation, chat interface, Talk, Audio Generation, Image Generation, Models Overview, API Overview, Login, and Swarm/P2P dashboard screens.
    +3 moreshow less

    • ›WebUI model gallery gains pagination and filtering, including separate filtering for text, TTS, and image models.
    • ›WebUI chat interface automatically detects model usage type and surfaces a link to relevant model documentation.
    • ›AIO (All-in-One) images updated: CPU AIO ships llama3.1 (text), granite-embeddings (embeddings), minicpm (vision); GPU AIO ships localai-functioncall-qwen2.5-7b-v0.5 (text), granite-embeddings (embeddings), minicpm (vision).
56. v2.26.0 Feb 15, 2025 · issue -369

    LocalAI v2.26.0 adds Kokoro/OuteTTS/Fast-Whisper backends, grammar triggers for llama.cpp, AVX512 support, and Nvidia L4T arm64 images.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.26.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.26.0

    └──▷ USE IT
    Enable grammar-triggered tool-call generation so a llama.cpp model only applies JSON grammar when it emits a <tool_call> token, keeping free-form responses unrestricted.
    yaml copy

    function:
      grammar:
        triggers:
          word: "<tool_call>"
          at_start: true

    Run a fine-tuned function-calling model from the gallery to get a conversational assistant that reliably executes tool calls.
    copy

    $ local-ai run LocalAI-functioncall-phi-4-v0.3

    Start LocalAI on an Nvidia AGX Orin (or similar L4T arm64 device) using the ready-made L4T container image with GPU passthrough.
    copy

    $ docker run -e DEBUG=true \
      -p 8080:8080 \
      -v $PWD/models:/build/models \
      -ti --restart=always --name local-ai \
      --runtime nvidia --gpus all quay.io/go-skynet/local-ai:master-nvidia-l4t-arm64-core

    • ›Adds function.grammar.triggers config block (with word and at_start fields) to model YAML config files for llama.cpp, enabling grammar-triggered JSON/tool-call generation only when a specific token (e.g. <tool_call>) is seen.
    • ›Adds hf.co and hf:// URI schemes to the model downloader, allowing models to be referenced directly by Hugging Face URLs.
    • ›New Kokoro TTS backend added for text-to-speech inference.
    • ›New OuteTTS backend added with voice cloning capabilities, available via the transformers backend.
    • ›New Fast-Whisper backend added for faster Whisper model inference.
    +10 moreshow less

    • ›Adds function argument parsing using named regular expressions, simplifying structured function call handling.
    • ›Adds tokenization support for llama.cpp.
    • ›Adds machine tag and inference timing tracking to monitor per-machine performance during inference.
    • ›Adds support for Sana pipelines in the diffusers backend.
    • ›Adds image generation option overrides to the diffusers backend.
    • ›Adds bundled AVX512 build support for CPUs with the AVX512 instruction set.
    • ›Adds Nvidia L4T arm64 container images for devices such as Nvidia AGX Orin, launchable via --runtime nvidia --gpus all quay.io/go-skynet/local-ai:master-nvidia-l4t-arm64-core.
    • ›Adds fine-tuned function-calling models (LocalAI-functioncall-phi-4-v0.3, LocalAI-functioncall-llama3.2-1b-v0.4, LocalAI-functioncall-llama3.2-3b-v0.5, localai-functioncall-qwen2.5-7b-v0.5) to the LocalAI gallery.
    • ›Adds new models to the gallery including DeepSeek-R1, Mistral-small-24b, nightwing3-10b, rombos-qwen2.5-writer, and negative_llama_70b.
    • ›Merges Mamba, Transformers-Musicgen, and Sentencetransformers backends into the unified transformers backend.
    └──▷ BREAKING ON UPGRADE

    • !The vall-e-x backend has been removed; use the CoquiTTS community fork, Kokoro, or OuteTTS instead.
    • !The openvoice backend has been removed; use Kokoro or OuteTTS instead.
    • !The stablediffusion-NCN (ONNX-based) backend has been removed and replaced by the stablediffusion-ggml backend.
    • !The llama-ggml (pre-GGUF) backend has been removed; only GGUF models are supported going forward.
    • !Mamba, Transformers-Musicgen, and Sentencetransformers backends now route through the transformers backend — existing config files referencing the old backend names may be incompatible.
    • !Mirostat is no longer enabled by default in llama.cpp (previously it was on by default).
57. v2.25.0 Jan 10, 2025 · issue -370

    LocalAI v2.25.0 adds KV cache quantization, Jinja templates from GGUF, streaming token usage, and resumable downloads

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.25.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.25.0

    └──▷ USE IT
    Quantize the KV cache to Q8_0 to cut VRAM usage when running large models locally.
    yaml copy

    cache_type_k: q8_0
    cache_type_v: q8_0

    Stream token usage stats in a chat completion request to monitor prompt and completion token counts in real time.
    copy

    $ curl http://localhost:8080/v1/chat/completions \
      -H 'Content-Type: application/json' \
      -d '{"model": "llama-3.3-70b-instruct", "stream": true, "stream_options": {"include_usage": true}, "messages": [{"role": "user", "content": "Hello"}]}'

    • ›Exposes cache_type_k and cache_type_v llama.cpp config keys to enable quantization of the KV cache, reducing VRAM usage for large models.
    • ›Supports reading Jinja templates directly from GGUF model files, eliminating the need for separate template configuration.
    • ›Streams token usage statistics alongside generated tokens during inference.
    • ›Adds UI path prefix support via HTTP header, enabling reverse-proxy deployments at non-root paths.
    • ›Allows skipping driver installation in the Dockerfile build process.
    +2 moreshow less

    • ›Enables resuming partial downloads when fetching models, avoiding full re-downloads on interrupted transfers.
    • ›Adds a large batch of new models to the model gallery, including phi-4, phi-3.5-moe-instruct, llama-3.3-70b-instruct, qwen2-vl-72b-instruct, qvq-72b-preview, falcon3 series (1b/3b/7b/10b), dolphin3.0 series, and many others.
58. v2.24.0 Dec 4, 2024 · issue -371

    LocalAI v2.24.0 adds bark.cpp and stablediffusion.cpp backends, silero-vad speech detection, WebUI API key auth, and new P2P config options.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.24.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.24.0

    └──▷ TRY IT
    Generate audio from text locally using the new bark.cpp backend in a single command.
    copy

    $ local-ai run bark-cpp-small

    Install the silero-vad model to enable speech segment detection in audio streams.
    copy

    $ local-ai models install silero-vad

    • ›Adds bark.cpp backend for transformer-based text-to-audio generation, installable via local-ai models install bark-cpp-small or runnable with local-ai run bark-cpp-small.
    • ›Adds stablediffusion.cpp backend for ggml-powered image generation from text descriptions.
    • ›Adds silero-vad backend for voice activity detection in audio streams, installable via local-ai models install silero-vad or configurable through the WebUI.
    • ›Adds P2P configuration keys listen_maddrs, dht_announce_maddrs, and bootstrap_peers for fine-grained control of peer-to-peer node addressing and connectivity.
    • ›WebUI now supports API key authentication, automatically prompting for a key when one or more API keys are configured.
    +1 moreshow less

    • ›Adds real-time token usage reporting during streamed outputs.
    └──▷ BREAKING ON UPGRADE

    • !The rwkv.cpp backend has been removed; RWKV model support now routes through llama.cpp.
    • !The bert.cpp backend has been dropped; its functionality is now integrated into llama.cpp.
59. v2.23.0 Nov 10, 2024 · issue -372

    LocalAI v2.23.0 adds vLLM load_format exposure, TTS response_format, multi-LoRA diffusers, Flux single-file support, and a disableable /metrics endpoint.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.23.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.23.0

    • ›Exposes load_format backend config key for the vLLM backend, allowing operators to control how model weights are loaded.
    • ›Implements response_format parameter on the TTS endpoint, letting callers specify the desired audio output format.
    • ›Allows disabling the /metrics endpoint for local stats collection via configuration.
    • ›Supports multiple LoRA adapters simultaneously in the diffusers backend.
    • ›Adds Flux single-file support to the diffusers backend.
    +1 moreshow less

    • ›Moves model detailed info to a modal in the UI, reducing clutter on the model list view.
    └──▷ BREAKING ON UPGRADE

    • !Multimodal message templates now use a single unified template; existing per-role multimodal templates will need to be migrated to the new single-template format.
60. v2.22.1 Oct 21, 2024 · issue -373

    LocalAI v2.22.1 adds Sprig templating and text extraction for multimodal requests, plus Phi-3 Vision and 30+ new gallery models.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.22.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.22.1

    • ›Adds Sprig template functions to multimodal templates, enabling richer data manipulation in prompt construction.
    • ›Adds text extraction from multimodal requests in templates, allowing templates to pull plain text out of mixed-media inputs.
    • ›Adds phi-3 vision to the model gallery, enabling multimodal vision inference locally.
    • ›Adds 30+ new models to the gallery, including llama-3.1-nemotron-70b-instruct-hf, supernova-medius, cursorcore-qw2.5-7b-i1, ml-ms-etheris-123b, and multiple CursorCore, Hermes-3, and Llama-3.x variants.
61. v2.22.0 Oct 12, 2024 · issue -373

    LocalAI v2.22.0 adds vLLM video/image-to-text, a tokenization endpoint, loaded-model listing via /system, and configurable multimodal placeholders.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.22.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.22.0

    └──▷ TRY IT
    Submit a video to a vLLM-backed model for description — extend the OpenAI chat API with the new video_url content type.
    copy

    $ curl http://localhost:8080/v1/chat/completions \
      -H "Content-Type: application/json" \
      -d '{
        "model": "gpt-4o",
        "messages": [
          {
            "role": "user",
            "content": [
              {"type": "text", "text": "What is in this video?"},
              {"type": "video_url", "video_url": {"url": "https://example.com/clip.mp4"}}
            ]
          }
        ],
        "max_tokens": 300
      }'

    Override the default image and video placeholder format for a vLLM model that uses a non-standard notation.
    yaml copy

    template:
      image: "<|image_{{.ID}}|> {{.Text}}"
      video: "<|video_{{.ID}}|> {{.Text}}"
      audio: "<|audio_{{.ID}}|> {{.Text}}"

    • ›Adds template.video, template.image, and template.audio keys to model YAML config to override per-backend multimodal placeholder notation (e.g. <|video_{{.ID}}|> {{.Text}}).
    • ›Extends the chat completions API to accept video_url and audio content types alongside images, enabling video and audio understanding via the vLLM backend.
    • ›Lists all currently loaded models via the GET /system endpoint for real-time model management.
    • ›Adds a tokenization endpoint to the API.
    • ›Adds token metrics retrieval to the GRPC server.
    +3 moreshow less

    • ›Adds a correlationID field to chat request tracking via the API.
    • ›Adds force-shutdown capability for backends.
    • ›The Transformers backend now reuses an already-downloaded model instead of re-downloading it.
62. v2.21.0 Sep 24, 2024 · issue -374

    LocalAI v2.21 adds a /system info endpoint, video/audio API inputs, ElevenLabs sound generation, vLLM embeddings, and auto model warm-up.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.21.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.21.0

    • ›Adds GET /system endpoint to list system information.
    • ›Adds ElevenLabs sound-generation API support via the elevenlabs backend.
    • ›Adds support for passing videos to backends via the API.
    • ›Adds support for passing audios to backends via the API.
    • ›Adds embedding support to the vllm backend.
    +4 moreshow less

    • ›Adds trust_remote_code configuration support for the sentencetransformers backend.
    • ›Adds regex-based output extraction from LLM responses.
    • ›Adds auto warm-up to load models into memory on startup.
    • ›AIO images now ship gpt-4o as the Vision API model alias.
    └──▷ BREAKING ON UPGRADE

    • !The exllama backend has been removed; any configuration referencing it will break on upgrade.
    • !The AIO image model alias gpt-4-vision-preview has been renamed to gpt-4o; configurations or API calls using gpt-4-vision-preview will need to be updated.
63. v2.20.1 Aug 23, 2024 · issue -375

    LocalAI v2.20 adds FLUX-1 image gen, strict JSON mode, multi-cluster P2P, gallery disable, and Hugging Face integration.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.20.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.20.1

    └──▷ TRY IT
    Run two logically separate P2P clusters on the same network by giving each a distinct network ID.
    copy

    $ export LOCALAI_P2P_NETWORK_ID=550e8400-e29b-41d4-a716-446655440000
    local-ai run --p2p

    Disable the model gallery endpoint to lock down a production deployment.
    copy

    $ export LOCALAI_DISABLE_GALLERY_ENDPOINT=true
    local-ai run

    • ›Adds LOCALAI_DISABLE_GALLERY_ENDPOINT environment variable to disable the gallery endpoint and its UI routes.
    • ›Adds LOCALAI_RANDOM_WORKER environment variable to disable load balancing (now enabled by default in federated mode).
    • ›Adds LOCALAI_TARGET_WORKER environment variable to target a specific worker in federated mode.
    • ›Adds LOCALAI_P2P_NETWORK_ID environment variable (set to a shared UUID) to run multiple logically separated P2P clusters within the same network using the same token.
    • ›Adds strict: true field in API calls to enforce grammar-compliant JSON outputs via BNF grammar rules, matching OpenAI's Structured Outputs spec.
    +6 moreshow less

    • ›Adds json_schema as a supported format_type; when set, BNF grammars are automatically generated from the schema.
    • ›Adds FLUX-1 image generation support (flux.1-dev and flux.1-schnell in the model gallery), including federated mode.
    • ›Adds a network explorer and community pools UI at explorer.localai.io, enabling sharing and discovery of LocalAI instances.
    • ›Adds P2P load balancing enabled by default across federated clusters.
    • ›LocalAI is now available as a selectable option inside Hugging Face's local apps for one-click model deployment.
    • ›Adds Gemma 2 to the model guesser for automatic backend selection.
    └──▷ BREAKING ON UPGRADE

    • !The gpt4all.cpp backend is removed; gpt4all models in ggml format are no longer compatible.
    • !The petals backend is removed; use LocalAI's P2P federation capabilities as a replacement.
64. v2.20.0 Aug 22, 2024 · issue -375

    LocalAI v2.20.0 adds FLUX-1 image generation, OpenAI strict mode, multi-cluster P2P networking, and a global community explorer.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.20.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.20.0

    └──▷ TRY IT
    Enforce structured JSON output that matches a schema in a single API call, without changing your model config.
    copy

    $ curl http://localhost:8080/v1/chat/completions \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "my-model",
        "messages": [{"role": "user", "content": "Return info about a user"}],
        "response_format": {
          "type": "json_schema",
          "json_schema": {
            "name": "user",
            "schema": {"type": "object", "properties": {"name": {"type": "string"}, "age": {"type": "integer"}}, "required": ["name", "age"]}
          }
        },
        "strict": true
      }'

    • ›Adds strict: true to API call payloads to enforce BNF-grammar-backed JSON outputs, matching OpenAI's Structured Outputs spec; when format_type is set to json_schema, BNF grammars are auto-generated from the schema.
    • ›Adds LOCALAI_DISABLE_GALLERY_ENDPOINT environment variable to disable the gallery endpoint and its UI routes.
    • ›Adds LOCALAI_P2P_NETWORK_ID environment variable (set to a shared UUID) to run multiple logically separated P2P clusters on the same network using the same token.
    • ›Adds LOCALAI_TARGET_WORKER environment variable to target a specific worker in federated mode.
    • ›Adds LOCALAI_RANDOM_WORKER environment variable to disable the new default load-balancing behavior across P2P workers.
    +4 moreshow less

    • ›Adds json_schema as a supported format_type value in API requests, enabling schema-driven structured output.
    • ›Adds FLUX-1 image generation support via the Diffusers backend, including flux.1-dev and flux.1-schnell gallery models, usable in federated mode.
    • ›Launches a network explorer at explorer.localai.io and a public demo instance at demo.localai.io, backed by community P2P pools.
    • ›LocalAI is now selectable as a local app within Hugging Face, enabling one-click model deployment.
    └──▷ BREAKING ON UPGRADE

    • !The gpt4all.cpp backend is removed; gpt4all models in ggml format are no longer compatible.
    • !The petals backend is removed; workloads must migrate to LocalAI's P2P federation capabilities.
    • !P2P load balancing is now enabled by default; existing setups that relied on random worker selection must set LOCALAI_RANDOM_WORKER to restore the previous behavior.
65. v2.19.3 Jul 28, 2024 · issue -376

    LocalAI v2.19.3 adds a Llama 3.1 grammar schema and expands the model gallery with 14 new models including Mistral Nemo and Llama 3.1 variants.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.19.3 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.19.3

    • ›Adds Llama 3.1 grammar schema for structured output generation.
    • ›Adds function calling template for Llama 3.1 models to the gallery.
    • ›Adds Llama 3.1 with enforced function calling via grammars to the model gallery.
    • ›Adds 13 new models to the gallery: llama3.1-claude, darkidol-llama3.1, gemmoy, mistral-nemo, llama3.1-8b-fireplace2, lumimaid-v0.2-12b, darkidol-llama-3.1-8b-instruct-1.1-uncensored, meta-llama-3.1-8b-instruct-abliterated, llama-3.1-70b-japanese-instruct-2407, llama-3.1-8b-instruct-fei-v1-uncensored, openbuddy-llama3.1-8b-v22.1-131k, lumimaid-8b.
66. v2.19.1 Jul 20, 2024 · issue -376

    LocalAI v2.19.1 adds P2P federated AI swarms, llama.cpp embeddings, model pull by URL, and a HuggingFace /scan endpoint.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.19.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.19.1

    └──▷ TRY IT
    Stand up a load-balanced federated AI cluster across multiple machines using a shared token.
    copy

    $ local-ai --p2p --federated

    Connect additional worker nodes to an existing P2P swarm using the shared token generated by the primary instance.
    copy

    $ local-ai --p2p

    • ›Adds --p2p and --p2p --federated CLI flags to launch federated LocalAI instances that auto-discover each other across networks using a shared token, with optional load balancing.
    • ›Adds --p2p flag support for distributing model weights across nodes as P2P workers, currently available on the llama.cpp backend.
    • ›Adds a HuggingFace /scan API endpoint for scanning HuggingFace resources.
    • ›Enables embedding support in the llama-cpp (gRPC) backend via the embeddings endpoints.
    • ›Adds model pulling directly from URLs via feat(models): pull models from urls.
    +6 moreshow less

    • ›Adds a 'Swarm' section to the WebUI providing a P2P dashboard with auto-discovery setup instructions and one-liner connection commands.
    • ›Adds TTS (Text-to-Speech) to binary releases.
    • ›Updates install.sh to support federated instance setup.
    • ›Adds the ability to remove a backend from the backend list in the WebUI.
    • ›Improves tool/function call handling by parsing broken JSON in raw results and using dynamic rules for grammar keys when grammars are disabled.
    • ›WebUI welcome page now shows models without a config file alongside configured models.
67. v2.19.0 Jul 19, 2024 · issue -376

    LocalAI v2.19.0 adds P2P federated instances, llama-cpp embeddings, HuggingFace /scan endpoint, and URL-based model pulling.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.19.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.19.0

    └──▷ TRY IT
    Start a P2P AI swarm node — the generated shared token lets other LocalAI instances auto-discover and join the cluster for load-balanced inference.
    copy

    $ local-ai --p2p --federated

    • ›Adds --p2p flag to start LocalAI and generate a shared token for auto-discovering and clustering multiple instances into a load-balanced AI swarm without Docker or Kubernetes.
    • ›Adds --p2p --federated flags to launch fully federated LocalAI instances that share requests across the cluster, with auto-discovery across different networks.
    • ›Adds HuggingFace /scan endpoint for scanning HuggingFace model assets.
    • ›Adds embeddings endpoint support to the llama-cpp (grpc) backend, enabling local embedding generation via llama.cpp.
    • ›Adds load-balanced option to the P2P federation mode.
    +7 moreshow less

    • ›Supports pulling models directly from URLs via the model management interface.
    • ›Includes TTS (Text-to-Speech) in binary releases, removing the need for a separate install.
    • ›Adds federated instance setup support to the install.sh installer script.
    • ›Adds a P2P 'Swarm' dashboard in the WebUI with one-liner connection instructions and auto-discovery guidance.
    • ›Improves function/tool calling by parsing broken JSON in raw results and using dynamic rules for grammar keys when grammars are disabled.
    • ›Allows removing a backend from the backend list via the WebUI.
    • ›Shows models without a config file in the WebUI welcome/model list page.
68. v2.18.1 Jul 1, 2024 · issue -376

    LocalAI v2.18.1 adds backend autodetect fallback and six new models to the gallery.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.18.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.18.1

    • ›Adds backend fallback with autodetection, allowing LocalAI to automatically select an available backend when the primary one is unavailable.
    • ›Adds new-dawn-llama to the model gallery.
    • ›Adds l3-aethora-15b-v2 to the model gallery.
    • ›Adds bungo-l3-8b-iq-imatrix to the model gallery.
    • ›Adds llama3-8b-darkidol-2.1-uncensored-1048k-iq-imatrix to the model gallery.
    +2 moreshow less

    • ›Adds llm-compiler to the model gallery.
    • ›Adds llama3-turbcat-instruct-8b to the model gallery.
69. v2.18.0 Jun 28, 2024 · issue -377

    LocalAI v2.18.0 adds OCI/Ollama model support, Vulkan backend, transcription translation, and new YAML config options.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.18.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.18.0

    └──▷ TRY IT
    Pull and run an Ollama-hosted model directly without a separate Ollama installation.
    copy

    $ local-ai run ollama://gemma:2b

    Control repetition and JSON property ordering in a llama.cpp model config.
    yaml copy

    parameters:
      repeat_last_n: 64
    function:
      grammar:
        properties_order: "name,arguments"

    • ›Adds oci:// and ollama:// URI prefixes for specifying models in YAML config files, enabling local-ai run ollama://gemma:2b and local-ai models install ollama://gemma:2b directly from the CLI.
    • ›Adds translate: true field to transcription requests, enabling the transcription endpoint to translate audio to English in a single call.
    • ›Adds repeat_last_n as a model YAML config parameter (specific to the llama.cpp backend) to control repetition penalty window.
    • ›Adds properties_order under function.grammar in model YAML config to force JSON responses to return properties in a specified order.
    • ›Introduces Vulkan support for the llama.cpp backend, available via new container image tags latest-vulkan-ffmpeg-core and v2.18.0-vulkan-ffmpeg-core.
    +3 moreshow less

    • ›Upgrades CUDA support to version 12.5 in container images.
    • ›Adds UI model selector allowing users to choose between all available models in the chat interface.
    • ›Adds gallery support for Gemma 2 models, runnable via local-ai run gemma-2-27b-it and local-ai run gemma-2-9b-it.
70. v2.17.1 Jun 19, 2024 · issue -377

    LocalAI v2.17.1 adds Stable Diffusion 3 support via a new StableDiffusion3Pipeline backend config.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.17.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.17.1

    └──▷ TRY IT
    Generate an image with Stable Diffusion 3 after placing the model config YAML in your model folder.
    copy

    $ curl http://localhost:9091/v1/images/generations -H "Content-Type: application/json" -d '{"prompt": "A cute baby sea otter", "model": "sd3"}'

    Configure Stable Diffusion 3 manually by dropping this YAML into your LocalAI model folder instead of using the gallery.
    yaml copy

    backend: diffusers
    diffusers:
      cuda: true
      enable_parameters: negative_prompt,num_inference_steps
      pipeline_type: StableDiffusion3Pipeline
    f16: false
    name: sd3
    parameters:
      model: v2ray/stable-diffusion-3-medium-diffusers
    step: 25

    • ›Adds Stable Diffusion 3 image generation support via the diffusers backend with pipeline_type: StableDiffusion3Pipeline; install from the gallery as stable-diffusion-3-medium or configure manually with backend, diffusers.cuda, diffusers.enable_parameters, f16, parameters.model, and step keys in a model YAML file.
    • ›Improves the talk UI with an informative display box and better colors.
71. v2.17.0 Jun 17, 2024 · issue -377

    LocalAI v2.17.0 adds auto-GGUF detection, voice chat UI, gallery CLI installs, and single binaries for ROCm/NVIDIA/Intel/ARM64.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.17.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.17.0

    └──▷ TRY IT
    Run the quick-start installer with a custom port and pre-set API key, pointing models at a non-default path.
    copy

    $ curl https://localai.io/install.sh | PORT=9090 API_KEY=mysecretkey MODELS_PATH=/data/localai/models sh

    Install a specific model from the gallery without launching it, for pre-staging models in CI or provisioning scripts.
    copy

    $ local-ai models install codestral

    • ›Adds local-ai run <model_name> to install and launch a model from the gallery directly via CLI.
    • ›Adds local-ai models install subcommand to install models from galleries via the CLI.
    • ›Adds install.sh quick-start installer configurable via environment variables DOCKER_INSTALL, USE_AIO, API_KEY, CORE_IMAGES, PORT, THREADS, VERSION, and MODELS_PATH (default port 8080, default MODELS_PATH /usr/share/local-ai/models).
    • ›Adds response_regex support as a list in function-calling grammar config, enabling multiple regex patterns for free-string matching.
    • ›Supports response_type field in OpenAI-compatible image API requests.
    +12 moreshow less

    • ›Adds CSRF protection support via the fiber CSRF feature, enabling CSRF enforcement at LocalAI startup.
    • ›Adds OpaqueErrors mode to hide internal error details from API responses.
    • ›Automatic identification and configuration of GGUF models — drop a GGUF file into the models folder and LocalAI infers defaults without requiring a YAML config.
    • ›Adds a util command to print GGUF file metadata and model information.
    • ›Automatic template detection from GGUF files with built-in defaults for Qwen2, Phi, Mistral, ChatML, Command-R, and Gemma model families.
    • ›New WebUI page for voice-based LLM interaction using audio transcription and TTS, accessible with a couple of clicks.
    • ›Linux single binaries now bundle ROCm (AMD), NVIDIA, and Intel GPU support in one binary (system GPU libraries still required separately).
    • ›Single-binary builds now available for ARM64 (Linux and macOS/Apple Silicon), with bundled gRPC and supporting libraries.
    • ›ARM64 Docker images added (e.g., localai/localai:v2.17.0-ffmpeg-core).
    • ›Parallel function calls with mixed or no grammar constraints now supported.
    • ›New model gallery site at https://models.localai.io for browsing available models without running LocalAI locally.
    • ›Gallery expanded with new models including Mirai Nova, Mahou, MiniCPM, Codestral, Gemma-2b, Llama3-11b, all Whisper variants, and others.
72. v2.16.0 May 24, 2024 · issue -378

    LocalAI v2.16.0 adds distributed llama.cpp inferencing, peer-to-peer LLM clusters, mixed JSON grammar function calling, and a single binary release.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.16.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.16.0

    └──▷ TRY IT
    Distribute a large model across two worker machines to run inference that exceeds single-node VRAM.
    copy

    $ LLAMACPP_GRPC_SERVERS="worker1.local:50052,worker2.local:50053" local-ai run

    Bootstrap a private peer-to-peer inference cluster — start the server, capture the generated token, then join workers from other hosts.
    copy

    $ # On the server node:
    ./local-ai run --p2p
    # Copy the printed token, then on each worker node:
    TOKEN=XXXXXXXXXXX ./local-ai p2p-llama-cpp-rpc

    Enable mixed-grammar function calling for a Hermes-family model so it can return both structured tool calls and free-text in the same response.
    yaml copy

    function:
      disable_no_action: true
      grammar:
        mixed_mode: true

    • ›New local-ai llamacpp-worker <listening_address> <listening_port> subcommand starts llama.cpp RPC workers to offload inferencing workload to remote nodes.
    • ›New LLAMACPP_GRPC_SERVERS environment variable accepts a comma-separated list of address:port pairs to distribute llama.cpp inference across multiple nodes when running local-ai run.
    • ›New --p2p flag on local-ai run enables fully decentralized peer-to-peer LLM inferencing over the libp2p protocol without manual IP configuration, using DHT and mDNS for discovery.
    • ›New --p2ptoken flag (and P2P_TOKEN environment variable) lets you supply a pre-shared token to rejoin an existing private p2p cluster on server restart.
    • ›New p2p-llama-cpp-rpc subcommand (with TOKEN=XXX env var or token argument) starts a worker node that joins the p2p cluster and contributes compute.
    +9 moreshow less

    • ›New function.grammar.mixed_mode: true config key in YAML model configuration enables mixed JSON BNF grammars, allowing models to output both structured JSON and free text in function-calling responses.
    • ›New function.grammar.disable: true config key turns off grammar enforcement entirely, letting users supply regex-based parsing instead.
    • ›New function.json_regex_match config key accepts a list of regex patterns to extract function-call results from raw LLM output (e.g. for Hermes-style <tool_call> tags).
    • ›New function.replace_llm_results and function.replace_function_results config keys accept key/value regex replacement lists to clean LLM output before OpenAI-spec compliance checks.
    • ›New function.return_name_in_function_response config key includes the function name in the response payload.
    • ›New function.disable_no_action config key suppresses injection of the default 'answer' tool in function-calling prompts.
    • ›Single binary release consolidates all variants (CUDA and non-CUDA) and dependencies into one portable executable, simplifying installation and upgrades.
    • ›Model gallery adds Aya-35b, Mistral-0.3, Hermes-Theta, Hermes-2-Pro-Mistral, Hermes-2-Theta-Llama-3, and a fine-tuned LocalAI-Llama3-8b-Function-Call-v0.2 model with enhanced out-of-the-box function-calling support.
    • ›Python backends migrated from Conda to UV, reducing setup time and dependency management complexity.
73. v2.15.0 May 9, 2024 · issue -378

    LocalAI v2.15.0 adds Vision API in Chat WebUI, single binary releases, --debug CPU/GPU info, and trust_remote_code UI flag.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.15.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.15.0

    • ›Adds --debug flag output to display CPU/GPU information at startup.
    • ›Exposes trust_remote_code as a configurable flag in the WebUI for model loading.
    • ›Adds llama.cpp backend autoloading without requiring explicit backend specification, plus llama.cpp variant support.
    • ›Integrates Vision API into the Chat WebUI, enabling image processing model testing directly in the browser.
    • ›Adds system prompt configuration in the WebUI chat interface.
    +6 moreshow less

    • ›Introduces single binary releases for simplified deployment without AVX/SSE instruction sets (CUDA builds planned).
    • ›Adds model gallery filtering by tag and category in the WebUI.
    • ›Adds a background operations indicator to the WebUI to show when tasks are running.
    • ›Adds a revamped welcome/onboarding page in the WebUI to guide new users through model installation.
    • ›Expands the model gallery with new one-click-install models including 'moondream2', 'llama3-llava', 'llama3-instruct-coder', 'lumimaid', 'openbiollm', 'Soliloquy', 'tess', 'aurora', 'kunocchini', 'tiamat', and several OpenVINO models.
    • ›Updates ROCM support with a smaller base image.
74. v2.14.0 May 3, 2024 · issue -378

    LocalAI v2.14.0 adds OpenVINO acceleration, user-defined inference devices, model deletion, llama3 AIO, and a new WebUI with chat/TTS/image-gen pages.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.14.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.14.0

    • ›Adds user-defined inference device selection for CUDA and OpenVINO backends, letting practitioners pin workloads to specific hardware.
    • ›Adds OpenVINO acceleration for embeddings in the transformer backend, enabling fast inference on Intel CPUs and GPUs.
    • ›Adds model deletion support to the gallery UI, allowing installed models to be removed directly from the interface.
    • ›Adds gallery job status display during navigation so model install progress is visible while browsing.
    • ›Adds Chat, TTS, and image-generation pages to the WebUI for quick interactive debugging and model assessment.
    +2 moreshow less

    • ›Switches the AIO image default LLM to a llama3-based model (Hermes-2-Pro-Llama-3-8B-GGUF), which supports function calling out of the box.
    • ›Adds numerous new models to the gallery including Einstein v6.1, SOVL, Hermes-2-Pro-Llama-3-8B, biomistral-7b, WizardLM2, llama3-32k, suzume-llama-3-8B-multilingual, and several OpenVINO-optimized models.
75. v2.13.0 Apr 25, 2024 · issue -379

    LocalAI v2.13.0 adds a model gallery UI, rerankers backend with Jina API compatibility, and a new parler-tts backend.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.13.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.13.0

    └──▷ TRY IT
    Rerank a set of documents against a query using the new Jina-compatible /v1/rerank endpoint with a locally running model.
    copy

    $ curl http://localhost:8080/v1/rerank \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "jina-reranker-v1-base-en",
        "query": "Organic skincare products for sensitive skin",
        "documents": [
          "Natural organic skincare range for sensitive skin",
          "Tech gadgets for smart homes: 2024 edition",
          "Sensitive skin-friendly facial cleansers and toners"
        ],
        "top_n": 2
      }'

    Configure the rerankers backend by pointing a model YAML at the cross-encoder model so LocalAI can serve reranking requests.
    yaml copy

    name: jina-reranker-v1-base-en
    backend: rerankers
    parameters:
      model: cross-encoder

    • ›Adds rerankers backend implementing the Jina reranker API at /v1/rerank, compatible with existing Jina clients; configured via a model YAML with backend: rerankers and parameters.model: cross-encoder.
    • ›Adds parler-tts backend for text-to-speech, installable from the gallery or via model config.
    • ›Adds tensor_parallel_size setting to vLLM backend configuration.
    • ›Adds use_tokenizer_template and stop_prompts options to the Transformers backend.
    • ›Adds ConfigURLs support to the gallery, enabling custom hosted model index repositories.
    +9 moreshow less

    • ›Adds a Golang client for the LocalAI store backend.
    • ›Adds trace log level and zerolog-based fiber request logging.
    • ›Adds flash-attn support in NVIDIA and ROCm environments.
    • ›Adds tokenizer.apply_chat_template() support in the vLLM backend.
    • ›Adds function calling support for models with no grammar.
    • ›Adds consumed token count reporting in GRPC backend responses.
    • ›Adds a model gallery UI in the WebUI with browseable models including stablediffusion, llama3, phi-3, hermes, tts, and embeddings.
    • ›Adds llama3, hermes, phi-3, and other models to the gallery.
    • ›Adds Transformers backend adherence to OpenAI API max_tokens behavior.
76. v2.12.3 Apr 10, 2024 · issue -379

    LocalAI v2.12.3 adds Assistant API, OpenVINO runtime, Swagger UI, Vision support for AutoGPTQ, and an Intel AIO profile.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.12.3 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.12.3

    • ›Adds Assistant and AssistantFiles API endpoints, enabling OpenAI-compatible assistant workflows.
    • ›Adds Swagger UI at the LocalAI front page for interactively exploring and testing API calls directly in the browser.
    • ›Adds OpenVINO runtime support for the transformer backend, enabling hardware-accelerated inference on Intel hardware.
    • ›Adds token streaming support for the transformer backend (including OpenVINO and CUDA paths).
    • ›Adds Intel GPU profile for AIO (all-in-one) images, enabling out-of-the-box Intel GPU acceleration.
    +5 moreshow less

    • ›Adds Vision (VL model) support to the AutoGPTQ backend.
    • ›Adds a landing welcome page when accessing the LocalAI front page, with a model list and quick-start guidance.
    • ›Web UI now shows which backends are associated with each model.
    • ›AIO CPU images now default to NousResearch/Hermes-2-Pro-Mistral-7B-GGUF, pre-configured for functions and tools API support.
    • ›Improves structured logging across the LocalAI server.
77. v2.12.1 Apr 9, 2024 · issue -379

    LocalAI v2.12.1 adds Assistant API, OpenVINO runtime, Swagger UI, vision support for AutoGPTQ, and an Intel AIO profile.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.12.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.12.1

    • ›Adds Assistant and AssistantFiles API endpoints, enabling OpenAI-compatible assistant workflows.
    • ›Adds Swagger UI at the LocalAI front page for interactively exploring and testing API calls directly in the browser.
    • ›Adds a landing welcome page served at the LocalAI front page root.
    • ›Adds OpenVINO runtime support for the transformer backend, including token streaming for both OpenVINO and CUDA.
    • ›Adds token streaming support for the transformer backend.
    +3 moreshow less

    • ›Adds Vision/VL model support to the AutoGPTQ backend.
    • ›Adds an Intel GPU profile for AIO images, with Hermes-2-Pro-Mistral-7B-GGUF as the new default CPU AIO model, pre-configured for functions and tools API support.
    • ›Web UI now shows which backends are associated with each model.
78. v2.12.0 Apr 9, 2024 · issue -379

    LocalAI v2.12.0 adds Assistant API, OpenVINO runtime, Swagger UI, Vision for AutoGPTQ, and an Intel AIO GPU profile.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.12.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.12.0

    • ›Adds Assistant and AssistantFiles API endpoints, enabling assistant-style workflows against a local inference server.
    • ›Adds Swagger UI for interactively exploring and testing LocalAI API calls directly from the browser.
    • ›Adds OpenVINO runtime support for the transformer backend, with token streaming for both OpenVINO and CUDA.
    • ›Adds Vision (VL model) support to the AutoGPTQ backend.
    • ›Adds an Intel GPU profile for AIO (all-in-one) images.
    +2 moreshow less

    • ›Adds a landing welcome page shown when accessing the LocalAI front page, with a model list and backend associations visible in the web UI.
    • ›AIO CPU images now default to NousResearch/Hermes-2-Pro-Mistral-7B-GGUF, pre-configured for functions and tools API support.
79. v2.11.0 Mar 26, 2024 · issue -380

    LocalAI v2.11.0 ships All-in-One Docker images and a new vector store backend for CPU and GPU environments.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.11.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.11.0

    └──▷ TRY IT
    Spin up a fully pre-configured local OpenAI-compatible server with LLM, TTS, STT, image generation, and function calling on a CPU-only machine.
    copy

    $ docker run -p 8080:8080 --name local-ai -ti localai/localai:latest-aio-cpu

    Run the same all-in-one stack with CUDA 12 GPU acceleration on an Nvidia host.
    copy

    $ docker run -p 8080:8080 --gpus all --name local-ai -ti localai/localai:latest-aio-gpu-cuda-12

    • ›Adds localai/localai:latest-aio-cpu, localai/localai:latest-aio-gpu-cuda-11, and localai/localai:latest-aio-gpu-cuda-12 All-in-One images that bundle LLM, TTS, speech-to-text, image generation, and function calling out of the box — no manual model setup required.
    • ›Adds localai/localai:latest-aio-gpu-hipblas and Intel GPU AIO images for AMD and Intel GPU environments.
    • ›Adds a new 'vector stores' backend type, enabling LocalAI to operate as an in-memory vector database.
    • ›Adds pre-configured model definitions for llava-1.6-vicuna, llava-1.6-mistral, hermes-2-pro-mistral, phi-2-chat, bakllava, and cerbero.
    • ›Changes llama.cpp model defaults to automatically offload the maximum number of GPU layers when a GPU is available.
80. v2.10.0 Mar 16, 2024 · issue -380

    LocalAI v2.10.0 adds Intel GPU support, Elevenlabs/OpenAI TTS compatibility, Bitsandbytes quantization, and vLLM streaming.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.10.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.10.0

    └──▷ USE IT
    Allow a transformers or AutoGPTQ model to execute remote code during loading — required after the new enforcement of trust_remote_code.
    yaml copy

    trust_remote_code: true

    • ›Now honors trust_remote_code in model YAML config files for the AutoGPTQ and transformers backends, controlling whether remote code execution is allowed during model loading.
    • ›Adds Bitsandbytes quantization support for the transformer backend, enabling lower-memory model loading on CUDA hardware.
    • ›Adds compatibility layers for Elevenlabs and OpenAI TTS, allowing LocalAI to serve as a drop-in endpoint for both TTS APIs.
    • ›Adds diffusers/transformers support for Intel GPUs, enabling image generation and transformer-based inference on Intel GPU hardware.
    • ›Enables stream: true for the vLLM backend via AsyncLLMEngine, supporting true token streaming from vLLM-served models.
    +2 moreshow less

    • ›Adds initial implementation of the Assistants API, beginning OpenAI-compatible assistant endpoint support.
    • ›Embeddings endpoints no longer require explicit model configuration to function.
    └──▷ BREAKING ON UPGRADE

    • !The trust_remote_code setting in model YAML config files is now enforced for AutoGPTQ and transformers backends. Models that previously loaded remote code without this setting will now be blocked; set trust_remote_code: true in the model YAML config file to restore the old behavior.
81. v2.9.0 Feb 24, 2024 · issue -381

    LocalAI v2.9.0 adds Tools API with parallel function calling, ROCm/SYCL GPU images, file upload API, and Gemma model support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.9.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.9.0

    • ›Adds Tools API endpoint support, including parallel function calling and SSE with function calling; the now-deprecated functions API call remains supported.
    • ›Adds initial implementation of the upload files API (early Assistant API groundwork).
    • ›Publishes ROCm container images for AMD GPUs, tagged hipblas (e.g. master-hipblas-ffmpeg-core).
    • ›Publishes SYCL container images for Intel GPUs in two precision flavors: sycl-f16 and sycl-f32 (e.g. master-sycl-f16-core).
    • ›Adds support for Google Gemma models via a llama.cpp dependency update.
82. v2.8.0 Feb 10, 2024 · issue -381

    LocalAI v2.8.0 adds Intel GPU support via SYCL container images and drops legacy ggml-based backends.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.8.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.8.0

    └──▷ TRY IT
    Run a model on an Intel GPU by selecting the SYCL-enabled image and passing through the DRI device.
    copy

    $ docker run -e DEBUG=true -ti -v $PWD/models:/build/models -p 8080:8080 -v /dev/dri:/dev/dri --rm quay.io/go-skynet/local-ai:master-sycl-f32-ffmpeg-core phi-2

    • ›Adds Intel GPU support via new container images tagged sycl-f16 and sycl-f32, e.g. quay.io/go-skynet/local-ai:master-sycl-f32-ffmpeg-core, enabling f16 or f32 inference on Intel hardware.
    • ›Container images are now based on Ubuntu 22.04 LTS instead of Debian Bullseye.
    └──▷ BREAKING ON UPGRADE

    • !The old ggml-based falcon backend has been removed; use the llama.cpp backend instead.
    • !The ggml-based gpt2 and starcoder backends have been removed; these architectures are now served by llama.cpp.
    • !The ggml-transformers backends have been dropped from container images.
83. v2.7.0 Jan 29, 2024 · issue -382

    LocalAI v2.7.0 adds transformer backend support for LLM text generation and remote model definition fetching.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.7.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.7.0

    └──▷ TRY IT
    Run a code-generation LLM via the transformer backend with GPU acceleration instead of llama.cpp.
    copy

    $ docker run -ti -p 8080:8080 --gpus all localai/localai:v2.7.0-cublas-cuda12 codellama-7b

    • ›Enables fetching model definitions remotely at startup, so models can be referenced by name without pre-downloading config files.
84. v2.6.1 Jan 23, 2024 · issue -382

    LocalAI v2.6.1 adds pluggable gRPC backend SPI in embedding mode

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.6.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.6.1

    • ›Enables gRPC backend SPI pluggability in embedding mode, allowing custom embedding backends to be wired in via the gRPC interface.
85. v2.6.0 Jan 20, 2024 · issue -382

    LocalAI v2.6.0 adds Mamba model support and allows building without GRPC backends.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.6.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.6.0

    • ›Adds Mamba architecture support as a new extra backend, enabling local inference with Mamba-based models.
    • ›Allows building LocalAI without GRPC_BACKENDS, reducing build complexity for minimal deployments.
86. v2.5.0 Jan 8, 2024 · issue -382

    LocalAI v2.5.0 adds phi-2 and more embedded models, plus URL-based YAML model loading at startup.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.5.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.5.0

    └──▷ TRY IT
    Run a popular embedded model by short-hand name without any manual config — useful for quick local inference.
    copy

    $ docker run -ti -p 8080:8080 localai/localai:v2.5.0-ffmpeg-core phi-2

    Load a model at startup from a remote YAML config URL (e.g. a GitHub Gist), enabling community-shared model definitions without rebuilding your image.
    copy

    $ docker run -ti -p 8080:8080 localai/localai:v2.5.0-ffmpeg-core https://raw.githubusercontent.com/mudler/LocalAI/master/embedded/models/llava.yaml

    • ›Supports passing a URL pointing to a valid YAML model config file (e.g. a GitHub Gist) directly as a startup argument to load models like llava without pre-bundling them.
    • ›Adds phi-2 and additional embedded models launchable by name as a direct CLI argument (e.g. phi-2) when starting LocalAI.
    • ›Adds model usage and description metadata to embedded model definitions.
87. v2.4.1 Jan 6, 2024 · issue -382

    LocalAI v2.4.1 ships embedded model configurations with popular model examples ready to use out of the box.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.4.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.4.1

    • ›Adds embedded model configurations bundled directly into LocalAI, including pre-built examples for popular models, removing the need to author config files from scratch.
88. v2.4.0 Jan 4, 2024 · issue -382

    LocalAI v2.4.0 adds gallery model preloading, URL-based model sharing, and additional file download support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.4.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.4.0

    • ›Supports specifying additional files to download alongside a model at prepare time.
    • ›Enables preloading and preparing models directly from galleries before first use.
    • ›Adds support for sharing models by URL, allowing remote model sources to be referenced directly.
    • ›Adds trimsuffix template function for model configuration templating.
89. v2.3.0 Dec 29, 2023 · issue -383

    LocalAI v2.3.0 adds Coqui TTS, tiny-dream Stable Diffusion backend, and partial model download support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.3.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.3.0

    • ›Adds Coqui TTS backend for text-to-speech generation.
    • ›Adds tiny-dream backend for Stable Diffusion image generation.
    • ›Adds partial download support, enabling resumable model downloads.
    • ›Switches the VALL-E-X conda environment to use the transformers library.
90. v2.2.0 Dec 21, 2023 · issue -383

    LocalAI v2.2.0 adds inline prompt templates and URL-based model loading in YAML config files.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.2.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.2.0

    └──▷ USE IT
    Load a GGUF model directly from HuggingFace and define its prompt template inline — no separate download or template file needed.
    yaml copy

    name: mixtral
    
    parameters:
      model: https://huggingface.co/TheBloke/Mixtral-8x7B-v0.1-GGUF/resolve/main/mixtral-8x7b-v0.1.Q2_K.gguf
    # or huggingface://TheBloke/Mixtral-8x7B-v0.1-GGUF/mixtral-8x7b-v0.1.Q2_K.gguf@main
    template:
      completion: |
        Complete the following: {{.Input}}

    • ›Supports specifying parameters.model as a direct HTTPS URL or huggingface:// URI in model YAML config files, enabling on-demand model downloads without pre-staging files.
    • ›Supports inline prompt templates under template.completion (and siblings) directly in model YAML config files, eliminating the need for separate template files.
    • ›Transformer-based backends now share a single conda environment, reducing container image size.
91. v2.1.0 Dec 16, 2023 · issue -383

    LocalAI v2.1.0 adds ExLlamaV2, music generation, image/video synthesis backends, and API key file validation

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.1.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.1.0

    • ›Adds API key file validation support to the API layer, enabling file-based key management.
    • ›Adds the transformers-musicgen backend for AI-powered music generation.
    • ›Adds exllamav2 backend support alongside restored exllamav1 compatibility.
    • ›Adds autopipeline and controlnet support to the diffusers backend.
    • ›Adds initial support for img2vid and txt2vid (image-to-video and text-to-video) generation.
    +2 moreshow less

    • ›Adds CUDA-accelerated transformers backend for GPU-accelerated inference.
    • ›Adds optional preparation of extra endpoints via the entrypoint configuration.
    └──▷ BREAKING ON UPGRADE

    • !The llama backend alias is renamed to llama-cpp; any configuration referencing the llama backend name will break on upgrade.
92. v2.0.0 Dec 4, 2023 · issue -383

    LocalAI v2.0.0 adds LLaVA/GPT Vision API, parallel requests, conda environments, watchdog, and GPU-accelerated Whisper.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v2.0.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v2.0.0

    • ›Adds LLaVA support and the GPT Vision API, enabling multimodal image+text inference via llama.cpp.
    • ›Adds .yml as an accepted extension for YAML config files (previously only .yaml was recognized).
    • ›Enables parallel request handling for llama.cpp, with automatic request queuing when parallelism is disabled.
    • ›Adds support for custom CA certificates for outbound connections.
    • ›Adds conda environment support for Python backends.
    +6 moreshow less

    • ›Adds an initial watchdog implementation to detect and recover stuck backends.
    • ›Adds embeddings support via the Transformers AutoModel backend.
    • ›Adds -core Docker images that ship without Python dependencies for leaner deployments.
    • ›Adds CUBLAS, HIPBLAS, Metal, OpenBLAS, and CLBlast acceleration support to whisper_cpp.
    • ›Adds the Petals backend for distributed model inference.
    • ›Adds LoRA support with scale and YaRN rope scaling in the llama.cpp backend.
    └──▷ BREAKING ON UPGRADE

    • !The llama-stable backend is renamed to llama-ggml; any config or tooling referencing llama-stable will break.
93. v1.40.0 Oct 30, 2023 · issue -385

    LocalAI v1.40.0 adds a native llama-cpp C++ backend, AMD ROCm/HIPBLAS support, new CLI subcommands, and Prometheus metrics.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.40.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v1.40.0

    └──▷ TRY IT
    Install a model from the gallery without starting the server, useful in automated provisioning or CI.
    copy

    $ local-ai models install <model-name>

    Quickly test text-to-speech output from the CLI without spinning up the full API server.
    copy

    $ local-ai tts --model <model-name> 'Hello, this is a test of LocalAI text to speech.'

    Transcribe an audio file directly from the CLI to validate a speech model before deploying it.
    copy

    $ local-ai transcript --model <model-name> <audio-file>

    • ›Adds models CLI subcommand with list and install sub-subcommands for managing the model gallery directly from the local-ai binary.
    • ›Adds tts CLI subcommand for running text-to-speech inference from the command line.
    • ›Adds transcript CLI subcommand for audio transcription from the command line.
    • ›Adds initial Prometheus metrics support for observability.
    • ›Introduces the llama-cpp backend — a native C++ gRPC backend that tracks recent llama.cpp versions more closely and reduces abstraction layers compared to the existing llama (Go/C++) backend.
    +2 moreshow less

    • ›Enables ROCm/HIPBLAS support, adding AMD GPU acceleration to LocalAI.
    • ›Drops the bloomz and ggllm backends, as their model formats are now covered by the llama.cpp backend.
    └──▷ BREAKING ON UPGRADE

    • !The bloomz and ggllm backends have been removed; configurations or workflows relying on them must migrate to the llama-cpp backend.
94. v1.30.0 Sep 25, 2023 · issue -386

    LocalAI v1.30.0 adds vllm and vall-e-x backends, speculative sampling, LoRA for diffusers, and new model/job API endpoints.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.30.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v1.30.0

    └──▷ TRY IT
    Poll all in-progress model download/load jobs to monitor async model operations via the API.
    copy

    $ curl http://localhost:8080/models/jobs

    • ›Adds GET /models/jobs endpoint to expose all background model jobs.
    • ›Adds PYTHON_GRPC_MAX_WORKERS environment variable to control the maximum number of workers for the Python gRPC backend.
    • ›Adds initial vllm backend implementation, including support for setting quantization.
    • ›Adds vall-e-x backend for audio generation.
    • ›Adds speculative sampling support via a draft model field in the model config.
    +3 moreshow less

    • ›Adds LoRA support to the diffusers backend.
    • ›Adds mutable/refactored Model Gallery endpoints.
    • ›Allows manually setting the seed for the Stable Diffusion pipeline.
95. v1.25.0 Aug 26, 2023 · issue -387

    LocalAI v1.25.0 adds GGUF support, LoRA adapters, img2img diffusion, a stable llama backend, and a backend monitor shutdown endpoint.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.25.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v1.25.0

    └──▷ TRY IT
    Prevent multiple backends from loading simultaneously to conserve GPU/CPU memory on a resource-constrained host.
    copy

    $ local-ai --single-active-backend

    • ›Adds --single-active-backend CLI flag to allow only one backend active at a time, limiting resource contention.
    • ›Adds llama-stable backend as an alternative to the llama.cpp backend for users who need a pinned, stable build.
    • ›Adds LoRA adapter loading support for llama.cpp backends via model configuration.
    • ›Adds backend monitor shutdown endpoint (process-based) for programmatic backend lifecycle management.
    • ›Adds GGUF model format support via bumped llama.cpp (see breaking changes).
    +7 moreshow less

    • ›Adds img2img and depthimg2img pipeline support to the diffusers backend, along with clip_skip and additional kernel schedulers.
    • ›Adds DPMSolverMultistepScheduler++, DPMSolverMultistepSchedulerSDE++, and guidance_scale options to the diffusers backend.
    • ›Overcomes the prompt length limit in the diffusers backend.
    • ›Adds customizable RWKV tokenizer support.
    • ›Adds usage tracking features (API-level usage collection).
    • ›Allows restricting which backends are built via Makefile targets.
    • ›Makes the gRPC backend initializer accept configurable delay times.
    └──▷ BREAKING ON UPGRADE

    • !llama.cpp has been bumped with GGUF support added; existing GGML-format models may no longer be compatible and will require conversion or replacement with GGUF variants.
96. v.1.24.0 Aug 12, 2023 · issue -387

    LocalAI v1.24.0 adds Bark, AutoGPTQ, Diffusers, ExLlama, API key list support, and pre-configured model galleries.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v.1.24.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v.1.24.0

    • ›Adds API_KEY list support, enabling multiple API keys to be accepted by the LocalAI server for access control.
    • ›Adds Bark text-to-audio and AutoGPTQ quantized model backends as new inference options.
    • ›Adds Diffusers backend, enabling image generation via Hugging Face Diffusers models.
    • ›Adds ExLlama backend for fast, memory-efficient inference with GPTQ-quantized LLaMA models.
    • ›Adds pre-configured LocalAI galleries, allowing one-step model discovery and installation from curated sources.
97. v1.23.2 Aug 5, 2023 · issue -387

    LocalAI v1.23.2 adds system prompt config templating, new model parameters, and expanded grammar primitive support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.23.2 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v1.23.2

    • ›Adds ngqa and RMSNormEps parameters to model configuration for finer control over model inference behavior.
    • ›Adds a 'simple' system prompt templating mechanism for chat, edit, and completion endpoints via model config files.
    • ›Expands grammar rules to allow integers, numbers, and strings as root types, enabling primitives as top-level output constraints.
98. v1.23.1 Aug 2, 2023 · issue -387

    LocalAI v1.23.1 adds a filter query string parameter to the model list endpoint.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.23.1 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v1.23.1

    └──▷ TRY IT
    Retrieve only models whose names match a filter string, avoiding manual client-side filtering of large model lists.
    copy

    $ curl 'http://localhost:8080/v1/models?filter=gpt4'

    • ›Adds a filter query string parameter to the model list API, enabling callers to narrow results without fetching the full list.
99. v1.23.0 Jul 29, 2023 · issue -388

    LocalAI v1.23.0 adds rope settings, negative prompt support, and stream cancellation when clients disconnect.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.23.0 https://github.com/mudler/LocalAI.git
    # already have the repo? check out this version:
    $ git checkout v1.23.0

    • ›Adds rope settings and negative prompt support to model configuration, replacing the grammar backend.
    • ›Cancels stream generation automatically when a streaming client disconnects, freeing backend resources.
    └──▷ BREAKING ON UPGRADE

    • !The grammar backend has been dropped; configurations relying on it will break on upgrade.
100. v1.22.0 Jul 23, 2023 · issue -388

     LocalAI v1.22.0 adds a new llama-master backend, JSONSchema ref resolution for planners, and a Llama 2 chat message template.

     └──▷ GET THIS VERSION
     copy

     $ git clone --branch v1.22.0 https://github.com/mudler/LocalAI.git
     # already have the repo? check out this version:
     $ git checkout v1.22.0

     • ›Adds the llama-master backend for running llama.cpp models via the latest upstream llama.cpp code.
     • ›Adds a chat message template for Llama 2 models to correctly format multi-turn conversation prompts.
     • ›Improves backend internals with general backends improvements in this release.
     └──▷ BREAKING ON UPGRADE

     • !The backend formerly named llama-master is renamed to llama; the backend formerly named llama is renamed to llama-grammar — any model config or API calls referencing these backend names must be updated.
101. v1.21.0 Jul 16, 2023 · issue -388

     LocalAI v1.21.0 adds gRPC backends, LocalAI functions, Falcon model support, and verbose_json transcription.

     └──▷ GET THIS VERSION
     copy

     $ git clone --branch v1.21.0 https://github.com/mudler/LocalAI.git
     # already have the repo? check out this version:
     $ git checkout v1.21.0

     • ›Adds gRPC-based backends, enabling a new extensible backend architecture for model serving.
     • ›Adds support for Falcon 7B and 40B models via ggllm.cpp.
     • ›Adds LocalAI functions capability for function-calling workflows.
     • ›Adds partial support for verbose_json format in the transcribe endpoint (Whisper).
102. v1.20.0 Jun 28, 2023 · issue -389

     LocalAI v1.20.0 adds TTS via go-piper, a model gallery repository, a /version endpoint, and NUMA support for llama.

     └──▷ GET THIS VERSION
     copy

     $ git clone --branch v1.20.0 https://github.com/mudler/LocalAI.git
     # already have the repo? check out this version:
     $ git checkout v1.20.0

     └──▷ TRY IT
     Check the running LocalAI server version from the CLI or a script to confirm which release is deployed.
     copy

     $ curl http://localhost:8080/version

     • ›Adds GET /version endpoint and displays the version in the CLI.
     • ›Renames /models/list to /models/available for listing available models.
     • ›Adds text-to-speech (TTS) support via go-piper integration.
     • ›Introduces a gallery repository for browsing and managing models.
     • ›Enables NUMA support in the updated llama backend, improving multi-socket CPU performance.
     +1 moreshow less

     • ›Supports $THREADS environment variable in entrypoint.sh to control parallel compilation at startup.
     └──▷ BREAKING ON UPGRADE

     • !The /models/list endpoint is renamed to /models/available; any client or script calling /models/list will break.
103. v1.19.2 Jun 20, 2023 · issue -389

     LocalAI v1.19.2 adds a LowVRAM option parameter for constrained GPU environments.

     └──▷ GET THIS VERSION
     copy

     $ git clone --branch v1.19.2 https://github.com/mudler/LocalAI.git
     # already have the repo? check out this version:
     $ git checkout v1.19.2

     • ›Adds LowVRAM option parameter to enable inference on systems with limited GPU memory.
104. v1.19.0 Jun 19, 2023 · issue -389

     LocalAI v1.19.0 adds typical_p sampling, extended llama.cpp config, full GPU offloading, and model download progress display.

     └──▷ GET THIS VERSION
     copy

     $ git clone --branch v1.19.0 https://github.com/mudler/LocalAI.git
     # already have the repo? check out this version:
     $ git checkout v1.19.0

     • ›Adds typical_p parameter to model configuration for llama.cpp-based models.
     • ›Extends model configuration options for llama.cpp backends.
     • ›Enables full GPU offloading via updated go-llama bindings, with fixed CUDA images.
     • ›Displays download progress when installing models via the model gallery.
105. v1.18.0 Jun 6, 2023 · issue -389

     LocalAI v1.18.0 adds streaming completions, esrgan upscaling, falcon-7b, Metal GPU, k-quants, and a HuggingFace backend via LangChainGo.

     └──▷ GET THIS VERSION
     copy

     $ git clone --branch v1.18.0 https://github.com/mudler/LocalAI.git
     # already have the repo? check out this version:
     $ git checkout v1.18.0

     └──▷ TRY IT
     Stream completion tokens in real time from a locally-hosted model, useful for low-latency chat UIs or piped processing.
     copy

     $ curl -s http://localhost:8080/v1/completions -H 'Content-Type: application/json' -d '{"model": "ggml-gpt4all-j", "prompt": "Explain zero-trust networking:", "stream": true}'

     • ›Adds Stream: true support to the completion endpoint, enabling server-sent event streaming for inference responses.
     • ›Adds LangChainGo HuggingFace backend, allowing LocalAI to route completions through HuggingFace-hosted models via LangChainGo.
     • ›Adds experimental support for falcon-7b models.
     • ›Adds upscaled image generation using esrgan.
     • ›Adds Metal GPU support for gpt4all on Apple Silicon.
     +5 moreshow less

     • ›Updates llama.cpp to support k-quants model quantization formats.
     • ›Enables Stable Diffusion by default in container images.
     • ›Adds ffmpeg-enabled Docker image variants for audio processing use cases.
     • ›Enables static builds for Linux binaries, removing dynamic library dependencies.
     • ›Builds multiple Docker image variants via CI, expanding deployment options.
     └──▷ BREAKING ON UPGRADE

     • !gpt4all is updated with support for multiple implementations selectable at runtime; existing gpt4all configurations may need to be updated to specify the desired implementation.
106. v1.17.1 May 30, 2023 · issue -390

     LocalAI v1.17.1 adds CuBLAS support to Docker images for GPU-accelerated inference.

     └──▷ GET THIS VERSION
     copy

     $ git clone --branch v1.17.1 https://github.com/mudler/LocalAI.git
     # already have the repo? check out this version:
     $ git checkout v1.17.1

     • ›Adds CuBLAS support to Docker images, enabling GPU-accelerated model inference without a separate build.
107. v1.17.0 May 29, 2023 · issue -390

     LocalAI v1.17.0 adds CLBlast GPU acceleration support and bumps the RWKV backend.

     └──▷ GET THIS VERSION
     copy

     $ git clone --branch v1.17.0 https://github.com/mudler/LocalAI.git
     # already have the repo? check out this version:
     $ git checkout v1.17.0

     • ›Adds CLBlast support, enabling OpenCL-based GPU acceleration for inference.
     • ›Bumps the RWKV backend to a newer revision.
     └──▷ BREAKING ON UPGRADE

     • !The RWKV backend has been bumped to a new version, which may break existing RWKV model configurations or bindings.
108. v1.16.0 May 27, 2023 · issue -390

     LocalAI v1.16.0 adds Kubernetes health endpoints, static builds, model preloading via env var, and prompt cache path support.

     └──▷ GET THIS VERSION
     copy

     $ git clone --branch v1.16.0 https://github.com/mudler/LocalAI.git
     # already have the repo? check out this version:
     $ git checkout v1.16.0

     └──▷ TRY IT
     Probe LocalAI readiness in a Kubernetes deployment without waiting for a model to be loaded.
     copy

     $ curl http://localhost:8080/readyz

     • ›Adds /healthz and /readyz HTTP endpoints for Kubernetes liveness and readiness probes.
     • ›Allows preloading models before startup via environment variable or config files.
     • ›Adds support for setting a prompt cache path and enabling state saving.
     • ›Adds static builds of LocalAI.
     • ›Adds Docker image HEALTHCHECK support.