◆ AI Model & Data Infrastructure

Darkbloom

v0.8.15 commercial

Private Inference Network on Idle Macs

source ↗ website ↗ rss ↗ last repo push Aug 25, 2026

curl https://api.darkbloom.dev/v1/chat/completions \
  -H "Authorization: Bearer sk-db-..." \
  -H "Content-Type: application/json" \
  -d '{"model":"gemma-4-26b","messages":[{"role":"user","content":"Hello!"}],"metadata_details":true,"stream":false}'
curl https://api.darkbloom.dev/v1/chat/completions \
  -H "Authorization: Bearer sk-db-..." \
  -H "Content-Type: application/json" \
  -H "X-Darkbloom-Metadata-Details: true" \
  -d '{"model":"gemma-4-26b","messages":[{"role":"user","content":"Hello!"}],"stream":false}'
mtp_mode = "off"
DARKBLOOM_CBV2_MAX_PARTIAL_PREFILLS=0 darkbloom restart
curl https://api.darkbloom.dev/v1/models/openrouter \
  -H 'Authorization: Bearer sk-db-...'
DARKBLOOM_CBV2_SOLO_PREFILL_STRIPE=0 darkbloom
DARKBLOOM_CBV2_MAX_PARTIAL_PREFILLS=1 darkbloom
curl http://localhost:<port>/metrics | grep mtp
darkbloom report --dry-run
EIGENINFERENCE_MEDIA_FETCH_ENABLED=false

Summary

Private Inference Network on Idle Macs

Command line

composed command

option surface across analysed releases

options at that release added, per release removed, per release

option	found in	since	description

Values are placeholders taken from each option’s declared default. Nothing is executed here — the output shown is a recording of a run that already happened.

Release history

v0.8.15 Aug 29, 2026 · issue 011

Darkbloom v0.8.15 adds opt-in JSON metadata on chat completions and integrates the Qwen3.8-27B VLM.
└──▷ GET THIS VERSION
```
$ git clone --branch v0.8.15 https://github.com/Layr-Labs/d-inference.git
# already have the repo? check out this version:
$ git checkout v0.8.15
```
└──▷ TRY IT
Retrieve attestation, timing, and job-identity details in the response body when your SDK strips custom HTTP headers.
```
$ curl https://api.darkbloom.dev/v1/chat/completions \
  -H "Authorization: Bearer sk-db-..." \
  -H "Content-Type: application/json" \
  -d '{"model":"gemma-4-26b","messages":[{"role":"user","content":"Hello!"}],"metadata_details":true,"stream":false}'
```
Opt in to metadata details via header (e.g. from a browser context that cannot set a custom JSON body key).
```
$ curl https://api.darkbloom.dev/v1/chat/completions \
  -H "Authorization: Bearer sk-db-..." \
  -H "Content-Type: application/json" \
  -H "X-Darkbloom-Metadata-Details: true" \
  -d '{"model":"gemma-4-26b","messages":[{"role":"user","content":"Hello!"}],"stream":false}'
```
- ›Adds metadata_details: true request body field to POST /v1/chat/completions to embed provider attestation, timing, and job identity details directly in the JSON response body — useful when OpenAI-compatible SDKs hide raw HTTP headers.
- ›Adds X-Darkbloom-Metadata-Details: true request header as an alternative opt-in for the same per-request metadata embedding, supporting browser preflight flows.
- ›When metadata is opted in, metadata.location surfaces the serving provider's GeoIP region as region, region_code, country, country_code, and timezone fields; city, coordinates, and raw IPs are intentionally omitted.
- ›Integrates Qwen3.8-27B full VLM (EigenLabs/Qwen3.8-27B-4bit) with vision and video (API) support, restricted to Apple M5 hardware with the approved NAX runtime.
- ›Adds EigenLabs/Qwen3.8-27B-MTP-4bit as an MTP proposal assistant for Qwen3.8, enabled by default; disable with mtp_mode = "off", darkbloom beta disable mtp, or the DARKBLOOM_CBV2_MTP=0 environment variable.
v0.8.14 Aug 27, 2026 · issue 009

Darkbloom v0.8.14 adds Qwen3-VL production serving, automatic Qwen MTP defaulting, and competing-inference warnings in darkbloom doctor.
└──▷ GET THIS VERSION
```
$ git clone --branch v0.8.14 https://github.com/Layr-Labs/d-inference.git
# already have the repo? check out this version:
$ git checkout v0.8.14
```
└──▷ USE IT
Opt a provider explicitly out of MTP on Qwen models, or force it on, without relying on the new automatic default.
toml
```
mtp_mode = "off"
```
- ›Adds mtp_mode config key accepting 'auto' | 'on' | 'off' to control Multi-Token Prediction; valid inline Qwen3.5/3.6 MTP artifacts now default to auto (on) while Gemma remains opt-in; DARKBLOOM_CBV2_MTP=0 env var remains an independent rollback path.
- ›Config schema v3 migrates legacy generated mtp = false values to auto and retains legacy mtp = true as on, so upgraded providers receive the new policy automatically.
- ›Adds competing inference check to darkbloom doctor that warns when local inference processes (e.g. Ollama on port 11434, llama-server) are detected consuming unified memory alongside the provider.
- ›Adds production serving of Qwen3-VL (qwen3_vl_moe architecture) through the ContinuousBatchingV2 path, with per-row M-RoPE for text, causal visual spans and every DeepStack level for image prefill, and fused homogeneous routed gate/up expert projections at load time.
v0.8.13 Aug 26, 2026 · issue 008

Darkbloom v0.8.13 adds Qwen 3.5 video inference, grounded image captions, and hardware serial number redaction from public APIs.
└──▷ GET THIS VERSION
```
$ git clone --branch v0.8.13 https://github.com/Layr-Labs/d-inference.git
# already have the repo? check out this version:
$ git checkout v0.8.13
```
- ›Supports Qwen 3.5 video inference via EngineV2, sampling at most 8 frames at no more than 512×512 pixels per frame and bounding the vision score tensor to 512 MiB, replacing the previous 400 invalid media input failure.
- ›Adds EXIF-orientation-aware full-raster image decode for Qwen vision requests, defaulting enable_thinking=false for media unless the client explicitly sets reasoning.enabled.
- ›Removes hardware serial numbers, UDIDs, and raw MDA certificates from public APIs, headers, URLs, and enrollment flows; public surfaces now expose only opaque provider IDs and trust-verdict status.
- ›Updates earnings calculators to require Mac model, chip family, and unified memory inputs before showing an estimate, grounded in bandwidth-limited capacity; adds M5 Ultra and M6 hardware profiles.
- ›Pins libs/mlx-swift-lm to commit fe01df9, adding Qwen3-VL 30B-A3B model support.
v0.8.12 Aug 25, 2026 · issue 008

Darkbloom v0.8.12 enables atomic first-token deadline admission by default, with typed TOML override support.
└──▷ GET THIS VERSION
```
$ git clone --branch v0.8.12 https://github.com/Layr-Labs/d-inference.git
# already have the repo? check out this version:
$ git checkout v0.8.12
```
└──▷ TRY IT
Fall back to unlimited partial-prefill interleave (bypassing forecast admission) while keeping hard absolute expiry in place.
```
$ DARKBLOOM_CBV2_MAX_PARTIAL_PREFILLS=0 darkbloom restart
```
- ›Sets atomic first-token deadline admission to default-on for eligible requests, returning a health-neutral deadline_unreachable before GPU prompt work begins so the coordinator can retry another provider.
v0.8.9 Aug 21, 2026 · issue 007

Darkbloom v0.8.9 adds concrete-source support for OpenRouter aliases, exposing them via a dedicated /v1/models/openrouter endpoint.
└──▷ GET THIS VERSION
```
$ git clone --branch v0.8.9 https://github.com/Layr-Labs/d-inference.git
# already have the repo? check out this version:
$ git checkout v0.8.9
```
└──▷ TRY IT
Fetch the OpenRouter-specific alias catalog, which now includes aliases backed by concrete catalog models.
```
$ curl https://api.darkbloom.dev/v1/models/openrouter \
  -H 'Authorization: Bearer sk-db-...'
```
- ›Adds support for OpenRouter-only aliases to reference concrete catalog models as their source, not just standard rollout aliases; alias source kind is persisted to prevent silent retargeting on later takeovers.
- ›Exposes OpenRouter-only aliases exclusively through /v1/models/openrouter, omitting them from the main /v1/models list while preserving exact GET /v1/models/{id} retrieval — including at zero connected providers.
- ›Shares source pricing, limits, features, readiness, capacity, and datacenter data in the dedicated OpenRouter alias feed.
v0.8.7 Aug 20, 2026 · issue 007

Darkbloom v0.8.7 exposes OpenRouter aliases in the model catalog via GET /v1/models.
└──▷ GET THIS VERSION
```
$ git clone --branch v0.8.7 https://github.com/Layr-Labs/d-inference.git
# already have the repo? check out this version:
$ git checkout v0.8.7
```
- ›Surfaces OpenRouter-only model aliases in GET /v1/models, making previously undiscoverable model IDs available for direct inference requests.
v0.8.6 Aug 20, 2026 · issue 007

Darkbloom v0.8.6 ships default-on CBv2 prefill stack (+38% prefill speed), OpenRouter model aliases, and new MTP metrics.
└──▷ GET THIS VERSION
```
$ git clone --branch v0.8.6 https://github.com/Layr-Labs/d-inference.git
# already have the repo? check out this version:
$ git checkout v0.8.6
```
└──▷ TRY IT
Disable the solo-prefill stripe on a battery-backed or Low Power Mode provider to avoid the known ~12% TTFT regression.
```
$ DARKBLOOM_CBV2_SOLO_PREFILL_STRIPE=0 darkbloom
```
Opt in to mean-TTFT prefill serialization to spread burst load as a staircase rather than forcing all concurrent requests to wait for the full prefill makespan.
```
$ DARKBLOOM_CBV2_MAX_PARTIAL_PREFILLS=1 darkbloom
```
- ›Adds DARKBLOOM_CBV2_SOLO_PREFILL_STRIPE=0 environment variable to disable the new solo-prefill stripe (2048-token chunks), which regresses TTFT ~12% under Low Power Mode — throttled or battery-backed providers should set this escape.
- ›Adds DARKBLOOM_CBV2_PREFILL_NARROWING=0 environment variable to restore pre-v0.8.6 behavior where intermediate chunks return the full [1,512,248320] logits tensor (242.5 MiB/chunk) instead of a one-element handle.
- ›Adds DARKBLOOM_CBV2_MAX_PARTIAL_PREFILLS=1 opt-in environment variable to cap prompt-work rows per scheduler step (FCFS), turning burst TTFTs into a staircase instead of all requests waiting for the full makespan.
- ›Enables CBv2 prefill stack by default, delivering ~1,766 tok/s at 8K on M4 Max (+38% vs v0.8.5 defaults) via expert-tile trust serving default, solo-prefill stripe, recurrent prompt narrowing, and packed prefill for Qwen3.6.
- ›Adds OpenRouter-only model aliases in the coordinator, expanding the set of model IDs routable through OpenRouter.
+1 moreshow less
- ›Promotes adaptive persistent-history MTP with persistent head KV and adaptive width selection onto the release pin (still behind the mtp beta flag).
v0.8.5 Aug 15, 2026 · issue 007

Darkbloom v0.8.5 adds MTP posture and acceptance counters to the local /metrics endpoint and human-approved agent production access.
└──▷ GET THIS VERSION
```
$ git clone --branch v0.8.5 https://github.com/Layr-Labs/d-inference.git
# already have the repo? check out this version:
$ git checkout v0.8.5
```
└──▷ TRY IT
Poll MTP posture and acceptance counters from a running local provider to verify speculative decoding is active and measure acceptance rates.
```
$ curl http://localhost:<port>/metrics | grep mtp
```
- ›Exposes mtp_enabled, mtp_active, mtp_rounds_total, mtp_tokens_proposed_total, mtp_tokens_accepted_total, and mtp_inactive_reason{model,reason} counters on the local /metrics endpoint in both --local and unified serving modes, making MTP acceptance observable without Datadog Logs.
- ›Allows human-approved agents to operate in production environments.
- ›Adds Qwen3.6 E=256 expert-tile prefill route with fused gate/up projection (SwitchGLU(fuseGateUp: true)) and an opt-in trust refinement, delivering up to +15.2% prefill throughput at 8k context on M4 Max.
- ›Adds adaptive persistent-history capture-verify MTP stack for Qwen3.6 (behind the mtp beta flag, default off), measured at 2.0077x throughput vs non-MTP in canary validation.
v0.8.3 Aug 13, 2026 · issue 007

Darkbloom v0.8.3 adds Qwen3.6-35B-A3B VLM with inline MTP on the production provider path.
└──▷ GET THIS VERSION
```
$ git clone --branch v0.8.3 https://github.com/Layr-Labs/d-inference.git
# already have the repo? check out this version:
$ git checkout v0.8.3
```
- ›Adds production-path text, image, and tool inference for the combined Qwen3.6-35B-A3B VLM artifact via the EngineV2 provider path, with request-owned recurrent state, three-axis mRoPE state, causal vision attention, exact rollback, and source-matched target/assistant memory accounting.
- ›Ships inline MTP (depth-one, serial, exact-target-verified) alongside the Qwen3.6-35B-A3B model; video, prefix reuse, paged KV, compiled decode, packed prefill, and rectangular MTP remain fail-closed.
v0.8.2 Aug 12, 2026 · issue 007

Darkbloom v0.8.2 adds self-serve payout bank updates for providers, Gemma 4 26B config-controlled optimizations, and tightened provider privacy controls.
└──▷ GET THIS VERSION
```
$ git clone --branch v0.8.2 https://github.com/Layr-Labs/d-inference.git
# already have the repo? check out this version:
$ git checkout v0.8.2
```
└──▷ TRY IT
Review what darkbloom report would upload before sending, preserving macOS unified-log privacy redaction.
```
$ darkbloom report --dry-run
```
- ›Enables [gemma_optimizations] config controls for Gemma 4 26B-A4B v0.8.2, gating layer-18 lazy prefill submission, coupled weighted-expert-unsort, and safe-R1 expert-QMM gate (both default-on).
- ›Providers can now update their own Stripe Express payout bank account via a new self-serve endpoint, removing the need to contact support for bank account changes.
- ›Keeps inline video (MP4/QuickTime) off disk by decoding coordinator-inlined bytes through a bounded, memory-backed AVFoundation asset and purging any legacy vlm-<UUID>.mp4 files on launch.
- ›Closes provider-derived plaintext egress paths: inference failures cross the WebSocket boundary only as closed-vocabulary codes/reasons; free-form telemetry and automatic provider log reporting are retired in favor of the operator-initiated darkbloom report command (with --dry-run support and authenticated upload).
v0.8.0 Jul 29, 2026 · issue 007

Darkbloom v0.8.0 completes PagedAttention migration and adds live-togglable remote media fetching at the coordinator.
└──▷ GET THIS VERSION
```
$ git clone --branch v0.8.0 https://github.com/Layr-Labs/d-inference.git
# already have the repo? check out this version:
$ git checkout v0.8.0
```
└──▷ TRY IT
Disable remote media fetching at runtime on a running coordinator without redeploying — useful when you want to enforce that clients must pre-inline all images.
```
$ EIGENINFERENCE_MEDIA_FETCH_ENABLED=false
```
- ›Adds EIGENINFERENCE_MEDIA_FETCH_ENABLED=false environment variable to disable remote media fetching live — no redeploy required — restoring the prior HTTP 400 response for requests with remote image or video URLs.
- ›Accepts OpenAI-style remote http(s) image_url and video_url parts on POST /v1/chat/completions: the coordinator fetches each URL once, structurally validates it, and rewrites it as an inline base64 data: URI before coordinator-to-provider encryption — keeping provider Macs from ever fetching request-controlled URLs.
- ›Rejects remote media URLs in sender-sealed requests with instructions to inline them, preserving the end-to-end encryption guarantee for sealed payloads.

Darkbloom

Summary

Examples

Command line

Release history