Heads up This site is currently under heavy development.
Subscribe Get it delivered — the daily firehose, filtered to the tools you run, plus the documentation changes vendors never announce. Compare plans →

The AI Toolchain — issue -318, October 2, 2025

THE AI TOOLCHAIN NO. -318
Tail
THE DAILY RELEASE FIREHOSE
PUBLISHED OCTOBER 2, 2025 · EVERY WEEKDAY
EDITIONS tail grep head diff uniq

The daily firehose — everything the toolchain shipped today, already filtered.

// HOW THIS ISSUE IS MADE

We read every release from the 174 tools on our watchlist at the source — GitHub and GitLab release notes, vendor release pages and changelogs, project blogs and feeds, vendor press releases, and the source code behind the tag. Bug-fix-only releases and non-product newsroom noise are dropped; what's left is summarized down to the new capability, how to try it, and any screenshots or videos the release itself published. Every entry links to the sources it was built from.

VIEW
ISSUE VIEW full issue
Do you prefer this view?
$ tct list   # 10 tools matched
AI & LLM Tooling
◆  AI Agent Frameworks

AutoGPT

Sources Release notes → autogpt-platform-beta-v0.6.31 NOTES

AutoGPT Platform v0.6.31 adds claude-sonnet-4.5 support, an AI Condition Block, and a table input UI builder block.

└──▷ GET THIS VERSION
$ git clone --branch autogpt-platform-beta-v0.6.31 https://github.com/Significant-Gravitas/AutoGPT.git
# already have the repo? check out this version:
$ git checkout autogpt-platform-beta-v0.6.31
  • Adds claude-sonnet-4.5 model support across the platform.
  • Introduces an AI Condition Block for evaluating conditions expressed in natural language, enabling no-code branching logic in agent graphs.
  • Adds a table input UI and builder block for structured tabular data entry in agent workflows.
Was this useful?

LangChain

Sources Release notes → langchain-qdrant==1.0.0a1 10 RELEASES · 2025-10-02 NOTES STABLE

langchain-qdrant 1.0.0a1 adds similarity_search_with_score_by_vector() and a new QdrantVectorStore with sparse embeddings support.

└──▷ GET THIS VERSION
$ git clone --branch langchain-qdrant==1.0.0a1 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout langchain-qdrant==1.0.0a1
└──▷ USE IT
Use the new QdrantVectorStore in SPARSE mode with a retriever, without needing a dense embedding model.
python
store = QdrantVectorStore.from_existing_collection(
    url="http://localhost:6333",
    collection_name="my_collection",
    sparse_embedding=my_sparse_embedder,
    retrieval_mode=RetrievalMode.SPARSE,
)
retriever = store.as_retriever()
  • Adds similarity_search_with_score_by_vector() method to QdrantVectorStore for direct vector-based similarity search with scores.
  • Adds _asimilarity_search_with_relevance_scores() async method to the Qdrant class for async relevance-scored search.
  • Introduces new QdrantVectorStore implementation as the primary vector store interface, replacing the legacy Qdrant class.
  • Adds sparse embeddings provider interface to QdrantVectorStore, enabling hybrid dense/sparse retrieval workflows.
  • Enables as_retriever() to work without embeddings when operating in SPARSE mode.
+2 moreshow less
  • Removes Python upper bound constraint in packaging, allowing compatibility with a broader range of Python environments.
  • Adds support for Python 3.13 in CI, signaling readiness for that runtime.
9 more releases in this issue · 2025-10-02
langchain-perplexity==1.0.0a1 NOTES STABLE

langchain-perplexity 1.0.0a1 adds Perplexity chat integration with search_results exposure in ChatPerplexity.

└──▷ GET THIS VERSION
$ git clone --branch langchain-perplexity==1.0.0a1 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout langchain-perplexity==1.0.0a1
  • Exposes search_results field in the ChatPerplexity chat model, giving callers access to Perplexity's cited search results alongside generated responses.
  • Adds initial ChatPerplexity integration, bringing Perplexity's chat API into the LangChain library as a first-class chat model.
langchain-groq==1.0.0a1 NOTES STABLE

langchain-groq 1.0.0a1 adds json_schema support, reasoning output access, service tier, and loosened reasoning_effort controls for ChatGroq.

└──▷ GET THIS VERSION
$ git clone --branch langchain-groq==1.0.0a1 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout langchain-groq==1.0.0a1
└──▷ USE IT
Use strict JSON schema-based structured output with a Groq reasoning model to get validated, typed responses.
python
from langchain_groq import ChatGroq
from pydantic import BaseModel

class Answer(BaseModel):
    reasoning: str
    result: str

llm = ChatGroq(model="deepseek-r1-distill-llama-70b", reasoning_effort="default")
structured = llm.with_structured_output(Answer, method="json_schema")
print(structured.invoke("Explain why the sky is blue."))
Stream a response and inspect reasoning output and usage metadata injected into response chunks.
python
from langchain_groq import ChatGroq

llm = ChatGroq(model="deepseek-r1-distill-llama-70b", reasoning_effort="default")
for chunk in llm.stream("Solve: what is 42 * 17?"):
    print(chunk.content, chunk.response_metadata)
  • Adds json_schema as a supported structured output method in ChatGroq, enabling strict schema-based response formatting.
  • Adds reasoning_effort parameter to ChatGroq with loosened restrictions and injection into response metadata, supporting Groq reasoning models.
  • Adds service tier option to ChatGroq for selecting Groq API service tiers.
  • Adds access to reasoning output from Groq models via response metadata in ChatGroq.
  • Adds response metadata when streaming from ChatGroq.
+11 moreshow less
  • Adds usage_metadata to invoke, ainvoke, stream, and astream responses in ChatGroq.
  • Adds support for tool_choice=any and tool_choice=required in ChatGroq.
  • Adds strict and method parameters to with_structured_output in ChatGroq.
  • Adds OpenAI-OSS compatible model support to ChatGroq.
  • Supports overriding ls_model_name from kwargs in model tracing.
  • Adds stop attribute to ChatGroq.
  • Adds streaming tool calls support to ChatGroq.
  • Adds tool calling support to ChatGroq via .tool_calls attribute.
  • Adds Groq proxy support to ChatGroq.
  • Adds user-agent header injection to ChatGroq requests.
  • Removes the default model requirement, with a warning emitted when no model is specified.
└──▷ BREAKING ON UPGRADE
  • !The default model is removed from ChatGroq; callers that relied on a default model must now explicitly specify one or a warning will be emitted.
langchain-deepseek==1.0.0a1 NOTES STABLE

LangChain ships langchain-deepseek 1.0.0a1, adding a ChatDeepSeek integration with structured output and reasoning support.

└──▷ GET THIS VERSION
$ git clone --branch langchain-deepseek==1.0.0a1 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout langchain-deepseek==1.0.0a1
└──▷ USE IT
Instantiate a DeepSeek chat model by provider string without importing the partner package explicitly.
python
from langchain.chat_models import init_chat_model

llm = init_chat_model("deepseek-chat", model_provider="deepseek")
response = llm.invoke("Explain zero-day exploits in one paragraph.")
print(response.content)
Extract structured findings from model output using strict JSON schema enforcement via with_structured_output.
python
from langchain_deepseek import ChatDeepSeek
from pydantic import BaseModel

class ThreatReport(BaseModel):
    cve_id: str
    severity: str
    summary: str

llm = ChatDeepSeek(model="deepseek-chat")
structured_llm = llm.with_structured_output(ThreatReport, method="json_schema", strict=True)
report = structured_llm.invoke("Summarize CVE-2024-1234 as a threat report.")
print(report)
  • Adds ChatDeepSeek chat model integration, accessible via the langchain-deepseek package, enabling DeepSeek models to be used as a drop-in LangChain chat model.
  • Supports strict and method parameters in with_structured_output for ChatDeepSeek, giving callers control over structured-output enforcement mode.
  • Registers DeepSeek as a named provider in LangChain's init_chat_model, so models can be instantiated by provider string without importing the partner package directly.
  • Surfaces reasoning_content in streamed chunks from DeepSeek-R1, exposing chain-of-thought reasoning alongside the final response.
langchain-chroma==1.0.0a1 NOTES STABLE

langchain-chroma 1.0.0a1 debuts with collection forking and Chroma Cloud support.

└──▷ GET THIS VERSION
$ git clone --branch langchain-chroma==1.0.0a1 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout langchain-chroma==1.0.0a1
  • Adds collection forking via feat(chroma): Add support for collection forking — enables branching an existing Chroma collection into a new one without duplicating the underlying data pipeline.
  • Adds Chroma Cloud support, allowing langchain-chroma to connect to hosted Chroma Cloud deployments in addition to local instances.
langchain-xai==1.0.0a1 NOTES STABLE

langchain-xai 1.0.0a1 adds xAI/Grok chat integration with live search, reasoning content, and structured output support.

└──▷ GET THIS VERSION
$ git clone --branch langchain-xai==1.0.0a1 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout langchain-xai==1.0.0a1
  • Adds langchain-xai partner integration package providing a LangChain chat model for xAI's Grok models.
  • Supports live search capability in the xAI chat integration.
  • Supports reasoning content in the xAI chat integration.
  • Supports dedicated structured output feature, including strict and method parameters in with_structured_output.
  • Supports tool_choice enforcement standards in the xAI chat integration.
langchain-text-splitters==1.0.0a1 NOTES STABLE

langchain-text-splitters 1.0.0a1 adds custom Markdown header patterns, Visual Basic 6 support, and keep_separator for HTML splitting.

└──▷ GET THIS VERSION
$ git clone --branch langchain-text-splitters==1.0.0a1 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout langchain-text-splitters==1.0.0a1
└──▷ USE IT
Split HTML content while keeping separator tokens in each chunk — useful when downstream models need boundary context.
python
from langchain_text_splitters import HTMLSemanticPreservingSplitter

splitter = HTMLSemanticPreservingSplitter(keep_separator=True)
chunks = splitter.split_text(html_content)
Split a JavaScript React component file into logical chunks for indexing or retrieval.
python
from langchain_text_splitters import JSFrameworkTextSplitter

splitter = JSFrameworkTextSplitter()
chunks = splitter.split_text(open('App.jsx').read())
Split Visual Basic 6 source code recursively by language-aware separators for code search or review workflows.
python
from langchain_text_splitters import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter.from_language(language='vb', chunk_size=500, chunk_overlap=50)
chunks = splitter.split_text(open('Module1.bas').read())
  • Adds keep_separator argument to HTMLSemanticPreservingSplitter to control whether separators are retained in output chunks.
  • Adds optional custom header pattern support to the Markdown splitter, allowing non-standard heading formats to be recognized.
  • Adds chunk_size and chunk_overlap validation to prevent misconfigured splitters from silently producing bad output.
  • Adds Visual Basic 6 as a supported language in RecursiveCharacterTextSplitter.
  • Adds JSFrameworkTextSplitter for splitting JavaScript framework code (React, Vue, etc.) into meaningful chunks.
+10 moreshow less
  • Adds HTMLSemanticPreservingSplitter for splitting HTML while preserving semantic structure and extracting metadata from tags.
  • Replaces lxml/XSLT with BeautifulSoup in HTMLHeaderTextSplitter for improved processing of large HTML files.
  • Adds PowerShell as a supported language in RecursiveCharacterTextSplitter.
  • Adds ExperimentalMarkdownSyntaxTextSplitter for finer-grained Markdown splitting based on syntax structure.
  • Adds Lua, Haskell, Elixir, and C language support to RecursiveCharacterTextSplitter.
  • Adds ensure_ascii parameter to text splitters to control ASCII encoding of output.
  • Adds add_start_index support and request parameters to HTMLHeaderTextSplitter.split_text.
  • Adds HTMLSectionSplitter, a section-aware splitter that segments HTML documents by structural sections.
  • Extends keep_separator functionality in TextSplitter to support additional separator-preservation modes.
  • Drops Python 3.9 support; minimum supported version is now Python 3.10.
└──▷ BREAKING ON UPGRADE
  • !Python 3.9 is no longer supported; the minimum required Python version is now 3.10.
  • !The xslt_path parameter has been removed from HTMLSectionSplitter and XML parsers have been hardened, removing XSLT-based processing paths.
  • !HTMLHeaderTextSplitter no longer uses lxml and XSLT internally; it now uses BeautifulSoup, which may produce different chunking output for some HTML inputs.
langchain-ollama==1.0.0a1 NOTES STABLE

langchain-ollama v1.0.0a1 adds basic auth, reasoning models, thinking/tool streaming, structured output, and async client kwargs.

└──▷ GET THIS VERSION
$ git clone --branch langchain-ollama==1.0.0a1 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout langchain-ollama==1.0.0a1
└──▷ USE IT
Authenticate against a protected Ollama server endpoint using basic auth credentials.
python
from langchain_ollama import ChatOllama

llm = ChatOllama(
    model="llama3",
    base_url="https://ollama.internal",
    auth=("myuser", "mypassword"),
)
print(llm.invoke("Summarize the OWASP Top 10").content)
Run a reasoning model (e.g. DeepSeek) with a custom reasoning intensity string for tunable chain-of-thought depth.
python
from langchain_ollama import ChatOllama

llm = ChatOllama(
    model="deepseek-r1",
    reasoning_effort="gpt-oss",
)
print(llm.invoke("Explain CVE triage prioritization").content)
Validate that the chosen model is available on the Ollama server at startup, failing fast before any inference requests are sent.
python
from langchain_ollama import ChatOllama

llm = ChatOllama(
    model="mistral",
    validate_model_on_init=True,
)
  • Adds basic auth support to ChatOllama and OllamaLLM via auth parameter in base_url, headers, and auth constructor arguments.
  • Adds validate_model_on_init parameter to ChatOllama to eagerly validate the model name at construction time and catch errors early.
  • Adds keep_alive parameter support to OllamaEmbeddings to control how long the model stays loaded in memory.
  • Adds separate async_client_kwargs parameter to ChatOllama for passing kwargs exclusively to the async Ollama client.
  • Supports reasoning model inference (e.g. DeepSeek) via ChatOllama, with reasoning_effort accepting string values for custom intensity levels such as 'gpt-oss'.
+11 moreshow less
  • Enables token-level streaming when using bind_tools with ChatOllama.
  • Adds streaming support for tool calls in ChatOllama.
  • Supports structured output (with_structured_output) in ChatOllama with an updated default method.
  • Supports passing arbitrary-role ChatMessage objects directly to ChatOllama.
  • Supports standard image input format in ChatOllama including ImageContentBlock.
  • Supports the seed parameter for both ChatOllama and OllamaLLM.
  • Adds model_name to response metadata returned by ChatOllama.
  • Adds backwards-compatible initialization for OllamaEmbeddings when migrating from langchain_community.embeddings to langchain_ollama.embeddings.
  • Adds num_gpu parameter support to the async OllamaEmbeddings method.
  • Emits a warning on empty load responses from the Ollama server.
  • Supports standard content blocks, message IDs, translators, and normalization across ChatOllama.
langchain-core==1.0.0a6 NOTES STABLE

LangChain Core 1.0.0a6 adds standardized GenAI content blocks, PDF tool message support, server tool call types, and new OpenAI/AWS content surface.

└──▷ GET THIS VERSION
$ git clone --branch langchain-core==1.0.0a6 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout langchain-core==1.0.0a6
└──▷ USE IT
Filter OpenAI data content blocks from a message using the now-public is_openai_data_block to strip non-text content before logging.
python
from langchain_core.messages.content import is_openai_data_block

filtered = [block for block in message.content if not is_openai_data_block(block)]
Use a Mustache-formatted prompt template instead of the default f-string format for richer templating syntax.
python
from langchain_core.prompts import PromptTemplate

template = PromptTemplate.from_template(
    'Hello, {{name}}! You are a {{role}}.',
    template_format='mustache'
)
print(template.invoke({'name': 'Alice', 'role': 'security analyst'}))
Sanitize user-supplied text before writing to PostgreSQL to avoid NUL-byte DataErrors at ingestion time.
python
from langchain_core.utils import sanitize_for_postgres

clean_text = sanitize_for_postgres(raw_text)
vectorstore.add_texts([clean_text])
  • Adds is_openai_data_block as a public API with filtering support for inspecting OpenAI data content blocks.
  • Adds id field to Document objects passed to the filter callback in InMemoryVectorStore similarity search.
  • Adds web_search to the OpenAI tools list recognized by the framework.
  • Adds sanitize_for_postgres utility function to strip PostgreSQL NUL bytes and prevent DataError on insert.
  • Adds support for overriding ls_model_name from kwargs when tracing LLM calls.
+14 moreshow less
  • Adds support for PromptTemplate formats other than f-string (e.g., mustache, jinja2) via the format parameter.
  • Adds support for AWS Bedrock document content blocks in msg_content_output.
  • Adds standard content blocks, IDs, translators, and normalization layer (feat: standard content, IDs, translators, & normalization).
  • Adds GenAI standard content block support (feat(core): genai standard content).
  • Adds PDF input support in ToolMessages including tracing.
  • Adds server tool call and result types for the v1 message surface.
  • Adds standard content block support for AWS Bedrock in the v1 message surface.
  • Adds reasoning_content parsing from additional_kwargs and support for the reasoning type in convert_to_openai_messages.
  • Adds a custom Mermaid diagram URL option, allowing the graph visualization endpoint to be overridden.
  • Adds an option to make deserialization more permissive for forward-compatibility.
  • Adds additional hashing options to the indexing API and warns on SHA-1 usage.
  • Adds tracing of response body on error for improved observability.
  • Zeros out token costs for cache hits in token usage tracking.
  • Drops support for Python 3.9 in the v1 release line.
└──▷ BREAKING ON UPGRADE
  • !Python 3.9 is no longer supported in langchain-core v1.0.x; the minimum supported version is Python 3.10.
  • !The example attribute has been removed from AIMessage and HumanMessage; code that sets or reads message.example will break.
  • !The beta namespace and context API have been removed (chore(core): remove beta namespace and context api).
langchain-ollama==0.3.9 NOTES STABLE

langchain-ollama 0.3.9 adds basic authentication support for Ollama connections.

└──▷ GET THIS VERSION
$ git clone --branch langchain-ollama==0.3.9 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout langchain-ollama==0.3.9
  • Adds basic auth support to the Ollama integration, enabling authenticated connections to Ollama endpoints.
Was this useful?

PydanticAI

Sources Release notes → v1.0.13 NOTES

PydanticAI v1.0.13 adds contextual agent instruction overrides, exposes MCPServer.server_info, and upgrades OTel instrumentation.

└──▷ GET THIS VERSION
$ git clone --branch v1.0.13 https://github.com/pydantic/pydantic-ai.git
# already have the repo? check out this version:
$ git checkout v1.0.13
└──▷ USE IT
Inspect MCP server metadata after connecting — useful for logging or validating server capabilities before dispatching tool calls.
python
info = await mcp_server.server_info
print(info)
  • Exposes server_info on MCPServer instances, giving access to MCP server metadata at runtime.
  • Supports contextually overriding agent instructions at runtime, enabling dynamic per-request instruction customization.
  • Upgrades OpenTelemetry instrumentation to version 3 with updated eval attributes for improved observability.
Was this useful?
◆  AI Coding Agents

Continue

Sources Release notes → @continuedev/[email protected] NOTES

Continue config-yaml 1.25.0 adds serverName option for registry MCP servers.

└──▷ GET THIS VERSION
$ git clone --branch @continuedev/[email protected] https://github.com/continuedev/continue.git
# already have the repo? check out this version:
$ git checkout @continuedev/[email protected]
└──▷ USE IT
Override the default name of a registry MCP server to avoid conflicts or clarify its role in your config.
yaml
mcpServers:
  - name: my-registry-server
    serverName: custom-server-name
  • Adds serverName option to registry MCP server entries in Continue's config YAML, allowing explicit server name overrides.
Was this useful?

GitHub Copilot CLI

Sources Release notes → v0.0.333 NOTES

GitHub Copilot CLI v0.0.333 adds image input, shell passthrough via !, /usage stats, and --continue session resumption.

└──▷ GET THIS VERSION
$ git clone --branch v0.0.333 https://github.com/github/copilot-cli.git
# already have the repo? check out this version:
$ git checkout v0.0.333
└──▷ TRY IT
Check how many Premium requests and tokens you have consumed mid-session without waiting for session end.
$ /usage
Pick up where you left off by resuming the most recently closed Copilot CLI session.
$ copilot --continue
Run a shell command directly without sending it to the model — useful for quick one-offs inside a Copilot session.
$ ! grep -r 'TODO' ./src
  • Adds --continue flag to resume the most recently closed session.
  • Adds /usage slash command to report Premium request usage, session time, code changes, and per-model token use — also printed automatically at session end.
  • Adds ! prefix to bypass the model and execute shell commands directly from the Copilot CLI prompt.
  • Adds image input support via @-mention to attach image files as model input.
  • Improves --screen-reader mode by replacing icons in the session timeline with descriptive text labels.
Was this useful?

Google gemini-cli

Sources Release notes → v0.9.0-nightly.20251002.0f465e88 5 RELEASES · 2025-10-02 NOTES STABLE

An open-source AI agent that brings the power of Gemini directly into your terminal.

gemini-cli v0.9.0-nightly adds debug logging for the IDE extension and GitHub repo URL trailing-slash support.

└──▷ GET THIS VERSION
$ git clone --branch v0.9.0-nightly.20251002.0f465e88 https://github.com/google-gemini/gemini-cli.git
# already have the repo? check out this version:
$ git checkout v0.9.0-nightly.20251002.0f465e88
  • Adds debug logging to the IDE extension for easier troubleshooting of extension behavior.
  • Supports GitHub repo URLs with a trailing slash, removing a previous input restriction.
4 more releases in this issue · 2025-10-02
v0.9.0-nightly.20251002.a6af7bbb NOTES STABLE

gemini-cli nightly adds IDE extension debug logging and agent submit_final_output tool

└──▷ GET THIS VERSION
$ git clone --branch v0.9.0-nightly.20251002.a6af7bbb https://github.com/google-gemini/gemini-cli.git
# already have the repo? check out this version:
$ git checkout v0.9.0-nightly.20251002.a6af7bbb
  • Introduces submit_final_output tool for agent completion workflows in the agents subsystem.
  • Adds debug logging to the IDE extension for easier troubleshooting of extension behavior.
  • Supports GitHub repo URLs with a trailing slash, removing a friction point when pasting URLs.
v0.9.0-nightly.20251002.4a70d6f2 NOTES STABLE

gemini-cli v0.9.0-nightly adds debug logging for the IDE extension and accepts GitHub repo URLs with trailing slashes.

└──▷ GET THIS VERSION
$ git clone --branch v0.9.0-nightly.20251002.4a70d6f2 https://github.com/google-gemini/gemini-cli.git
# already have the repo? check out this version:
$ git checkout v0.9.0-nightly.20251002.4a70d6f2
  • Adds debug logging to the IDE extension for easier troubleshooting of extension behavior.
  • Supports GitHub repo URLs with a trailing slash, removing a common friction point when pasting URLs.
v0.9.0-nightly.20251002.460ec602 NOTES STABLE

gemini-cli v0.9.0-nightly adds debug logging for IDE extensions and accepts GitHub repo URLs with trailing slashes.

└──▷ GET THIS VERSION
$ git clone --branch v0.9.0-nightly.20251002.460ec602 https://github.com/google-gemini/gemini-cli.git
# already have the repo? check out this version:
$ git checkout v0.9.0-nightly.20251002.460ec602
  • Supports GitHub repo URLs with a trailing slash as valid input.
  • Adds debug logging to the IDE extension.
v0.9.0-nightly.20251002.99958c68 NOTES STABLE

gemini-cli now accepts GitHub repo URLs with a trailing slash

└──▷ GET THIS VERSION
$ git clone --branch v0.9.0-nightly.20251002.99958c68 https://github.com/google-gemini/gemini-cli.git
# already have the repo? check out this version:
$ git checkout v0.9.0-nightly.20251002.99958c68
  • Supports GitHub repo URLs with a trailing slash as valid input
Was this useful?
◆  Local LLM Runtimes

Jan AI Jan

Sources Release notes → v0.7.0 NOTES

Jan v0.7.0 adds MCP support for web, Azure as a first-class provider, Swagger UI for the API server, thread folders, and an MSI installer.

└──▷ GET THIS VERSION
$ git clone --branch v0.7.0 https://github.com/janhq/jan.git
# already have the repo? check out this version:
$ git checkout v0.7.0
  • Adds configurable model response timeout for the local API server.
  • Adds Swagger UI for the Jan local API server, making the OpenAPI spec browsable and testable in-browser.
  • Adds getTokensCount method to compute token usage.
  • Adds .zip archive support for manual llama.cpp backend installation.
  • Adds Azure as a first-class provider alongside existing remote providers.
+13 moreshow less
  • Adds MCP (Model Context Protocol) support for the Jan web interface.
  • Adds Exa MCP server as a built-in MCP integration.
  • Adds authentication with Google Auth provider for Jan web.
  • Adds thread organization into folders for managing conversation history.
  • Adds MSI installer option for Windows.
  • Adds model selector that fetches available models from v1/models when configuring a provider.
  • Adds custom downloaded model naming.
  • Adds prompt-progress indicator during streaming responses.
  • Enables vision capability for remote providers.
  • Adds LaTeX fragment normalization in markdown rendering.
  • Adds system tray icon build flag for desktop builds.
  • Adjusts RAM/VRAM calculation for unified memory systems (Apple Silicon and similar).
  • Adds web remote conversation support, syncing threads across devices on Jan web.
Was this useful?

vLLM

Sources Release notes → v0.11.0 NOTES

vLLM v0.11.0 ships V1-only engine, CPU KV offloading, new model architectures, FP8/FP4 quant expansions, and major API/CLI additions.

└──▷ GET THIS VERSION
$ git clone --branch v0.11.0 https://github.com/vllm-project/vllm.git
# already have the repo? check out this version:
$ git checkout v0.11.0
└──▷ TRY IT
Request full-vocabulary logprobs for every output token to analyze model confidence across the entire vocab.
$ curl http://localhost:8000/v1/completions -H 'Content-Type: application/json' -d '{"model": "meta-llama/Llama-3-8B-Instruct", "prompt": "Hello world", "logprobs": -1}'
  • Adds --enable-logging CLI flag for controlling logging output.
  • Adds logprobs=-1 support to return logprobs for the full vocabulary via the OpenAI-compatible API.
  • Adds FULL_AND_PIECEWISE as the new default CUDA graph mode (previously PIECEWISE), improving out-of-the-box performance especially for fine-grained MoEs.
  • Adds speculative model engine args to the Config system for configuring speculative decoding at the engine level.
  • Adds NVTX profiling support via config, enabling GPU timeline tracing for performance analysis.
+57 moreshow less
  • Adds LLM.apply_model method for direct model access in the V1 engine.
  • Adds KV cache metrics reporting in GiB units via updated metrics fields.
  • Adds V1 TPOT (Time Per Output Token) histogram metric.
  • Adds KV transfer metrics for disaggregated serving.
  • Adds prompt logprobs for all tokens in OpenAI-compatible completions API.
  • Adds reasoning streaming events to the OpenAI-compatible API.
  • Adds MCP tools support to the Responses API.
  • Adds XML tool-call parser for Qwen3-Coder and Hermes-style tool-call token support.
  • Adds health endpoint 503 response when the engine is dead.
  • Adds image path format support for multimodal inputs via the API.
  • Adds media UUID caching: clients can skip re-uploading media data when UUIDs are provided.
  • Adds torchrun launcher support for data-parallel large-scale serving.
  • Adds CPU KV cache offloading with LRU management.
  • Adds prompt embeddings support in V1 engine.
  • Adds sharded state loading in V1 engine.
  • Adds FlexAttention sliding window attention in V1.
  • Adds shared-memory-based multimodal data caching and IPC between processes.
  • Adds BERT token classification / NER task support.
  • Adds multimodal model support for pooling tasks.
  • Enables DeepGEMM by default, delivering ~5.5% throughput improvement.
  • Enables NCCL symmetric memory by default for tensor parallelism, with 3-4% throughput improvement.
  • Adds NVFP4 quantization support for dense models including Gemma3 and Llama 3.1 405B.
  • Adds FP8 per-token-group quantization and hardware-accelerated FP8 instructions.
  • Adds FP8 FlashInfer MLA decode support for NVIDIA GPUs.
  • Adds BF16 fused MoE for Hopper/Blackwell expert parallel.
  • Adds FlashAttention 3 support for Vision Transformer (ViT) inference.
  • Adds Dual-Batch Overlap (DBO) for overlapping prefill and decode computation.
  • Adds EAGLE3 speculative decoding support for MiniCPM3 and GPT-OSS.
  • Adds SeedOSS reasoning parser.
  • Adds EVS video token pruning for video multimodal models.
  • Adds data-parallel support for vision encoders in InternVL, Qwen2-VL, and Qwen3-VL.
  • Adds RADIO encoder support.
  • Adds Transformers backend support for encoder-only models.
  • Adds new model architectures: DeepSeek-V3.2-Exp, Qwen3-VL, Qwen3-Next, OLMo3, LongCat-Flash, Dots OCR, Ling2.0, CWM.
  • Adds RISC-V 64-bit and ARM non-x86 CPU backend support.
  • Adds ROCm 7.0 support.
  • Adds ARM 4-bit fused MoE kernel.
  • Adds Whisper model support on Intel XPU.
  • Adds Hybrid SSM/Attention support in Triton.
  • Adds torch.compile CUDA graph Inductor partition integration.
  • Adds EPLB (Expert-Parallel Load Balancing) support for Hunyuan V1, Mixtral, and static placement.
  • Adds Mamba2 support with tensor parallelism and quantization.
  • Adds MRoPE + YaRN support for long-context models.
  • Adds --help improvements to the CLI for better discoverability.
  • Adds env validation for configuration, catching misconfigured environment variables at startup.
  • Adds guided decoding backward compatibility fixes.
  • Adds NIXL MLA latent dimension support for disaggregated serving.
  • Adds Ray placement group support for data-parallel deployments.
  • Adds Triton DP/EP kernels for large-scale data/expert parallelism.
  • Adds FlashInfer speculative decoding backend with 1.14x speedup.
  • Adds CUDA graph Inductor partition integration for torch.compile.
  • Adds optimized LoRA weight loading.
  • Adds blocked FP8 for MoE via compressed tensors.
  • Adds W4A8 faster preprocessing.
  • Adds FP8 torch.compile KV cache support.
  • Adds EPLB reduced overhead and shared expert overlap optimization for MoE.
  • V0 engine (AsyncLLMEngine, LLMEngine, MQLLMEngine) and all V0 attention backends have been fully removed; V1 is now the sole engine.
└──▷ BREAKING ON UPGRADE
  • !AsyncLLMEngine, LLMEngine, and MQLLMEngine have been removed entirely — any code importing or instantiating these classes will break.
  • !All V0 attention backends have been removed; deployments relying on V0 backend selection will break.
  • !The default CUDA graph mode is now FULL_AND_PIECEWISE instead of PIECEWISE; models that only support PIECEWISE mode may need explicit configuration.
  • !C++17 is now enforced globally as a build requirement; builds using older C++ standards will fail.
  • !TPU: xm.mark_step is deprecated in favor of torch_xla.sync; code calling xm.mark_step will produce deprecation errors.
  • !max_seq_len_to_capture interface has been removed.
  • !V0 components removed: encoder-decoder support, V0 output processor, V0 sampling metadata, V0 Sequence/Sampler, V0 async output processor, MultiModalPlaceholderMap, V0 seq group methods, placeholder attention, V0 input embeddings, V0 multimodal registry, V0 attention classes, V0 hybrid model support, V0 backend suffixes, V0 compilation fallbacks, and V0 default args.
  • !--async-scheduling produces incorrect (gibberish) output in v0.11.0 under preemption and other scenarios; use v0.10.1 if this flag is required.
Was this useful?
Other / Uncategorized
◆  AI OBSERVABILITY

Arize Phoenix

Sources Release notes → arize-phoenix-evals-v2.4.0 NOTES

Arize Phoenix Evals 2.4.0 adds coroutine (async) function support to create_evaluator.

└──▷ GET THIS VERSION
$ git clone --branch arize-phoenix-evals-v2.4.0 https://github.com/Arize-ai/phoenix.git
# already have the repo? check out this version:
$ git checkout arize-phoenix-evals-v2.4.0
└──▷ USE IT
Wrap an async LLM-calling function as a Phoenix evaluator to run evaluations concurrently.
python
import asyncio
import phoenix.evals as evals

async def my_async_eval_fn(input):
    # async call to an LLM or scoring service
    return {"score": 1.0, "label": "correct"}

evaluator = evals.create_evaluator(my_async_eval_fn)
  • Adds coroutine (async) function support to phoenix.evals.create_evaluator, enabling async callables to be used directly as custom evaluators.
Was this useful?

Langfuse

Sources Release notes → v3.115.0 NOTES

Langfuse v3.115.0 adds structured output support to prompt experiments.

└──▷ GET THIS VERSION
$ git clone --branch v3.115.0 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v3.115.0
  • Supports structured output in prompt experiments, enabling schema-constrained LLM responses during experiment runs.
Was this useful?
my-toolchain — 0 tools
paste an install list to detect your tools

A brew list, a Brewfile, requirements.txt, a Dockerfile — or just the product names, free-form. Nothing leaves your browser.

    browse all tools →