The AI Toolchain — issue -383, December 31, 2023

Zed v0.115.3 adds an option to include gitignored files in project search.

$ git clone --branch v0.115.3 https://github.com/zed-industries/zed.git
# already have the repo? check out this version:
$ git checkout v0.115.3

• ›Adds an option to include gitignored files when running a project search.

LangChain v0.0.352 adds MistralAI, Together, NVIDIA TRT, GPTRouter, Jaguar, Aphrodite, and Qdrant sparse vector support.

$ git clone --branch v0.0.352 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout v0.0.352

• ›Adds langchain-mistralai partner package, bringing MistralAI models as a first-class LangChain integration.
• ›Adds together partner package with embedding model support for Together AI.
• ›Adds anthropic beta messages integration.
• ›Adds NVIDIA TRT partner package for TensorRT-backed LLM inference.
• ›Adds GPTRouter integration (LLM routing across multiple providers).

LangChain v0.0.351 adds Gemini, NVIDIA AI Playground, SurrealDB, YAML output parsing, logprobs, and multi-modal retrieval templates.

$ git clone --branch v0.0.351 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout v0.0.351

from langchain_google_genai import ChatGoogleGenerativeAI

llm = ChatGoogleGenerativeAI(model="gemini-pro")
response = llm.invoke("Explain chain-of-thought prompting in one paragraph.")
print(response.content)

from langchain_community.chat_models import ChatOllama

llm = ChatOllama(model="llava")
response = llm.invoke(
    [
        {"type": "text", "text": "Describe any security-relevant content in this image."},
        {"type": "image_url", "image_url": "<path_to_image>"},
    ]
)
print(response.content)

• ›Adds langchain-google-genai partner package with ChatGoogleGenerativeAI and Gemini Embeddings for direct Gemini model access.
• ›Adds NVIDIA AI Playground integration (langchain-nvidia-aiplay package) for accessing NVIDIA foundation models.
• ›Adds YamlOutputParser for parsing LLM output as structured YAML.
• ›Adds SurrealDB as a supported vector store integration.
• ›Adds similarity_score_threshold search mode to MongoDB Atlas vector store.

LangChain v0.0.349 adds SmartLLMChain output key customization and promotes RunnableContext to beta.

$ git clone --branch v0.0.349 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout v0.0.349

• ›Adds output key customization to SmartLLMChain, letting callers control the key used to retrieve the chain's result.
• ›Promotes RunnableContext from experimental to beta, signaling a more stable API surface for context-passing in Runnable pipelines.
• ›Switches MultiVectorRetriever to use a byte store backend instead of the previous store implementation.

LangChain v0.0.349-rc.1 adds output key customization for SmartLLMChain.

$ git clone --branch v0.0.349-rc.1 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout v0.0.349-rc.1

• ›Adds output key customization to SmartLLMChain, allowing callers to control the key name used in the chain's output.

LangChain v0.0.347 adds Cloudflare Workers AI, text-embeddings-inference, a context API for Runnables, multi-modal RAG, and new pgvector/AzureSearch capabilities.

$ git clone --branch v0.0.347 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout v0.0.347

• ›Adds retry_min_seconds and retry_max_seconds parameters to OpenAIEmbeddings for configurable retry back-off.
• ›Adds NIN metadata filter operator to pgvector, enabling set-absence checks in vector store queries.
• ›Adds CORS options support for AzureSearch integration.
• ›Adds metadata field to Blob objects for richer document-loading pipelines.
• ›Adds BaseChatMessageHistory.__str__ method for human-readable inspection of chat history objects.

LangChain v0.0.346 adds Slack toolkit, Steam/NASA/SearchAPI tools, Couchbase loader, Yellowbrick vector store, CometTracer, and more new integrations.

$ git clone --branch v0.0.346 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout v0.0.346

from langchain.chains import RetrievalQA

qa = RetrievalQA.from_llm(
    llm=llm,
    retriever=retriever,
    llm_chain_kwargs={"verbose": True}
)

• ›Adds llm_chain_kwargs parameter to BaseRetrievalQA.from_llm for passing additional keyword arguments to the underlying LLM chain.
• ›Adds response kwarg to the on_llm_error callback in core, giving error handlers access to the LLM response at the time of failure.
• ›Adds input_type override to Cohere embeddings integration.
• ›Adds support for custom Hugging Face inference endpoint URLs.
• ›Adds Python logging-based tracer for chain and LLM observability.

LangChain v0.0.345 adds OllamaFunctions, IBM integration, Azure AI Data loader, and Ollama multi-query retriever template.

$ git clone --branch v0.0.345 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout v0.0.345

• ›Adds OllamaFunctions wrapper, enabling function-calling capabilities with Ollama-hosted models.
• ›Adds add azure ai data document loader integration for loading documents from Azure AI Data sources.
• ›Adds support for passing parameters to llms.Databricks and llms.Mlflow LLM integrations.
• ›Adds BaseTracer helper method for Run lookup, simplifying custom tracer development.
• ›Adds IBM integration (Harrison/ibm) as a new LLM/model provider.

LangChain v0.0.344 adds Volcengine LLM, Reddit search, Merriam-Webster tool, MongoDB Atlas self-query, Pandas DataFrame output parser, and more.

$ git clone --branch v0.0.344 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout v0.0.344

• ›Adds PandasDataFrameOutputParser to parse LLM outputs directly into Pandas DataFrames.
• ›Adds Volcengine endpoint support for LLM integrations.
• ›Adds multi-input Reddit search tool for agent use.
• ›Adds Merriam-Webster Dictionary Tool for agent use.
• ›Adds MongoDB Atlas Self-Query Retriever for structured metadata filtering over Atlas vector search.

Letta 0.2.8 adds free hosted LLM endpoints, memgpt quickstart, memgpt server, memgpt folder, /summarize, and a REST API.

$ git clone --branch 0.2.8 https://github.com/letta-ai/letta.git
# already have the repo? check out this version:
$ git checkout 0.2.8

$ memgpt quickstart --latest

$ memgpt quickstart --latest --backend openai

$ memgpt server

• ›Adds memgpt quickstart --latest to auto-configure MemGPT for the free hosted endpoint, and memgpt quickstart --latest --backend openai for OpenAI; writes defaults to ~/.memgpt/config.
• ›Adds memgpt server command to launch the REST API server, with support for passing a custom host.
• ›Adds memgpt folder command for folder management.
• ›Adds /summarize in-chat command.
• ›Introduces a REST API via API server refactor, with local APIs updated to return usage info.

Letta 0.2.7 adds Chroma vector storage integration and bundles lancedb and chroma as default dependencies.

$ git clone --branch 0.2.7 https://github.com/letta-ai/letta.git
# already have the repo? check out this version:
$ git checkout 0.2.7

• ›Adds skip_verify parameter to autogen constructors for bypassing verification.
• ›New Chroma vector storage integration for persistent memory backends.
• ›Bundles lancedb and chroma as default package dependencies, removing the need for separate installation.

Letta 0.2.5 adds HuggingFace TEI embedding support and vLLM integration, while removing legacy python main.py and BACKEND_TYPE configuration.

$ git clone --branch 0.2.5 https://github.com/letta-ai/letta.git
# already have the repo? check out this version:
$ git checkout 0.2.5

• ›Removes BACKEND_TYPE and OPENAI_BASE_URL environment variables for configuring local/custom LLMs — use memgpt configure and memgpt run flags instead.
• ›Removes the python main.py command — replaced by memgpt run.
• ›Adds support for HuggingFace Text Embeddings Inference (TEI) endpoints as a custom embedding model backend.
• ›Adds documentation and support for vLLM OpenAI-compatible endpoints, including the user field for vLLM requests.
• ›Adds a warning when no data sources are loaded on the /attach command.

LlamaIndex v0.9.12 adds vLLM support, Python 3.12 compatibility, and claude-2.1 model name alongside a new async client option.

$ git clone --branch v0.9.12 https://github.com/run-llama/llama_index.git
# already have the repo? check out this version:
$ git checkout v0.9.12

• ›Adds reuse_client option to OpenAI/Azure integrations — set to False to reduce async timeout errors.
• ›Adds support for vLLM as an LLM backend.
• ›Adds support for the claude-2.1 model name in the Anthropic integration.
• ›Adds support for Python 3.12.

AutoGen v0.2.1 adds AutoBuild, Function Inception, async human input, and verbose GPT assistant logging.

$ git clone --branch v0.2.1 https://github.com/microsoft/autogen.git
# already have the repo? check out this version:
$ git checkout v0.2.1

from autogen.agentchat.contrib.gpt_assistant_agent import GPTAssistantAgent

agent = GPTAssistantAgent(
    name="analyst",
    llm_config={"config_list": config_list},
    verbose=True
)

• ›Adds a verbose flag to the GPT assistant agent to print more detailed logs during execution.
• ›Enables agents to register async human input handlers, supporting non-blocking input flows.
• ›Introduces Function Inception: agents can now define, update, or remove functions dynamically during a conversation after agent creation.
• ›Adds AutoBuild for automatically constructing multi-agent systems from a task description.
• ›Adds cost calculation and cost summary to the client-based inference layer, restoring a v0.1 capability.

Semantic Kernel Python adds an Azure OpenAI on Your Data connector backed by Azure AI Search.

$ git clone --branch python-0.4.2.dev https://github.com/microsoft/semantic-kernel.git
# already have the repo? check out this version:
$ git checkout python-0.4.2.dev

• ›Adds a Python connector for Azure OpenAI on Your Data, allowing an Azure AI Search index to be configured as a data source so the model answers queries with index-retrieved content, including vector search retrieval modes.

Semantic Kernel Python 0.4.1.dev upgrades to Pydantic v2.5.2 and adds custom AzureOpenAI/OpenAI client support with configurable default headers.

$ git clone --branch python-0.4.1.dev https://github.com/microsoft/semantic-kernel.git
# already have the repo? check out this version:
$ git checkout python-0.4.1.dev

• ›Allows default_headers configuration and passing custom AzureOpenAI/OpenAI client instances directly into the kernel.
• ›Upgrades to Pydantic v2.5.2, consolidating skbasemodel, skgeneric, and pydantic field classes into a unified skbasemodel, and replacing the dict function with model_dump and model_dump_json.
• ›Adds a grounding sample as a standalone Python script (in addition to the existing notebook).

Jan v0.4.2 adds theme persistence, a combined stop/send button, and a model-loading indicator.

$ git clone --branch v0.4.2 https://github.com/janhq/jan.git
# already have the repo? check out this version:
$ git checkout v0.4.2

• ›App theme now persists via local storage instead of following the native Electron system theme, so user theme choices survive restarts.
• ›Adds a loading indicator while a model is starting up, giving visual feedback before inference begins.

Jan v0.4.1 adds NVIDIA Triton TRT-LLM extension, prompt template inference, multiple engine support, and model parameter controls

$ git clone --branch v0.4.1 https://github.com/janhq/jan.git
# already have the repo? check out this version:
$ git checkout v0.4.1

• ›Adds NVIDIA Triton TRT-LLM extension for accelerated inference via @vuonghoainam's new engine integration.
• ›Supports multiple inference engines for Nitro and OpenAI, enabling engine selection per model.
• ›Adds model parameters configuration for inference extensions.
• ›Adds JSON schema definitions for engine and model parameters.
• ›Enables inference with prompt templates in Nitro.

Jan v0.4.0 revamps the thread and hub screens and boots a local Express server on localhost:1337

$ git clone --branch v0.4.0 https://github.com/janhq/jan.git
# already have the repo? check out this version:
$ git checkout v0.4.0

• ›Adds a boilerplate Express server running on localhost 1337 for local API access
• ›Pre-populates Jan's /models folder with model.json files so models are available without manual setup
• ›Revamps the Model Hub screen with a new cover image and updated layout
• ›Revamps the thread screen UI
• ›Revamps the landing page

KoboldCpp v1.52.2 adds a NoScript WebUI, per-layer KV offloading for CUDA, QWEN/Mixtral support, and richer API responses.

$ git clone --branch v1.52.2 https://github.com/LostRuins/koboldcpp.git
# already have the repo? check out this version:
$ git checkout v1.52.2

$ koboldcpp.exe --model mixtral-8x7b.gguf --blasbatchsize -1

• ›Adds a new NoScript WebUI at /noscript (e.g. http://localhost:5001/noscript) — W3C HTML-compliant, works in text-based browsers like Lynx over SSH, requires no JavaScript.
• ›The --usecublas lowvram flag (equivalent to -nkvo in llama.cpp) disables per-layer KV offloading for CUDA when the reduced GPU layer budget is undesirable.
• ›Merges partial per-layer KV offloading for CUDA, changing how GPU layer capacity is calculated — fully offloaded models are unaffected.
• ›The /api/extra/tokencount endpoint now returns an array of token IDs in the response body alongside the token count.
• ›Adds a new Horde Worker error-count field to the /api/extra/perf endpoint for monitoring embedded Horde Worker health.

KoboldCpp v1.51.1 adds --quiet console suppression and extends --multiuser with a concurrency queue limit.

$ git clone --branch v1.51.1 https://github.com/LostRuins/koboldcpp.git
# already have the repo? check out this version:
$ git checkout v1.51.1

$ koboldcpp.exe --model my_model.gguf --quiet

$ koboldcpp.exe --model my_model.gguf --multiuser 10

• ›Adds --quiet flag to suppress all input and output from appearing in the console.
• ›Extends --multiuser with an optional integer parameter to set how many concurrent requests may queue (defaults to 5 when unset or set to 1).
• ›API URL printouts on startup now include the tunnel URL when --remotetunnel is in use.
• ›Allocates ~80 tokens of reserved KV cache space on top of the specified --contextsize (e.g. --contextsize 2048 allocates 2128 internally) to reduce 'Failed to predict' errors caused by KV cache fragmentation during context shift.
• ›Displays available KAI and OAI endpoint URLs in the terminal on startup.

LocalAI v2.2.0 adds inline prompt templates and URL-based model loading in YAML config files.

$ git clone --branch v2.2.0 https://github.com/mudler/LocalAI.git
# already have the repo? check out this version:
$ git checkout v2.2.0

name: mixtral

parameters:
  model: https://huggingface.co/TheBloke/Mixtral-8x7B-v0.1-GGUF/resolve/main/mixtral-8x7b-v0.1.Q2_K.gguf
# or huggingface://TheBloke/Mixtral-8x7B-v0.1-GGUF/mixtral-8x7b-v0.1.Q2_K.gguf@main
template:
  completion: |
    Complete the following: {{.Input}}

• ›Supports specifying parameters.model as a direct HTTPS URL or huggingface:// URI in model YAML config files, enabling on-demand model downloads without pre-staging files.
• ›Supports inline prompt templates under template.completion (and siblings) directly in model YAML config files, eliminating the need for separate template files.
• ›Transformer-based backends now share a single conda environment, reducing container image size.

LocalAI v2.1.0 adds ExLlamaV2, music generation, image/video synthesis backends, and API key file validation

$ git clone --branch v2.1.0 https://github.com/mudler/LocalAI.git
# already have the repo? check out this version:
$ git checkout v2.1.0

• ›Adds API key file validation support to the API layer, enabling file-based key management.
• ›Adds the transformers-musicgen backend for AI-powered music generation.
• ›Adds exllamav2 backend support alongside restored exllamav1 compatibility.
• ›Adds autopipeline and controlnet support to the diffusers backend.
• ›Adds initial support for img2vid and txt2vid (image-to-video and text-to-video) generation.

LocalAI v2.0.0 adds LLaVA/GPT Vision API, parallel requests, conda environments, watchdog, and GPU-accelerated Whisper.

$ git clone --branch v2.0.0 https://github.com/mudler/LocalAI.git
# already have the repo? check out this version:
$ git checkout v2.0.0

• ›Adds LLaVA support and the GPT Vision API, enabling multimodal image+text inference via llama.cpp.
• ›Adds .yml as an accepted extension for YAML config files (previously only .yaml was recognized).
• ›Enables parallel request handling for llama.cpp, with automatic request queuing when parallelism is disabled.
• ›Adds support for custom CA certificates for outbound connections.
• ›Adds conda environment support for Python backends.

Adds HQQ quant loader, GGUF instruction-template auto-detection, tensor-core llama.cpp wheels, and safetensor LoRA support for ExLlama v1.

$ git clone --branch snapshot-2023-12-24 https://github.com/oobabooga/textgen.git
# already have the repo? check out this version:
$ git checkout snapshot-2023-12-24

• ›Adds HQQ quantization loader for loading HQQ-quantized models.
• ›Adds auto-detection of Eric Hartford's Dolphin models via models/config.yaml.
• ›Adds Orca-Vicuna instruction template.
• ›Reads instruction templates automatically from GGUF metadata for llama.cpp models, removing the need to set them manually.
• ›Adds llama-cpp-python wheels built with tensor cores support, unlocking faster inference on compatible GPUs.

Adds Jinja templates for chat/instruct, OpenAI embeddings endpoint, max_logits param, and --num_experts_per_token for ExLlamav2.

$ git clone --branch snapshot-2023-12-17 https://github.com/oobabooga/textgen.git
# already have the repo? check out this version:
$ git checkout snapshot-2023-12-17

$ python server.py --loader exllamav2 --num_experts_per_token 2 --model <your-moe-model>

• ›Adds --num_experts_per_token CLI parameter for ExLlamav2 to control mixture-of-experts routing per token.
• ›Adds OpenAI-compatible embeddings endpoint via the OpenAI extension.
• ›Adds max_logits parameter to the OpenAI extension logits endpoint.
• ›Introduces Jinja2 template support for Instruct and Chat prompt formatting.
• ›Supports symlinked folders within the root directory for model/asset organization.

Adds QuIP# quantization support and gallery improvements; changes defaults for max_new_tokens and repetition_penalty_range.

$ git clone --branch snapshot-2023-12-10 https://github.com/oobabooga/textgen.git
# already have the repo? check out this version:
$ git checkout snapshot-2023-12-10

• ›Changes default values for max_new_tokens and repetition_penalty_range parameters.
• ›Adds QuIP# quantization format support for model loading.
• ›Improves the model Gallery UI.

Adds OpenHermes Mistral model support and Docker container CLI improvements.

$ git clone --branch snapshot-2023-12-03 https://github.com/oobabooga/textgen.git
# already have the repo? check out this version:
$ git checkout snapshot-2023-12-03

• ›Adds support for the OpenHermes Mistral model.
• ›Improves Docker container CLI arguments for more practical containerized deployments.

vLLM v0.2.5 adds Mixtral expert parallelism and grouped-query attention for MPT models.

$ git clone --branch v0.2.5 https://github.com/vllm-project/vllm.git
# already have the repo? check out this version:
$ git checkout v0.2.5

• ›Adds expert parallelism support for Mixtral models, improving multi-GPU inference throughput.
• ›Adds grouped-query attention (GQA) support for MPT model attention layers.
• ›Removes the Python 3.10 minimum version requirement, broadening supported runtime environments.

vLLM v0.2.4 adds Mixtral 8x7B and AMD GPU support, plus ChatGLM and custom server params.

$ git clone --branch v0.2.4 https://github.com/vllm-project/vllm.git
# already have the repo? check out this version:
$ git checkout v0.2.4

• ›Adds custom server parameters support via the OpenAI-compatible server.
• ›Supports Mixtral 8x7B (Mixture-of-Experts) model, requiring the megablocks dependency.
• ›Adds AMD GPU (ROCm) support in collaboration with EmbeddedLLM.
• ›Adds support for ChatGLMForConditionalGeneration models.
• ›Adds CUDA architecture list as a Docker build option.

vLLM v0.2.3 adds Prometheus metrics, echo support for the OpenAI chat API, and better Ray Serve integration.

$ git clone --branch v0.2.3 https://github.com/vllm-project/vllm.git
# already have the repo? check out this version:
$ git checkout v0.2.3

• ›Adds production metrics exported in Prometheus format for operational observability.
• ›Adds echo parameter support to the OpenAI-compatible chat API, mirroring the completions API behavior.
• ›Supports chat templates in the OpenAI-compatible chat API.
• ›Adds --profile option to the latency benchmark tool.
• ›Supports max-model-len argument in the throughput benchmark tool.

Ollama v0.1.16 adds Mixtral and Mixture of Experts (MoE) architecture support.

$ git clone --branch v0.1.16 https://github.com/ollama/ollama.git
# already have the repo? check out this version:
$ git checkout v0.1.16

$ ollama run mixtral

• ›Adds support for the Mixtral model via ollama run mixtral (requires at least 48 GB of memory), enabling use of a high-quality open-weights mixture of experts model.
• ›Adds support for Mixture of Experts (MoE) architecture broadly, enabling other MoE-based models beyond Mixtral.
• ›Adds Dolphin Mixtral (jmorgan/dolphin-mixtral), an uncensored fine-tuned MoE model optimized for coding tasks.

Ollama v0.1.15 adds multimodal model support with image inputs via CLI and API, plus enriched model details from the show endpoint.

$ git clone --branch v0.1.15 https://github.com/ollama/ollama.git
# already have the repo? check out this version:
$ git checkout v0.1.15

$ ollama run llava
>>> What does the text in this image say? /Users/you/Downloads/image.png

$ curl http://localhost:11434/api/generate -d '{"model": "llava", "prompt": "What is in this picture?", "images": ["<base64-encoded-png-or-jpeg>"]}'

$ curl http://localhost:11434/api/show -d '{"name": "llava"}'

• ›Adds images parameter to the Generate API (POST /api/generate) accepting a list of base64-encoded png or jpeg images up to 100MB each, enabling vision-capable models like LLaVA.
• ›Adds images field to user role messages in the Chat API (POST /api/chat), allowing image inputs in multi-turn conversations.
• ›Extends POST /api/show to return a details object with format, families, parameter_size, and quantization_level fields for the requested model.
• ›Adds multimodal image input to ollama run: include a full path to a png or jpeg file directly in the prompt to pass images to vision models.
• ›Adds two new multimodal models to the library: llava (7B and 13B) and bakllava (Mistral 7B + LLaVA architecture).

Ollama v0.1.14 adds a Chat API with message history, a version endpoint, and two new models.

$ git clone --branch v0.1.14 https://github.com/ollama/ollama.git
# already have the repo? check out this version:
$ git checkout v0.1.14

$ curl http://localhost:11434/api/chat -d '{"model": "mistral", "messages": [{"role": "system", "content": "You are a helpful assistant that answers concisely."}, {"role": "user", "content": "why is the sky blue?"}]}'

• ›New POST /api/chat endpoint accepts a messages array with role and content fields, enabling multi-turn chat with full conversation history passed per request.
• ›New GET /api/version endpoint for programmatically checking the running Ollama version.
• ›Adds version-mismatch warning when connecting to a remote instance via OLLAMA_HOST.
• ›Adds StableLM Zephyr (stablelm-zephyr), a lightweight chat model designed for low-end hardware.
• ›Adds Magicoder (magicoder), a family of 7B models trained on 75K synthetic OSS-Instruct code snippets.

Triton v2.40.0 adds iterative sequence scheduling, HTTP endpoint access restrictions, vLLM auto-complete, and TRT-LLM request cancellation.

$ git clone --branch v2.40.0 https://github.com/triton-inference-server/server.git
# already have the repo? check out this version:
$ git checkout v2.40.0

backend: "vllm"

• ›Adds backend: "vllm" auto-complete support so Triton can automatically fill in vLLM backend model configuration without a full manual config.
• ›Enhances the HTTP endpoint to support access restriction (beta), limiting which endpoints are reachable — see the inference protocols customization guide.
• ›Enhances implicit state management to support growing buffers and a single shared buffer for both input and output states, configurable via model_config.proto.
• ›Adds iterative scheduling support to the sequence batcher, enabling sequences that require multiple scheduling passes.
• ›Extends the backend API to support request rescheduling, available in the Python backend and custom C++ backends.

Phoenix v2.1.0 adds a v1/traces HTTP endpoint, function-calling evals, and criteria-based evaluator instantiation.

$ git clone --branch v2.1.0 https://github.com/Arize-ai/phoenix.git
# already have the repo? check out this version:
$ git checkout v2.1.0

import phoenix.evals as evals

results = evals.run_evals(
    dataframe=df,
    evaluators=[evals.QAEvaluator(model=model)],
    provide_explanation=True,
    use_function_calling_if_available=True,
)

• ›Adds v1/traces HTTP endpoint to accept ExportTraceServiceRequest payloads, enabling OTLP-compatible trace ingestion over HTTP.
• ›Supports function calling for run_evals, allowing LLM evaluators to use structured function-call outputs during evaluation.
• ›Enables instantiating evaluators by criteria, letting practitioners define custom evaluation criteria directly rather than using only preset evaluators.
• ›Adds retrieval summary to the traces UI header for at-a-glance retrieval metrics.
• ›Adds evaluation summary to the traces UI header for at-a-glance eval metrics.

Phoenix v2.0.0 adds async llm_generate, multi-eval parallelism, explanation support, and OpenAI streaming tool calls.

$ git clone --branch v2.0.0 https://github.com/Arize-ai/phoenix.git
# already have the repo? check out this version:
$ git checkout v2.0.0

• ›Adds async submission to llm_generate, enabling non-blocking LLM generation calls.
• ›Adds explanation support to run_evals, allowing evaluations to return rationale alongside labels.
• ›Supports running multiple evals at once via run_evals, enabling parallel evaluation pipelines.
• ›Updates llm_classify and llm_generate interfaces with revised calling conventions.
• ›Adds evaluation column selectors for more granular control over which columns feed into evaluations.

Arize Phoenix v1.9.0 adds automatic retries for Amazon Bedrock calls.

$ git clone --branch v1.9.0 https://github.com/Arize-ai/phoenix.git
# already have the repo? check out this version:
$ git checkout v1.9.0

• ›Adds automatic retry support for Amazon Bedrock LLM calls, improving resilience against transient API failures.

Phoenix v1.8.0 adds audio embedding support and OpenAI streaming function call message tracing.

$ git clone --branch v1.8.0 https://github.com/Arize-ai/phoenix.git
# already have the repo? check out this version:
$ git checkout v1.8.0

• ›Supports audio modality for embeddings, enabling audio data to be visualized and analyzed in the embeddings view.
• ›Supports OpenAI streaming function call messages, enabling tracing of streamed function-call interactions in LLM workflows.

Phoenix v1.7.0 adds streaming instrumentation for LlamaIndex and OpenAI async, plus span-to-dataframe querying.

$ git clone --branch v1.7.0 https://github.com/Arize-ai/phoenix.git
# already have the repo? check out this version:
$ git checkout v1.7.0

• ›Adds px.Client().query_spans() (or equivalent trace query API) to pull spans directly into pandas DataFrames for offline analysis.
• ›Adds instrumentation for LlamaIndex streaming responses, capturing trace spans across streamed LLM calls.
• ›Adds instrumentation for OpenAI async streaming, enabling tracing of async streamed completions.

Phoenix v1.6.0 adds OpenAI streaming instrumentation, document retrieval metrics, and filterable evaluation summaries in the trace UI.

$ git clone --branch v1.6.0 https://github.com/Arize-ai/phoenix.git
# already have the repo? check out this version:
$ git checkout v1.6.0

• ›Adds GraphQL query for document evaluation summary, enabling programmatic retrieval of per-document evaluation metrics.
• ›Adds instrumentation support for OpenAI synchronous streaming, capturing streaming spans in traces.
• ›OpenAI streaming spans now appear in the Phoenix UI alongside other trace data.
• ›Displays document retrieval metrics directly on the trace details panel.
• ›Adds filterable span and document evaluation summaries in the traces view.

Phoenix v1.5.0 adds Human vs AI evals, span eval filtering, document retrieval metrics, and async OpenAI instrumentation.

$ git clone --branch v1.5.0 https://github.com/Arize-ai/phoenix.git
# already have the repo? check out this version:
$ git checkout v1.5.0

• ›Adds server-side span filtering by evaluation result values in the traces UI, enabling targeted review of spans that meet specific eval criteria.
• ›Adds evaluation annotations on traces to associate spans with eval metrics, surfacing eval scores and labels directly alongside trace data.
• ›Adds span evaluation summary with aggregation metrics (scores and labels) across spans in the trace and span tables.
• ›Adds document retrieval metrics derived from document evaluation scores, visible in trace and span tables.
• ›Adds semantic conventions for the tool_calls array in OpenAI ChatCompletion messages.

Langfuse v1.24.2 introduces more compact data tables with reduced padding in the UI.

$ git clone --branch v1.24.2 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v1.24.2

• ›Data tables in the UI are now more compact with less padding, improving information density.

Langfuse v1.24.0 adds AzureAD authentication support and custom plan limit visibility in project settings.

$ git clone --branch v1.24.0 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v1.24.0

• ›Adds AzureAD as a supported authentication provider.
• ›Displays custom plan limits in project settings for cloud deployments.

Langfuse v1.23.0 adds middle-click/open-in-new-window support in tables and host name display on project settings.

$ git clone --branch v1.23.0 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v1.23.0

• ›Displays host name on the project settings page for easier environment identification.
• ›Enables middle-click and 'open in new window' navigation for table rows throughout the UI.

Langfuse v1.22.0 adds optional SHADOW_DATABASE_URL env var support and in-app version display.

$ git clone --branch v1.22.0 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v1.22.0

• ›Adds optional SHADOW_DATABASE_URL environment variable support for Docker deployments to configure a shadow database.
• ›Displays the current application version in the UI to aid debugging of version conflicts and upgrades.

Langfuse v1.21.0 adds optional S3-backed data exports and a DIRECT_URL env var for pooled database connections.

$ git clone --branch v1.21.0 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v1.21.0

$ DATABASE_URL=postgres://limited_user:pass@pgbouncer:6432/langfuse DIRECT_URL=postgres://admin:pass@postgres:5432/langfuse docker compose up

• ›Adds DIRECT_URL environment variable to set a direct Postgres connection URL when DATABASE_URL points to a connection pooler or a limited-privilege user.
• ›Adds optional S3 backend for data exports, configurable via environment variables.

Langfuse v1.20.0 adds SDK log event support to the ingestion API.

$ git clone --branch v1.20.0 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v1.20.0

• ›Adds sdk-log event type to the ingestion API, enabling SDKs to send log events alongside traces and observations.

Langfuse v1.19.3 adds a cost column to the generations table.

$ git clone --branch v1.19.3 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v1.19.3

• ›Adds a cost column to the generations table for at-a-glance LLM spend visibility.

Langfuse v1.19.1 promotes Sessions to general availability, removing the beta label from the main menu.

$ git clone --branch v1.19.1 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v1.19.1

• ›Sessions feature is now generally available, removed from beta status in the main menu UI.

Langfuse v1.19.0 adds multi-select trace deletion, enter-to-search in tables, and metadata support in the ingestion API.

$ git clone --branch v1.19.0 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v1.19.0

• ›Adds metadata field to the ingestion API, allowing structured metadata to be attached to trace ingestion payloads.
• ›Adds multi-select and bulk delete for traces in the UI.
• ›Adds enter-key-triggered search across tables in the UI.

Langfuse v1.18.0 adds API support for v2 SDKs.

$ git clone --branch v1.18.0 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v1.18.0

• ›Adds API support for v2 SDKs, enabling clients using the v2 SDK to communicate with the Langfuse server.

Langfuse v1.17.0 adds session tracking with I/O on traces and score comments visible on hover across the UI.

$ git clone --branch v1.17.0 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v1.17.0

• ›Adds session tracking with input/output capture on traces, enabling grouping and replay of multi-turn LLM interactions.
• ›Score comments are now visible on hover throughout the UI, surfacing annotation context inline without navigating away.

Langfuse v1.16.0 adds project renaming in settings and X-Frame-Options security header.

$ git clone --branch v1.16.0 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v1.16.0

• ›Adds X-Frame-Options HTTP response header to protect against clickjacking attacks.
• ›Enables renaming a project from the project settings UI.

Langfuse generations table now surfaces level, status message, and metadata columns.

$ git clone --branch v1.15.0 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v1.15.0

• ›Adds level, status message, and metadata columns to the generations table in the UI.

Langfuse v1.14.0 adds latency filtering for traces in the UI.

$ git clone --branch v1.14.0 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v1.14.0

• ›Adds latency filtering for traces in the UI, enabling practitioners to quickly isolate slow LLM calls.

Langfuse v1.13.3 adds individual database connection environment variables as an alternative to DATABASE_URL

$ git clone --branch v1.13.3 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v1.13.3

• ›Adds DATABASE_HOST, DATABASE_USERNAME, DATABASE_PASSWORD, and DATABASE_NAME environment variables as an alternative to the single DATABASE_URL environment variable for database configuration.

Langfuse v1.13.0 adds API/SDK trace publishing and configurable table column visibility.

$ git clone --branch v1.13.0 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v1.13.0

• ›Enables publishing traces via the API and SDK.
• ›Adds configurable visibility of table columns in the UI.

Langfuse v1.12.0 adds one-click formatted JSON copying in the UI.

$ git clone --branch v1.12.0 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v1.12.0

• ›Adds a 'copy formatted JSON' action in the UI for cleaner, human-readable JSON output when inspecting trace data.

Langfuse v1.11.0 adds env-var-driven default project assignment on sign-up and improved project invitation emails.

$ git clone --branch v1.11.0 https://github.com/langfuse/langfuse.git
# already have the repo? check out this version:
$ git checkout v1.11.0

$ LANGFUSE_DEFAULT_PROJECT_ID=your-project-id
LANGFUSE_DEFAULT_PROJECT_ROLE=VIEWER

• ›Adds LANGFUSE_DEFAULT_PROJECT_ID and LANGFUSE_DEFAULT_PROJECT_ROLE environment variables to automatically grant new sign-ups access to a default project with a specified role.
• ›Sends styled HTML email invitations to non-account users invited to a project.

Chroma 0.4.19 adds Jina AI and Hugging Face embedding functions, a $not_contains filter, cloud client, and OpenTelemetry tracing.

$ git clone --branch 0.4.19 https://github.com/chroma-core/chroma.git
# already have the repo? check out this version:
$ git checkout 0.4.19

• ›Adds $not_contains filter operator to where clause queries, enabling exclusion-based metadata filtering.
• ›Adds AdminClient to the Python API and tenancy support to the JavaScript client.
• ›Adds Jina AI embedding function for use as a drop-in embedding provider.
• ›Adds Hugging Face Text Embedding Server embedding function.
• ›Allows default headers to be passed through to the OpenAI API in the OpenAI embedding function.

LanceDB python-v0.4.3 adds batch query support for the remote API.

$ git clone --branch python-v0.4.3 https://github.com/lancedb/lancedb.git
# already have the repo? check out this version:
$ git checkout python-v0.4.3

• ›Adds batch queries for the remote API, enabling multiple vector searches to be submitted in a single call.

LanceDB v0.4.2 adds post-filtering for full-text search, list-of-list Pydantic fields, and timezone-aware datetime support.

$ git clone --branch python-v0.4.2 https://github.com/lancedb/lancedb.git
# already have the repo? check out this version:
$ git checkout python-v0.4.2

• ›Adds post-filtering support for full-text search queries, enabling result refinement after FTS retrieval.
• ›Supports list-of-list fields when defining schemas via Pydantic models.
• ›Adds timezone handling for datetime fields in Pydantic schemas.

LanceDB v0.4.1 adds scalar index creation, FTS nested field references, and a pandas flatten option.

$ git clone --branch python-v0.4.1 https://github.com/lancedb/lancedb.git
# already have the repo? check out this version:
$ git checkout python-v0.4.1

df = table.search(query_vector).to_pandas(flatten=True)

table.create_scalar_index("price")

• ›Adds create_scalar_index() capability to create scalar indices on table columns, enabling faster filtered queries.
• ›Adds flatten option to to_pandas() to flatten nested/struct output into a flat DataFrame.
• ›Supports nested field references in full-text search (FTS) queries, allowing search over nested document fields.

LanceDB v0.4.1 adds Node.js Schema, index creation, scalar indices, and paginated table listing APIs.

$ git clone --branch v0.4.1 https://github.com/lancedb/lancedb.git
# already have the repo? check out this version:
$ git checkout v0.4.1

• ›Adds scalar index creation support via the new scalar indices API, enabling fast filtering on non-vector columns.
• ›Adds Node.js Schema API for inspecting and working with table schemas in JavaScript/TypeScript.
• ›Adds Node.js createIndex API, bringing vector index creation to the Node client.
• ›Adds pagination support for listTables in the Node.js client to handle large numbers of tables.

LanceDB v0.4.0 adds GPU index creation, scalar indexes, prefiltering, update queries, Cohere embeddings, and remote table operations.

$ git clone --branch v0.4.0 https://github.com/lancedb/lancedb.git
# already have the repo? check out this version:
$ git checkout v0.4.0

results = (
    table.search([0.1, 0.2, 0.3])
        .where("category = 'public'")
        .prefilter(True)
        .limit(10)
        .to_list()
)

table.update(where="status = 'pending'", values={"status": "reviewed"})

• ›Adds prefilter flag to vector search queries, enabling pre-filtering with an index before performing ANN search (Python, Node.js, and Rust).
• ›Adds update query support for Python via a new update query API, and implements update for remote clients.
• ›Adds to_list and to_pandas APIs for retrieving query results in Python.
• ›Adds RemoteTable.version property in Python to inspect the version of a remote table.
• ›Adds index cache size exposure in Python for tuning ANN search memory usage.

LanceDB v0.4.0 adds GPU indexing, scalar indexes, prefilter support, Cohere embeddings, update queries, and remote table operations.

$ git clone --branch python-v0.4.0 https://github.com/lancedb/lancedb.git
# already have the repo? check out this version:
$ git checkout python-v0.4.0

$ table.update(where="status = 'pending'", values={"status": "processed"})

$ results = table.search().where("score > 0.9").to_list()

• ›Adds prefilter flag to queries, enabling pre-filtering with an index before vector search (available in Python, Node.js, and Rust).
• ›Adds update query support for Python via update API, and implements update for remote clients.
• ›Adds to_list and to_pandas APIs for query result retrieval.
• ›Adds RemoteTable.version property in Python to inspect the version of a remote table.
• ›Adds index cache size configuration via expose index cache size API in Python.

LanceDB v0.3.6 adds nested Pydantic schema support, custom vector column queries, and remote update operations.

$ git clone --branch python-v0.3.6 https://github.com/lancedb/lancedb.git
# already have the repo? check out this version:
$ git checkout python-v0.3.6

• ›Supports nested Pydantic schemas for table definitions, enabling richer data models to be used directly with LanceDB.
• ›Allows a custom vector column name to be specified in queries, letting users target non-default vector columns during search.
• ›Passes the vector column name through to the remote backend, enabling custom column naming in remote query workflows.
• ›Implements update operations for remote clients, bringing remote LanceDB deployments to parity with local update support.

LanceDB v0.3.11 adds custom vector column naming in queries and update support for remote clients.

$ git clone --branch v0.3.11 https://github.com/lancedb/lancedb.git
# already have the repo? check out this version:
$ git checkout v0.3.11

• ›Enables custom column names in vector queries instead of requiring a fixed default column.
• ›Implements update for remote clients, bringing remote table mutations to parity with local usage.

LanceDB v0.3.10 adds filter-only table scans and row updates to the Node.js client.

$ git clone --branch v0.3.10 https://github.com/lancedb/lancedb.git
# already have the repo? check out this version:
$ git checkout v0.3.10

await tbl.update({
  filter: "id = 2",
  updates: { vector: [2, 2], name: "Michael" },
})

• ›Adds .filter(<expression>).execute() to the Node.js table API, enabling table scans with a predicate but without a vector search.
• ›Adds .update({ filter, updates }) to the Node.js tbl API (OSS only), allowing in-place row updates by filter expression.

LanceDB Python v0.3.5 promotes update queries out of experimental with a new Python API.

$ git clone --branch python-v0.3.5 https://github.com/lancedb/lancedb.git
# already have the repo? check out this version:
$ git checkout python-v0.3.5

• ›Adds update query support for Python via the new update API, allowing in-place modification of table records without experimental caveats.

LanceDB v0.3.9 exposes prefilter in Rust and Node.js clients for pre-query filter application.

$ git clone --branch v0.3.9 https://github.com/lancedb/lancedb.git
# already have the repo? check out this version:
$ git checkout v0.3.9

• ›Exposes prefilter option in the Rust client, enabling filter application before vector search rather than post-filtering.
• ›Enables prefilter support in the Node.js client, bringing pre-query filtering parity with other LanceDB clients.