The AI Toolchain — issue -395, December 27, 2022

Haystack v1.12.1 adds PromptNode for LLM integration, BM25 support in InMemoryDocumentStore, and parallel dense batch search for Elasticsearch/OpenSearch.

$ git clone --branch v1.12.1 https://github.com/deepset-ai/haystack.git
# already have the repo? check out this version:
$ git checkout v1.12.1

from haystack.nodes import EmbeddingRetriever

# OpenAI
retriever = EmbeddingRetriever(
    embedding_model="text-embedding-ada-002",
    batch_size=32,
    api_key=api_key,
    max_seq_len=8191
)

# Cohere multilingual
retriever = EmbeddingRetriever(
    embedding_model="multilingual-22-12",
    batch_size=16,
    api_key=api_key
)

• ›Introduces PromptNode (in haystack.nodes.prompt) with PromptModel and PromptTemplate, enabling LLM-powered NLP tasks via prompt templates; supports Google Flan-T5 and OpenAI GPT-3 models (e.g. google/flan-t5-base, text-davinci-003) standalone or chained in pipelines.
• ›Adds all_terms_must_match parameter to BM25Retriever, configurable at runtime.
• ›Adds query_by_embedding_batch to ElasticsearchDocumentStore and OpenSearchDocumentStore, enabling parallel dense searches via msearch — up to 49% faster for run_batch, eval_batch, and MostSimilarDocumentsPipeline.
• ›Extends EmbeddingRetriever to support Cohere multilingual embedding models (e.g. multilingual-22-12) and OpenAI embedding models (e.g. text-embedding-ada-002 with max_seq_len=8191).
• ›Adds BM25Retriever support to InMemoryDocumentStore, making it the first dependency-free document store to support all Haystack retrievers.

Triton Inference Server v2.29.0 adds improved Windows builds and a beta Windows package with HTTP/REST, GRPC, ONNX, and OpenVINO support.

$ git clone --branch v2.29.0 https://github.com/triton-inference-server/server.git
# already have the repo? check out this version:
$ git checkout v2.29.0

• ›Releases a beta Windows package (tritonserver2.29.0-win.zip) supporting HTTP/REST and GRPC endpoints, ONNX models via ONNX Runtime 1.13.1 (CPU, CUDA, and TensorRT execution providers), and OpenVINO models via OpenVINO 2021.4.
• ›Bundles CUDA 11.8.0, cuDNN 8.7.0.84, and TensorRT 8.5.1.7 dependencies for Windows deployments.
• ›Improves container and non-container builds on Windows.
• ›Includes Ubuntu 20.04 client libraries and examples in v2.29.0_ubuntu2004.clients.tar.gz and Windows SDK in tritonserver2.29.0-sdk-win.zip.

Zed v0.66.0 adds active_pane_magnification to visually enlarge the active pane.

$ git clone --branch v0.66.0 https://github.com/zed-industries/zed.git
# already have the repo? check out this version:
$ git checkout v0.66.0

• ›Adds active_pane_magnification setting to scale the active pane larger than surrounding panes by a configurable factor.

Milvus 2.2.1 adds Pulsar tenant auth, TLS for etcd, configurable TopK limits, and 30%+ search performance gains.

$ git clone --branch v2.2.1 https://github.com/milvus-io/milvus.git
# already have the repo? check out this version:
$ git checkout v2.2.1

• ›Adds MaxWatchDuration as a configurable setting to prevent shards with large data loads from timing out.
• ›Makes TopK limit configurable to control maximum search result size.
• ›Supports Pulsar tenant and authentication for message queue connections.
• ›Supports TLS in etcd config source for encrypted coordination-plane communication.
• ›Improves search performance by over 30% via an upgraded Knowhere vector engine and revised parallelism strategy.

Qdrant v0.11.6 adds cross-collection vector lookup in the recommendation API for item-to-user recommendation scenarios.

$ git clone --branch v0.11.6 https://github.com/qdrant/qdrant.git
# already have the repo? check out this version:
$ git checkout v0.11.6

• ›Enables lookup of vectors from a different collection inside the recommendation API, supporting item-to-user recommendations where user and item embeddings live in separate collections with independent index configurations.

$ git clone --branch v0.11.5 https://github.com/qdrant/qdrant.git
# already have the repo? check out this version:
$ git checkout v0.11.5

Weaviate v1.17 adds leaderless replication with tunable consistency and hybrid BM25F + dense-vector search.

$ git clone --branch v1.17.0 https://github.com/weaviate/weaviate.git
# already have the repo? check out this version:
$ git checkout v1.17.0

• ›Introduces leaderless replication with tunable consistency, enabling high availability and horizontal read-throughput scaling across a Weaviate cluster.
• ›Adds hybrid search combining BM25F keyword scoring and dense vector search with rank fusion, plus standalone pure BM25 and BM25F search modes.
• ›Supports dynamically adding nodes to a running cluster after data has already been imported.
• ›Adds TTLs to cluster-wide transactions covering schema and classification operations.
• ›Adjusts memtable size dynamically based on workload conditions.

$ git clone --branch v1.16.8 https://github.com/weaviate/weaviate.git
# already have the repo? check out this version:
$ git checkout v1.16.8