The AI Toolchain — issue -419, December 18, 2020

Haystack v0.6.0 introduces DAG-based Pipelines, an OpenDistro DocumentStore, and new QA pipeline types including Generative and FAQ.

$ git clone --branch v0.6.0 https://github.com/deepset-ai/haystack.git
# already have the repo? check out this version:
$ git checkout v0.6.0

from haystack.pipeline import Pipeline, JoinDocuments

class QueryClassifier:
    outgoing_edges = 2
    def run(self, **kwargs):
        if '?' in kwargs['query']:
            return (kwargs, 'output_1')
        else:
            return (kwargs, 'output_2')

pipe = Pipeline()
pipe.add_node(component=QueryClassifier(), name='QueryClassifier', inputs=['Query'])
pipe.add_node(component=es_retriever, name='ESRetriever', inputs=['QueryClassifier.output_1'])
pipe.add_node(component=dpr_retriever, name='DPRRetriever', inputs=['QueryClassifier.output_2'])
pipe.add_node(component=JoinDocuments(join_mode='concatenate'), name='JoinResults', inputs=['ESRetriever', 'DPRRetriever'])
pipe.add_node(component=reader, name='QAReader', inputs=['JoinResults'])
res = pipe.run(query='What did Einstein work on?', top_k_retriever=1)

from haystack.pipeline import GenerativeQAPipeline

pipe = GenerativeQAPipeline(generator=rag_generator, retriever=retriever)
res = pipe.run(query='What causes aurora borealis?', top_k_retriever=3)

• ›Adds Pipeline class with add_node(), run(), draw(), and set_node() methods for composing search pipelines as Directed Acyclic Graphs (DAGs) with Retrievers, Readers, Generators, and custom nodes.
• ›Adds JoinDocuments(join_mode=...) node with score aggregation support to merge results from multiple Retrievers in a single Pipeline.
• ›Adds ExtractiveQAPipeline, DocumentSearchPipeline, GenerativeQAPipeline, and FAQPipeline default pipeline classes in haystack.pipeline, replacing the deprecated Finder class.
• ›Adds OpenDistroElasticsearchDocumentStore to support Open Distro / AWS-hosted Elasticsearch deployments.
• ›Adds refresh_type parameter to ElasticsearchDocumentStore.update_embeddings().

ONNX Runtime v1.6.0 adds opset 13, NNAPI Android acceleration, Apple Silicon support, new contrib ops, and expanded quantization.

$ git clone --branch v1.6.0 https://github.com/microsoft/onnxruntime.git
# already have the repo? check out this version:
$ git checkout v1.6.0

• ›Exposes AddInitializer API to share initializers between sessions, eliminating duplicate memory allocation when multiple sessions load models with common weights.
• ›Adds Python binding for the OrtValue data structure, enabling allocation and management of CUDA device memory directly within ORT without third-party allocators.
• ›Adds OrtValue instances as bindable inputs/outputs in the Python I/O Binding interface, including support for ORT-allocated device memory.
• ›Adds new session option to disable denormal floating-point numbers on SSE3-capable CPUs, eliminating denormal-induced performance degradation without model retraining.
• ›Adds support for ONNX 1.8 / opset 13.

Triton 2.6.0 adds alpha Windows support, Model Analyzer, and a reorganized SDK container with renamed image tags.

$ git clone --branch v2.6.0 https://github.com/triton-inference-server/server.git
# already have the repo? check out this version:
$ git checkout v2.6.0

$ tritonserver --model-repository=/path/to/model_repo --backend-directory=/path/to/tritonserver/backends --backend-config=tensorflow,version=2

$ python3 -m pip install --upgrade clients/python/tritonclient-2.6.0-py3-none-linux_aarch64.whl[all]

• ›Renames the *-py3-clientsdk container to *-py3-sdk; the renamed image now bundles the Model Analyzer alongside client libraries and examples.
• ›Initial release of the Model Analyzer tool, available in the Triton SDK container and as the nvidia-triton-model-analyzer PIP package on the NVIDIA Py Index.
• ›Moves the PyTorch backend to a separate repository (https://github.com/triton-inference-server/pytorch_backend), enabling it to be added or removed without rebuilding Triton via the compose workflow.
• ›Alpha release of Triton for Windows (tritonserver2.6.0-win.zip), supporting TensorRT 7.2.2 models over the GRPC endpoint (HTTP/REST, Prometheus metrics, and shared memory are not yet supported).
• ›Adds --backend-directory flag to explicitly set the backend path, enabling Jetson/JetPack deployments where the default path is not used.

Weaviate 0.23.0 goes standalone: drops Elasticsearch and etcd for a custom vector-first storage engine with HNSW indexing.

$ git clone --branch 0.23.0 https://github.com/weaviate/weaviate.git
# already have the repo? check out this version:
$ git checkout 0.23.0

• ›Replaces Elasticsearch and etcd runtime dependencies with Weaviate's own vector-first storage system, making the service fully standalone.
• ›Introduces a custom HNSW vector index implementation with full CRUD support, Write-Ahead-Commit-Log persistence, and ongoing maintenance tasks — enabling sub-50ms 20NN-vector queries on datasets of 1–100M objects.
• ›Adds a pluggable vector index architecture (HNSW is the first supported plugin) backed by bolt/bbolt for inverted index and object storage disk operations.
• ›Supports running with available memory smaller than total vector size via a cache-based mem/disk strategy — no requirement to keep all vectors in RAM.
• ›Explicitly defines the behavior of the Like operator (wildcard semantics, modelled after Elasticsearch wildcards).