The AI Toolchain — issue -410, September 29, 2021

Haystack v0.10.0 adds RayPipeline for distributed scaling, SAS evaluation metric, and new FARMClassifier, SentenceTransformersRanker, and QuestionGenerator nodes.

$ git clone --branch v0.10.0 https://github.com/deepset-ai/haystack.git
# already have the repo? check out this version:
$ git checkout v0.10.0

from haystack.pipeline import RayPipeline
pipeline = RayPipeline.load_from_yaml(path="my_pipelines.yaml", pipeline_name="ray_query_pipeline")
pipeline.run(query="What is the capital of Germany?")

from haystack.nodes import EvalAnswers
eval_reader = EvalAnswers(sas_model="sentence-transformers/paraphrase-multilingual-mpnet-base-v2")

• ›Adds RayPipeline class (imported from haystack.pipeline) enabling distributed pipeline execution across a Ray cluster, with per-node replicas configured in YAML pipeline config.
• ›Adds params dict argument to Pipeline.run() supporting node-targeted parameter routing such as params={"Retriever": {"top_k": 10}, "Reader": {"top_k": 5}}.
• ›Adds sas_model parameter to EvalAnswers node enabling cross-encoder-based Semantic Answer Similarity (SAS) evaluation metric.
• ›Adds ImageToTextConverter and PDFToTextOCRConverter classes providing OCR-based document conversion.
• ›Adds language parameter to PreProcessor for optional language-specific preprocessing.

ONNX Runtime v1.9 adds opset 15, sparse tensor APIs, TensorRT V2 provider API, and CUDA/TensorRT bundling in GPU packages.

$ git clone --branch v1.9.0 https://github.com/microsoft/onnxruntime.git
# already have the repo? check out this version:
$ git checkout v1.9.0

OrtTensorRTProviderOptionsV2* trt_options;
api->CreateTensorRTProviderOptions(&trt_options);
const char* keys[] = {"device_id", "trt_max_workspace_size"};
const char* values[] = {"0", "1073741824"};
api->UpdateTensorRTProviderOptions(trt_options, keys, values, 2);
api->SessionOptionsAppendExecutionProvider_TensorRT_V2(session_options, trt_options);
api->ReleaseTensorRTProviderOptions(trt_options);

$ ./build.sh --config Release --_use_extensions

from onnxruntime.training.ortmodule import ORTModule, DebugOptions, LogLevel

model = ORTModule(pt_model, DebugOptions(log_level=LogLevel.VERBOSE))

• ›Adds --_use_extensions build option to statically link onnxruntime-extensions for running models with custom operators.
• ›Adds RegisterAllocator and UnregisterAllocator C APIs (in onnxruntime_c_api.h) for sharing a custom allocator across multiple sessions.
• ›Adds SessionOptionsAppendExecutionProvider_TensorRT_V2, CreateTensorRTProviderOptions, UpdateTensorRTProviderOptions, GetTensorRTProviderOptionsAsString, and ReleaseTensorRTProviderOptions C APIs for configuring TensorRT EP.
• ›Adds EnableOrtCustomOps C API to enable custom operator support in a session.
• ›Adds sparse tensor C APIs: IsSparseTensor, CreateSparseTensorAsOrtValue, FillSparseTensorCoo, FillSparseTensorCsr, FillSparseTensorBlockSparse, CreateSparseTensorWithValuesAsOrtValue, UseCooIndices, UseCsrIndices, UseBlockSparseIndices, GetSparseTensorFormat, GetSparseTensorValuesTypeAndShape, GetSparseTensorValues, GetSparseTensorIndicesTypeShape, and GetSparseTensorIndices.

Triton v2.14.0 ships BLS beta, Java client beta, PyPI SDK wheel, and TensorRT as an optional backend.

$ git clone --branch v2.14.0 https://github.com/triton-inference-server/server.git
# already have the repo? check out this version:
$ git checkout v2.14.0

• ›Adds --gpus flag support for CUDA Device Index in addition to GPU UUID in Model Analyzer, giving teams more flexibility when targeting specific devices.
• ›Full-featured beta of Business Logic Scripting (BLS) released for the Python backend, enabling custom inference logic within the server.
• ›Beta Java client released with initial support for Triton's inference API.
• ›Triton Client SDK wheel now available directly from PyPI for both Ubuntu and Windows, installable via pip.
• ›TensorRT backend is now optional; the compose utility can build a Triton container without it, just like all other backends.

Zed v0.3 adds collaborative editing with a new people panel and .zed.toml collaborator config.

$ git clone --branch v0.3 https://github.com/zed-industries/zed.git
# already have the repo? check out this version:
$ git checkout v0.3

collaborators = ["nathansobo", "as-cii", "maxbrunsfeld"]

• ›Adds .zed.toml config file (placed in the project root) with a collaborators key listing GitHub usernames to grant access to a shared source tree.
• ›New people panel UI lets you share a local folder or join a collaborator's shared remote folder with a single click.
• ›Avatar icon in the upper-right corner enables GitHub-based authentication required to appear in collaborators' people panels.
• ›Collaborators listed in .zed.toml are automatically granted download access to Zed at https://zed.dev if they don't already have it.

$ git clone --branch v0.2.1 https://github.com/zed-industries/zed.git
# already have the repo? check out this version:
$ git checkout v0.2.1

$ git clone --branch v0.2 https://github.com/zed-industries/zed.git
# already have the repo? check out this version:
$ git checkout v0.2

Weaviate v1.7.0 adds array datatypes, a spellcheck module with auto-correct, and transformer-based NER at query time.

$ git clone --branch v1.7.0 https://github.com/weaviate/weaviate.git
# already have the repo? check out this version:
$ git checkout v1.7.0

{
  Get {
    Post(nearText: { concepts: "missspelled text" }) {
      content
      _additional {
        spellCheck {
          changes { corrected original }
          didYouMean
          location
          originalText
        }
      }
    }
  }
}

{
  Get {
    Post {
      content
      _additional {
        tokens(
          properties: ["content"],
          limit: 10,
          certainty: 0.8
        ) {
          certainty
          endPosition
          entity
          property
          startPosition
          word
        }
      }
    }
  }
}

• ›Adds array primitive datatypes (string[], text[], int[], number[]) to the schema, enabling lists of primitives to be stored, filtered, and aggregated like scalar properties; auto-schema automatically recognizes lists of string/text and number/int.
• ›New text-spellcheck module exposes a spellCheck field under _additional in GraphQL queries, returning per-term corrections (corrected, original), a didYouMean suggestion, location, and originalText at query time without altering results.
• ›New ner-transformers module exposes a tokens field under _additional in GraphQL queries for on-the-fly named-entity extraction from object properties, with optional properties, limit, and certainty parameters returning entity, word, startPosition, endPosition, certainty, and property per token.