The AI Toolchain — issue -417, February 27, 2021

Triton v2.7.0 adds Java/Scala gRPC clients, TFServing/TorchServe perf_analyzer support, and YAML-based Model Analyzer configuration.

$ git clone --branch v2.7.0 https://github.com/triton-inference-server/server.git
# already have the repo? check out this version:
$ git checkout v2.7.0

• ›Extends perf_analyzer to work with TFServing and TorchServe, enabling benchmarking across non-Triton serving backends.
• ›Model Analyzer parameters and test model configurations can now be specified via a YAML configuration file.
• ›Model Analyzer now reports performance metrics for end-to-end latency and CPU memory usage.
• ›Adds example Java and Scala clients based on the gRPC-generated API.
• ›Alpha Windows release (tritonserver2.7.0-win.zip) ships with TensorRT 7.2.2 support and a GRPC endpoint.

Milvus v0.10.6 adds nbits parameter for IVF_PQ indexing and Prometheus label configuration support.

$ git clone --branch v0.10.6 https://github.com/milvus-io/milvus.git
# already have the repo? check out this version:
$ git checkout v0.10.6

• ›Adds optional nbits argument to the create_index() method for the IVF_PQ index, enabling finer control over product quantization bit-width.
• ›Supports configuring Prometheus labels cluster_label and instance_label under the metric config section.
• ›Improves FLAT search performance on binary vectors using the AVX2 instruction set.

Weaviate v1.1.0 adds nearObject GraphQL search and delivers 30–50% faster cross-reference batch imports.

$ git clone --branch v1.1.0 https://github.com/weaviate/weaviate.git
# already have the repo? check out this version:
$ git checkout v1.1.0

{
  Get{
    Publication(
      nearObject: {
        id: "27b5213d-e152-4fea-bd63-2063d529024d",
        certainty: 0.7
      }
    ){
      name
      _additional {
        certainty
      }
    }
  }
}

• ›Adds nearObject search parameter to GraphQL Get queries, letting you find the most similar objects to a given id or beacon in a single step — no need to first retrieve the vector and run a separate nearVector search; supports a certainty threshold.
• ›Supports combining nearObject with movement operations in the text2vec-contextionary module.
• ›Cross-reference batch imports are now 30–50% faster on cross-reference-heavy datasets by recognising that reference updates do not change vector positions and skipping a full re-index of affected objects.