The AI Toolchain — issue -428, March 26, 2020

ONNX Runtime v1.2.0 adds WinML APIs, TensorRT 7, new C API surfaces, quantized ops, and Android AAR builds.

$ git clone --branch v1.2.0 https://github.com/microsoft/onnxruntime.git
# already have the repo? check out this version:
$ git checkout v1.2.0

OrtModelMetadata* metadata;
ort_api->SessionGetModelMetadata(session, &metadata);

char* producer;
ort_api->ModelMetadataGetProducerName(metadata, allocator, &producer);

char* custom_val;
ort_api->ModelMetadataLookupCustomMetadataMap(metadata, allocator, "my_key", &custom_val);

ort_api->ReleaseModelMetadata(metadata);

• ›Adds C API functions GetDenotationFromTypeInfo, CastTypeInfoToMapTypeInfo, CastTypeInfoToSequenceTypeInfo, GetMapKeyType, GetMapValueType, GetSequenceElementType, ReleaseMapTypeInfo, ReleaseSequenceTypeInfo, SessionEndProfiling, SessionGetModelMetadata, ModelMetadataGetProducerName, ModelMetadataGetGraphName, ModelMetadataGetDomain, ModelMetadataGetDescription, ModelMetadataLookupCustomMetadataMap, ModelMetadataGetVersion, and ReleaseModelMetadata for richer session introspection.
• ›Adds new Contrib ops QLinearAdd, QLinearMul, QLinearReduceMean, MulInteger, and QLinearAveragePool for quantized inference workflows.
• ›Adds Int8 type support for the Where op.
• ›Default value of graph_optimization_level in SessionOptions changed to ORT_ENABLE_ALL (99), and default max_num_graph_transformation_steps increased to 10.
• ›Introduces a model opset number and IR version check — ONNX Runtime will now reject models with ONNX opset versions higher than those supported in the built version.

Triton v1.12.0 adds dynamic batching queue policies, ONNX large-model support, and an experimental GRPC inferencing API.

$ git clone --branch v1.12.0 https://github.com/triton-inference-server/server.git
# already have the repo? check out this version:
$ git checkout v1.12.0

• ›Adds --min-supported-compute-capability flag to allow Triton Server to run on older, otherwise unsupported GPUs.
• ›Adds queuing policies for the dynamic batching scheduler, configurable per model via the model configuration: maximum queue size, timeouts, and priority levels for inference requests.
• ›Supports configuring ONNX Runtime optimization level via the model configuration optimization setting.
• ›Supports large ONNX models where weights are stored in separate files.
• ›Experimental Python client and server support for the community standard GRPC inferencing API.

Milvus v0.7.0 adds vector deletion, RESTful API, Go SDK, HNSW index, WAL, and Jaccard/Hamming/Tanimoto distance support.

$ git clone --branch v0.7.0 https://github.com/milvus-io/milvus.git
# already have the repo? check out this version:
$ git checkout v0.7.0

wal:
  enable: true
  recovery_error_ignore: false
  buffer_size: 256MB

• ›Adds WAL (Write-Ahead Logging) configurable via server_config.yaml to significantly improve reliability of data operations.
• ›Adds RESTful API for interacting with Milvus over HTTP.
• ›Adds Go SDK, available at github.com/milvus-io/milvus-sdk-go.
• ›Adds support for HNSW index type for approximate nearest neighbor search.
• ›Adds vector deletion support for one or multiple vectors (available with FLAT, IVFLAT, IVFSQ8, and related index types).

Weaviate 0.22.4 adds contextionary language support for German, Dutch, Italian, and Czech.

$ git clone --branch 0.22.4 https://github.com/weaviate/weaviate.git
# already have the repo? check out this version:
$ git checkout 0.22.4

• ›Adds contextionary language support for German, Dutch, Italian, and Czech in contextionary version xx0.13.0-v0.4.7, with example Docker Compose files provided for each language.

$ git clone --branch 0.22.3 https://github.com/weaviate/weaviate.git
# already have the repo? check out this version:
$ git checkout 0.22.3

$ curl 'http://localhost:8080/v1/things?meta=true'