Backfill issue -431 — The AI Toolchain

EDITIONS tail grep head diff uniq

The daily firehose — everything the toolchain shipped today, already filtered.

// HOW THIS ISSUE IS MADE

We read every release from the 174 tools on our watchlist at the source — GitHub and GitLab release notes, vendor release pages and changelogs, project blogs and feeds, vendor press releases, and the source code behind the tag. Bug-fix-only releases and non-product newsroom noise are dropped; what's left is summarized down to the new capability, how to try it, and any screenshots or videos the release itself published. Every entry links to the sources it was built from.

VIEW

ISSUE VIEW full issue

Do you prefer this view?

$ tct list   # 4 tools matched

AI & LLM Tooling

◆ AI Model & Data Infrastructure

Microsoft ONNX Runtime

v1.1.0 NOTES

ONNX Runtime v1.1.0 adds Android NNAPI and Java API previews, shared-library custom ops, and BERT/TensorRT performance upgrades.

└──▷ GET THIS VERSION

$ git clone --branch v1.1.0 https://github.com/microsoft/onnxruntime.git
# already have the repo? check out this version:
$ git checkout v1.1.0

›Adds preview NN API Execution Provider for Android, enabling on-device inference via Android's Neural Networks API.
›Adds preview Java API for ONNX Runtime, bringing Java application support to the runtime.
›Custom ops can now be packaged in shared libraries and loaded at runtime, allowing distribution across multiple applications without modification.
›Adds a Python tool that automatically maps a pandas DataFrame to ONNX graph inputs based on schema information, located at onnxruntime/python/tools/featurizer_ops.
›TensorRT Execution Provider gains support for inputs with dynamic shapes.

+2 moreshow less

›Fused CPU kernels added for Attention, EmbedLayerNormalization, SkipLayerNormalization, and FastGelu operators to accelerate BERT model inference.
›MKL-DNN (now renamed DNNL) updated to v1.1.

Was this useful?

NVIDIA Triton Inference Server

v1.9.0 NOTES

Triton v1.9.0 adds model warmup, a new repository index API, Oldest sequence-batcher strategy, and enhanced tracing for ensembles.

└──▷ GET THIS VERSION

$ git clone --branch v1.9.0 https://github.com/triton-inference-server/server.git
# already have the repo? check out this version:
$ git checkout v1.9.0

›Adds a model warmup option to model configuration, allowing models to be pre-tuned with synthetic or realistic workloads before serving inference requests — particularly useful for frameworks like TensorFlow that optimize on initial inference.
›New repository index API (available via shared library API, HTTP, and GRPC) returns an index of all models visible in the model repositories, enabling discovery of models available for loading.
›Server status API response now includes the timestamp of the last inference request received for each model.
›Adds an Oldest scheduling strategy to the sequence batcher, integrating with the dynamic batcher to improve throughput for models that do not require all requests in a sequence to share the same batch slot.
›Adds a perf_client option to generate requests using a Poisson distribution or a user-provided distribution for more realistic load testing.

+2 moreshow less

›Extends tracing support to cover ensembles and their contained models.
›Adds a community-contributed Dockerfile for building inference server clients on CentOS.

└──▷ BREAKING ON UPGRADE

!The beta custom backend API v2 CustomGetNextInputV2Fn_t function signature adds the memory_type_id argument — existing custom backends using this function will break.
!The beta custom backend API v2 CustomGetOutputV2Fn_t function signature adds the memory_type_id argument — existing custom backends using this function will break.
!The beta library API TRTSERVER_ResponseAllocatorAllocFn_t function signature and behavior has changed — see src/core/trtserver.h for the new behavior; existing callers will break.
!The beta library API TRTSERVER_InferenceRequestProviderSetInputData function signature adds the memory_type_id argument — existing callers will break.
!The beta library API TRTSERVER_InferenceResponseOutputData function signature adds the memory_type_id argument — existing callers will break.

Was this useful?

Other / Uncategorized

◆ VECTOR DB RAG

Milvus

v0.6.0 NOTES

Milvus v0.6.0 adds table partitioning, Mishards sharding middleware, new index types, and multi-GPU index building via build_index_resources.

└──▷ GET THIS VERSION

$ git clone --branch v0.6.0 https://github.com/milvus-io/milvus.git
# already have the repo? check out this version:
$ git checkout v0.6.0

└──▷ USE IT

Speed up index building on a multi-GPU machine by assigning specific GPU devices via the Milvus server config.

yaml

build_index_resources:
  - gpu0
  - gpu1

›Adds build_index_resources configuration parameter to enable multi-GPU index building, reducing index build and overall query time.
›Adds table partitioning APIs across Python, Java, and C++ SDKs — supporting partition creation, vector insertion into a specified partition, and partition-scoped queries.
›Introduces experimental index types SPTAG-KDT, SPTAG-BKT, RNSG, and IVFPQ for broader ANN search coverage.
›Introduces Mishards, an experimental Milvus sharding middleware enabling distributed deployment with request forwarding, read/write splitting, horizontal scalability, and dynamic extension.
›Publishes performance test reports for IVFFLAT, IVFSQ8, and IVFSQ8H indexes.

+2 moreshow less

›Open-sources Milvus internal FAISS with deep optimizations, adding support for the IVFSQ8H index type.
›Provides Docker images for both CPU-only and GPU-enabled Milvus, with Docker compilation supported on machines with or without a GPU.

Was this useful?

Weaviate

0.21.6 3 RELEASES · 2019-12-06 → 2019-12-18 NOTES STABLE

Weaviate 0.21.6 adds configurable sharding, replication, and supernode threshold controls for the vector index.

└──▷ GET THIS VERSION

$ git clone --branch 0.21.6 https://github.com/weaviate/weaviate.git
# already have the repo? check out this version:
$ git checkout 0.21.6

›Adds vectorIndex.numberOfShards (integer) and vectorIndex.autoExpandReplicas (string) config keys to control Elasticsearch shard and replica defaults per class, mirroring Elasticsearch index-module settings.
›Adds vectorIndex.supernodeThreshold (integer) config key to override the default threshold (100 outgoing references) at which a class is treated as a supernode.

2 more releases in this issue · 2019-12-06 → 2019-12-18

0.21.5 NOTES STABLE

Weaviate 0.21.5 adds sourceWhere, trainingSetWhere, and targetWhere filters to narrow classification runs.

└──▷ GET THIS VERSION

$ git clone --branch 0.21.5 https://github.com/weaviate/weaviate.git
# already have the repo? check out this version:
$ git checkout 0.21.5

›Adds sourceWhere filter to classification API to limit which unclassified objects are processed during a classification run.
›Adds trainingSetWhere filter to classification API to restrict the training set, usable with training-set-based types such as 'type': 'knn'.
›Adds targetWhere filter to classification API to restrict potential label targets, usable with direct-relationship types such as 'type': 'contextual'.

0.21.4 NOTES STABLE

Weaviate 0.21.4 adds contextual classification — no training data required, targets chosen by vector distance.

└──▷ GET THIS VERSION

$ git clone --branch 0.21.4 https://github.com/weaviate/weaviate.git
# already have the repo? check out this version:
$ git checkout 0.21.4

›Adds type: contextual classification to the classification API payload, enabling vector-distance-based classification without training data; omit the k field (which is knn-only) when using this type.

Was this useful?

// END OF TRANSMISSION

The AI Toolchain · curated daily · December 21, 2019
Releases & features from the open-source and commercial AI tools that practitioners actually run.

The AI Toolchain — issue -431, December 21, 2019

Microsoft ONNX Runtime

NVIDIA Triton Inference Server

Milvus

Weaviate