Heads up This site is currently under heavy development.
Subscribe Get it delivered — the daily firehose, filtered to the tools you run, plus the documentation changes vendors never announce. Compare plans →

The AI Toolchain — issue -433, October 30, 2019

THE AI TOOLCHAIN NO. -433
Tail
THE DAILY RELEASE FIREHOSE
PUBLISHED OCTOBER 30, 2019 · EVERY WEEKDAY
EDITIONS tail grep head diff uniq

The daily firehose — everything the toolchain shipped today, already filtered.

// HOW THIS ISSUE IS MADE

We read every release from the 174 tools on our watchlist at the source — GitHub and GitLab release notes, vendor release pages and changelogs, project blogs and feeds, vendor press releases, and the source code behind the tag. Bug-fix-only releases and non-product newsroom noise are dropped; what's left is summarized down to the new capability, how to try it, and any screenshots or videos the release itself published. Every entry links to the sources it was built from.

VIEW
ISSUE VIEW full issue
Do you prefer this view? 
$ tct list   # 4 tools matched
AI & LLM Tooling
◆  AI Model & Data Infrastructure

Microsoft ONNX Runtime

Sources Release notes → v1.0.0 NOTES

ONNX Runtime v1.0.0 adds stable C ABI, four new execution providers, free dimension override, and new build/perf options.

└──▷ GET THIS VERSION
$ git clone --branch v1.0.0 https://github.com/microsoft/onnxruntime.git
# already have the repo? check out this version:
$ git checkout v1.0.0
└──▷ TRY IT
Tune thread counts for a high-throughput concurrent inference server, controlling both intra- and inter-op parallelism via onnxruntime_perf_tool.
$ onnxruntime_perf_tool -m model.onnx -o 99 -y 4
  • ›Adds free dimension override, letting callers supply statically-unknown tensor shapes (e.g. batch size) before inference to unlock ahead-of-time optimizations.
  • ›Adds ability to set intra- and inter-operator thread counts independently to tune sequential vs. concurrent inferencing workloads.
  • ›Adds new cmake build options onnxruntime_USE_GEMMLOWP, onnxruntime_USE_AUTOML, and onnxruntime_USE_DML for controlling optional EP dependencies.
  • ›Adds -y flag to onnxruntime_perf_tool for controlling inter_op_num_threads; maximum optimization level is now 99 (previously 3).
  • ›Adds new session option for serializing optimized ONNX models to disk.
+12 moreshow less
  • ›Enables registration of execution providers through the Python API and setting additional run options through the C# API.
  • ›Stabilizes the C API at v1.0 with ABI compatibility and Semantic Versioning guarantees — programs linked against this release will not require re-linking for future releases.
  • ›Adds [Preview] NUPHAR execution provider, a TVM- and LLVM-based EP that JIT-compiles subgraph nodes into optimized functions for model acceleration.
  • ›Adds [Preview] DirectML execution provider for hardware-accelerated GPU inference via DirectX 12 on Windows across a broad range of hardware and drivers.
  • ›Adds [Preview] ARM Compute Library (ACL) execution provider targeting ARM CPUs and GPUs.
  • ›Adds [Preview] OpenVINO support for Intel Arria 10 FPGA.
  • ›Promotes OpenVINO EP to General Availability for Intel CPU, Integrated Graphics, Neural Compute Stick 2, and Movidius Myriad VPU targets.
  • ›Adds quantization tool support for selective node quantization, bias quantization for Conv nodes, and node fusion for dynamic quantization.
  • ›Adds ONNX 1.6 / opset 11 operator support on CPU, including Sequence ops.
  • ›Adds component-level telemetry via Windows Trace Logging for Windows builds.
  • ›Adds Gelu fusion optimizer for CPU/GPU inference.
  • ›Switches default thread pool to Eigen ThreadPool; adds ability to disable multi-threading by setting pool size to 1 and onnxruntime_USE_OPENMP to OFF.
└──▷ BREAKING ON UPGRADE
  • !cmake options onnxruntime_USE_MLAS and onnxruntime_USE_EIGEN_THREADPOOL are removed; both are now always enabled and specifying them will break builds.
  • !In onnxruntime_perf_tool, optimization level 3 is no longer valid; the maximum is now 99.
  • !Minimum supported gcc version is raised to 4.8.2.
Was this useful?

NVIDIA Triton Inference Server

Sources Release notes → v1.7.0 NOTES

Triton v1.7.0 adds TensorRT optimization for TensorFlow, ONNXRuntime TensorRT/OpenVINO providers, multi-repo support, and a Go GRPC client example.

└──▷ GET THIS VERSION
$ git clone --branch v1.7.0 https://github.com/triton-inference-server/server.git
# already have the repo? check out this version:
$ git checkout v1.7.0
  • ›Adds --strict-model-config=false support for TensorRT models with variable-sized inputs and/or outputs, enabling automatic configuration generation for those models.
  • ›Multiple model repositories can now be specified on the command line, with optional flags to explicitly load specific models from each repository.
  • ›TensorRT optimization can now be enabled for any TensorFlow model via the optimization section of the model configuration.
  • ›The ONNXRuntime backend now includes TensorRT and OpenVINO execution providers, configurable via the optimization section of the model configuration.
  • ›Ensemble models are now pruned dynamically so only models required to compute the requested outputs are executed.
+2 moreshow less
  • ›A Client SDK container is now available on NGC, packaging client libraries and examples separately from the inference server container.
  • ›Example clients now include a Go example using the GRPC API.
Was this useful?
Other / Uncategorized
◆  VECTOR DB RAG

Milvus

Sources Release notes → v0.5.0 NOTES

Milvus v0.5.0 adds IVFSQ8 Hybrid index, startup data preloading, and a new Java SDK

└──▷ GET THIS VERSION
$ git clone --branch v0.5.0 https://github.com/milvus-io/milvus.git
# already have the repo? check out this version:
$ git checkout v0.5.0
  • ›Adds new IVFSQ8 Hybrid index type for approximate nearest-neighbor search.
  • ›Adds preloading of data during system startup phase to reduce cold-start query latency.
  • ›Adds a new Java SDK for interacting with Milvus.
  • ›Improves result merge performance.
└──▷ BREAKING ON UPGRADE
  • !The DeleteByDate API has been removed.
Was this useful?

Weaviate

Sources Release notes → 0.20.3 2 RELEASES · 2019-10-10 → 2019-10-11 NOTES STABLE

Weaviate 0.20.3 lets you disable vectorization and search indexing per property via index: false in the schema.

└──▷ GET THIS VERSION
$ git clone --branch 0.20.3 https://github.com/weaviate/weaviate.git
# already have the repo? check out this version:
$ git checkout 0.20.3
  • ›Adds index: false to property schema definitions, allowing specific properties to be excluded from both vectorization and Elasticsearch text-based indexing; properties default to indexed when index: true or omitted.
1 more release in this issue · 2019-10-10 → 2019-10-11
0.20.2 NOTES STABLE

Weaviate 0.20.2 adds kNN-based classification via /v1/classifications/ and a new meta=true option on thing retrieval.

└──▷ GET THIS VERSION
$ git clone --branch 0.20.2 https://github.com/weaviate/weaviate.git
# already have the repo? check out this version:
$ git checkout 0.20.2
└──▷ TRY IT
Inspect whether a retrieved object's cross-reference was set by a user or assigned automatically by classification.
$ curl 'http://localhost:8080/v1/things/Dish/<id>?meta=true'
  • ›Adds POST /v1/classifications/ endpoint to trigger kNN-based classification of data objects using cross-referenced schema classes as training data.
  • ›Adds ?meta=true query parameter on GET /things/{kinds}/{id} to expose additional fields, including classification provenance (whether a reference was set by user input or by classification).
Was this useful?
// END OF TRANSMISSION

The AI Toolchain · curated daily · October 30, 2019
Releases & features from the open-source and commercial AI tools that practitioners actually run.

▷ Subscribe
my-toolchain — 0 tools
paste an install list to detect your tools

A brew list, a Brewfile, requirements.txt, a Dockerfile — or just the product names, free-form. Nothing leaves your browser.

    browse all tools →