Backfill issue -440 — The AI Toolchain

EDITIONS tail grep head diff uniq

The daily firehose — everything the toolchain shipped today, already filtered.

// HOW THIS ISSUE IS MADE

We read every release from the 174 tools on our watchlist at the source — GitHub and GitLab release notes, vendor release pages and changelogs, project blogs and feeds, vendor press releases, and the source code behind the tag. Bug-fix-only releases and non-product newsroom noise are dropped; what's left is summarized down to the new capability, how to try it, and any screenshots or videos the release itself published. Every entry links to the sources it was built from.

VIEW

ISSUE VIEW full issue

Do you prefer this view?

$ tct list   # 2 tools matched

AI & LLM Tooling

◆ AI Model & Data Infrastructure

Microsoft ONNX Runtime

v0.3.0 NOTES

ONNX Runtime v0.3.0 adds ONNX 1.4/Opset 9 support, custom op registration via C API, and new OS X/x86 build packages.

└──▷ GET THIS VERSION

$ git clone --branch v0.3.0 https://github.com/microsoft/onnxruntime.git
# already have the repo? check out this version:
$ git checkout v0.3.0

›Adds custom op registration via the C API, enabling users to plug in their own operators at runtime.
›Adds Non-Tensor type support for input/output in the C and C# APIs.
›Supports ONNX 1.4 compatibility including Opset 9 operators and large models over 2 GB.
›Adds new prebuilt packages: C/C# on OS X x64 CPU, C on Linux x86 CPU, and C on Windows x86 CPU.
›Default CPU execution provider is now Eigen and MLAS (previously MKL-DNN).

└──▷ BREAKING ON UPGRADE

!The default execution provider for CPU has changed from MKL-DNN to Eigen and MLAS; existing builds relying on MKL-DNN behavior must explicitly select it via build options.
!OpenMP is now required for prebuilt binaries; setups without OpenMP installed will fail to run.

Was this useful?

NVIDIA Triton Inference Server

v1.0.0 NOTES

TensorRT Inference Server 1.0.0 GA adds stateful model scheduling, GRPC streaming, and async HTTP for lower-latency inference.

└──▷ GET THIS VERSION

$ git clone --branch v1.0.0 https://github.com/triton-inference-server/server.git
# already have the repo? check out this version:
$ git checkout v1.0.0

›Adds the *sequence batcher* to schedule and batch stateful models that require multiple inference requests routed to the same model instance/batch slot.
›Adds GRPC streaming protocol support for inference requests.
›HTTP front-end is now asynchronous, enabling lower-latency and higher-throughput inference handling.
›Enhances perf_client to support stateful models and backends.

Was this useful?

// END OF TRANSMISSION

The AI Toolchain · curated daily · March 18, 2019
Releases & features from the open-source and commercial AI tools that practitioners actually run.

The AI Toolchain — issue -440, March 18, 2019

Microsoft ONNX Runtime

NVIDIA Triton Inference Server