Backfill issue -414 — The AI Toolchain

EDITIONS tail grep head diff uniq

The daily firehose — everything the toolchain shipped today, already filtered.

// HOW THIS ISSUE IS MADE

We read every release from the 174 tools on our watchlist at the source — GitHub and GitLab release notes, vendor release pages and changelogs, project blogs and feeds, vendor press releases, and the source code behind the tag. Bug-fix-only releases and non-product newsroom noise are dropped; what's left is summarized down to the new capability, how to try it, and any screenshots or videos the release itself published. Every entry links to the sources it was built from.

VIEW

ISSUE VIEW full issue

Do you prefer this view?

$ tct list   # 2 tools matched

AI & LLM Tooling

◆ AI Model & Data Infrastructure

NVIDIA Triton Inference Server

v2.10.0 NOTES

Triton v2.10.0 adds ONNX on Jetson, HTTP compression, ragged batching, and Model Analyzer CLI subcommands.

└──▷ GET THIS VERSION

$ git clone --branch v2.10.0 https://github.com/triton-inference-server/server.git
# already have the repo? check out this version:
$ git checkout v2.10.0

└──▷ TRY IT

Run Triton on Jetson with ONNX Runtime backend and TensorFlow 2.x, explicitly setting the backend directory.

$ tritonserver --model-repository=/path/to/model_repo --backend-directory=/path/to/tritonserver/backends --backend-config=tensorflow,version=2

›Adds profile, analyze, and report subcommands to the Model Analyzer CLI for structured profiling and reporting workflows.
›Adds --backend-directory flag to tritonserver to explicitly set the backend directory, required on Jetson deployments.
›Adds --backend-config=tensorflow,version=2 flag to select TensorFlow 2.x on Jetson (where TF 1.x is the default).
›Triton server and HTTP clients (Python and C++) now support request/response compression.
›Ragged batching is now supported for ONNX models.

+4 moreshow less

›Triton on Jetson (JetPack 4.5) now supports ONNX via the ONNX Runtime backend (version 1.7.1), including the TensorRT execution provider.
›Model Analyzer gains a report subcommand to generate a detailed report for any specific model configuration.
›Model Analyzer now supports CPU-only mode.
›Windows alpha release adds GRPC endpoint support for TensorRT and ONNX Runtime (CPU, CUDA, TensorRT execution providers) models.

└──▷ BREAKING ON UPGRADE

!Legacy custom backend support is removed; all custom backends must be reimplemented using the TRITONBACKEND API.
!Model Analyzer CLI subcommands (profile, analyze, report) require updating existing Model Analyzer config files and CLI flags.
!The Triton client libraries have moved to a separate repository at https://github.com/triton-inference-server/client.
!utils.serialize_byte_tensor() and utils.deserialize_byte_tensor() in the Python client shared-memory support library now return np.object_ arrays instead of np.bytes_ arrays; code depending on np.bytes_ must be updated.

Was this useful?

Other / Uncategorized

◆ VECTOR DB RAG

Milvus

v1.1.0 NOTES

Milvus v1.1.0 adds partition-scoped entity lookup and deletion, plus a new method to unload collections from cache.

└──▷ GET THIS VERSION

$ git clone --branch v1.1.0 https://github.com/milvus-io/milvus.git
# already have the repo? check out this version:
$ git checkout v1.1.0

›Adds partition targeting to get_entity_by_id(), allowing entity lookups to be scoped to a specific partition.
›Adds partition targeting to delete_entity_by_id(), allowing entity deletions to be scoped to a specific partition.
›Adds new release_collection() method to explicitly unload a specific collection from cache.

Was this useful?

// END OF TRANSMISSION

The AI Toolchain · curated daily · May 21, 2021
Releases & features from the open-source and commercial AI tools that practitioners actually run.

The AI Toolchain — issue -414, May 21, 2021

NVIDIA Triton Inference Server

Milvus