Triton v1.1.0 adds zero-sized tensor support, TensorFlow custom C++ ops, and a standalone client build system.
$ git clone --branch v1.1.0 https://github.com/triton-inference-server/server.git # already have the repo? check out this version: $ git checkout v1.1.0
- ›Supports zero-sized input and output tensors for batching models, enabling models that require inputs and outputs with shape
[ batch-size ]. - ›Supports variable-size tensor dimensions (indicated by
-1in model configuration) where the variable dimension has value 0. - ›TensorFlow custom operations (C++) can now be built directly into the inference server, with an included example and documentation.
- ›Client libraries and examples now build via a separate Makefile (with an accompanying Dockerfile), decoupled from the main server build.