Triton v2.13.0 adds Business Logic Scripting beta, a Container Composition Utility, and Model Analyzer support for custom ops.
$ git clone --branch v2.13.0 https://github.com/triton-inference-server/server.git # already have the repo? check out this version: $ git checkout v2.13.0
$ tritonserver --model-repository=/path/to/model_repo --backend-directory=/path/to/tritonserver/backends --backend-config=tensorflow,version=2
- ›Adds
--backend-directoryflag and--backend-config=tensorflow,version=2argument totritonserverfor explicit backend path and TensorFlow version selection on Jetson. - ›Initial beta release of Business Logic Scripting — a new set of utility functions allowing a Python model to execute inference requests on other models being served by Triton.
- ›New Container Composition Utility (
docs/compose.md) for building custom Triton containers with specific backends and repository agents. - ›Publishes two new focused NGC containers:
nvcr.io/nvidia/tritonserver:21.08-tf-python-py3(TensorFlow 2.x + Python backends) andnvcr.io/nvidia/tritonserver:21.08-pyt-python-py3(PyTorch + Python backends). - ›Adds Model Analyzer support for models with custom operations.
- !In the Python client shared-memory support library, utils.serialize_byte_tensor() and utils.deserialize_byte_tensor() now return
np.object_numpy arrays instead ofnp.bytes_arrays; code depending onnp.bytes_must be updated.