Triton Inference Server v0.10.0 adds custom backend support for models implemented outside deep-learning frameworks.
$ git clone --branch v0.10.0 https://github.com/triton-inference-server/server.git # already have the repo? check out this version: $ git checkout v0.10.0
- ›Adds custom backend support, allowing individual models to implement arbitrary logic while retaining GPU support, concurrent execution, and dynamic batching from the server.