Triton v1.10.0 adds JSON-format server status via REST and ordered batching across multiple model instances.
$ git clone --branch v1.10.0 https://github.com/triton-inference-server/server.git # already have the repo? check out this version: $ git checkout v1.10.0
$ curl http://<server>:8000/api/status?format=json - ›Adds JSON-format server status via the HTTP/REST endpoint
GET /api/status?format=json. - ›Adds a preserve-ordering option to the dynamic batcher so batched requests maintain sequence across multiple model instances (configured via
model_config.proto).