THE AI TOOLCHAIN TRTIS 0.9.0 adds live model repository reloading, priority controls, and a --strict-model-config=false auto-config flag.
└──▷ GET THIS VERSION
$ git clone --branch v0.9.0 https://github.com/triton-inference-server/server.git # already have the repo? check out this version: $ git checkout v0.9.0
└──▷ TRY IT
Start the inference server without requiring a hand-authored config.pbtxt for supported model types, letting the server auto-derive configuration.
$ trtserver --strict-model-config=false --model-store=/models
- ›Adds
--strict-model-config=falseflag to allow automatic derivation of model configuration properties, removing the need for aconfig.pbtxtfile for some model types. - ›Adds a model priority parameter to model configuration, controlling CPU thread priority for all models and CUDA stream priority for TensorRT models.
- ›Live model repository monitoring now dynamically reloads models without a server restart — supports adding/removing model versions, adding/removing entire models, modifying
config.pbtxt, and updating model labels at runtime. - ›Asynchronous GRPC frontend delivers improved inference throughput.
└──▷ BREAKING ON UPGRADE
- !The GRPC API model version parameter type has changed from string to int — existing clients passing a string model version will break.
Was this useful?