ONNX Runtime v0.5.0 adds C++ API, HTTP/gRPC inference server, Intel OpenVINO EP, Python custom operators, and CUDA 10 support.
$ git clone --branch v0.5.0 https://github.com/microsoft/onnxruntime.git # already have the repo? check out this version: $ git checkout v0.5.0
- ›Adds a C++ API (
onnxruntime_cxx_api.h) for inferencing, wrapping the existing C API. - ›Introduces ONNX Runtime Server (Beta) for serving models over HTTP and gRPC endpoints.
- ›Adds Python Operator (Beta) support to embed custom Python code in a single ONNX graph node, enabling experimentation with custom operators.
- ›Intel OpenVINO Execution Provider is now available for Public Preview.
- ›MKL-DNN Execution Provider now uses subgraph-based execution for improved performance.
+4 moreshow less
- ›Updates CUDA support to CUDA 10 for GPU inferencing with NVIDIA GPUs.
- ›Base CPU Execution Provider gains faster convolution performance via NCHWc blocked layout, enabled by setting graph optimization level to
3in session options. - ›All C APIs now return an
ORTstatus code for consistency. - ›Adds support for Keras-based Mask R-CNN model inferencing.