ONNX Runtime

Summary

ONNX Runtime is an open-source library that acts as a cross-platform inference and training machine-learning accelerator, and it is free under its license. It is used as a library imported into other code, supporting model formats from frameworks like PyTorch and TensorFlow, as well as classical ML libraries. This is aimed at application developers looking to run inference or train models across various hardware and operating systems, and its documentation does not name a comparable product. The project maintains active repositories and a roadmap for updates.

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

Command line

Values are placeholders taken from each option’s declared default. Nothing is executed here — the output shown is a recording of a run that already happened.

Release history

1. plugin-ep-webgpu/v0.3.0 Aug 24, 2026 · issue 007

   ONNX Runtime WebGPU Plugin EP 0.3.0 adds new operators, quantized KV cache, int64/uint8 type expansion, and configurable buffer-cache options.

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch plugin-ep-webgpu/v0.3.0 https://github.com/microsoft/onnxruntime.git
   # already have the repo? check out this version:
   $ git checkout plugin-ep-webgpu/v0.3.0

   • ›Exposes WebGPU buffer-cache options through the JavaScript API, adds a robustness provider option, and makes the pending-dispatch limit configurable.
   • ›Adds device-free compile-only sessions for offline graph transformation without requiring a physical GPU.
   • ›Adds initial PagedAttention support, plus new operators: MRotaryEmbedding, GRU, DFT, PRelu, HardSwish, Trilu, Max, Min, and MatMulBnb4.
   • ›Adds 2-bit GatherBlockQuantized support and integrates ONNX 1.22 with opset 27.
   • ›Expands integer data-type coverage: int64 for Add, Cast, Clip, Concat, Equal, Gather, Min, Max, ReduceSum, Reshape, Sub, Tile, and Where; uint8 for Cast, Expand, Gather, and Reshape; int32/uint32 for CumSum and Tile.
   +5 moreshow less

   • ›Adds quantized KV cache support and expands GQA with sliding-window cache, batched right-padded prompts, and FlashAttention graph capture for Gemma 4.
   • ›Adds deferred dispatch to parallelize shader compilation during cold start, reducing first-inference latency.
   • ›Adds Intel subgroup-matrix MatMul and FP16 Gemm kernels with batched-B and odd-N weight support and additional tiling optimizations.
   • ›Switches Softmax and non-flash Attention to an online softmax algorithm and adds tuned FlashAttention decode.
   • ›Removes the dynamic Duktape/Node WGSL-generation path in favor of statically generated shader sources.
   └──▷ BREAKING ON UPGRADE

   • !The dynamic Duktape/Node WGSL-generation path has been removed and replaced with generated shader sources, which may break build setups that relied on runtime WGSL generation.
2. v1.28.1 Aug 18, 2026 · issue -001

   ONNX Runtime v1.28.1 adds device-free WebGPU compile-only sessions for offline model optimization without GPU hardware.

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch v1.28.1 https://github.com/microsoft/onnxruntime.git
   # already have the repo? check out this version:
   $ git checkout v1.28.1

   • ›Adds device-free compile-only sessions in the WebGPU EP, enabling offline graph transformation and optimized-model serialization without access to GPU hardware.
3. v1.28.1 Aug 18, 2026 · issue 002

   ONNX Runtime v1.28.1 adds device-free WebGPU compile-only sessions for offline graph transformation without GPU hardware.

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch v1.28.1 https://github.com/microsoft/onnxruntime.git
   # already have the repo? check out this version:
   $ git checkout v1.28.1

   • ›Adds support for device-free compile-only sessions in the WebGPU EP, enabling offline graph transformation and optimized-model serialization without access to GPU hardware.
4. plugin-ep-cuda/v0.1.0 Aug 17, 2026 · issue -002

   ONNX Runtime ships its first CUDA Plugin EP release, delivering CUDA execution as a separately packaged, independently installable provider.

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch plugin-ep-cuda/v0.1.0 https://github.com/microsoft/onnxruntime.git
   # already have the repo? check out this version:
   $ git checkout plugin-ep-cuda/v0.1.0

   • ›Introduces the CUDA Plugin EP as a separately packaged execution provider, now the default CUDA provider implementation for ONNX Runtime, distributed as the onnxruntime-ep-cuda Python wheel and NuGet packages.
   • ›Adds provider options for user compute streams, copy behavior, EP-level unified streams, and external allocators, with compatibility back to ONNX Runtime 1.24.4 via version-gated callbacks.
   • ›Adds plugin profiling APIs and memory statistics in profiling output.
   • ›Adds CUDA Graph capture and replay, including user compute stream support, synchronization controls, and a kernel sync stream for scratch allocation.
   • ›Adds arena allocation, resource accounting, available-resource reporting, and IOBinding synchronization.
   +4 moreshow less

   • ›Expands operator coverage with cuDNN SDPA for MHA/GQA, quantized MoE kernels, block-quantized FP4 MatMul, native NVFP4 QMoE prefill on SM120, and NHWC hardening.
   • ›Adds cuDNN-free implementations of ArgMax, ArgMin, and ReduceSum.
   • ›Reduces runtime dependencies by making cuDNN and cuFFT optional and removing the nvrtc link requirement.
   • ›Adds Windows ARM64 packages and build coverage across CUDA 12.8/13, SM90, and SM120.
5. plugin-ep-cuda/v0.1.0 Aug 17, 2026 · issue 002

   ONNX Runtime ships its first CUDA Plugin EP release, decoupling CUDA execution into a separately packaged provider with graph capture, profiling APIs, and expanded operator coverage.

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch plugin-ep-cuda/v0.1.0 https://github.com/microsoft/onnxruntime.git
   # already have the repo? check out this version:
   $ git checkout plugin-ep-cuda/v0.1.0

   • ›Introduces the onnxruntime-ep-cuda Python wheel and NuGet packages, delivering CUDA execution as a separately installable plugin execution provider.
   • ›Adds provider options for user compute streams, copy behavior, EP-level unified streams, and external allocators, with compatibility back to ONNX Runtime 1.24.4 via version-gated callbacks.
   • ›Adds plugin profiling APIs and memory statistics in profiling output.
   • ›Adds CUDA Graph capture and replay, including user compute stream support, synchronization controls, and a kernel sync stream for scratch allocation.
   • ›Adds Windows ARM64 packages for the CUDA Plugin EP.
   +5 moreshow less

   • ›Adds arena allocation, resource accounting, available-resource reporting, and IOBinding synchronization.
   • ›Expands operator coverage with NHWC hardening, cuDNN SDPA for MHA/GQA, quantized MoE kernels, block-quantized FP4 MatMul, and native NVFP4 QMoE prefill on SM120.
   • ›Adds cuDNN-free implementations of ArgMax, ArgMin, and ReduceSum.
   • ›Reduces runtime dependencies by making cuDNN and cuFFT optional and removing the nvrtc link requirement.
   • ›Supports CUDA 12.8/13, SM90, and SM120 in build and CI coverage.
6. v1.29.0 Aug 12, 2026 · issue -007

   ONNX Runtime v1.29.0 adds env-var thread-pool control, NVFP4 QMoE, PagedAttention with quantized KV cache, and broad WebGPU operator expansion.

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch v1.29.0 https://github.com/microsoft/onnxruntime.git
   # already have the repo? check out this version:
   $ git checkout v1.29.0

   └──▷ TRY IT
   Cap ONNX Runtime to a fixed thread count at startup to prevent a latency-sensitive service from consuming all cores.
   copy

   $ export ORT_INTRA_OP_NUM_THREADS=4
   export ORT_INTER_OP_NUM_THREADS=2
   python my_inference_server.py

   Opt out of POSIX telemetry on Linux before any session is created, e.g. in a privacy-sensitive deployment.
   copy

   $ export ORT_DISABLE_TELEMETRY=1
   python run_inference.py

   • ›Adds ORT_INTRA_OP_NUM_THREADS and ORT_INTER_OP_NUM_THREADS environment variables to set default intra-op and inter-op thread-pool sizes at process startup; explicit thread settings still take precedence, and 0 preserves machine-sized defaults.
   • ›Adds ORT_DISABLE_TELEMETRY=1 environment variable to disable new POSIX telemetry (now available on Linux, macOS, Android, and iOS when built with telemetry enabled) before initialization.
   • ›Adds --data_shape flag to onnxruntime_perf_test for multi-shape profiling in a single run.
   • ›Adds MatMulBlockQuantizedFp4Weight and MatMulBlockQuantizedFp8Weight contrib operators on the CUDA EP, with block-scaled tensor-core/GEMV decode paths, packed FP4 decode, M-tiling, and folded W8A8 activation QDQ.
   • ›Adds LinearAttentionGate, GatedRMSNorm, and GatedAdd contrib operators on the CUDA EP.
   +14 moreshow less

   • ›Adds fused MRotaryEmbedding contrib operator for Qwen mRoPE variants, with ONNX-domain rotary embedding support.
   • ›Adds PagedAttention CUDA kernel with quantized KV cache, XQA decode, MLA, QK-Norm, and head-sink support.
   • ›Extends quantized KV-cache support with attention sinks, independent and per-channel scales, sliding-window cache, and a fused K/V dequantization launch on the CUDA EP.
   • ›Adds a cuDNN SDPA decode tier to the standard ONNX Attention CUDA kernel and enables cuDNN SDPA for the contrib Attention operator.
   • ›Adds attention_bias support to the GroupQueryAttention unfused path and state_window support to LinearAttention and CausalConvWithState for MTP on the CUDA EP.
   • ›Adds NVFP4 QMoE support on the CUDA EP, including native FP4xFP4 prefill on SM120, faster decode GEMV, fused routing/finalization paths, and reduced activation and weight-dequantization overhead.
   • ›Adds bfloat16 support to AllReduce, AllGather, and AllToAll collective operators on the CUDA EP.
   • ›Adds initial WebGPU PagedAttention implementation and moves Softmax and non-flash Attention to online algorithms on the WebGPU EP.
   • ›Adds DFT, HardSwish, Max/Min, Trilu, GRU, PRelu, MatMulBnb4, and MRotaryEmbedding operator support to the WebGPU EP.
   • ›Expands integer-type support across Clip, Reshape, Cast, Add, Tile, Concat, Expand, Gather, CumSum, Max, and Min on the WebGPU EP.
   • ›Adds and extends Intel subgroup-matrix MatMul/Gemm kernels on the WebGPU EP, including f16, batched-B, and odd-N support.
   • ›Adds weightless-model support for all initializer types and allows zero-input EpContext nodes, with maximum-shape inference wired into workspace estimation.
   • ›Adds device-free compile-only sessions on the WebGPU EP for offline graph transformation.
   • ›Expands CUDA plugin EP packaging to include a Windows ARM64 package with size options, with aligned architecture selections across Python, C API, TensorRT, Node.js, and plugin packages.
7. v1.29.0 Aug 12, 2026 · issue 002

   ONNX Runtime v1.29.0 adds env-var thread-pool sizing, POSIX telemetry, PagedAttention, NVFP4 QMoE, and broad WebGPU operator expansion.

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch v1.29.0 https://github.com/microsoft/onnxruntime.git
   # already have the repo? check out this version:
   $ git checkout v1.29.0

   └──▷ TRY IT
   Cap ONNX Runtime to a fixed thread count on a shared inference host to avoid competing with other processes for CPU cores.
   copy

   $ export ORT_INTRA_OP_NUM_THREADS=4
   export ORT_INTER_OP_NUM_THREADS=2
   python my_inference_script.py

   Disable POSIX telemetry on a Linux deployment before the ONNX Runtime session is created.
   copy

   $ export ORT_DISABLE_TELEMETRY=1
   python my_inference_script.py

   Profile a model across multiple input shapes in one pass to catch shape-dependent performance regressions in CI.
   copy

   $ onnxruntime_perf_test --data_shape 1x3x224x224 --data_shape 8x3x224x224 model.onnx

   • ›Adds ORT_INTRA_OP_NUM_THREADS and ORT_INTER_OP_NUM_THREADS environment variables to set default intra-op and inter-op thread-pool sizes at process startup; explicit thread settings still take precedence, and 0 preserves machine-sized defaults.
   • ›Adds ORT_DISABLE_TELEMETRY=1 environment variable to disable non-Windows POSIX telemetry (now available on Linux, macOS, Android, and iOS when built with telemetry enabled) before initialization; WebAssembly remains telemetry-free.
   • ›Adds --data_shape flag to onnxruntime_perf_test to enable multi-shape profiling in a single run.
   • ›Adds PagedAttention contrib operator to the CUDA EP with quantized KV cache, XQA decode, MLA, QK-Norm, and head-sink support.
   • ›Adds MatMulBlockQuantizedFp4Weight and MatMulBlockQuantizedFp8Weight contrib operators to the CUDA EP with block-scaled tensor-core/GEMV decode paths, packed FP4 decode, and M-tiling.
   +14 moreshow less

   • ›Adds LinearAttentionGate, GatedRMSNorm, and GatedAdd contrib operators to the CUDA EP.
   • ›Adds NVFP4 QMoE support to the CUDA EP, including native FP4xFP4 prefill on SM120, faster decode GEMV, and fused routing/finalization paths.
   • ›Extends quantized KV-cache support in the CUDA EP with attention sinks, independent and per-channel scales, sliding-window cache, and a fused K/V dequantization launch.
   • ›Adds a cuDNN SDPA decode tier to the standard ONNX Attention CUDA kernel and enables cuDNN SDPA for contrib Attention.
   • ›Adds attention_bias support to the GroupQueryAttention unfused path and state_window support to LinearAttention and CausalConvWithState for MTP.
   • ›Adds bfloat16 support to AllReduce, AllGather, and AllToAll CUDA collective operators.
   • ›Adds ONNX-domain support for rotary embedding and a fused MRotaryEmbedding contrib operator for Qwen mRoPE variants.
   • ›Adds weightless-model support for all initializer types and allows zero-input EpContext nodes.
   • ›Adds WebGPU device-free compile-only sessions for offline graph transformation.
   • ›Adds initial WebGPU PagedAttention implementation and moves Softmax and non-flash Attention to online algorithms.
   • ›Adds DFT, HardSwish, Max/Min, Trilu, GRU, PRelu, MatMulBnb4, and MRotaryEmbedding operator support to the WebGPU EP.
   • ›Expands integer-type support across Clip, Reshape, Cast, Add, Tile, Concat, Expand, Gather, CumSum, Max, and Min in the WebGPU EP.
   • ›Adds and extends Intel subgroup-matrix MatMul/Gemm kernels in the WebGPU EP, including f16, batched-B, and odd-N support.
   • ›Expands CUDA plugin EP packaging to include a Windows ARM64 package with size options, aligned across Python, C API, TensorRT, Node.js, and plugin packages.
8. plugin-ep-webgpu/v0.2.1 Jul 30, 2026 · issue -020

   WebGPU EP v0.2.1 adds 2-bit GatherBlockQuantized, Gemma 4 support, Qwen3 fusions, and expanded FlashAttention capabilities.

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch plugin-ep-webgpu/v0.2.1 https://github.com/microsoft/onnxruntime.git
   # already have the repo? check out this version:
   $ git checkout plugin-ep-webgpu/v0.2.1

   • ›Adds 2-bit support to GatherBlockQuantized for more aggressive quantized-path compression on WebGPU.
   • ›Adds Opset 24 and KV-shared decoder layer support for Gemma 4 model paths.
   • ›Adds QKV and MLP fusions for Qwen3-style models, plus Q/K RMSNorm fusion into GroupQueryAttention.
   • ›Extends FlashAttention decode kernels to handle any sequence length, removing previous length constraints.
   • ›Adds QKV bias support for FlashAttention in MultiHeadAttention.
   +6 moreshow less

   • ›Enables dynamic max_k_step for NVIDIA hardware in the FlashAttention path.
   • ›Adds M4 Max-specific FlashAttention optimizations.
   • ›Generalizes the FlashAttention prefill shared-memory path for broader hardware coverage.
   • ›GroupQueryAttention now supports optional present-key/value outputs.
   • ›Adds LinearAttention subgroup optimizations and larger tile_v with subgroup support.
   • ›Introduces per-graph buffer manager and session-level buffer pool for graph-capture reuse.
9. plugin-ep-webgpu/v0.2.1 Jul 30, 2026 · issue 002

   WebGPU EP v0.2.1 adds 2-bit GatherBlockQuantized, Gemma 4 / Qwen3 fusions, and extended FlashAttention kernels.

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch plugin-ep-webgpu/v0.2.1 https://github.com/microsoft/onnxruntime.git
   # already have the repo? check out this version:
   $ git checkout plugin-ep-webgpu/v0.2.1

   • ›Extends FlashAttention decode kernels to support any sequence length, enabling longer-context LLM inference on the WebGPU EP.
   • ›Adds QKV bias support to FlashAttention in MultiHeadAttention.
   • ›Enables dynamic max_k_step for NVIDIA GPUs in the FlashAttention path.
   • ›Generalizes the FlashAttention prefill shared-memory path for broader hardware coverage.
   • ›Lands M4 Max-specific FlashAttention optimization for Apple Silicon.
   +7 moreshow less

   • ›Adds QKV and MLP op fusions for Qwen3-style models.
   • ›Adds Q/K RMSNorm fusion into GroupQueryAttention for Qwen3-style models.
   • ›Adds Opset 24 and KV-shared decoder layer support for Gemma 4 models.
   • ›GroupQueryAttention now supports optional present-key/value outputs.
   • ›GatherBlockQuantized gains 2-bit quantization support, expanding low-bit inference options.
   • ›Adds LinearAttention subgroup optimizations and larger tile_v with subgroup support.
   • ›Introduces per-graph buffer manager and session-level buffer pool for graph-capture reuse.
10. v1.28.0 Jul 25, 2026 · issue -025

    ONNX Runtime v1.28.0 adds CudaQuantizer, OrtModelPackageApi experimental C API, Linux NPU sysfs discovery, and CUDA EP footprint reductions.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.28.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.28.0

    • ›Adds OrtErrorCode::ORT_DEVICE_RESET to the OrtErrorCode enum, with documentation single-sourced so StatusCode stays in sync.
    • ›Adds model_external_initializers_file_folder_path session option, now honored for file-path model loads.
    • ›Adds CudaQuantizer to onnxruntime.quantization for GPU-side quantization workflows.
    • ›Adds a Python API for HOST_ACCESSIBLE OrtValue allocation.
    • ›Introduces experimental OrtModelPackageApi in the C/C++ experimental API surface, including authoring tools, schema versioning, and folding external_data into session options — note this API may change in future releases.
    +14 moreshow less

    • ›CUDA EP: cuDNN and cuFFT are now optional at runtime and nvrtc is no longer linked, significantly reducing the required CUDA redistributable footprint.
    • ›CUDA plugin EP gains support for user compute stream with CUDA graph, kernel sync stream exposed for scratch allocation, and Windows ARM64 packages.
    • ›Adds Linux NPU discovery through sysfs accel devices.
    • ›Adds an API to select the best compiled-model compatibility info from candidate strings.
    • ›Adds crypto support: applications can supply I/O callbacks to an EP, with callback and fallback helpers.
    • ›Implements name-based partitioning in the plugin EP framework.
    • ›Registers Flatten as a Direct8Bit op in the Python QDQ static quantizer.
    • ›Adds Float16, BFloat16, and Float8 support in the TensorArray custom op.
    • ›Adds memory statistics to profiling output.
    • ›Adds EP version logging on inference failure, in the EpDeviceUsage event, and ORT version logging.
    • ›User-supplied external initializers are now used in place when already on the planned device, avoiding unnecessary copies.
    • ›Upgrades to ONNX 1.22.0 and protobuf 6.33.5.
    • ›NPM packages are now published from the CUDA 13 pipeline.
    • ›WebGPU plugin EP version bumped to 0.3.0.
    └──▷ BREAKING ON UPGRADE

    • !CUDA_QUANT_PREPROCESS is now off by default; workflows relying on the previous default-on behavior must explicitly enable it.
    • !The TensorRT fused causal attention kernels were removed from the CUDA EP; models depending on them will need an alternative path.
    • !The dynamic WGSL generator (duktape/Node) path was removed; the Python wgsl-gen implementation is now the only supported path.
    • !OrtModelPackageApi now lives in the experimental C API and may change in future releases; callers should treat its signatures as unstable.
    • !SkipLayerNorm strict mode is deprecated and may be removed in a future release.
11. v1.28.0 Jul 25, 2026 · issue 002

    ONNX Runtime v1.28.0 adds CudaQuantizer, HOST_ACCESSIBLE OrtValue Python API, ORT_DEVICE_RESET error code, and CUDA plugin EP gains user compute stream with CUDA graph support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.28.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.28.0

    • ›Adds CudaQuantizer to onnxruntime.quantization for GPU-side quantization workflows.
    • ›Adds a Python API for HOST_ACCESSIBLE OrtValue allocation, enabling CPU-accessible device memory from Python.
    • ›Adds OrtErrorCode::ORT_DEVICE_RESET to the C API error code surface, with single-sourced StatusCode sync.
    • ›Adds model_external_initializers_file_folder_path session option, now honored for file-path model loads.
    • ›CUDA plugin EP now supports user-supplied compute streams with CUDA graph and exposes a kernel sync stream for scratch allocation.
    +12 moreshow less

    • ›Adds crypto support to plugin EPs: applications can supply I/O callbacks with callback and fallback helpers.
    • ›Adds Linux NPU discovery through sysfs accel devices.
    • ›Implements name-based graph partitioning for plugin EPs.
    • ›Adds memory statistics to profiling output.
    • ›Adds EP version logging on inference failure via the EpDeviceUsage event.
    • ›Registers Flatten as a Direct8Bit op in the Python QDQ static quantizer.
    • ›Adds Float16, BFloat16, and Float8 support in the TensorArray custom op.
    • ›cuDNN and cuFFT are now optional at runtime for the CUDA EP, and nvrtc is no longer linked, significantly reducing the CUDA redistributable footprint.
    • ›WebGPU plugin EP bumped to version 0.3.0.
    • ›NPM packages are now published from the CUDA 13 pipeline.
    • ›Adds an API to select the best compiled-model compatibility info from candidate strings.
    • ›User-supplied external initializers are now used in-place when already on the planned device, avoiding unnecessary copies.
    └──▷ BREAKING ON UPGRADE

    • !CUDA_QUANT_PREPROCESS is now off by default; workflows that relied on the previous default-on behavior must explicitly enable it.
    • !The TensorRT fused causal attention kernels were removed from the CUDA EP; models relying on them will no longer use that path.
    • !The dynamic WGSL generator (duktape/Node) path was removed; WebGPU users must use the Python wgsl-gen implementation instead.
    • !SkipLayerNorm strict mode is deprecated and its behavior may change in future releases.
12. v1.27.0 Jun 19, 2026 · issue -061

    ONNX Runtime v1.27.0 adds FLOAT8E8M0, quantized KV-cache, calibration cache, new EP plugin APIs, and broad CoreML/CUDA op coverage.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.27.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.27.0

    └──▷ USE IT
    Use the calibration cache with static quantization to skip re-running calibration on repeated quantization runs of the same model.
    python copy

    from onnxruntime.quantization import quantize_static, CalibrationDataReader
    
    quantize_static(
        model_input='model.onnx',
        model_output='model_quantized.onnx',
        calibration_data_reader=reader,
        calibration_cache_path='calib_cache.json'
    )

    • ›Adds OrtEp::OnSessionInitializationEnd() callback to the Execution Provider Plugin API for post-initialization hooks.
    • ›Adds plugin EP session-options getters to the Execution Provider Plugin API.
    • ›Adds CUDA Plugin EP provider options for streams and external allocators.
    • ›Adds kOrtEpDevice_EpMetadataKey_OSDriverVersion metadata key with example and docs for EP device metadata.
    • ›Adds calibration cache support to quantize_static in the quantization tooling.
    +29 moreshow less

    • ›Adds ActivationRestrictedAsymmetric quantization option.
    • ›Adds opset-21 block_size attribute support to QDQ quantization.
    • ›Adds PEP 561 py.typed marker to the onnxruntime Python package, enabling type-checker support.
    • ›Makes sympy an optional runtime dependency in the Python package.
    • ›Adds zero-copy I/O for plugin EPs with HOST_ACCESSIBLE memory.
    • ›Adds FLOAT8E8M0 datatype support to ONNX Runtime core, including CPU Cast ops.
    • ›Adds support for ONNX overloaded functions (IR v10+).
    • ›Adds CPU fallback for FusedAdam optimizer in ORT Training.
    • ›Adds chunked dequant+GEMM for MatMulNBits on CUDA EP to reduce peak GPU memory.
    • ›Adds ConvTranspose-22 support on the CUDA EP.
    • ›Fills CUDA opset gaps for LSTM, RNN, Reshape, Cast, Round/Equal, ReduceMax/ReduceMin, Sin/Cos, and Random* ops.
    • ›Adds LpNormalization support on the CUDA EP.
    • ›Adds LSTM support on the WebGPU EP.
    • ›Adds QKV and MLP layer fusions for Qwen3-style models on the WebGPU EP.
    • ›Adds QKV bias support in FlashAttention for MultiHeadAttention on the WebGPU EP.
    • ›Adds shader dump-to-file environment variable for WebGPU EP debugging.
    • ›Adds opset-24 and KV-shared decoder support (Gemma 4) on the WebGPU EP.
    • ›Adds FusedConv, Identity, Ceil, Tile, Cast(bool), Sin, Cos, and GatherND op support on the CoreML EP.
    • ›Adds support for pre-opset-13 Split (attribute path) and scalar Gather indices on the CoreML EP.
    • ›Adds NHWC convolution path in MLAS to reduce transpose overhead.
    • ›Adds CPU QMoE 2-bit support with LUT GEMM fast path.
    • ›Adds quantized KV-cache support for CPU GroupQueryAttention with SIMD optimizations and tiled compute.
    • ›Adds RVV-optimized NCHWc convolution/pooling and LLM operators for RISC-V targets.
    • ›Parallelizes CPU ScatterElements and optimizes MatMulNBits 2-bit float-zero-point path.
    • ›Adds optional bias support in MatMulNBits CPU LUT GEMM path.
    • ›Adds DiT attention fusion for F5-TTS and diffusion transformer models.
    • ›Adds support for optional present_key/present_value outputs in GQA and Gemma4.
    • ›Adds diagnostics for DML failure paths.
    • ›Adds per-graph buffer manager for multi-graph capture on the WebGPU EP.
    └──▷ BREAKING ON UPGRADE

    • !CUDA 12 package files are now explicitly named as such — existing references to unversioned CUDA package names will need to be updated.
13. plugin-ep-webgpu/v0.1.0 May 29, 2026 · issue -082

    ONNX Runtime ships the WebGPU Execution Provider as a standalone plugin (onnxruntime_providers_webgpu) compatible with ORT 1.24.4+.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch plugin-ep-webgpu/v0.1.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout plugin-ep-webgpu/v0.1.0

    • ›Introduces the onnxruntime_providers_webgpu plugin EP, registered into an existing ONNX Runtime 1.24.4+ installation at runtime via onnxruntime.register_execution_provider_library(...) (Python) or OrtEnv.RegisterExecutionProviderLibrary(...) (.NET) — no core ORT package swap required.
    • ›Adds the onnxruntime-ep-webgpu Python wheel (PyPI: onnxruntime-ep-webgpu) for installing the WebGPU EP alongside the onnxruntime package.
    • ›Adds the Microsoft.ML.OnnxRuntime.EP.WebGpu NuGet package for .NET, referenced alongside Microsoft.ML.OnnxRuntime.
    • ›Ships cross-platform native binaries for Windows x64/arm64 (bundled with dxil.dll and dxcompiler.dll), Linux x64, and macOS arm64.
    • ›Provides WebGPU kernels for transformer and LLM contrib ops: Attention, MultiHeadAttention, GroupQueryAttention, RotaryEmbedding, SkipLayerNormalization, SkipSimplifiedLayerNormalization, SimplifiedLayerNormalization, BiasAdd, BiasGelu, BiasSplitGelu, FastGelu, Gelu, QuickGelu, CausalConvWithState, and LinearAttention.
    +4 moreshow less

    • ›Supports quantized inference via MatMulNBits (with DP4A and subgroup-matrix acceleration paths), DequantizeLinear, GatherBlockQuantized, and QMoE (quantized Mixture-of-Experts) on WebGPU.
    • ›Includes a FlashAttention kernel and Intel-optimized MatMul/Gemm code paths for improved throughput on Intel GPUs.
    • ›Adds program caching to amortize WebGPU shader compilation costs across inference runs.
    • ›Adds optional PIX frame capture and WebGPU profiler integration for GPU performance investigation.
14. v1.26.0 May 8, 2026 · issue -103

    ONNX Runtime v1.26.0 adds RISC-V Vector CPU support, CUDA plugin graph/profiling APIs, WebGPU GridSample, and CoreML new ops.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.26.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.26.0

    • ›Adds optional memory mapping for .ort model loads, reducing load-time I/O overhead.
    • ›Adds RISC-V Vector (RVV) support for the CPU Execution Provider, enabling inference on RVV-capable hardware.
    • ›Adds graph capture/replay support and a profiling API to the CUDA plugin EP.
    • ›Adds sync support for IOBinding in the CUDA plugin EP.
    • ›Adds GridSample operator support to the WebGPU EP.
    +12 moreshow less

    • ›Extends WebGPU Split-K support to batch size > 1, improving batched matrix-multiply throughput.
    • ›Adds HardSigmoid and QuickGelu operator support to the CoreML EP.
    • ›Adds Pad reflect mode support to the CoreML EP.
    • ›Adds CUDA TopK support for INT8, INT16, and UINT8 data types.
    • ›Adds CUDA LabelEncoder support for numeric types.
    • ›Adds CUDA QuantizeLinear/DequantizeLinear opset 25 support.
    • ›Extends CUDA Transpose support to opsets 23–25.
    • ›Adds FP32 QK accumulation path for unfused GQA attention on CUDA.
    • ›Adds FP16 Gelu enablement in MLAS.
    • ›Adds Arm64 BF16 fast-math convolution kernels for NCHW/NCHWc paths in MLAS.
    • ›Upgrades OpenVINO EP for the 1.26.0 development release.
    • ›Replaces unrestricted Python setattr configuration with an allowlist, hardening the session configuration surface.
    └──▷ BREAKING ON UPGRADE

    • !The WebNN pool2d property roundingType is renamed to outputShapeRounding; any code or config referencing roundingType will break.
    • !Support for CUDA 12 will be removed in v1.27.0; the published CUDA 13 package is onnxruntime-<os>-<arch>-gpu_cuda13-<version>.<ext>.
15. v1.25.1 Apr 27, 2026 · issue -114

    ONNX Runtime 1.25.1 adds LinearAttention, CausalConvState, RotaryEmbedding, and RMSNorm operators with Qwen3.5 WebGPU support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.25.1 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.25.1

    • ›Adds LinearAttention and CausalConvState operators to support Qwen3.5 model inference.
    • ›Adds RotaryEmbedding (RotEMB) and RMSNorm operators.
    • ›Adds Linear Attention signature support.
    • ›Updates Reshape to opset 25 and Transpose to opset 24.
    • ›Enables Qwen3.5 model execution on the WebGPU execution provider.
    +1 moreshow less

    • ›Optimizes QMoE 1-token decode path on WebGPU via fused operations to reduce GPU dispatches.
16. v1.25.0 Apr 20, 2026 · issue -121

    ONNX Runtime 1.25 ships a CUDA Plugin EP, new Attention opsets 23/24 on CUDA, per-session profiling in RunOptions, and expanded WebGPU op coverage.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.25.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.25.0

    • ›Adds enable_profiling to RunOptions for per-session profiling control without a global session option.
    • ›Adds KernelInfo string-array attribute APIs for C and C++, enabling custom ops to read string-array attributes via the C and C++ EP API.
    • ›Adds OpKernelInfo::GetConfigOptions and LoggingManager::HasDefaultLogger() to the EP API adapter.
    • ›Adds OrtModel input support to the Compile API, allowing compiled model workflows to accept OrtModel objects directly.
    • ›Adds a session config key to create weightless EPContext models during compilation.
    +27 moreshow less

    • ›Introduces the CUDA Plugin EP — a new core implementation allowing third-party CUDA-backed execution providers to be delivered as dynamically loaded plugins without rebuilding ORT.
    • ›Adds BFC-style arena and CUDA mempool allocators to the CUDA Plugin EP for stream-aware memory management.
    • ›Adds a Plugin EP Sync API for synchronous execution.
    • ›Adds Plugin EP event profiling APIs.
    • ›Adds Plugin EP APIs to retrieve ONNX operator schemas.
    • ›Adds annotation-based graph partitioning with resource accounting to the EP plugin API.
    • ›Adds per-session thread pool work callbacks API to core APIs.
    • ›Introduces Model Package support (preview): automatically selects compiled EPContext model variants from a packaged collection based on EP, device, and hardware constraints.
    • ›Implements Attention opset 23 on CUDA with GQA, boolean masks, softcap, and softmax precision control.
    • ›Implements Attention opset 24 on CUDA (disjoint from contrib op) and nonpad KV sequence length on CPU.
    • ›Adds TensorScatter-24 op support for CPU and CUDA.
    • ›Adds DeformConv op support for CPU and CUDA.
    • ›Adds LpNormalization-22 op support.
    • ›Fills CUDA opset gaps: Flatten, Identity, If, Loop, Scan, ConstantOfShape, Size (opsets 21/23); GlobalAveragePool/GlobalMaxPool (→22); Shape (→25); Squeeze/Unsqueeze (→25); TopK (→24, BF16); GRU (→22); Pad (→25, wrap mode); Resize v19; RoiAlign v16/v22.
    • ›Adds GQA with XQA and quantized KV cache to the NVIDIA CUDA EP, including FP8 (E4M3) KV cache support.
    • ›Adds CUDA graph capture compatibility for LLM ops and pre-compiled paths in the CUDA EP.
    • ›Adds volumetric (3D) GridSample support to the CUDA EP.
    • ›Adds optional router_weights input to the QMoE contrib op on CUDA.
    • ›Adds D3D12 external resource import support to the NVIDIA TensorRT RTX EP.
    • ›Adds TopK, Softplus, and Identity op support to the WebGPU EP.
    • ›Adds Conv3D and LpNorm op support to the WebGPU EP.
    • ›Adds int64/bool type support for Range, Expand, Flatten, Gather, and Unsqueeze in the WebGPU EP.
    • ›Optimizes WebGPU Gemm/MatMul using subgroup features.
    • ›Adds 2-bit zero-point support and higher K-parallelism to MatMulNBits in the WebGPU EP.
    • ›Adds head_sink support and configurable multi-rotary cache concat offset to Flash Attention in the WebGPU EP.
    • ›Makes WebGPU EP compatible with the EP plugin API.
    • ›Upgrades bundled ONNX to 1.21.0.
    └──▷ BREAKING ON UPGRADE

    • !Building ORT from source now requires C++20; minimum toolchains are MSVC 19.29+, GCC 10+, Clang 10+. Prebuilt-package users are unaffected.
    • !CUDA minimum version raised to 12.0 — CUDA 11.x is no longer supported. Users pinned to CUDA 11.x must stay on ORT 1.24.x or upgrade their CUDA toolkit and driver.
    • !ArmNN EP has been removed. Any --use_armnn build flags must be removed; workloads should migrate to the MLAS/KleidiAI-backed CPU EP or QNN EP.
    • !ORT_API_VERSION is now 25, which may affect EP plugins or native bindings that check the API version at load time.
17. v1.24.3 Mar 5, 2026 · issue -166

    ONNX Runtime v1.24.3 adds DQ→MatMulNBits fusion for NvTensorRtRtx, OrtEnv.DisableDllImportResolver for NuGet, and DynamicQGemm ukernel in MLAS.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.24.3 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.24.3

    • ›Adds OrtEnv.DisableDllImportResolver API to prevent fatal errors when a DLL import resolver conflict is detected in NuGet environments.
    • ›Adds 'library_path' metadata entry to OrtEpDevice instances for plugin and provider bridge execution providers.
    • ›Adds DynamicQGemm function pointers and ukernel interface to MLAS.
    • ›Adds DQ→MatMulNBits fusion transformer to the NvTensorRtRtx execution provider.
    • ›Increases kMaxValueLength to 8192 in config.
    +5 moreshow less

    • ›Builds Windows ARM64X binaries as part of the packaging pipeline.
    • ›Adds Python 3.14 CI support with upgraded dependencies.
    • ›Delivers up to 4x QMoE CPU performance improvement on 4-bit workloads.
    • ›Supports using the embedded WASM module in Blob URL workers when wasmBinary is provided in WebGPU EP.
    • ›Allows a new memory info name for WebGPU EP.
18. v1.24.1 Feb 6, 2026 · issue -192

    ONNX Runtime v1.24.1 adds plugin EP APIs, free-threaded Python 3.13t/3.14t, CUDA Graph on by default, and broad new op coverage across CUDA, QNN, WebGPU, and CPU.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.24.1 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.24.1

    └──▷ TRY IT
    Build ONNX Runtime from source with NCHWc layout enabled to improve Conv model inference throughput on ARM NEON hardware.
    copy

    $ ./build.sh --config Release --enable_arm_neon_nchwc

    • ›Adds OrtApi::CreateEnvWithOptions() and OrtEpApi::GetEnvConfigEntries() for programmatic environment configuration in plugin execution providers.
    • ›Adds OrtKernelInfo APIs for kernel-based plugin EPs, enabling kernel authors to query session configuration from KernelInfo.
    • ›Adds add_external_initializers_from_files to the Python binding for loading external initializers directly from file paths.
    • ›Adds --enable_arm_neon_nchwc build flag to enable NCHWc layout support for Conv models on ARM NEON, improving convolutional inference performance.
    • ›Enables CUDA Graph by default in the CUDA EP, reducing per-inference launch overhead without additional configuration.
    +19 moreshow less

    • ›Introduces a plugin-based Execution Provider (EP) infrastructure with dynamic loading, including initial kernel support, weight pre-packing, EP context model support, and control flow kernel APIs.
    • ›Adds EP Device Compatibility APIs for matching execution providers to available hardware at runtime.
    • ›Adds External Resource Importer API for importing D3D12 shared resources into the DirectML/DX12 execution path.
    • ›Adds Python 3.14 support and free-threaded Python (PEP 703) support for Python 3.13t and 3.14t on Linux.
    • ›Upgrades ONNX dependency to 1.20.1 and Protobuf from 3.20.3 to 4.25.8.
    • ›Upgrades ORT_API_VERSION to 24.
    • ›Adds Java bindings for Auto EP, compiled model support (OrtCompiledModelCompatibility), and compile model APIs.
    • ›Adds .NET 9.0 MAUI targets to the C# binding.
    • ›CUDA EP gains Flash Attention updates, GQA kernel fusion, BF16 support for MoE/qMoE/MatMulNBits, and CUDA 13.0 support.
    • ›TensorRT EP upgraded to TensorRT 10.14 with automatic plugin loading and NVFP4 custom ops.
    • ›TensorRT RTX EP gains RTX runtime caching, CUDA graph support, BFloat16, and memory-mapped engine support.
    • ›QNN EP upgraded to SDK 2.42.0, adding RMSNorm, ScatterElements, GatherND, STFT, and RandomUniformLike ops, plus Gelu pattern fusion, LPBQ quantization, ARM64 wheel builds, and v81 device support.
    • ›OpenVINO EP upgraded to 2025.4.1.
    • ›MIGraphX EP adds QuickGelu, multihead attention, and QLinear pooling ops.
    • ›WebGPU EP gains Flash Attention optimizations, graph capture, Split-K MatMul, qMoE support, and WGSL templates.
    • ›WebNN EP adds GQA local attention, GatherBlockQuantized, ConvInteger, and MatMulInteger ops.
    • ›Node.js binding updated to Node.js v22 with JSI for React Native and JSPI build support.
    • ›CPU backend adds MoE/qMoE kernels, RotaryEmbeddings (opset 23), LayerNorm/RMSNorm broadcasting, SME1/SME2 Convolution and SGemm kernels via KleidiAI, S390x SIMD, and LoongArch64 4-bit quantization.
    • ›Arm NN EP is formally deprecated and will be removed in an upcoming release; all --use_armnn build options will be dropped.
    └──▷ BREAKING ON UPGRADE

    • !Python 3.10 wheels are no longer published; upgrade to Python 3.11 or later is required.
    • !x86_64 binaries for macOS and iOS are no longer provided, and the minimum macOS version is raised to 14.0.
    • !Builds using --use_armnn will fail once the upstream ArmNN EP removal lands (release TBD); adjust configurations in advance.
19. v1.23.1 Oct 8, 2025 · issue -313

    ONNX Runtime v1.23.1 adds new Python and C# APIs, CPU MoE kernels, and a dynamic QNN HTP performance mode option.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.23.1 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.23.1

    • ›Adds a dynamic option to set HTP performance mode in the QNN EP (QNN EP).
    • ›Adds MemcpyFromHost and MemcpyToHost support for plugin execution providers.
    • ›Adds CPU Mixture-of-Experts (MoE) kernel for CPU execution.
    • ›Adds block-wise QMoE kernel for CPU execution.
    • ›Implements new Python APIs.
    +3 moreshow less

    • ›Implements missing C# APIs.
    • ›Adds session_id_ to LogEvaluationStart, LogEvaluationStop, and LogSessionCreationStart for improved session tracing.
    • ›Adds EP ABI check to verify whether nodes specified in GetCapability() have already been assigned.
20. v1.23.0 Sep 26, 2025 · issue -324

    ONNX Runtime v1.23.0 adds an EP Plugin API, dynamic EP downloading on Windows 11 25H2, and a new Nvidia TensorRT RTX execution provider.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.23.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.23.0

    • ›Introduces the Execution Provider (EP) Plugin API, a new infrastructure for building plugin-based execution providers.
    • ›Adds dynamic download and installation of execution providers via the WinML build on Windows 11 version 25H2 or later; C/C++/C# users consume this through the Windows App SDK, Python users via the onnxruntime-winml package.
    • ›Adds AutoEP/Device Management capability for automatic discovery of computing devices and selection of the best execution providers to download and register.
    • ›Adds a new Nvidia TensorRT RTX execution provider.
    • ›Adds support for QNN SDK 2.37 in the QNN EP.
    +4 moreshow less

    • ›Adds WGSL template support to the WebGPU EP.
    • ›Upgrades EMSCRIPTEN SDK (EMDSK) from 4.0.4 to 4.0.8 in the Web EP.
    • ›Enhances KleidiAI performance for SGEMM, IGEMM, and Dynamic Quantized MatMul operations, with improvements for Conv2D operators on hardware supporting SME2 (Scalable Matrix Extension v2).
    • ›Removes the ROCM EP from the source tree; AMD users are directed to the MIGraphX or Vitis AI EPs.
    └──▷ BREAKING ON UPGRADE

    • !The ROCM EP has been removed from the source tree; existing setups relying on it must migrate to the MIGraphX or Vitis AI EPs.
21. v1.22.2 Aug 13, 2025 · issue -363

    ONNX Runtime v1.22.2 adds --client_package_build, vectorized DequantizeLinear, and QNN EP operator support for Upsample, Einsum, LSTM, and CumSum.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.22.2 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.22.2

    • ›Adds --client_package_build build option that enables defaults tuned for client/on-device workloads, including disabling thread spinning by default.
    • ›Adds multithreaded, vectorized CPU/MLAS implementation of DequantizeLinear for int8 and uint8 inputs using SSE2 and NEON instruction sets.
    • ›Adds QNN EP support for the Upsample, Einsum, LSTM, and CumSum operators.
    • ›Enables DSP queue polling in QNN EP when performance mode is set to 'burst'.
    • ›Updates bundled QNN SDK to version 2.36.1 in the QNN execution provider.
    +1 moreshow less

    • ›Fuses scale into Softmax in the QNN EP for improved operator fusion.
22. v1.22.0 May 10, 2025 · issue -366

    ONNX Runtime v1.22.0 adds Model Editor, Compile, and Auto EP APIs plus a new TensorRT RTX execution provider.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.22.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.22.0

    • ›New OrtCompileApi struct enables explicit AOT compilation of ONNX models via a dedicated Compile API.
    • ›New OrtModelEditorApi struct exposes an API for programmatically creating and editing ONNX models in-process.
    • ›Adds Auto EP Selection infrastructure that automatically chooses Execution Providers via configurable selection policies, reducing manual EP configuration.
    • ›Introduces the NV TensorRT RTX Execution Provider, a new EP targeting Nvidia RTX GPUs via TensorRT.
    • ›Adds support for MatMulNBits (8-bit weight-only quantization) in both the CPU/MLAS and CUDA EPs.
    +7 moreshow less

    • ›Integrates KleidiAI into ONNX Runtime/MLAS for improved performance on Arm architectures.
    • ›Adds constrained decoding support for generative AI models, giving finer control over output token selection.
    • ›QNN EP gains support for QNN SDK 2.33.2, operator coverage for Sum, Softmax, Upsample, Expand, ScatterND, and Einsum, QnnGpu backend, and the ability to build as a shared or static library.
    • ›TensorRT EP adds support for TensorRT 10.9, a new EP option to enable TRT Preview Features, and support for loading TensorRT V3 plugins.
    • ›OpenVINO EP adds support up to OpenVINO 2025.1, Intel compiler-level optimizations for QDQ models, device selection by LUID, and Load_config support for AUTO, HETERO, and MULTI plugins.
    • ›WebGPU support extended to the Node.js package on Windows and macOS, and enabled when building from source on macOS, Linux, and Windows.
    • ›QNN NuGet package is now built as ARM64x.
    └──▷ BREAKING ON UPGRADE

    • !CUDA 11.x GPU packages are no longer published; only CUDA 12.x packages are available.
    • !The minimum supported Windows version is now 10.0.19041.
23. v1.21.0 Mar 8, 2025 · issue -368

    ONNX Runtime v1.21.0 adds chat mode, TensorRT 10.8, QNN shared memory, OpenVINO weights sharing, and expanded tokenizer support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.21.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.21.0

    └──▷ TRY IT
    Build ONNX Runtime with the WebGPU EP using an external Dawn source and Vulkan backend enabled.
    copy

    $ cmake .. \
      -Donnxruntime_USE_EXTERNAL_DAWN=ON \
      -Donnxruntime_CUSTOM_DAWN_SRC_PATH=/path/to/dawn \
      -Donnxruntime_ENABLE_DAWN_BACKEND_VULKAN=ON \
      -Donnxruntime_BUILD_DAWN_MONOLITHIC_LIBRARY=ON

    • ›Introduces trt_op_types_to_exclude option to exclude specific ops from TensorRT assignment, giving fine-grained control over TensorRT EP graph partitioning.
    • ›Adds preload_dlls Python API for the CUDA EP to coexist with PyTorch without DLL conflicts.
    • ›Adds CMake option onnxruntime_BUILD_QNN_EP_STATIC_LIB for building QNN EP as a static library; QNN EP is now built as a shared library/DLL by default (use --use_qnn static_lib to retain old behavior).
    • ›Adds CMake options onnxruntime_USE_EXTERNAL_DAWN, onnxruntime_CUSTOM_DAWN_SRC_PATH, onnxruntime_BUILD_DAWN_MONOLITHIC_LIBRARY, onnxruntime_ENABLE_PIX_FOR_WEBGPU_EP, onnxruntime_ENABLE_DAWN_BACKEND_VULKAN, and onnxruntime_ENABLE_DAWN_BACKEND_D3D12 for WebGPU EP build configuration.
    • ›Enables onnxruntime_USE_CUDA_NHWC_OPS by default for CUDA builds.
    +19 moreshow less

    • ›Adds support for TensorRT 10.8, including default assignment of DDS ops NMS, RoiAlign, and NonZero to TensorRT.
    • ›Adds chat mode support for CPU, GPU, and WebGPU execution.
    • ›Adds support for decoder model pipelines.
    • ›Adds Java API support for MultiLoRA.
    • ›Introduces QNN shared memory support for the QNN EP.
    • ›Adds support for QAIRT/QNN SDK 2.31 and a Python 3.13 package for QNN EP.
    • ›Introduces OpenVINO EP Weights Sharing feature.
    • ›Adds contrib op support in OpenVINO EP: SkipLayerNormalization, MatMulNBits, FusedGemm, FusedConv, EmbedLayerNormalization, BiasGelu, Attention, DynamicQuantizeMatMul, FusedMatMul, QuickGelu, SkipSimplifiedLayerNormalization.
    • ›Adds support for caching generated CoreML models.
    • ›Expands tokenizer support to include ChatGLM, Baichuan2, and Phi-4 models.
    • ›Adds full Phi-4 pre/post-processing support for text, vision, and audio.
    • ›Introduces RegEx pattern loading from tokenizer.json.
    • ›Adds support for loading tokenizer data from a memory blob in the C API.
    • ›Introduces a new tokenizer op schema to unify the tokenizer codebase.
    • ›Adds support for WASM64 (build from source).
    • ›Increases minimum required CMake version from 3.26 to 3.28 and adds support for CMake 4.0.
    • ›Increases minimum required Python version from 3.8 to 3.10 for building ONNX Runtime from source.
    • ›Removes the onnxruntime_USE_PREINSTALLED_EIGEN CMake option.
    • ›Upgrades Gradle from 7.x to 8.x and JDK from 11 to 17 for Android builds.
    └──▷ BREAKING ON UPGRADE

    • !Chat mode introduces breaking API changes — see the migration guide at https://onnxruntime.ai/docs/genai/howto/migrate.html.
    • !QNN EP is now built as a shared library/DLL by default; builds that previously relied on static linking must add --use_qnn static_lib.
    • !All prebuilt Windows packages now require VC++ Runtime >= 14.40 (previously 14.38); lower versions will cause a crash on initialization.
    • !Minimum iOS SDK requirement raised to >= 15.1.
    • !Minimum Android API requirement raised to >= 24 (Android 7).
    • !All macOS packages now require macOS >= 13.3.
    • !Minimum CMake version increased from 3.26 to 3.28.
    • !Minimum Python version for building from source increased from 3.8 to 3.10.
    • !The onnxruntime_USE_PREINSTALLED_EIGEN CMake option has been removed.
    • !TVM EP has been removed from the source tree.
    • !NNAPI EP is marked for deprecation following Google's deprecation of NNAPI.
24. v1.20.1 Nov 21, 2024 · issue -372

    ONNX Runtime v1.20.1 adds get_qdq_config() helper, QNN SDK v2.28.2 support, and reworked native library packaging.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.20.1 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.20.1

    └──▷ USE IT
    Retrieve a ready-made QDQ configuration with a reduced quantization range to improve compatibility on narrow-range hardware.
    python copy

    from onnxruntime.quantization import get_qdq_config
    
    qdq_config = get_qdq_config(model_input='model.onnx', reduce_range=True)
    print(qdq_config)

    • ›Introduces get_qdq_config() helper function in the Python Quantization Tool to retrieve QDQ configurations programmatically.
    • ›Adds reduce_range option to get_qdq_config() for finer control over quantization range in QDQ workflows.
    • ›Adds support for QNN SDK v2.28.2 in the QNN Execution Provider.
    • ›Reworks native library loading so a pre-built ORT native package can be consumed without modification.
    • ›Updates QDQ operator handling for Pad, Slice, and Softmax nodes in the Python Quantization Tool.
    +1 moreshow less

    • ›Excludes DDS ops from running on the TensorRT Execution Provider.
25. v1.20.0 Nov 1, 2024 · issue -372

    ONNX Runtime v1.20 adds MultiLoRA, INT4 embedding, cuDNN Flash Attention, QNN weight sharing, and WebGPU on mobile.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.20.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.20.0

    • ›Adds ability to serialize pre-packed weights to avoid memory growth on model load.
    • ›Adds bfloat16 and float8 data type support in the Python I/O binding API.
    • ›Adds INT4 quantized embedding support on CPU and CUDA execution providers.
    • ›Adds FP16 support for MatMulNbits, Clip, and LayerNormalization ops on the CPU EP.
    • ›Adds cuDNN frontend integration for convolution operators on the CUDA EP.
    +24 moreshow less

    • ›Adds cuDNN Flash Attention and Lean Attention support in the MultiHeadAttention op on the CUDA EP.
    • ›Adds TensorRT 10.4 and 10.5 support in the TensorRT EP.
    • ›Adds QNN HTP weight sharing across multiple ORT inference sessions in the QNN EP.
    • ›Adds support for QNN SDK 2.27.
    • ›Adds support up to OpenVINO 2024.4.1 in the OpenVINO EP.
    • ›Adds remote tensors for direct memory access during NPU inferencing in the OpenVINO EP.
    • ›Adds DirectML 1.15.2 support.
    • ›Adds FP16 support for ML Program models with the CoreML EP on mobile.
    • ›Adds FP16 XNNPACK kernels as a CoreML fallback option on mobile.
    • ›Adds initial support for the native WebGPU EP on Android and iOS.
    • ›Adds on-demand weight loading for Web/Wasm32, enabling 8B-parameter LLMs by offloading the Wasm32 heap.
    • ›Adds Opset-21 support (Reshape, Shape, Gelu) for the Web target.
    • ›Adds quantized embedding support for Web.
    • ›Adds MultiLoRA support in Core and GenAI.
    • ›Adds support for mid-loop generation termination in GenAI.
    • ›Adds logit soft capping support in Group Query Attention (GQA) for GenAI.
    • ›Adds model support for Phi-3.5 Vision Multi-Frame, ChatGLM3, and Nemotron-Mini in GenAI.
    • ›Adds GenAI Python package for Mac and Mac/iOS NuGet packages.
    • ›Adds Unigram tokenization model support in Extensions.
    • ›Adds support for the latest Hugging Face tokenization JSON format (transformers>=4.45) in Extensions.
    • ›Adds Olive CLI for executing well-defined workflows without manual config creation or editing.
    • ›Adds YAML-based workflow config support in Olive.
    • ›Adds Python 3.13 support in PyPI packages.
    • ›Adds experimental vcpkg support for the CPU EP.
    └──▷ BREAKING ON UPGRADE

    • !All ONNX Runtime Training packages are deprecated and will no longer be published: onnxruntime-training (PyPI), onnxruntime-training-cpu (PyPI), Microsoft.ML.OnnxRuntime.Training (NuGet), onnxruntime-training-c (CocoaPods), onnxruntime-training-objc (CocoaPods), and onnxruntime-training-android (Maven Central). ORT 1.19.2 was the last release to include them.
    • !Python 3.8 and Python 3.9 are no longer supported in ORT packages. Users on those versions must stay on ORT 1.19.2 or earlier.
    • !CUDA packages now require additional shared libraries that were not needed in 1.19: libcudnn_adv.so.9, libcudnn_cnn.so.9, libcudnn_engines_precompiled.so.9, libcudnn_engines_runtime_compiled.so.9, libcudnn_graph.so.9, libcudnn_heuristic.so.9, libcudnn_ops.so.9, libnvrtc.so.12, and libz.so.1.
26. v1.19.2 Sep 4, 2024 · issue -374

    ONNX Runtime v1.19.2 adds CUDA Phi3 MoE kernel, smooth softmax for GroupQueryAttention, and causal support in MultiHeadAttention.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.19.2 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.19.2

    • ›Adds CUDA kernel for Phi3 MoE (Mixture of Experts) inference.
    • ›Adds smooth softmax support in CUDA and CPU kernels for the GroupQueryAttention operator.
    • ›Enables causal support in the MultiHeadAttention CUDA operator.
    • ›Restores support for CUDA compute capability 7.0 and 7.5 with CUDA 12, and 6.0 and 6.1 with CUDA 11 in training workloads.
27. v1.19.0 Aug 19, 2024 · issue -375

    ONNX Runtime v1.19.0 adds TensorRT 10.2, CUDA 12/CuDNN 9 defaults, INT4 QDQ, FlashAttention on CPU, and new GenAI model support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.19.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.19.0

    • ›Adds sdpa_kernel provider option to the CUDA Execution Provider to choose the kernel used for Scaled Dot-Product Attention.
    • ›Adds QDQ support for INT4 quantization in CPU and CUDA Execution Providers.
    • ›Adds C APIs for language, vision, and audio processors in Extensions, including a new FeatureExtractor for Whisper.
    • ›Adds new CUDA custom operators in Extensions: MulSigmoid, Transpose2DCast, ReplaceZero, AddSharedInput, and MulSharedInput.
    • ›Introduces Java CUDA 12 packages published on Maven.
    +22 moreshow less

    • ›Adds Java bindings for GenAI (build from source).
    • ›Supports enabling EpContext via session options in the OpenVINO Execution Provider.
    • ›Enables refittable embedded engine in TensorRT when an ONNX model is provided as a byte stream.
    • ›Implements FlashAttention on CPU to improve performance for GenAI prompt workloads.
    • ›Implements DeformConv operator support.
    • ›Expands CUDA Execution Provider op support to include Tile (bf16).
    • ›Expands CPU Execution Provider op support to include GroupQueryAttention and SparseAttention (for Phi-3 small).
    • ›Expands QNN Execution Provider op support: HardSigmoid, ConvTranspose 3d, Clip (int32 data), Matmul (int4 weights), Conv (int4 weights), prelu (fp16).
    • ›Adds Conv + Clip/Relu fusion support in the QNN Execution Provider.
    • ›Adds support for OpenVINO 2024.3.
    • ›Updates TensorRT Execution Provider to support TensorRT 10.2.
    • ›Updates DirectML from 1.14.1 to 1.15.1 with ONNX opset support expanded from 17 to 20.
    • ›Adds support for building quantized GenAI models using AWQ and GPTQ methods.
    • ›Adds GenAI support for new models: Qwen, Llama 3.1, Gemma 2, and Phi-3 small.
    • ›Adds Android support for GenAI (build from source).
    • ›Publishes GenAI packages for Windows ARM.
    • ›Adds support for the Phi-3 Small Tokenizer and a new OpenAI tiktoken format for fast loading of BPE tokenizers in Extensions.
    • ›Adds NumPy 2.x support.
    • ›Updates default GPU packages to use CUDA 12.x and CuDNN 9.x (previously CUDA 11.x/CuDNN 8.x); CUDA 11.x/CuDNN 8.x packages moved to the aiinfra VS feed.
    • ›Updates QNN Execution Provider to support QNN SDK 2.25.
    • ›Improves CPU operator coverage for WebNN (now supported by Chrome).
    • ›Adds additional CoreML ML Program operators for mobile.
    └──▷ BREAKING ON UPGRADE

    • !Default GPU packages now use CUDA 12.x and CuDNN 9.x; CUDA 11.x/CuDNN 8.x packages are no longer in the default feed and have been moved to the aiinfra VS feed.
    • !Xamarin support has been removed; projects targeting Xamarin will no longer build.
    • !macOS 11 is no longer supported; the minimum supported macOS version is now 12.
    • !iOS 12 is no longer supported; the minimum supported iOS version is now 13.
    • !JavaScript packaging for onnxruntime-web has been updated in ways that introduce slight incompatibilities when apps bundle onnxruntime-web.
28. v1.18.1 Jun 28, 2024 · issue -377

    ONNX Runtime v1.18.1 adds TensorRT Weightless API, hardware-compatible engines, and INT64 calibration support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.18.1 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.18.1

    • ›Adds TensorRT Weightless API integration to the TensorRT Execution Provider.
    • ›Adds support for TensorRT hardware compatible engines in the TensorRT EP.
    • ›Adds support for INT64 types in TensorRT constant layer calibration.
    • ›CUDA 12.x GPU packages are now built against cuDNN 9.x (previously cuDNN 8.x); CUDA 11.x packages continue to use cuDNN 8.x.
    • ›Publishes CUDA 12 Java packages to the Azure DevOps feed.
    +1 moreshow less

    • ›Updates to the latest commit of the onnx-tensorrt parser, incorporating upstream improvements.
    └──▷ BREAKING ON UPGRADE

    • !CUDA 12.x ONNX Runtime GPU packages now depend on cuDNN 9.x instead of cuDNN 8.x, requiring a cuDNN upgrade for existing CUDA 12.x installations.
    • !Windows packages now require Microsoft Visual C++ Redistributable Runtime 14.38 or newer.
    • !ONNX Runtime Python packages now require numpy >=1.21.6, <2.0; numpy 2.0 is not supported.
29. v1.18.0 May 21, 2024 · issue -378

    ONNX Runtime v1.18.0 adds TensorRT 10, WebNN EP preview, RISC-V support, QNN mixed-precision, and GenAI model expansion.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.18.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.18.0

    └──▷ TRY IT
    Cross-compile ONNX Runtime for a 64-bit RISC-V target using QEMU for emulation during build.
    copy

    $ ./build.sh --rv64 --riscv_toolchain_root /opt/riscv --riscv_qemu_path /usr/bin/qemu-riscv64

    Build a minimal CUDA EP that includes only memcpy ops, reducing binary size for inference pipelines that handle data movement outside the runtime.
    copy

    $ cmake .. -Donnxruntime_CUDA_MINIMAL=ON && cmake --build . --config Release

    • ›Adds --rv64, --riscv_toolchain_root, and --riscv_qemu_path build options for initial RISC-V architecture support.
    • ›Adds --use_binskim_compliant_compile_flags build option to opt into security-related compile/link flags (default OFF for source builds; ON for all release binaries).
    • ›Adds onnxruntime_CUDA_MINIMAL CMake option to build the CUDA execution provider with only memcpy ops, enabling minimal CUDA builds.
    • ›Adds enable_htp_fp16 provider option to the QNN EP for fp16 execution on HTP.
    • ›Adds a provider option to the CUDA EP to disable TF32.
    +28 moreshow less

    • ›Adds SessionOptions.DisablePerSessionThreads to the C# API, enabling threadpool sharing between sessions.
    • ›Adds a new SessionOptions config entry to disable specific graph transformers and rules.
    • ›Exposes Reserve() in OrtAllocator to allow custom allocators to work when session.use_device_allocator_for_initializers is specified.
    • ›Adds WebNN EP as a preview execution provider for Web targets.
    • ›Adds TensorRT 10 support to the TensorRT EP.
    • ›Adds Python support for user-provided CUDA streams in both the CUDA and TensorRT EPs.
    • ›Adds support for multiple CUDA graphs in the CUDA EP.
    • ›Extends MoE in the CUDA EP to support Tensor Parallelism and int4 quantization.
    • ›Adds QNN SDK support up to version 2.22, with mixed 8/16-bit precision configurability per layer (upgraded from A16W8).
    • ›Adds multiple partition support for QNN context binary.
    • ›Adds per-channel quantized weights support for Conv in the QNN EP.
    • ›Integrates QNN EP with Qualcomm's AIHub.
    • ›Adds OpenVINO 2024.1 support, including import of pre-compiled blobs as EPContext blobs.
    • ›Separates device and precision as distinct inputs in the OpenVINO EP, adding precision as a separate CLI option and removing device_id from provider options.
    • ›Adds DirectML operator support for Resize-18, Resize-19, Col2Im-18, IsNaN-20, IsInf-20, and ReduceMax-20, plus contrib ops SimplifiedLayerNormalization, SkipSimplifiedLayerNormalization, QLinearAveragePool, MatMulIntegerToFloat, GroupQueryAttention, DynamicQuantizeMatMul, and QAttention.
    • ›Adds HQQ quantization support for 4-bit quant to improve accuracy on GPU.
    • ›Adds support for models larger than 2 GB in on-device training, enabling SLM training on edge devices.
    • ›Adds GenAI model support for Phi-3, Gemma, and LLama-3.
    • ›Adds DML EP support for GenAI.
    • ›Adds support for building ONNX Runtime with QNN on Android.
    • ›Adds visionOS support for mobile builds.
    • ›Adds initial support for creating ML Program format CoreML models.
    • ›Adds 1D Conv and ConvTranspose support to the XNNPACK EP.
    • ›Adds MacCatalyst (Catalyst) support for macOS builds.
    • ›Adds Mixtral integration using the ORT training backend.
    • ›Adds support for Hugging Face FastTokenizer conversion into an ONNX custom operator.
    • ›Adds Java CUDA 12 support and a Java packaging pipeline published to Maven repository.
    • ›Enables eager execution for custom operators in Extensions.
    └──▷ BREAKING ON UPGRADE

    • !Windows ARM32 support has been dropped at the source code level.
    • !Python version >=3.8 is now required for build.bat/build.sh (previously >=3.7).
    • !The onnxruntime-mobile Android package and onnxruntime-mobile-c/onnxruntime-mobile-objc iOS CocoaPods are deprecated; users must migrate to onnxruntime-android and onnxruntime-c/onnxruntime-objc.
    • !OpenVINO EP removes device_id from provider options; device and precision are now separate inputs, and CPU_FP32/GPU_FP32 terminology is replaced with CPU/GPU.
    • !Security-related compile/link flags have been moved out of the default build settings into the new --use_binskim_compliant_compile_flags option, which is OFF by default when building from source.
    • !Windows OneCore build now uses 'Reverse forwarding' apisets instead of 'Direct forwarding', causing onnxruntime.dll in NuGet packages to depend on kernel32.dll; systems without kernel32.dll require reverse forwarders.
30. v1.17.3 Apr 18, 2024 · issue -379

    ONNX Runtime v1.17.3 adds WebGPU/WebNN capabilities, new op support, and packed QKV with Rotary Embedding for sm<80 GPUs.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.17.3 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.17.3

    • ›Adds hardSigmoid op support and hardSigmoid activation for fusedConv in the Web (WebGPU) execution provider.
    • ›Adds LeakyRelu activation for fusedConv in the Web execution provider.
    • ›Adds FastGelu custom op support in the Web execution provider.
    • ›Adds MatMulNBits op with optimizations for the Web execution provider.
    • ›Adds support for WebNN async API via Asyncify in the Web execution provider.
    +8 moreshow less

    • ›Adds capture and replay support for the JS execution provider.
    • ›Allows uint8 tensors for WebGPU.
    • ›Enables ort-web with any Float16Array polyfill.
    • ›Adds uniform support for conv, conv transpose, conv grouped, and fp16 in the Web execution provider.
    • ›Adds support for packed QKV input and Rotary Embedding with sm<80 GPUs using the Memory Efficient Attention kernel.
    • ›Updates replacement logic for MultiHeadAttention (MHA) and GroupQueryAttention (GQA) kernel optimizations.
    • ›Adds benchmarking support for LLaMA model end-to-end performance.
    • ›Adds example demonstrating export of OpenAI Whisper implementation with batched prompts.
31. v1.17.1 Feb 27, 2024 · issue -381

    ONNX Runtime v1.17.1 adds DirectML/CUDA EP support for Node.js, ETW session logging, and a new DefaultTensorType quantization option.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.17.1 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.17.1

    • ›Adds DefaultTensorType option to the quantization API to specify the default tensor type to quantize.
    • ›Adds capturestate/rundown ETW support logging for session and provider options.
    • ›Enables DirectML on Windows and CUDA on Linux for the Node.js binding.
    • ›Adds x64 Python wheels build job for QNN EP, enabling cached QNN model creation on Windows x64.
    • ›Adds ATen fallback support for the bicubic interpolation algorithm in onnxruntime-training.
    +3 moreshow less

    • ›Adds contrib Q/DQ ops to the symbolic shape inference tool.
    • ›Updates Q/DQ quantization to ensure Microsoft opset compliance.
    • ›Reduces onnxruntime-training package size to allow publication on PyPI.
32. v1.17.0 Feb 3, 2024 · issue -381

    ONNX Runtime v1.17 adds ONNX 1.15 opsets, QNN mixed-precision, WebGPU GA, QLoRA training, and new EP APIs.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.17.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.17.0

    └──▷ TRY IT
    Re-enable link-time optimization when building ONNX Runtime from source to match the performance profile of prebuilt binaries.
    copy

    $ python build.py --config Release --build_shared_lib --enable_lto

    • ›Adds AppendExecutionProvider_OpenVINO_V2 API for configuring new OpenVINO EP options.
    • ›Adds QNN EP device-level session options soc_model, htp_arch, device_id, extreme_power_saver for htp_performance_mode, and vtcm_mb.
    • ›Adds --enable_lto build flag to opt back in to link-time code generation/LTO, which is now disabled by default when building from source.
    • ›Adds new dynamic ETW provider for tracing and diagnosing ONNX Runtime internals at runtime with low performance overhead.
    • ›Adds support for 12 new ONNX 1.15 opsets: IsInf-20, IsNaN-20, DFT-20, ReduceMax-20, ReduceMin-20, AffineGrid-20, GridSample, ConstantOfShape-20, RegexFullMatch, StringConcat, StringSplit, and ai.onnx.ml.LabelEncoder-4.
    +31 moreshow less

    • ›Adds CUDA 12 packages for Python and NuGet.
    • ›Adds Python 3.12 packages for ONNX Runtime Inference.
    • ›Adds AMD GPU package with ROCm and MiGraphX (Python + Linux only).
    • ›Splits the ONNX Runtime GPU NuGet package into two separate packages.
    • ›Adds Windows ARM64X build support when building ONNX Runtime from source.
    • ›Adds support for supplying a custom logger at the session level.
    • ›Adds new logging and tracing of session and execution provider options.
    • ›Adds 4-bit quantization support on NVIDIA GPU and ARM64.
    • ›Adds support for direct load of precompiled TensorRT engines and customizable engine prefix in the TensorRT EP.
    • ›Adds Python support for TensorRT plugins via ORT custom ops.
    • ›Adds support for QNN SDK 2.18.
    • ›Adds QNN EP context binary caching and model initialization optimizations.
    • ›Adds mixed precision (8/16-bit) quantization support for the QNN EP.
    • ›Enables QNN NPU profiling dynamically via ETW or CSV output.
    • ›Adds support for OpenVINO 2023.2.
    • ›Adds RotaryEmbedding operator support to the DirectML EP.
    • ›Adds support for fusing subgraphs into DirectML execution plans.
    • ›Adds new Python API to select a specific GPU on multi-GPU devices with the DirectML EP.
    • ›Adds initial 4-bit quantization support on ARM64 for mobile.
    • ›Adds support for YOLOv8 pose detection pre/post processing on mobile.
    • ›Adds support for macOS in the CocoaPods package.
    • ›Adds Web support for external data format, I/O bindings, and training.
    • ›Transitions WebGPU backend out of experimental (now generally available) and adds FP16 support for WebGPU.
    • ›Enables QLoRA training support with BFloat16.
    • ›Enables memory-efficient gradient management for large model training (~10 GB memory reduction observed on Mistral).
    • ›Adds support for Aten efficient attention and Triton Flash Attention for large model training.
    • ›Adds training support on the web (On-Device Training), targeting federated learning and developer exploration.
    • ›Allows custom ops to invoke ONNX Runtime's internal thread pool for parallelism.
    • ›Adds CUDA kernel custom operator support in extensions, including two example operators.
    • ›Introduces fairseq flag in extensions to improve compatibility with certain Hugging Face tokenizers.
    • ›Incorporates added_token attribute into the BPE tokenizer in extensions to improve CodeGen tokenizer support.
    └──▷ BREAKING ON UPGRADE

    • !All Linux binaries now require glibc >= 2.28; CentOS 7 is no longer supported as a prebuilt target.
    • !Windows ARM32 binaries are removed from official packages; users must build from source.
    • !Android NDK 24.x support is removed; NDK 25.x or 26.x is required.
    • !Building for Linux ARM64 or Android now requires the C/C++ compiler to support BFloat16.
    • !Link-time code generation (LTCG/LTO) is now disabled by default when building from source; re-enable with --enable_lto.
33. v1.16.0 Sep 20, 2023 · issue -386

    ONNX Runtime v1.16.0 adds fp8/4-bit CPU quant, FlashAttention v2, Azure EP on mobile, Swift Package Manager support, and major LLM training optimizations.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.16.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.16.0

    └──▷ USE IT
    Prevent silent CPU fallback during inference to ensure a model runs only on the intended EP (e.g., CUDA) and fails fast if an op is unsupported.
    python copy

    import onnxruntime as ort
    
    opts = ort.SessionOptions()
    opts.add_session_config_entry('session.disable_cpu_ep_fallback', '1')
    
    session = ort.InferenceSession('model.onnx', sess_options=opts, providers=['CUDAExecutionProvider'])

    Export a tokenizer processing graph for a LLaMA model as an ONNX model for on-device or pipeline use.
    python copy

    from onnxruntime_extensions import gen_processing_models
    from transformers import AutoTokenizer
    
    tokenizer = AutoTokenizer.from_pretrained('huggyllama/llama-7b')
    onnx_model = gen_processing_models(tokenizer, pre_kwargs={}, post_kwargs={})
    # Save the exported ONNX processing model
    with open('llama_tokenizer.onnx', 'wb') as f:
        f.write(onnx_model[0].SerializeToString())

    • ›Adds session.disable_cpu_ep_fallback session option to prevent automatic fallback to the CPU execution provider.
    • ›Adds ORTMODULE_ENABLE_EMBEDDING_SPARSE_OPTIMIZER environment variable (disabled by default) for experimental embedding sparsity optimizations, improving Roberta training performance by 20-30%.
    • ›Adds ORTMODULE_ENABLE_COMPUTE_OPTIMIZER for Label Sparsity compute optimization, now enabled by default.
    • ›Adds ORTMODULE_CACHE_DIR model cache for exported ONNX models to avoid repeated exports when the model is unchanged.
    • ›Adds gen_processing_models Python API in onnxruntime-extensions to export ONNX data processing models from Hugging Face tokenizers (LLaMA, CLIP, XLM-Roberta, Falcon, BERT, etc.).
    +32 moreshow less

    • ›Supports serialization of models 2 GB and larger.
    • ›Java API gains support for fp16 and bf16 tensors as inputs/outputs, with conversion utilities leveraging hardware-accelerated Float.float16ToFloat and Float.floatToFloat16 on JDK 20+.
    • ›Java API adds support for external initializers, enabling large models to be instantiated without filesystem access.
    • ›C# API exposes OrtValue as the new preferred inference API, reducing garbage collection overhead and providing direct native memory access via Slice-like interfaces.
    • ›C# Float16 and BFloat16 types become full-featured interfaces supporting conversion and floating-point properties such as IsNaN and IsInfinity.
    • ›C++ Float16_t and BFloat16_t types become full-featured interfaces supporting conversion and floating-point properties such as IsNaN and IsInfinity.
    • ›Adds 4-bit quantization support on CPU.
    • ›Adds LLM quantization accuracy improvement via smoothquant.
    • ›Adds FlashAttention v2 support for Attention, MultiHeadAttention, and PackedMultiHeadAttention ops.
    • ›CUDA EP gains initial fp8 support covering QDQ, Cast, and MatMul operations.
    • ›CUDA EP relaxes CUDA Graph constraints to allow more models to utilize CUDA Graphs.
    • ›CUDA EP allows the CUDA allocator to be registered with ONNX Runtime externally.
    • ›TensorRT EP adds CUDA Graph support and user-provided CUDA compute stream.
    • ›OpenVINO EP adds support for OpenVINO 2023.1.
    • ›QNN EP enables context binary cache to reduce initialization time.
    • ›QNN EP adds support for QNN 2.12 and resize with asymmetric transformation mode on the HTP backend.
    • ›QNN EP adds op support for Equal, Less, LessOrEqual, Greater, GreaterOrEqual, LayerNorm, Asin, Sign, DepthToSpace, SpaceToDepth, and 1D Conv/ConvTranspose.
    • ›Mobile gains initial Azure EP support.
    • ›Mobile adds dynamic shape support for CoreML.
    • ›Mobile adds Swift Package Manager support for ONNX Runtime inference and extensions via onnxruntime-swift-package-manager.
    • ›Mobile adds support for CLIPImageProcessor pre-processing and CLIP inference scenarios.
    • ›React Native performance improved via JSI.
    • ›WebGPU ops coverage expanded to support SAM, T5, and Whisper models.
    • ›WebNN ops coverage expanded to support SAM and Stable Diffusion models.
    • ›ORTModule + OpenAI Triton integration now available for computing ONNX sub-graphs during large model training.
    • ›On-Device Training adds iOS support.
    • ›On-Device Training adds a minimal build (~1.5 MB binary) for resource-constrained environments.
    • ›On-Device Training enables ORT-Extensions custom op support via onnxblock.
    • ›ORT Extensions adds TrieTokenizer operator for RWKV-like LLM models.
    • ›ORT Extensions adds new Azure EP operators: AzureAudioToText, AzureTextToText, and AzureTritonInvoker for Python and NuGet packages.
    • ›LLaMAv2 training achieves ~10% acceleration; OpenAI Whisper training optimizations also included.
    • ›PythonOp enhancements include bool and tuple[bool] constants, materialize grads, empty inputs, save-in-context, customized shape inference, and full-qualified name export.
34. v1.15.0 May 25, 2023 · issue -390

    ONNX Runtime v1.15.0 adds JS and QNN execution providers, on-device training, WebGPU preview, and multi-GPU collective support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.15.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.15.0

    └──▷ TRY IT
    Build ONNX Runtime on a POSIX system as root (e.g., inside a Docker container running as root) — required now that root builds are disallowed by default.
    copy

    $ ./build.sh --config Release --allow_running_as_root

    • ›Adds --allow_running_as_root flag to the build command to permit building as root on POSIX systems (default now disallows it).
    • ›Adds ONNX Optional type support in the C# API.
    • ›Adds collective operations to support multi-GPU inferencing.
    • ›Adds Python 3.11 support across onnxruntime CPU, onnxruntime-gpu, onnxruntime-directml, and onnxruntime-training packages (drops 3.7).
    • ›Introduces two new execution providers: JS EP (JavaScript) and QNN EP (Qualcomm Neural Network).
    +23 moreshow less

    • ›Adds initial public preview of QNN EP, available as a NuGet package (Microsoft.ML.OnnxRuntime.QNN).
    • ›Adds official support for TensorRT 8.6, including explicit shape profile overrides, TensorRT plugin support via ORT custom ops, and timing cache.
    • ›Adds support for TensorRT options: heuristics, sparsity, optimization level, auxiliary stream, and tactic source selection.
    • ›Adds support for OpenVINO 2023.0 and dynamic shapes for iGPU in OpenVINO EP.
    • ›Adds OpenAI Whisper model support in Azure EP, now also available as a NuGet package.
    • ›Adds DirectML 1.12 support with opset 16–17 coverage in DirectML EP.
    • ›Adds Swift Package Manager package for onnxruntime on mobile.
    • ›Adds NuGet package for onnxruntime-extensions with Android/iOS support for MAUI/Xamarin.
    • ›Adds React Native package for onnxruntime with optional onnxruntime-extensions inclusion.
    • ›Adds built-in pre/post processing for NLP scenarios (classification, question-answering, text-prediction) on mobile.
    • ›Adds built-in pre/post processing for Speech Recognition (Whisper) on mobile.
    • ›Adds built-in post processing for Object Detection (YOLO) including non-max suppression and bounding box drawing on mobile.
    • ›Adds NNAPI kernels for BatchNormalization and LRN; CoreML kernels for Div, Flatten, LeakyRelu, LRN, Mul, Pad, Pow, and Sub.
    • ›Adds [preview] WebGPU support in the Web build.
    • ›Adds official On-Device Training package with APIs and language bindings for C, C++, Python, C#, and Java; packages available for Desktop and Android.
    • ›Adds graph optimizations leveraging label-data sparsity for ORT Training, yielding 4%–15% performance gains on popular Hugging Face models.
    • ›Adds native Windows ARM64 build support using Visual Studio 2022.
    • ›Updates to CUDA 11.8 (source remains compatible with CUDA 11.4 and 12.x).
    • ›Adds a lock-free queue build option for threadpool to improve CPU utilization.
    • ›Adds fused decoder multi-head attention kernel improving GPT and decoder model (T5, Whisper) performance.
    • ›Adds packing mode to improve encoder model performance with inputs of large padding ratio.
    • ›Upgrades DNNL from 2.7.1 to 3.0.
    • ›Adds cutlass as a new dependency for CUDA/TensorRT packages.
    └──▷ BREAKING ON UPGRADE

    • !The onnxruntime_ENABLE_EAGER_MODE CMake option and eager mode code are deleted — builds using this option will fail.
    • !Dropped support for Windows 8.1 and below.
    • !In v1.16.0 (announced here): support for iOS 11 and below will be dropped; iOS 12 will be the minimum.
    • !In v1.16.0 (announced here): support for CentOS 7, Ubuntu 18.04, and Linux distros without glibc >= 2.28 will be dropped.
    • !In v1.16.0 (announced here): support for GCC <= 9 and Visual Studio 2019 will be dropped.
    • !In v1.16.0 (announced here): the onnxruntime_DISABLE_ABSEIL build option will be removed.
35. v1.14.0 Feb 11, 2023 · issue -393

    ONNX Runtime v1.14.0 adds NUMA-aware threading, multi-stream GPU execution, Azure EP, Rust API preview, and ONNX opset 18 support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.14.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.14.0

    • ›Adds new custom operator APIs enabling a custom op to wrap an entire model for inferencing with an external API or runtime.
    • ›Adds new API to set intra-op thread affinity for fine-grained CPU pinning.
    • ›Changes cudnn_conv_use_max_workspace default value to 1, improving CUDA EP transformer model performance.
    • ›Supports ONNX 1.13 (opset 18), expanding the set of recognized operators and model formats.
    • ›ORT thread pool is now NUMA-aware, improving multi-socket CPU performance.
    +19 moreshow less

    • ›Multi-stream execution provider refactoring improves GPU utilization and memory reuse across parallel inference streams for CUDA, TensorRT, and ROCm EPs.
    • ›Adds Stream interface in ExecutionProvider API, allowing EP developers to customize stream implementations.
    • ›New [Preview] Rust API for ORT, available to build from the main branch.
    • ›Adds [Preview] Azure Execution Provider supporting AzureML-hosted models via Triton for hybrid on-device and on-cloud inferencing.
    • ›Adds quantization support with AMX on Sapphire Rapids processors.
    • ›ROCm EP now supports ROCm 5.4 and is GA-ready.
    • ›TensorRT EP adds support for TensorRT 8.5 GA versions.
    • ›OpenVINO EP adds support for OpenVINO 2022.3.
    • ›DirectML EP updated to DML 1.10.1, adding operators NonZero, Shape, Size, Attention, EmbedLayerNorm, SkipLayerNorm, and BiasGelu.
    • ›DirectML EP adds additional data type support for Abs, Sign, and Where operators.
    • ›DirectML EP enables SetOptimizedFilePath export/reload.
    • ›XNNPACK EP adds iOS build support and is now included in the onnxruntime-c iOS package.
    • ›XNNPACK EP adds support for using the ORT allocator in XNNPACK kernels to minimize memory usage.
    • ›XNNPACK EP adds support for additional commonly used operators.
    • ›Web build now includes onnxruntime-extensions (NLP-centric) by default.
    • ›Adds experimental utility functions for exchanging data between images and tensors in the JavaScript API.
    • ›FP16 optimizer exposed in torch-ort for use with DeepSpeed/Apex.
    • ›Mobile pre/post processing pipeline supports updating MobileNet and super-resolution models to embed pre/post processing (including JPG/PNG conversion custom ops) directly into the model via the onnxruntime-extensions Python package.
    • ›Stable Diffusion training optimizations, including support for Resize and InstanceNorm gradients, with ORT-enabled examples added to the Hugging Face diffusers library.
    └──▷ BREAKING ON UPGRADE

    • !Building ORT from source now requires cmake version >=3.24 instead of >=3.18.
36. v1.13.1 Oct 24, 2022 · issue -397

    ONNX Runtime v1.13.1 adds CANN EP, new quantization APIs, XNNPACK threadpool support, and 4 GB WebAssembly memory.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.13.1 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.13.1

    └──▷ USE IT
    Reduce peak memory on mobile by having the runtime read initializers directly from the ORT model byte buffer instead of copying them.
    python copy

    session_options.add_session_config_entry('session.use_ort_model_bytes_directly', '1')
    session_options.add_session_config_entry('session.use_ort_model_bytes_for_initializers', '1')
    session = onnxruntime.InferenceSession('model.ort', sess_options=session_options)

    • ›Sets session.use_ort_model_bytes_directly and session.use_ort_model_bytes_for_initializers to 1 in SessionOptions config to load ORT format model initializers directly from bytes, reducing peak memory usage on mobile.
    • ›Adds new QuantConfig-based quantize API for general quantization workflows.
    • ›Exposes all arena configs in the Python API in an extensible way.
    • ›Adds new quantized operators: SoftMax, Split, and Where.
    • ›Adds a quantization debugging tool to identify sensitive nodes/layers from accuracy-drop discrepancies.
    +15 moreshow less

    • ›New CANN Execution Provider (CANN EP) contributed by Huawei for initial support of Ascend 310 hardware.
    • ›TensorRT EP adds a build option to link against a pre-built onnx-tensorrt parser, enabling no-code TensorRT minor version upgrades and support for TensorRT 8.5 EA.
    • ›TensorRT EP adds uint8 data type support.
    • ›OpenVINO EP adds support for INT8 QDQ models from NNCF, Intel 13th Gen Core (Raptor Lake), and preview support for Intel Data Center GPU Flex Series and Intel Arc GPU.
    • ›DirectML EP adds new ops: LayerNormalization, Gelu, MatMulScale, DFT, and FusedMatMul (contrib).
    • ›SNPE EP NuGet package (Microsoft.ML.OnnxRuntime.Snpe) is now available and adds support for Windows Dev Kit 2023.
    • ›XNNPACK EP adds QDQ format model support with new kernels: AveragePool, Softmax, QLinearConv, QLinearAveragePool, and QLinearSoftmax.
    • ›XNNPACK EP adds threadpool support for configurable parallelism.
    • ›WebAssembly build now supports up to 4 GB memory.
    • ›Web build adds initial XNNPACK support for Wasm optimizations.
    • ›Web build adds source build support for onnxruntime-extensions and sentencepiece.
    • ›Adds fused attention CUDA kernels for BERT, fusing Add (bias) and Transpose of Q/K/V into one kernel for Attention and LongformerAttention.
    • ›Reduces GEMM computation in LongformerAttention via a new weight format.
    • ›Official ONNX Runtime GPU packages are now built with CUDA 11.6.
    • ›Training packages updated to CUDA 11.6 with Aten support added for GroupNorm, InstanceNormalization, and Upsample nearest.
    └──▷ BREAKING ON UPGRADE

    • !ORT format models must be regenerated: the mobile EP kernel format changed from hashes to operator constraint information and is NOT backwards compatible.
    • !Official ONNX Runtime GPU packages now require CUDA version >=11.6 (previously 11.4).
    • !NUPHAR EP code has been removed.
    • !Building ORT from source now requires a C++17 compiler; on Linux, GCC >=7.0 is required.
    • !Minimal numpy version for ONNX Runtime Python packages is now 1.21.6 (raised from 1.21.0).
    • !Windows 8.x is no longer supported in NuGet/C API prebuilt binaries (source builds retain Windows 7+ support).
    • !CUDA 10.x support removed at the source code level.
    • !Training packages drop CUDA 10.2 and 11.3 support.
37. v1.12.0 Jul 22, 2022 · issue -400

    ONNX Runtime v1.12 adds SNPE EP, XNNPACK EP preview, opset 17, BeamSearch op, and FusedAdam optimizer for training.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.12.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.12.0

    • ›New enable_dynamic_shapes runtime configuration option for the OpenVINO EP to enable dynamic shapes per inference iteration.
    • ›Adds new BeamSearch operator for encoder-decoder transformer models.
    • ›Supports invoking individual ops without creating a separate graph, enabling custom op development to reuse ORT code.
    • ›Supports feeding external initializers (for large models) as byte arrays for model inferencing.
    • ›New SNPE Execution Provider (production release).
    +19 moreshow less

    • ›New XNNPACK Execution Provider (preview) with initial operator support (Conv and MaxPool) for ORT Mobile and ORT Web.
    • ›Adds ONNX opset 17 support.
    • ›Adds onnx-ml opset 3 support (TreeEnsemble update).
    • ›New FusedAdam optimizer available through the torch-ort package for training integration.
    • ›ORT Training acceleration now natively available through HuggingFace Optimum.
    • ›TensorRT EP gains TensorRT 8.4 support and an option to share execution context memory between TensorRT subgraphs.
    • ›TVM EP adds model .dll ingestion and execution on Windows.
    • ›DirectML EP updated to DirectML 1.9 with opset 13–15 support.
    • ›Adds Python 3.10 support.
    • ›Adds Mac M1 support in Python and Java packages.
    • ›Adds .NET 6/MAUI support in the NuGet C# package with target frameworks net6.0, net6.0-android, net6.0-ios, and net6.0-macos.
    • ›New onnxruntime-openvino package available on PyPI for OpenVINO EP.
    • ›New quantized kernels for weight symmetry to improve GEMM and Conv performance on ARM64 little cores.
    • ›Adds gradient ops for ReduceMax and ReduceMin in ORT Training.
    • ›Adds bitmask versions of Dropout, BiasDropout, and DropoutGrad with approximately 8x space savings for the mask output.
    • ›Adds Bfloat16 support for AtenOp in ORT Training.
    • ›NNAPI EP gains new operator support: DepthToSpace, PRelu, Gather, Unsqueeze, Pad.
    • ›CoreML EP gains new operator support: DepthToSpace, PRelu.
    • ›Build switch to disable the abseil library dependency.
    └──▷ BREAKING ON UPGRADE

    • !The lightweight compile API is now the default compiler API for all Execution Providers; the legacy compiler API (previously at execution_provider.h#L237) will be deprecated in ORT 1.13 — EP maintainers must migrate to the lightweight compile API now.
    • !netstandard1.1 support is deprecated in this release and will be removed in ORT 1.13.
38. v1.10.0 Dec 8, 2021 · issue -407

    ONNX Runtime v1.10.0 adds TensorRT+CUDA in one GPU package, new WinML/DirectML APIs, Mac M1 Universal2 builds, and ARM64 Linux support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.10.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.10.0

    └──▷ USE IT
    Use the combined TensorRT+CUDA GPU package to run inference with TensorRT as the primary EP and CUDA as fallback — no separate TensorRT package install needed.
    python copy

    import onnxruntime as ort
    sess = ort.InferenceSession('model.onnx', providers=['TensorrtExecutionProvider', 'CUDAExecutionProvider'])
    result = sess.run(None, {'input': input_data})

    Create an OrtValue backed by a D3D12 GPU resource via the new DirectML C-API extensions, enabling zero-copy interop between your DirectX pipeline and ONNX Runtime.
    c copy

    // Allocate a GPU buffer from an existing D3D12 resource
    void* dml_alloc = DmlCreateGPUAllocationFromD3DResource(p_d3d12_resource);
    // ... use dml_alloc as OrtValue memory ...
    // Retrieve D3D12 resource back
    ID3D12Resource* resource = DmlGetD3D12ResourceFromAllocation(allocator, dml_alloc);
    // Free when done
    DmlFreeGPUAllocation(dml_alloc);

    • ›Adds OrtSessionOptionsAppendExecutionProviderEx_DML, DmlCreateGPUAllocationFromD3DResource, DmlFreeGPUAllocation, and DmlGetD3D12ResourceFromAllocation WinML APIs to create OrtValues from Windows ID3D12Resource objects via DirectML EP-specific C-API extensions.
    • ›New C/C++ API to query the CUDA stream for launching custom kernels, enabling implicit synchronization between custom ops in shared libraries and ORT CUDA kernels.
    • ›Python InferenceSession now requires the providers parameter to be set explicitly when enabling non-default Execution Providers (e.g. providers=['CUDAExecutionProvider']).
    • ›Python GPU package (onnxruntime-gpu) now bundles both TensorrtExecutionProvider and CUDAExecutionProvider in a single install — EPs must be explicitly registered via the providers argument.
    • ›Added Mac M1 Universal2 build: a single binary that runs natively on both Apple Silicon and Intel-based Macs, included in official NuGet packages.
    +14 moreshow less

    • ›NuGet package now supports ARM64 Linux C#.
    • ›Added Xamarin support to ORT C# NuGet packages, with iOS and Android binaries included in the native package.
    • ›ORT format models now carry a backwards compatibility guarantee.
    • ›Supports plug-in custom thread creation and join functions to enable usage of external threads.
    • ›Adds Optional type support from op set 15.
    • ›Introduced indirect Convolution method for QLinearConv with symmetrically quantized (int8, zero-point=0) filters on x64 (AVX2, AVXVNNI, AVX-512, AVX-512 VNNI) and ARM64, eliminating memcpy and per-pixel output-image sum computation.
    • ›Added transpose optimizer to push and cancel transpose ops, significantly improving performance for models requiring layout transformation.
    • ›DirectML EP updated from DirectML.dll 1.5.1 to 1.8.0, adding full-precision uint64/int64 support for 48 operators, 8D support for 7 additional operators, and DynamicQuantizeLinear op.
    • ›OpenVINO EP adds support for OpenVINO 2021.4.x, Auto Plugin, and IO Buffer/Copy Avoidance Optimizations for the GPU plugin.
    • ›DNNL EP adds Softmaxgrad, Transpose, Reshape, Pow, LeakyRelu, DynamicQuantizeLinear, Squeeze, and Unsqueeze ops.
    • ›TensorRT EP Python GPU packages now include TensorRT 8.0 support.
    • ›Web backend adds WebAssembly SIMD support for the qgemm kernel, accelerating quantized model inference.
    • ›Windows C API symbols are now uploaded to the Microsoft public symbol server.
    • ›Optimized WebAssembly bundle size to support WebAssembly-only or WebGL-only production scenarios.
    └──▷ BREAKING ON UPGRADE

    • !InferenceSession now requires the providers parameter to be explicitly set when using any Execution Provider other than the default CPUExecutionProvider; omitting it will no longer silently activate other EPs.
    • !Python 3.6 support removed for Mac builds.
    • !Invalid allocator error code renamed to OrtInvalidAllocator in the C/C++ API.
    • !Every item in OrtCudnnConvAlgoSearch has been updated to a safer global name.
39. v1.9.0 Sep 23, 2021 · issue -410

    ONNX Runtime v1.9 adds opset 15, sparse tensor APIs, TensorRT V2 provider API, and CUDA/TensorRT bundling in GPU packages.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.9.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.9.0

    └──▷ USE IT
    Configure a TensorRT execution provider session with the new V2 API, which supports explicit quantization and finer-grained options.
    c copy

    OrtTensorRTProviderOptionsV2* trt_options;
    api->CreateTensorRTProviderOptions(&trt_options);
    const char* keys[] = {"device_id", "trt_max_workspace_size"};
    const char* values[] = {"0", "1073741824"};
    api->UpdateTensorRTProviderOptions(trt_options, keys, values, 2);
    api->SessionOptionsAppendExecutionProvider_TensorRT_V2(session_options, trt_options);
    api->ReleaseTensorRTProviderOptions(trt_options);

    Build a custom ONNX Runtime binary that statically links onnxruntime-extensions so models using custom operators work without external dependencies.
    copy

    $ ./build.sh --config Release --_use_extensions

    Enable debug logging in ORTModule training to inspect graph transformations and execution issues.
    python copy

    from onnxruntime.training.ortmodule import ORTModule, DebugOptions, LogLevel
    
    model = ORTModule(pt_model, DebugOptions(log_level=LogLevel.VERBOSE))

    • ›Adds --_use_extensions build option to statically link onnxruntime-extensions for running models with custom operators.
    • ›Adds RegisterAllocator and UnregisterAllocator C APIs (in onnxruntime_c_api.h) for sharing a custom allocator across multiple sessions.
    • ›Adds SessionOptionsAppendExecutionProvider_TensorRT_V2, CreateTensorRTProviderOptions, UpdateTensorRTProviderOptions, GetTensorRTProviderOptionsAsString, and ReleaseTensorRTProviderOptions C APIs for configuring TensorRT EP.
    • ›Adds EnableOrtCustomOps C API to enable custom operator support in a session.
    • ›Adds sparse tensor C APIs: IsSparseTensor, CreateSparseTensorAsOrtValue, FillSparseTensorCoo, FillSparseTensorCsr, FillSparseTensorBlockSparse, CreateSparseTensorWithValuesAsOrtValue, UseCooIndices, UseCsrIndices, UseBlockSparseIndices, GetSparseTensorFormat, GetSparseTensorValuesTypeAndShape, GetSparseTensorValues, GetSparseTensorIndicesTypeShape, and GetSparseTensorIndices.
    +17 moreshow less

    • ›Supports ONNX 1.10 with opset 15 and ONNX IR 8 (SparseTensor type, model-local function protos).
    • ›Official ORT GPU packages (non-Python) now bundle both CUDA and TensorRT Execution Providers in a single package, built against CUDA 11.4.
    • ›New onnxruntime-directml Python package available on PyPI for DirectML-accelerated inference on Windows.
    • ›C# NuGet package adds netstandard2.0 as a supported target framework.
    • ›Adds DebugOptions and LogLevels to the ORTModule API for training debuggability.
    • ›ORT Training supports user-defined autograd functions and fallback to PyTorch for execution.
    • ›ORT Training adds support for deterministic compute to enable reproducibility with ORTModule.
    • ›ORT Training expands accepted input formats to include dictionaries and lists.
    • ›ORT Training adds ROCm 4.3.1 support on AMD GPUs.
    • ›ORT Web adds SIMD support in WebAssembly and an option to load WebAssembly from a worker thread to avoid blocking the main UI thread.
    • ›TensorRT EP adds support for TensorRT 8.0 (x64 Windows/Linux, ARM Jetson), including explicit-quantization features (ONNX Q/DQ support).
    • ›OpenVINO EP adds support for OpenVINO 2021.4.
    • ›CUDA EP adds support for sequence ops for models using the sequence type.
    • ›ORT Mobile iOS package switches to xcframework, supporting arm64 iPhone simulator on Apple silicon Macs.
    • ›Adds new quantized operator QGemm for direct quantization of Gemm.
    • ›IBM Power platform support added.
    • ›Quantization tool gains subgraph support.
    └──▷ BREAKING ON UPGRADE

    • !GCC versions below 7 are no longer supported.
    • !CMAKE_SYSTEM_PROCESSOR must be explicitly set when cross-compiling on Linux (set to the output of uname -m on the target device) due to the new pytorch cpuinfo dependency for ARM big.LITTLE support.
    • !SessionOptionsAppendExecutionProvider_TensorRT is deprecated; callers must migrate to SessionOptionsAppendExecutionProvider_TensorRT_V2.
    • !Windows symbol (PDB) files are no longer included in the NuGet package; they must be downloaded separately from GitHub artifacts.
40. v1.8.1 Jul 8, 2021 · issue -412

    ONNX Runtime v1.8.1 adds CoreML EP on iOS, React Native package, and ROCm 4.2 preview for training.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.8.1 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.8.1

    • ›Enables CoreML Execution Provider in the iOS mobile package, bringing hardware-accelerated inference to Apple devices.
    • ›Publishes the onnxruntime-react-native package on npm, enabling ONNX Runtime inference in React Native applications.
    • ›Adds support for PyTorch 1.8.1 and 1.9 in ONNX Runtime for PyTorch training acceleration.
    • ›Adds support for CUDA 10.2 and 11.1 in the training acceleration stack.
    • ›Releases preview packages for ROCm 4.2, enabling AMD GPU training acceleration.
    +2 moreshow less

    • ›Extends training acceleration coverage to GPT-Neo, Microsoft TNLG, and TNLU models in addition to popular Hugging Face models.
    • ›Adds additional operators to the ORT Mobile package.
41. v1.8.0 Jun 3, 2021 · issue -413

    ONNX Runtime v1.8.0 adds ONNX opset 14, dynamically loadable CUDA EP, ORT Web JS API, mobile packages, and a wave of new C APIs.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.8.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.8.0

    • ›Adds --enable_type_reduction flag when creating ORT format models and --enable_reduced_operator_type_support flag for minimal builds to support per-type reduction across all operators in the ai.onnx domain.
    • ›Adds new C inferencing APIs: KernelInfoGetAttributeArray_float, KernelInfoGetAttributeArray_int64, CreateArenaCfgV2, AddRunConfigEntry, CreatePrepackedWeightsContainer, PrepackedWeightsContainer, CreateSessionWithPrepackedWeightsContainer, and CreateSessionFromArrayWithPrepackedWeightsContainer.
    • ›Adds SetIntraOpThreadSpinning native WinML API to toggle intra-op thread spin behavior (enabled by default), improving throughput for current workloads.
    • ›Supports ONNX opset 14 with new and updated operators from the ONNX 1.9 release.
    • ›Introduces a dynamically loadable CUDA execution provider, allowing a single build to serve both CPU and GPU inference (excluding Python packages).
    +14 moreshow less

    • ›Releases the new [ONNX Runtime JavaScript API](https://github.com/microsoft/onnxruntime/tree/master/js#onnxruntime-web) (ORT Web) with WebAssembly and WebGL backends, Web Worker multi-threaded WebAssembly support, and ORT model format support.
    • ›Releases pre-built Android and iOS mobile packages with an Objective-C API for iOS (preview) and expanded NNAPI and CoreML execution provider operator coverage.
    • ›Releases the torch-ort package as the ONNX Runtime backend for PyTorch training.
    • ›Releases onnxruntime-training-gpu and onnxruntime-training-rocm packages for distributed training on NVIDIA and AMD GPUs.
    • ›Releases the experimental onnxruntime-extensions package providing a crowd-sourced library of custom operators (including string ops and tokenizers) loadable at runtime.
    • ›Adds TensorRT EP configuration via session options (replacing environment variables) and DLA support on Jetson Xavier (AGX, NX).
    • ›Adds support for Loading/Saving compiled model blobs on MyriadX (OpenVINO) devices to avoid expensive compilation at runtime.
    • ›Adds OpenVINO 2021.3 support (drops OpenVINO 2020.4 support).
    • ›Extends DirectML execution provider to ARM/ARM64 architectures in WinML and NuGet packages, with support for 8-dimensional tensors and fusion activations for Conv, ConvTranspose, BatchNormalization, MeanVarianceNormalization, Gemm, and MatMul.
    • ›Adds memory footprint reduction via shared pre-packed weights across sessions using CreatePrepackedWeightsContainer / CreateSessionWithPrepackedWeightsContainer.
    • ›Adds a memory arena shrinkage RunOption to return unused memory to the system after inference runs, particularly useful for dynamic-shape models with large outlier requests.
    • ›Adds native Quantize-Dequantize (QDQ) format support for CPU and per-column quantization for MatMul.
    • ›Adds GPT-2 and beam search integration for transformer models.
    • ›Extends profiler tool with threadpool usage metrics: multi-threading preparation time, run time, and wait time.
    └──▷ BREAKING ON UPGRADE

    • !Building ONNX Runtime from source now requires a C++ compiler with full C++14 support.
    • !OpenMP builds are no longer published; the default threadpool is used instead (OpenMP builds can still be compiled from source).
    • !Python package now requires flatbuffers as a new dependency.
    • !OpenVINO 2020.4 is no longer supported; OpenVINO 2021.3 is the new minimum.
42. v1.7.0 Mar 3, 2021 · issue -416

    ONNX Runtime v1.7.0 adds audio operators, QDQ quantization, Model Building API, CUDA 11, and broader platform support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.7.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.7.0

    • ›Experimental audio operators — Fourier Transforms (DFT, IDFT, STFT), Windowing Functions (Hann, Hamming, Blackman), and MelWeightMatrix — added under the com.microsoft.experimental domain, buildable with the ms_experimental build flag.
    • ›Experimental Model Building API available under the Microsoft.AI.MachineLearning.Experimental namespace (included in the Microsoft.AI.MachineLearning NuGet package) for creating dynamic models on the fly with hardware-accelerated tensor featurization.
    • ›WinML ImageFeatureDescriptor runtimeclass gains a new ImageNominalPixelRange property exposing native support for pixel ranges [0..1] and [-1..1] in image models, alongside existing BitmapPixelFormat and BitmapAlphaMode properties.
    • ›Quantization tool now supports QDQ (QuantizeLinear-DequantizeLinear) format models.
    • ›Python transformer optimizer tool now supports fusion for BART models.
    +12 moreshow less

    • ›New CPU profiling tool (profiler.py) added for transformer models.
    • ›Python 3.8 and 3.9 support added across all platforms.
    • ›ARM32/64 Windows builds now included in the CPU NuGet and zip packages.
    • ›WinML gains .NET 5 support via .NET Standard 2.0 projections.
    • ›Custom operators now supported in the ONNX Runtime Mobile build.
    • ›Custom op support extended to dynamic input.
    • ›CUDA EP official GPU build now targets CUDA 11.
    • ›OpenVINO EP updated to support OpenVINO 2021.2 and adds OpenVINO EP options in the onnxruntime_perf_test tool.
    • ›TensorRT EP updated to support TensorRT 7.2.
    • ›DirectML EP updated to support DirectML 1.4.2 and adds PIX markers for operator-level profiling.
    • ›NNAPI EP adds support for per-channel quantization for QlinearConv and new operators Min, Max, and Pow.
    • ›Mobile build gains ability to reduce operator kernel types to only those required by target models, yielding a 25–33% binary size reduction in tested models (MobileNet v2, SSD MobileNet, MobileBERT).
    └──▷ BREAKING ON UPGRADE

    • !All ONNX Runtime CPU packages are now built without OpenMP by default; users requiring OpenMP must switch to the Microsoft.ML.OnnxRuntime.OpenMP NuGet package or the onnxruntime-openmp PyPI package.
    • !Python 3.5 support has been removed from all platforms.
    • !MKLML, openblas, and jemalloc build configurations have been removed.
    • !Dependency on gemmlowp has been removed.
    • !OpenVINO 2020.2 support has been deprecated in the OpenVINO EP.
43. v1.6.0 Dec 11, 2020 · issue -419

    ONNX Runtime v1.6.0 adds opset 13, NNAPI Android acceleration, Apple Silicon support, new contrib ops, and expanded quantization.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.6.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.6.0

    • ›Exposes AddInitializer API to share initializers between sessions, eliminating duplicate memory allocation when multiple sessions load models with common weights.
    • ›Adds Python binding for the OrtValue data structure, enabling allocation and management of CUDA device memory directly within ORT without third-party allocators.
    • ›Adds OrtValue instances as bindable inputs/outputs in the Python I/O Binding interface, including support for ORT-allocated device memory.
    • ›Adds new session option to disable denormal floating-point numbers on SSE3-capable CPUs, eliminating denormal-induced performance degradation without model retraining.
    • ›Adds support for ONNX 1.8 / opset 13.
    +22 moreshow less

    • ›Adds new contrib ops: BiasSoftmax, MatMulIntegerToFloat, QLinearSigmoid, Trilu.
    • ›Adds LongformerAttention CUDA operator for Longformer transformer model optimization.
    • ›ORT Mobile now supports NNAPI for accelerating model execution on Android devices.
    • ›Adds build support for Mac with Apple Silicon (CPU only).
    • ›Adds support for loading sparse tensor initializers in pruned models.
    • ›Adds support for setting execution priority of individual nodes.
    • ›Adds support for selection of cuDNN convolution algorithms.
    • ›Adds BERT model profiling tool at onnxruntime/python/tools/transformers/profiler.py.
    • ›Python optimizer adds support for additional transformer model families: openai-GPT, ALBERT, and FlauBERT.
    • ›Adds TensorRT EP experimental Int8 quantization support.
    • ›Adds per-channel QuantizeLinear and DeQuantizeLinear quantization support.
    • ›Adds LSTM quantization support.
    • ›Adds CNN quantization optimizations including u8s8 support and NHWC transformer in QLinearConv.
    • ›Adds C# support for float16 and bfloat16 data types.
    • ›DNNL EP updated from version 1.1.1 to 1.7.
    • ›NNAPI EP adds support for CNN models and additional operators: Resize, Flatten, Clip.
    • ›OpenVINO EP updated to OpenVINO 2021.1 with added multi-threaded inferencing, fp16 input type, multi-device plugin, hetero plugin, shared library build, and ARM64 build support.
    • ›DirectML EP updated from 1.3.0 to 1.4.0, now using the standalone DirectML NuGet package Microsoft.AI.DirectML.
    • ›Windows ML NuGet package now supports UWP applications targeting Windows Store deployment for both CPU and GPU.
    • ›Windows ML gains ability to bind IIterable<Buffers> as inputs and outputs and to create Tensor* via multiple buffers.
    • ›NoOpenMP build of ONNX Runtime now available on NuGet (Microsoft.ML.OnnxRuntime.NoOpenMP) and PyPI (onnxruntime) for C/C++/C#/Python users.
    • ›Removes nGraph EP; OpenVINO EP is the recommended replacement.
    └──▷ BREAKING ON UPGRADE

    • !The destructor of OrtEnv is now non-trivial and may perform DLL unloading — calling ReleaseEnv from DllMain or placing OrtEnv in global variables is no longer safe.
    • !The nGraph EP has been removed; users must migrate to the OpenVINO EP.
44. v1.5.2 Oct 15, 2020 · issue -421

    ONNX Runtime v1.5.2 adds trilinear sampling in Resize, flatbuffers verification for ORT format, and config keys header for Linux/Mac.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.5.2 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.5.2

    • ›Adds flatbuffers verifier for ORT format buffers, enabling integrity validation of serialized models.
    • ›Adds config keys header file to Linux and Mac packages, exposing session configuration constants for native consumers.
    • ›Supports trilinear sampling mode in the Resize operator.
    • ›Removes the cudnn64_7.dll dependency from the GPU C# NuGet package, simplifying deployment on Windows.
45. v1.5.1 Sep 29, 2020 · issue -422

    ONNX Runtime v1.5.1 adds mobile-optimized builds, IO Binding APIs, TensorRT engine caching, and expanded execution provider support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.5.1 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.5.1

    └──▷ TRY IT
    Enable TensorRT engine caching to avoid recompilation on every process restart when running TensorRT-accelerated inference.
    copy

    $ export ORT_TENSORRT_ENGINE_CACHE_ENABLE=1
    python my_ort_inference.py

    • ›Adds ORT_TENSORRT_ENGINE_CACHE_ENABLE=1 environment variable to enable TensorRT engine caching, reducing model load time on repeated runs.
    • ›Adds Python API support for RegisterCustomOpsLibrary, enabling custom operator libraries to be registered at runtime.
    • ›Adds IO Binding API for C/C++/C# bindings, allowing use of pre-allocated buffers on targeted devices and specifying the target device for unknown output shapes.
    • ›Adds IntraopNumThreads and SetNamedDimensionOverrides APIs on LearningModelSessionOptions in Windows ML for fine-grained thread and dimension control.
    • ›Adds allocator sharing across multiple sessions, reducing per-session arena overhead in multi-session processes.
    +17 moreshow less

    • ›Adds Reduced Operator Kernel build support, enabling ORT binaries to be compiled with only the operators required by specific model(s).
    • ›Adds preview support for ORT on Mobile Platforms, minimizing build size for mobile and embedded devices.
    • ›Adds new static quantization operators: QLinearMul, QLinearAdd, QLinearSigmoid, and QLinearLeakyRelu.
    • ›Extends quantization tooling to support quantization-aware training models and models larger than 2 GB.
    • ›Adds Python 3.8 support for official Python packages (excluding Windows GPU and Linux ARM builds).
    • ›Adds CUDA 11 build-from-source support and upgrades the official package to CUDA 10.2 / cuDNN 8.0.
    • ›Adds TensorRT 7.1 support and delivers the TensorRT EP as a separate shared library DLL.
    • ›Adds OpenVINO 2020.4 support, VPU hardware selection runtime options, fast model compilation for VPU, and C# binding support for the OpenVINO EP.
    • ›Adds DirectML EP Python API (build from source) and C# NuGet package Microsoft.ML.OnnxRuntime.DirectML.
    • ›Adds 7 new DirectML operators for ONNX opset 12: Celu, GreaterOrEqual, LessOrEqual, ArgMin/ArgMax with select_last_index, GatherND with batch_dim, and RoiAlign.
    • ›Adds 1D–8D dimension support for DirectML ElementWise*, Activation*, Reduce*, ArgMin/ArgMax, Gather*, Scatter*, and OneHot operators.
    • ›Adds Android NNAPI EP support for dynamic input shapes, Int32/float32/uint8 data types, Uint8 static quantization, and CPU fallback for Android API level 26 and below, expanding supported operators to 36 (50% increase).
    • ›Adds MiGraphX EP operators: GatherElements, NonZero, Equal, and Where, plus Boolean data type support.
    • ›Adds Windows ML NuGet support for UWP (Windows Store, CPU only), .NET, and .NET Framework applications.
    • ›Adds Rust developer support for Windows ML, with sample and documentation available.
    • ›Adds Windows ML support for additional ONNX image type denotations: Gray8, normalized [0..1], and normalized [-1..1].
    • ›Enables telemetry in Java and Node.js packages for Windows builds via the Windows OS trace event system.
    └──▷ BREAKING ON UPGRADE

    • !macOS binaries now require OpenMP to be installed as an external dependency.
46. v1.4.0 Jul 17, 2020 · issue -424

    ONNX Runtime v1.4.0 adds Python IOBinding APIs, new quantization ops, AMD MIGraphX and ARM NN execution providers, and CUDA Einsum support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.4.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.4.0

    └──▷ USE IT
    Copy inference outputs from a GPU device back to CPU for result verification without manually managing device memory.
    python copy

    import onnxruntime as ort
    
    sess = ort.InferenceSession('model.onnx', providers=['CUDAExecutionProvider'])
    io_binding = sess.io_binding()
    io_binding.bind_cpu_input('input', input_array)
    io_binding.bind_output('output')
    sess.run_with_iobinding(io_binding)
    outputs = io_binding.copy_outputs_to_cpu()

    • ›Adds Python IOBinding methods bind_cpu_input, bind_output, and copy_outputs_to_cpu to simplify benchmarking on non-CPU devices without manual allocation or copy via third-party allocators.
    • ›Adds quantization operators QLinearAdd and QAttention to the quantization operator set.
    • ›Introduces pre-packing for constant Matrix B in DynamicQuantizeMatMul and QAttention to improve CPU transformer quantization performance.
    • ›Adds CUDA execution support for the Einsum operator (opset 12).
    • ›Adds preview-tier AMD MIGraphX Execution Provider for AMD GPU inference.
    +3 moreshow less

    • ›Adds preview-tier ARM NN Execution Provider for ARM-based inference.
    • ›Adds opset 12 support in ONNX Runtime Training.
    • ›Telemetry is now enabled by default for Python packages and GitHub release zip files (C API).
47. v1.3.1 Jun 16, 2020 · issue -425

    ONNX Runtime v1.3.1 publishes Java (CPU/GPU) and Node.js API packages to Maven and NPM.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.3.1 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.3.1

    • ›Publishes Java CPU inference library com.microsoft.onnxruntime/onnxruntime to Maven Central.
    • ›Publishes Java GPU inference library com.microsoft.onnxruntime/onnxruntime_gpu to Maven Central.
    • ›Publishes onnxruntime Node.js API package to NPM.
48. v1.3.0 May 19, 2020 · issue -426

    ONNX Runtime v1.3.0 adds training preview, new execution providers, Java/JS APIs, I/O binding for Python, and ONNX 1.7/opset 12 support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.3.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.3.0

    • ›Adds I/O binding support for the Python API, allowing inputs/outputs to be pre-staged on the GPU before model execution to significantly reduce execution time.
    • ›Adds API to specify free dimensions by both denotations and symbolic names.
    • ›Adds support for sessions to share a global threadpool, configurable via the ORT C APIs; when built with OpenMP, thread count is controlled via OpenMP environment variables.
    • ›Adds capability to build execution providers as shared library DLLs (initially supported for DNNL EP), enabling dynamic loading per model with no runtime cost when the EP is not loaded.
    • ›Adds OpenVINO v2.0 Execution Provider.
    +12 moreshow less

    • ›Adds preview Rockchip NPU Execution Provider (RKNPU EP).
    • ›Adds preview Xilinx FPGA Vitis-AI Execution Provider.
    • ›Promotes Windows Machine Learning APIs (Microsoft.AI.MachineLearning) to General Availability on NuGet.
    • ›Promotes ONNX Runtime DirectML package (Microsoft.ML.OnnxRuntime.DirectML) to General Availability on NuGet.
    • ›Promotes Java API to General Availability (Maven package forthcoming).
    • ›Adds preview JavaScript (Node.js) API, buildable from the master branch.
    • ›Adds ARM64 Linux CPU Python package, now available on PyPI.
    • ›Adds ONNX Runtime Training (preview) for accelerating transformer model training.
    • ›Adds ONNX 1.7 and opset 12 support, including function expansion for ops such as NegativeLogLikelihoodLoss, GreaterOrEqual, LessOrEqual, and Celu without requiring a kernel implementation.
    • ›Updates DirectML EP to support ONNX opset 10 and 11, and adds initial support for 8-bit and quantized operators.
    • ›Enables Windows telemetry collection.
    • ›Delivers ~10% average latency improvement across key models (ONNX Model Zoo, MLPerf, production models), with further improvements for Transformer models on CPU and GPU and for large-batch scikit-learn inference (TreeEnsembleRegressor, TreeEnsembleClassifier, LinearRegressor, LinearClassifier, SVMRegressor, SVMClassifier, TopK).
    └──▷ BREAKING ON UPGRADE

    • !This version requires the Visual C++ 2019 runtime; previous versions required Visual C++ 2017.
49. v1.2.0 Mar 10, 2020 · issue -428

    ONNX Runtime v1.2.0 adds WinML APIs, TensorRT 7, new C API surfaces, quantized ops, and Android AAR builds.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.2.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.2.0

    └──▷ USE IT
    Retrieve model metadata (producer name, graph name, custom metadata) from a session at runtime using the new C API.
    c copy

    OrtModelMetadata* metadata;
    ort_api->SessionGetModelMetadata(session, &metadata);
    
    char* producer;
    ort_api->ModelMetadataGetProducerName(metadata, allocator, &producer);
    
    char* custom_val;
    ort_api->ModelMetadataLookupCustomMetadataMap(metadata, allocator, "my_key", &custom_val);
    
    ort_api->ReleaseModelMetadata(metadata);

    • ›Adds C API functions GetDenotationFromTypeInfo, CastTypeInfoToMapTypeInfo, CastTypeInfoToSequenceTypeInfo, GetMapKeyType, GetMapValueType, GetSequenceElementType, ReleaseMapTypeInfo, ReleaseSequenceTypeInfo, SessionEndProfiling, SessionGetModelMetadata, ModelMetadataGetProducerName, ModelMetadataGetGraphName, ModelMetadataGetDomain, ModelMetadataGetDescription, ModelMetadataLookupCustomMetadataMap, ModelMetadataGetVersion, and ReleaseModelMetadata for richer session introspection.
    • ›Adds new Contrib ops QLinearAdd, QLinearMul, QLinearReduceMean, MulInteger, and QLinearAveragePool for quantized inference workflows.
    • ›Adds Int8 type support for the Where op.
    • ›Default value of graph_optimization_level in SessionOptions changed to ORT_ENABLE_ALL (99), and default max_num_graph_transformation_steps increased to 10.
    • ›Introduces a model opset number and IR version check — ONNX Runtime will now reject models with ONNX opset versions higher than those supported in the built version.
    +9 moreshow less

    • ›[Preview] Windows Machine Learning (WinML) WinRT APIs now available in Windows builds of ONNX Runtime, with DirectML execution provider for GPU acceleration; compatible with Windows 8.1 (CPU) and Windows 10 1709+ (GPU), distributed as windows.ai.machinelearning.dll via Nuget.
    • ›TensorRT Execution Provider upgraded to TRT 7.
    • ›CUDA updated to 10.1; Linux builds now require CUDA Runtime 10.1.243, cublas10-10.2.1.243, and CUDNN 7.6.5.32 (cublas 10.1.x is not supported); adds dependency on curand lib.
    • ›Adds capability to generate an ONNX Runtime Android Archive (AAR) file from source, importable directly into Android Studio.
    • ›Java API now available on Android; Gradle is now required to build the Java API.
    • ›Nuget package restructured to introduce a separate managed assembly Microsoft.ML.OnnxRuntime.Managed shared between CPU and GPU packages.
    • ›Build system now supports fetching dependencies from vcpkg (C++ package manager for Windows, Linux, and macOS).
    • ›Adds experimental featurizer operators as an expansion of Contrib operators (not part of the official build).
    • ›OrtEnv can now be created and destroyed multiple times within the same process.
    └──▷ BREAKING ON UPGRADE

    • !The default graph_optimization_level in SessionOptions is changed to ORT_ENABLE_ALL (99); existing code relying on the previous default will now apply full graph optimizations automatically.
    • !ONNX Runtime will now reject models whose opset version is higher than the supported opset for the installed version — models previously loaded under the old forward-compatibility behavior may now fail to load.
    • !Linux CUDA builds now require exactly CUDA Runtime 10.1.243, cublas10-10.2.1.243, and CUDNN 7.6.5.32; cublas 10.1.x versions will not work.
50. v1.1.0 Dec 19, 2019 · issue -431

    ONNX Runtime v1.1.0 adds Android NNAPI and Java API previews, shared-library custom ops, and BERT/TensorRT performance upgrades.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.1.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.1.0

    • ›Adds preview NN API Execution Provider for Android, enabling on-device inference via Android's Neural Networks API.
    • ›Adds preview Java API for ONNX Runtime, bringing Java application support to the runtime.
    • ›Custom ops can now be packaged in shared libraries and loaded at runtime, allowing distribution across multiple applications without modification.
    • ›Adds a Python tool that automatically maps a pandas DataFrame to ONNX graph inputs based on schema information, located at onnxruntime/python/tools/featurizer_ops.
    • ›TensorRT Execution Provider gains support for inputs with dynamic shapes.
    +2 moreshow less

    • ›Fused CPU kernels added for Attention, EmbedLayerNormalization, SkipLayerNormalization, and FastGelu operators to accelerate BERT model inference.
    • ›MKL-DNN (now renamed DNNL) updated to v1.1.
51. v1.0.0 Oct 30, 2019 · issue -433

    ONNX Runtime v1.0.0 adds stable C ABI, four new execution providers, free dimension override, and new build/perf options.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v1.0.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v1.0.0

    └──▷ TRY IT
    Tune thread counts for a high-throughput concurrent inference server, controlling both intra- and inter-op parallelism via onnxruntime_perf_tool.
    copy

    $ onnxruntime_perf_tool -m model.onnx -o 99 -y 4

    • ›Adds free dimension override, letting callers supply statically-unknown tensor shapes (e.g. batch size) before inference to unlock ahead-of-time optimizations.
    • ›Adds ability to set intra- and inter-operator thread counts independently to tune sequential vs. concurrent inferencing workloads.
    • ›Adds new cmake build options onnxruntime_USE_GEMMLOWP, onnxruntime_USE_AUTOML, and onnxruntime_USE_DML for controlling optional EP dependencies.
    • ›Adds -y flag to onnxruntime_perf_tool for controlling inter_op_num_threads; maximum optimization level is now 99 (previously 3).
    • ›Adds new session option for serializing optimized ONNX models to disk.
    +12 moreshow less

    • ›Enables registration of execution providers through the Python API and setting additional run options through the C# API.
    • ›Stabilizes the C API at v1.0 with ABI compatibility and Semantic Versioning guarantees — programs linked against this release will not require re-linking for future releases.
    • ›Adds [Preview] NUPHAR execution provider, a TVM- and LLVM-based EP that JIT-compiles subgraph nodes into optimized functions for model acceleration.
    • ›Adds [Preview] DirectML execution provider for hardware-accelerated GPU inference via DirectX 12 on Windows across a broad range of hardware and drivers.
    • ›Adds [Preview] ARM Compute Library (ACL) execution provider targeting ARM CPUs and GPUs.
    • ›Adds [Preview] OpenVINO support for Intel Arria 10 FPGA.
    • ›Promotes OpenVINO EP to General Availability for Intel CPU, Integrated Graphics, Neural Compute Stick 2, and Movidius Myriad VPU targets.
    • ›Adds quantization tool support for selective node quantization, bias quantization for Conv nodes, and node fusion for dynamic quantization.
    • ›Adds ONNX 1.6 / opset 11 operator support on CPU, including Sequence ops.
    • ›Adds component-level telemetry via Windows Trace Logging for Windows builds.
    • ›Adds Gelu fusion optimizer for CPU/GPU inference.
    • ›Switches default thread pool to Eigen ThreadPool; adds ability to disable multi-threading by setting pool size to 1 and onnxruntime_USE_OPENMP to OFF.
    └──▷ BREAKING ON UPGRADE

    • !cmake options onnxruntime_USE_MLAS and onnxruntime_USE_EIGEN_THREADPOOL are removed; both are now always enabled and specifying them will break builds.
    • !In onnxruntime_perf_tool, optimization level 3 is no longer valid; the maximum is now 99.
    • !Minimum supported gcc version is raised to 4.8.2.
52. v0.5.0 Aug 1, 2019 · issue -435

    ONNX Runtime v0.5.0 adds C++ API, HTTP/gRPC inference server, Intel OpenVINO EP, Python custom operators, and CUDA 10 support.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v0.5.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v0.5.0

    • ›Adds a C++ API (onnxruntime_cxx_api.h) for inferencing, wrapping the existing C API.
    • ›Introduces ONNX Runtime Server (Beta) for serving models over HTTP and gRPC endpoints.
    • ›Adds Python Operator (Beta) support to embed custom Python code in a single ONNX graph node, enabling experimentation with custom operators.
    • ›Intel OpenVINO Execution Provider is now available for Public Preview.
    • ›MKL-DNN Execution Provider now uses subgraph-based execution for improved performance.
    +4 moreshow less

    • ›Updates CUDA support to CUDA 10 for GPU inferencing with NVIDIA GPUs.
    • ›Base CPU Execution Provider gains faster convolution performance via NCHWc blocked layout, enabled by setting graph optimization level to 3 in session options.
    • ›All C APIs now return an ORT status code for consistency.
    • ›Adds support for Keras-based Mask R-CNN model inferencing.
53. v0.4.0 May 2, 2019 · issue -438

    ONNX Runtime v0.4.0 adds Intel nGraph and NVIDIA TensorRT execution providers, ONNX 1.5/Opset 10 support, and quantization operators.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v0.4.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v0.4.0

    • ›Adds Intel nGraph execution provider for accelerated inference on Intel hardware.
    • ›Adds NVIDIA TensorRT execution provider for GPU-accelerated inference.
    • ›Supports ONNX 1.5 and Opset 10 operators, enabling use of newly added ONNX Model Zoo object detection models (YOLO v3, SSD).
    • ›Adds quantization operators from ONNX 1.5.
    • ›Extends the C API for Custom Operators to support allocation of outputs during compute, includes a C++ wrapper to simplify implementation, and enables custom op DLLs when ONNX Runtime is compiled statically.
    +3 moreshow less

    • ›Adds graph optimization via Constant Folding for improved inference performance.
    • ›Publishes additional NuGet binaries built with MKL-ML.
    • ›Reduces binary sizes: 700 KB+ on Windows, 65% on Linux, 45% on macOS.
54. v0.3.0 Mar 14, 2019 · issue -440

    ONNX Runtime v0.3.0 adds ONNX 1.4/Opset 9 support, custom op registration via C API, and new OS X/x86 build packages.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v0.3.0 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v0.3.0

    • ›Adds custom op registration via the C API, enabling users to plug in their own operators at runtime.
    • ›Adds Non-Tensor type support for input/output in the C and C# APIs.
    • ›Supports ONNX 1.4 compatibility including Opset 9 operators and large models over 2 GB.
    • ›Adds new prebuilt packages: C/C# on OS X x64 CPU, C on Linux x86 CPU, and C on Windows x86 CPU.
    • ›Default CPU execution provider is now Eigen and MLAS (previously MKL-DNN).
    └──▷ BREAKING ON UPGRADE

    • !The default execution provider for CPU has changed from MKL-DNN to Eigen and MLAS; existing builds relying on MKL-DNN behavior must explicitly select it via build options.
    • !OpenMP is now required for prebuilt binaries; setups without OpenMP installed will fail to run.
55. v0.2.1 Feb 11, 2019 · issue -441

    ONNX Runtime v0.2.1 adds Linux C# packages, GPU support on Linux/Windows, and promotes the C API to Beta.

    └──▷ GET THIS VERSION
    copy

    $ git clone --branch v0.2.1 https://github.com/microsoft/onnxruntime.git
    # already have the repo? check out this version:
    $ git checkout v0.2.1

    • ›C# packages now available for Linux, with GPU support for both Windows and Linux.
    • ›C API promoted from experimental to Beta status, signaling expected stability for production use.
    • ›Adds MKL-DNN acceleration support for Relu, Sum, and BatchNormalization operators.