At Graiphic, we’re committed to advancing industrial AI deployment by promoting transparent and comprehensive insights into ONNX Runtime’s capabilities. Recognizing a clear industry need, we’ve initiated an extensive open-source project to systematically test every ONNX node across all execution providers (EP).
Addressing an Industry-Wide Need
The ONNX Runtime ecosystem is crucial for industrial and embedded AI deployments, providing high-performance inference across diverse hardware platforms. However, one significant challenge has persisted: the lack of a clear, real-time view of node-level compatibility across different execution providers, such as NVIDIA’s CUDA and TensorRT, Intel’s oneDNN and OpenVINO, Microsoft’s DirectML, and more.
In our experience, the official OperatorKernels.md documentation, while useful, often doesn’t reflect actual node compatibility and coverage status. This discrepancy creates uncertainty for developers, hardware providers, and companies considering ONNX Runtime for critical deployments.
Comprehensive Node-by-Node Testing
To bridge this gap, Graiphic created a dedicated repository, available publicly on GitHub, providing detailed test outcomes for each node and execution provider. This initiative covers:
- CPU
- CUDA
- TensorRT
- OpenVINO
- oneDNN
- DirectML
Further execution providers like Vitis AI and ROCm will be included soon.
Each node is tested systematically, categorizing outcomes clearly:
- SUCCESS: Fully supported and operational.
- FALLBACK: Partial support or relying on fallback mechanisms.
- FAIL: Not supported or operational errors.
- NOT TESTED: Nodes requiring specific conditions or models not currently available.
Real-Time Insights and Transparent Development
By providing these detailed insights, our project helps:
- Developers quickly identify the support status of specific nodes.
- Microsoft, Intel, NVIDIA, AMD, and other hardware providers pinpoint areas requiring further development.
- Industrial decision-makers clearly assess ONNX Runtime’s maturity and readiness for specific use cases.
Contribution and Collaboration
Graiphic is proud to contribute actively and sustainably to the ONNX Runtime community. Our project is entirely open-source, fostering transparency and collaboration, thus enabling smoother and wider adoption of ONNX Runtime in the industry.
Next Steps
Our goal is continuous improvement and real-time updates. We encourage the community to utilize this resource, offer feedback, and contribute. By aligning closely with the official ONNX Runtime repository, we strive to ensure accuracy and visibility, ultimately facilitating robust industrial AI deployments.
Explore the project on GitHub: Graiphic ONNX Runtime Execution Provider Testing
