Enhance your ParaView and VTK pipelines with Artificial Neural Networks

The Visualization Toolkit (VTK) has recently expanded its capabilities by introducing support for ONNX Runtime, a cross-platform engine designed for efficient execution of machine learning models. This integration opens new avenues for embedding machine learning inference directly within scientific visualization workflows. The same functionality is accessible in ParaView, a widely used open-source application for data analysis and visualization, through an official plugin named ONNXPlugin. ONNX, which stands for Open Neural Network eXchange, is an open standard file format that facilitates interoperability between different machine learning frameworks. It defines a common set of operators and ensures that models trained in frameworks like PyTorch, TensorFlow, or Scikit-learn can be exported and used across other compatible platforms without compatibility issues. ONNX Runtime complements this by serving as an efficient inference engine capable of running ONNX models across diverse hardware setups and operating systems. Though originally developed as a C++ library, it supports bindings for languages such as Python, C#, and JavaScript, thus broadening its applicability. The new VTK module utilizes ONNX Runtime to allow users to run ONNX models directly within VTK pipelines. This filter enables inference on cell or point data within any given DataSet input, regardless of its shape or geometry, treating it as a container for the output. Users can specify the input parameters for the neural network as vectors, including an optional time-dependent parameter that leverages VTK and ParaView’s temporal processing capabilities. The Python API snippet demonstrates how to set the input data, load the ONNX model file, set input parameters, and execute the model to obtain inference results, which are appended as new point or cell data to the dataset. In ParaView, the ONNXPlugin enhances usability by introducing dynamic Qt user interfaces. By loading a JSON description of the network parameters, the plugin automatically generates UI elements, including input fields with valid ranges tailored to the model’s requirements. This dynamic interface simplifies user interaction with various neural networks, adapting seamlessly without the need for manual UI development. To showcase practical applications, a multi-layer perceptron (MLP) was trained to serve as a surrogate model to replicate the Von-Mises Stress (SVM) results of a mechanical simulation performed with EasyFEA. This surrogate model approximates simulation output but delivers results roughly 2000 times faster — 5 milliseconds compared to 10 seconds for the actual simulation. This dramatic speedup facilitates rapid parameter studies, such as evaluating the effect of varying material properties like the Poisson ratio on stress distribution, which would otherwise be time-consuming and impractical using traditional simulations. Moreover, the efficiency of ONNX-enabled inference opens possibilities for tackling inverse problems, where outputs or desired results are known but corresponding input parameters must be determined. For instance, using the trained network, it was possible to quickly estimate the maximum force applied to an object while ensuring the Von-Mises Stress remains below a safety threshold. Such computations, which typically demand numerous forward evaluations, were completed within seconds, highlighting the potential for accelerated optimization and real-time decision-making workflows. While the initial VTK module release does not yet support certain advanced functionalities, such as using field data as inputs or generating new geometries, it represents a significant step towards seamless integration of machine learning into visualization and simulation pipelines. The feature is slated for official release in VTK 9.6.0 and ParaView 6.1, currently available in the ParaView master branch and requiring compilation from source. Users and organizations interested in leveraging this capability can reach out to Kitware’s experts for guidance and potential collaboration to expand post-processing and visualization features. This development was partially funded by EDF and Kitware Europe, emphasizing the growing industry interest and collaborative effort in advancing scientific visualization through AI-driven enhancements.