Preallocated CUDA (Local RAM)

 Data in : array, is an array of clusters, where each cluster represents a single model input. Each cluster contains metadata and raw data required to describe and pass an input tensor to the model.

 input_order : integer, defines the position of the input within the data array. It corresponds to the index assigned to the input when it is created (via the index parameter).
 Inputs Info : cluster

 inputs_data : array, contains the raw byte representation of the input tensor data, stored as a 1D flattened buffer.
 inputs_shapes : array, specifies the shape of the input tensor. Since the data is stored as a flattened 1D buffer, this shape is necessary to reconstruct the original dimensions.
 inputs string length : array, used when the tensor type is string. If the tensor has shape [5,3], this field contains 15 values, each representing the length of a corresponding string element. This ensures that the actual size of inputs_data is known despite variable string lengths.
 inputs_ranks : array, indicates the rank of the tensor, i.e. the number of dimensions (Scalar = 0, 1D = 1, 2D = 2, etc.).
 inputs_types : array, defines the ONNX tensor type as an enumerated value (e.g. FLOAT, INT64, STRING).

​

 Data out : array,

 output_order : integer, defines the position of the output within the data array. It corresponds to the index assigned to the output when it is created (via the index parameter).
 Outputs Info : cluster

 outputs_ptr : integer, contains the raw byte representation of the input tensor data, stored as a 1D flattened buffer.
 outputs_shapes : array, specifies the shape of the output tensor. Since the data is written into a pre-allocated device buffer, this shape allows the runtime to interpret the memory layout correctly.
 outputs_ranks : integer, indicates the rank of the tensor, i.e. the number of dimensions (Scalar = 0, 1D = 1, 2D = 2, etc.).
 outputs_types : enum, defines the ONNX tensor type as an enumerated value (e.g. FLOAT, INT64, STRING).

​