QOrderedLongformerAttention

 Graphs in : cluster, ONNX model architecture.

input (heterogeneous) – Q : object, 3D input tensor with shape (batch_size, sequence_length, hidden_size), hidden_size = num_heads * head_size.
scale_input (heterogeneous) – S : object, scale of the input.
weight (heterogeneous) – Q : object, 2D input tensor with shape (hidden_size, 3 * hidden_size).
scale_weight (heterogeneous) – S : object, scale of the weight.
bias (heterogeneous) – S : object, 1D input tensor with shape (3 * hidden_size), fp32 only currently.
scale_bias (heterogeneous) – S : object, reserved. (not used as add bias need float value in cublasLt for normal order).
scale_qkv_gemm (heterogeneous) – S : object, scale of the output for fused kqv gemm.
mask (heterogeneous) – F : object, attention mask with shape (batch_size, sequence_length).
global_weight (heterogeneous) – Q : object, 2D input tensor with shape (hidden_size, 3 * hidden_size).
scale_global_weight (heterogeneous) – S : object, scale of the global_weight.
global_bias (heterogeneous) – S : object, scale of the weight (scalar for per-tensor quantization or 1-D of dims [hidden_size] for per-channel quantization).
scale_global_gemm (heterogeneous) – S : object, 1D input tensor with shape (3 * hidden_size).
global (heterogeneous) – G : object, global attention flags with shape (batch_size, sequence_length).
scale_output (heterogeneous) – S : object, scale of the output.

 Parameters : cluster,

num_heads : integer, number of attention heads.
Default value “0”.
order_global_weight : integer, cublasLt order of weight matrix.
Default value “0”.
order_input : integer, cublasLt order of input matrix. See the schema of QuantizeWithOrder for order definition.
Default value “0”.
order_output : integer, cublasLt order of global bias.
Default value “0”.
order_weight : integer, cublasLt order of weight matrix.
Default value “0”.
window : integer, one sided attention windows length W, or half of total window length.
Default value “0”.
 training? : boolean, whether the layer is in training mode (can store data for backward).
Default value “True”.
 lda coeff : float, defines the coefficient by which the loss derivative will be multiplied before being sent to the previous layer (since during the backward run we go backwards).
Default value “1”.

 name (optional) : string, name of the node.