LongformerAttention

 Graphs in : cluster, ONNX model architecture.

input (heterogeneous) – T : object, 3D input tensor with shape (batch_size, sequence_length, hidden_size), hidden_size = num_heads * head_size.
weight (heterogeneous) – T : object, 2D input tensor with shape (hidden_size, 3 * hidden_size).
bias (heterogeneous) – T : object, 1D input tensor with shape (3 * hidden_size).
mask (heterogeneous) – T : object, attention mask with shape (batch_size, sequence_length).
global_weight (heterogeneous) – T : object, 2D input tensor with shape (hidden_size, 3 * hidden_size).
global_bias (heterogeneous) – T : object, 1D input tensor with shape (3 * hidden_size).
global (heterogeneous) – G : object, global attention flags with shape (batch_size, sequence_length).

 Parameters : cluster,

num_heads : integer, number of attention heads.
Default value “0”.
window : integer, one sided attention windows length W, or half of total window length.
Default value “0”.
 training? : boolean, whether the layer is in training mode (can store data for backward).
Default value “True”.
 lda coeff : float, defines the coefficient by which the loss derivative will be multiplied before being sent to the previous layer (since during the backward run we go backwards).
Default value “1”.

 name (optional) : string, name of the node.