SkipLayerNormalization

 Graphs in : cluster, ONNX model architecture.

input (heterogeneous) – T : object, 3D input tensor with shape (batch_size, sequence_length, hidden_size).
skip (heterogeneous) – T : object, 3D skip tensor with shape (batch_size, sequence_length, hidden_size) or (1, sequence_length, hidden_size) or (sequence_length, hidden_size).
gamma (heterogeneous) – T : object, 1D input tensor with shape (hidden_size).
beta (optional, heterogeneous) – T : object, 1D skip tensor with shape (hidden_size).
bias (optional, heterogeneous) – T : object, 1D bias tensor with shape (hidden_size).

 Parameters : cluster,

epsilon : float, the epsilon value to use to avoid division by zero.
Default value “1e-5”.
 training? : boolean, whether the layer is in training mode (can store data for backward).
Default value “True”.
 lda coeff : float, defines the coefficient by which the loss derivative will be multiplied before being sent to the previous layer (since during the backward run we go backwards).
Default value “1”.

 name (optional) : string, name of the node.

 Graphs out : cluster, ONNX model architecture.

output (heterogeneous) – T : object, 3D output tensor with shape (batch_size, sequence_length, hidden_size).
mean (optional, heterogeneous) – U : object, saved mean used during training to speed up gradient computation.
inv_std_var (optional, heterogeneous) – U : object, saved inverse standard variance used during training to speed up gradient computation.
input_skip_bias_sum (optional, heterogeneous) – T : object, sum of the input and skip inputs (and bias if it exists) with shape (batch_size, sequence_length, hidden_size).