LayerNormalization

Public technical reference documentation from Graiphic. For current SOTA installation and toolkit guides, visit the documentation center.

Description

Returns the LayerNormalization layer weights. Type : polymorphic.

Documentation illustration (illustration unavailable in the archive).

Input parameters

weights : cluster

index : integer, index of layer.
name : string, name of layer.
Documentation illustration (illustration unavailable in the archive). weight : variant, weight of layer.

Documentation illustration (illustration unavailable in the archive).

Output parameters

weights_info : cluster

index : integer, index of layer.
name : string, name of layer.
weights : cluster

gamma : array, 1D values. gamma = [input_dim1].
beta : array, 1D values. beta = [input_dim1].

Documentation illustration (illustration unavailable in the archive).

Dimension

gamma = [input_dim1]

The size depends on the input to the LayerNormalization layer.
For example, if the layer input has a size of [batch_size = 10, input_dim1 = 5, input_dim2 = 4, input_dim3 = 2] then gamma will have a size of [input_dim1 = 5].
Another example, if the input of the layer has a size of [batch_size = 12, input_dim1 = 8, input_dim2 = 5, input_dim3 = 3] then gamma will have a size of [input_dim1 = 8].

beta = [input_dim1]

The beta size is based on the same principle as the gamma size.

Example

All these exemples are snippets PNG, you can drop these Snippet onto the block diagram and get the depicted code added to your VI (Do not forget to install Deep Learning library to run it).