Forum
Login
Register

Welcome to our Support Center
Updated

LayerNormalization

Description

Adds the weights of the LayerNormalization layer to the weights table. Type : polymorphic.

 

Input parameters

 

Weights in : array

Β index :Β integer,Β index of layer.
Β weights :Β variant,Β weights values.

Β index :Β integer,Β index of layer.
Β gamma :Β array,Β 1D values. gamma = [input_dim1].
Β beta :Β array,Β 1D values. beta = [input_dim1].

Output parameters

 

Β Weights out : array

Β index :Β integer,Β index of layer.
Β weights :Β variant,Β weights values.

Dimension

  • gamma = [input_dim1]

The size depends on the input to theΒ LayerNormalizationΒ layer.
For example, if the layer input has a size of [batch_size = 10, input_dim1 = 5, input_dim2 = 4, input_dim3 = 2] then gamma will have a size of [input_dim1 = 5].
Another example, if the input of the layer has a size of [batch_size = 12, input_dim1 = 8, input_dim2 = 5, input_dim3 = 3] then gamma will have a size of [input_dim1 = 8].

 

  • beta = [input_dim1]

The beta size is based on the same principle as the gamma size.

Example

All these exemples are snippets PNG, you can drop these Snippet onto the block diagram and get the depicted code added to your VI (Do not forget to install Deep Learning library to run it).

Tags:
Table of Contents