Forum
Login
Register

Welcome to our Support Center
Updated

LRN

Description

Local Response Normalization proposed in theΒ AlexNet paper. It normalizes over local input regions.

 

 

The local region is defined across the channels. For an elementΒ X[n,Β c,Β d1,Β ...,Β dk]Β in a tensor of shapeΒ (NΒ xΒ CΒ xΒ D1Β xΒ D2,Β ...,Β Dk), its region isΒ {X[n,Β i,Β d1,Β ...,Β dk]Β |Β max(0,Β cΒ -Β floor((sizeΒ -Β 1)Β /Β 2))Β <=Β iΒ <=Β min(CΒ -Β 1,Β cΒ +Β ceil((sizeΒ -Β 1)Β /Β 2))}.

square_sum[n,Β c,Β d1,Β ...,Β dk]Β =Β sum(X[n,Β i,Β d1,Β ...,Β dk]Β ^Β 2), whereΒ max(0,Β cΒ -Β floor((sizeΒ -Β 1)Β /Β 2))Β <=Β iΒ <=Β min(CΒ -Β 1,Β cΒ +Β ceil((sizeΒ -Β 1)Β /Β 2)).

Y[n,Β c,Β d1,Β ...,Β dk]Β =Β X[n,Β c,Β d1,Β ...,Β dk]Β /Β (biasΒ +Β alphaΒ /Β sizeΒ *Β square_sum[n,Β c,Β d1,Β ...,Β dk]Β )Β ^Β beta

 

Input parameters

 

specified_outputs_name :Β array, this parameter lets you manually assign custom names to the output tensors of a node.
X (heterogeneous) – T : object, input data tensor from the previous operator; dimensions for image case are (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. For non image case, the dimensions are in the form of (N x C x D1 x D2 … Dn), where N is the batch size. Optionally, if dimension denotation is in effect, the operation expects the input data tensor to arrive with the dimension denotation of [DATA_BATCH, DATA_CHANNEL, DATA_FEATURE, DATA_FEATURE …].

Β Parameters :Β cluster,

alpha : float, scaling parameter.
Default value β€œ1E-4”.
beta :Β float, the exponent.
Default value β€œ0.75”.
bias : float, aΒ constant added to the denominator in the normalization calculation. It is used to avoid division by zero or very small values that could destabilize learning.
Default value β€œ1”.
size : integer, the number of channels to sum over.
Default value β€œ0”.
Β training?Β :Β boolean, whether the layer is in training mode (can store data for backward).
Default value β€œTrue”.
Β lda coeff :Β float, defines the coefficient by which the loss derivative will be multiplied before being sent to the previous layer (since during the backward run we go backwards).
Default value β€œ1”.

Β name (optional) :Β string, name of the node.

Output parameters

 

Β Y (heterogeneous) – T : object, output tensor, which has the shape and type as input tensor.

Type Constraints

T in (tensor(bfloat16),Β tensor(double),Β tensor(float),Β tensor(float16)) : Constrain input and output types to float tensors.

Example

All these exemples are snippets PNG, you can drop these Snippet onto the block diagram and get the depicted code added to your VI (Do not forget to install Deep Learning library to run it).
Table of Contents