DequantizeWithOrder
Description
Dequantize input matrix to specific layout used in cublaslt.
Documentation illustration (illustration unavailable in the archive).
Input parameters
specified_outputs_name : array, this parameter lets you manually assign custom names to the output tensors of a node.
Graphs in : cluster, ONNX model architecture.
input (heterogeneous) – Q : object, input tensor of (ROWS, COLS). if less than 2d, will broadcast to (1, X). If 3d, it is treated as (B, ROWS, COS). scale_input (heterogeneous) – S : object, scale of the input.
Documentation illustration (illustration unavailable in the archive).
Parameters : cluster,
order_input : enum, cublasLt order of input matrix. See the schema of QuantizeWithOrder for order definition.
Default value “ORDER_ROW”. order_output : enum, cublasLt order of output matrix.
Default value “ORDER_ROW”. to : enum, the output data type, only support FLOAT and FLOAT16.
Default value “FLOAT”.
training? : boolean, whether the layer is in training mode (can store data for backward).
Default value “True”.
lda coeff : float, defines the coefficient by which the loss derivative will be multiplied before being sent to the previous layer (since during the backward run we go backwards).
Default value “1”.
name (optional) : string, name of the node.
Documentation illustration (illustration unavailable in the archive).
Output parameters
output (heterogeneous) – F : object, output tensor.
Type Constraints
tensor(int8)) : Constrain input and output types to int8 tensors.
F in (tensor(float), tensor(float16)) : Constrain to float types.
S in (tensor(float)) : Constrain Scale to float32 types.
Example
All these exemples are snippets PNG, you can drop these Snippet onto the block diagram and get the depicted code added to your VI (Do not forget to install Deep Learning library to run it).