Forum
Login
Register

Welcome to our Support Center
Updated

BifurcationDetector

Description

Component for aggressive decoding. Find the bifurcation index of predicted tokens, between source tokens, starting from previous suffix match index, and predicted tokens. Concat predicted tokens, starting from bifurcation index, to the back of current tokens. This forms the output tokens. Detect suffix match index in source tokens, between source tokens and output tokens. Detection is based on finding the appearances of last n-gram in output tokens in source tokens. A match is considered found if source tokens contain a single matching n-gram. Return the index of the start of the n-gram in source tokens. No matching if found if src tokens contain multiple or zero matching n-grams. Return -1.

 

Input parameters

 

specified_outputs_name : array, this parameter lets you manually assign custom names to the output tensors of a node.

 Graphs in : cluster, ONNX model architecture.

src tokens – T : object, encoder input ids.
cur tokens – T : object, decoder input ids.
prev suffix match idx – T : object, previous suffix match index.
pred tokens – T : object, predicted token ids from aggressive decoding.

 Parameters : cluster,

max ngram size : integer, the maximum NGram size for suffix matching.
Default value “0”.
min ngram size : integer, the minimum NGram size for suffix matching.
Default value “0”.
 training? : boolean, whether the layer is in training mode (can store data for backward).
Default value “True”.
 lda coeff : float, defines the coefficient by which the loss derivative will be multiplied before being sent to the previous layer (since during the backward run we go backwards).
Default value “1”.

 name (optional) : string, name of the node.

Output parameters

 

 Graphs out : cluster, ONNX model architecture.

tokens – T : object, decoder input ids after merging predicted tokens.
suffix_match_idx – T : object, new suffix match index.

Type Constraints

T in (tensor(int64)) : Constrain to integer types.

Example

All these exemples are snippets PNG, you can drop these Snippet onto the block diagram and get the depicted code added to your VI (Do not forget to install Deep Learning library to run it).
Table of Contents