TI Deep Learning Product User Guide
tidl_dataflow.h File Reference

Introduction

This file contains interface to use TIDL's DMA dataflow to pipeline optimized kernel processing with DMA.

Usage Scenario 1 : User wants to write a best optimized version where they have some processing task which is not required to be repeated with every time kernel is called with new input within a layer's processing. To allow user to have this one time kind of processing an initialization function option is provided. User can implement such processing and then provide the function pointer to TIDL_DataflowInit function as part of TIDL_DataflowInitParams->initFuncPtr. User is expected to adhere to the signature of initFuncPtr and the same can be found in TIDL_DataflowKernelInitFuncPtr definition When user provides initFuncPtr, TIDL will internally call this function to create multiple kernel handles based on which region of the tensor processing is happening ( e.g. whether its last block of processing where dimension of processing block can be different). The information of the dimension is provided by srcAddr and dstAddr arguments to the init function and user is expected to use these dimension for the implementation of this function. Typically things which are needed only once per tensor/frame can be computed inside this function and stored in the handle. This will avoid their computation during execute call. Eg. of such information can be SE/SA templates creation which are used during kernel implementation, or some one time buffer used by the kernel. This is the preferred usage as it will be optimal without user needing to handle various regions of processing.

Below code shows a sample usage of these API's for this scenario

TIDL_DataflowInitParams initParams; TIDL_CustomMaxPoolIxXOxXInitArgs kernelInitArgs;

initParams.dataFlowType = TIDL_DataFlowTypeOneTensorInProcOneChannel; initParams.getHandleSize = TIDL_customMaxPool_ixX_oxX_getHandleSize; initParams.initFuncPtr = TIDL_customMaxPool_ixX_oxX_init; initParams.execFuncPtr = TIDL_customMaxPool_ixX_oxX_exec; initParams.kernelInitArgs = & kernelInitArgs;

status = TIDL_DataflowInit(tidlHandle, &initParams);

status = TIDL_DataflowProcess(tidlHandle, inPtrs, outPtrs);

Usage Scenario 2 : User want to quick start with a basic implementation without separating initialization and core repetitive processing. Then they can avoid providing initFuncPtr and the same can be set it to NULL. In this scenario user is expected to adhere to the signature of execFuncPtr and the same can be found in TIDL_DataflowKernelExecFuncPtr located in this file. Mainly srcAddr and dstAddr should be used for implemenation. These two parameters can be ignored ( they will be NULL) if initFuncPtr != NULL

srcAddr and dstAddr uses a structure TIDL_bufParams3D_t which can be found inside ti_dl\tidl_bufParams.h. It has following fields : data_type dim_x : The width of the buffer in X dimension in elements. This corresponds to width of the tensor which is going to processed in a given kernel call dim_y : The Height of the buffer in Y dimension in elements. This corresponds to height of the tensor which is going to processed in a given kernel call dim_z : The depth of the buffer in Z dimension in elements. This corresponds to number of channels which are going to processed in a given kernel call stride_y : Jump in bytes to move in dimesntion Y stride_z : Jump in bytes to move in dimesntion Z

Note :

  • Currently this API only supports layer which takes single input and generates single output.
  • Upscale kind of layers which increases the output resolution have not been validated
  • Ratio of input and output dimension is expected to be an integer.
  • Current implementation does not handle optimization for large resolution

Go to the source code of this file.

Data Structures

struct  TIDL_bufParams3D_t
 
struct  TIDL_DataflowInitParams
 A structure for a 3 dimensional buffer descriptor. More...
 

Macros

TIDL Data Type

Group containing all data types

#define TIDL_INT8   ((uint32_t) 0)
 
#define TIDL_INT16   ((uint32_t) 1)
 
#define TIDL_INT32   ((uint32_t) 2)
 
#define TIDL_INT64   ((uint32_t) 3)
 
#define TIDL_UINT8   ((uint32_t) 4)
 
#define TIDL_UINT16   ((uint32_t) 5)
 
#define TIDL_UINT24   ((uint32_t) 6)
 
#define TIDL_UINT32   ((uint32_t) 7)
 
#define TIDL_UINT64   ((uint32_t) 8)
 
#define TIDL_FLOAT16   ((uint32_t) 9)
 
#define TIDL_FLOAT32   ((uint32_t) 10)
 
#define TIDL_FLOAT64   ((uint32_t) 11)
 
TIDL various dataflow types supported

This group defines the different types of dataflow types that are supported

TIDL_DataFlowTypeMulInProcOneByOne

#define TIDL_DataFlowTypeOneTensorInProcOneChannel   ((int32_t) 0)
 Kernel processing happens with one input tensor and one input channel is used to produce one output channel - example depth wise separable convolution, pooling etc. Note that processing can happen on more than one channel at time but for producing an output of a channel is not dependent on the other channel. More...
 
#define TIDL_DataFlowTypeOneTensorInProcMultiChannel   ((int32_t) 1)
 Kernel processing happens with one input tensor but all input channels are used to produce one output channel - example convolution, argmax. More...
 
#define TIDL_DataFlowTypeMultiTensorInProcOneTensor   ((int32_t) 2)
 Number of input tensors are more than one but processing happens with one tensor at a time producing one output tensor, also dimension (C,H,W) of all tensors can be different - example Concat. More...
 
#define TIDL_DataFlowTypeMultiTensorInProcMultiTensor   ((int32_t) 3)
 Number of input tensors are more than one and all tensors are used to produce output, also dimension (C,H,W) of all tensors should be same. example Eltwise. More...
 

Typedefs

typedef int32_t(* TIDL_DataflowKernelGetHandleSize) (void *pKerInitArgs)
 This function call is required to know the memroy required by kernel handle. More...
 
typedef int32_t(* TIDL_DataflowKernelInitFuncPtr) (void *kernelHandle, const TIDL_bufParams3D_t *srcAddr, const TIDL_bufParams3D_t *dstAddr, void *pKerInitArgs)
 This function call is required to initialize the handle. In this function most of the one time operation are performed and results are stored in handle. More...
 
typedef int32_t(* TIDL_DataflowKernelExecFuncPtr) (void *kernelHandle, const TIDL_bufParams3D_t *srcAddr, const TIDL_bufParams3D_t *dstAddr, const void *srcPtr[], void *dstPtr)
 This function is the main compute function, and performs the argmaxing operation for CNN. It is called multiple times. More...
 

Functions

int32_t TIDL_DataflowInit (void *tidlHandle, TIDL_DataflowInitParams *initParams)
 This function initialized the dataflow handle. More...
 
int32_t TIDL_DataflowProcess (void *tidlHandle, void *inPtrs[], void *outPtr[])
 This function initialized the dataflow handle. More...
 

Macro Definition Documentation

◆ TIDL_INT8

#define TIDL_INT8   ((uint32_t) 0)

◆ TIDL_INT16

#define TIDL_INT16   ((uint32_t) 1)

◆ TIDL_INT32

#define TIDL_INT32   ((uint32_t) 2)

◆ TIDL_INT64

#define TIDL_INT64   ((uint32_t) 3)

◆ TIDL_UINT8

#define TIDL_UINT8   ((uint32_t) 4)

◆ TIDL_UINT16

#define TIDL_UINT16   ((uint32_t) 5)

◆ TIDL_UINT24

#define TIDL_UINT24   ((uint32_t) 6)

◆ TIDL_UINT32

#define TIDL_UINT32   ((uint32_t) 7)

◆ TIDL_UINT64

#define TIDL_UINT64   ((uint32_t) 8)

◆ TIDL_FLOAT16

#define TIDL_FLOAT16   ((uint32_t) 9)

◆ TIDL_FLOAT32

#define TIDL_FLOAT32   ((uint32_t) 10)

◆ TIDL_FLOAT64

#define TIDL_FLOAT64   ((uint32_t) 11)

◆ TIDL_DataFlowTypeOneTensorInProcOneChannel

#define TIDL_DataFlowTypeOneTensorInProcOneChannel   ((int32_t) 0)

Kernel processing happens with one input tensor and one input channel is used to produce one output channel - example depth wise separable convolution, pooling etc. Note that processing can happen on more than one channel at time but for producing an output of a channel is not dependent on the other channel.

◆ TIDL_DataFlowTypeOneTensorInProcMultiChannel

#define TIDL_DataFlowTypeOneTensorInProcMultiChannel   ((int32_t) 1)

Kernel processing happens with one input tensor but all input channels are used to produce one output channel - example convolution, argmax.

◆ TIDL_DataFlowTypeMultiTensorInProcOneTensor

#define TIDL_DataFlowTypeMultiTensorInProcOneTensor   ((int32_t) 2)

Number of input tensors are more than one but processing happens with one tensor at a time producing one output tensor, also dimension (C,H,W) of all tensors can be different - example Concat.

◆ TIDL_DataFlowTypeMultiTensorInProcMultiTensor

#define TIDL_DataFlowTypeMultiTensorInProcMultiTensor   ((int32_t) 3)

Number of input tensors are more than one and all tensors are used to produce output, also dimension (C,H,W) of all tensors should be same. example Eltwise.

Typedef Documentation

◆ TIDL_DataflowKernelGetHandleSize

typedef int32_t(* TIDL_DataflowKernelGetHandleSize) (void *pKerInitArgs)

This function call is required to know the memroy required by kernel handle.

Parameters
[in]pKerInitArgs: Pointer to structure holding init parameters
Returns
Size of the handle
Remarks

◆ TIDL_DataflowKernelInitFuncPtr

typedef int32_t(* TIDL_DataflowKernelInitFuncPtr) (void *kernelHandle, const TIDL_bufParams3D_t *srcAddr, const TIDL_bufParams3D_t *dstAddr, void *pKerInitArgs)

This function call is required to initialize the handle. In this function most of the one time operation are performed and results are stored in handle.

Parameters
[in]kernelHandle: Active handle to the kernel
[in]srcAddr: Pointer to structure containing dimensional information of source data
[in]dstAddr: Pointer to structure containing dimensional information of dst
[in]pKerInitArgs: Pointer to structure holding init parameters
Returns
Status of success or failure
Remarks

◆ TIDL_DataflowKernelExecFuncPtr

typedef int32_t(* TIDL_DataflowKernelExecFuncPtr) (void *kernelHandle, const TIDL_bufParams3D_t *srcAddr, const TIDL_bufParams3D_t *dstAddr, const void *srcPtr[], void *dstPtr)

This function is the main compute function, and performs the argmaxing operation for CNN. It is called multiple times.

Parameters
[in]kernelHandleActive handle to the kernel
[in]srcAddr: Pointer to structure containing dimensional information of source data.
[in]dstAddr: Pointer to structure containing dimensional information of dst
[in]srcPtr[]: List of Pointers to buffer holding input data
[in]dstPtr: Pointer to buffer holding output feature map data
Returns
Status of success or failure

Function Documentation

◆ TIDL_DataflowInit()

int32_t TIDL_DataflowInit ( void *  tidlHandle,
TIDL_DataflowInitParams initParams 
)

This function initialized the dataflow handle.

Parameters
tidlHandle[IN] Handle for the dataFlow ( this is created via TIDL_DataflowInit API)
initParams[IN] Pointer to initParams
Returns
Error Status

◆ TIDL_DataflowProcess()

int32_t TIDL_DataflowProcess ( void *  tidlHandle,
void *  inPtrs[],
void *  outPtr[] 
)

This function initialized the dataflow handle.

Parameters
tidlHandle[IN] Handle for the dataFlow ( this is created via TIDL_DataflowInit API)
inPtrs[][IN] List of pointers input tensors
outPtr[IN] Pointer to the output tensors
Returns
Error Status