Introduction

TIDL releases include import tool source code. Any user can change the code and recompile it to implement their own design.

TIDL import process contains following steps(All these steps are done within one command line, import tool will handle all automatically):

Read import config file.
Translate/import layers/operators to TIDL net file, Calculate layer size & buffer size. Merge layers if possible.
Generate quant config file, call quant tool to do Range Collection, and update the TIDL net file.
Generate config file for network compiler, and calls complier to do performance optimization.
[Optional] call GraphVisualiser to generate a image of net structure.
Import tool will check the model at the end of import process.
Finally, if there is no error, it is ready to deploy.

TIDL Import Process

Purpose

Different deep learning framework will generate different file type to describe models. TIDL library accept TI format deep learning model describe file. This import tool is to translate/convert/import different file type to a universal descriptor. And this tool will utilize all the acceleration mechanism to optimize input model. After import, quant tool & network compiler tool will be called to do range collection & optimization.

TIDL library has already supported some popular layers/operations. For new layer/operators which can be converted to supported layers/operators, users can modify the source code of this tool to enable them. Also, users can easily adapt this tool to his customized training framework.

Input and Output format

This application reads import config file to collect necessary information listed in User's Guide Import Parameter.
Import tool will mainly generate 2 files: net description file & io buf description file.
Other generated files: layer name file, tempDir folder.

Directory Structure

TIDL import tool is at below path in TIDL release package::

├── ti_dl                                           # TIDL release package
│   ├── utils                                       # Utilities
│   │   ├── tidlModelImport                         # Import tool
│   │   │   └── ti_dl.h                             # Import tool intermediate structure
│   │   │   └── tidl_caffeImport.cpp                # Old caffe import code, default unused, keep for fail backup.
│   │   │   └── tidl_caffeImport_param.cpp          # New caffe import code for map functions.
│   │   │   └── tidl_caffeImport_param.h
│   │   │   └── tidl_caffeImport_v2.cpp             # New caffe import code for net searching.
│   │   │   └── tidl_import_common.cpp              # Common code for layer merging, reshape functions, utilities.
│   │   │   └── tidl_import_common.h
│   │   │   └── tidl_import_common_model_check.cpp  # Common code for model sanity check.
│   │   │   └── tidl_import_common_model_check.h
│   │   │   └── tidl_import_config.cpp              # Common code for parsing import config file.
│   │   │   └── tidl_import_config.h
│   │   │   └── tidl_import_include.h
│   │   │   └── tidl_import_main.cpp                # Main function of Import tool
│   │   │   └── tidl_onnxImport.cpp                 # ONNX import code
│   │   │   └── tidl_tfImport.cpp                   # Tensorflow import code
│   │   │   └── tidl_tfLiteImport.cpp               # tfLite import code
│   │   │   └── tidl_tfMetaArchImport.cpp           # MetaArch code

Diagrams

Main flow

Following flow shows the main execution flow of TIDL import tool(tidl_import_main.cpp).

Inside import process

Following flow shows the internal execution flow of import process(same import flow for different framework).

Implement details

Reverse search mechanism

Import tool uses a reverse search mechanism to add all related layers into internal list.

Map functions

Map functions aim to translate layer/operator string name into TIDL layer name. Import tool match each string to corresponding map functions.
sTIDL_caffeLayerParamMap_t tidl_caffeLayerParamMapTable[] =
{
{ "Concat", TIDL_caffeMapConcatParams },
{ "Convolution", TIDL_caffeMapConvParams },
{ "ConvolutionDepthwise", TIDL_caffeMapConvParams },
{ "Pooling", TIDL_caffeMapPoolingParams },
{ "ReLU", TIDL_caffeMapReluParams },
{ "PReLU", TIDL_caffeMapPReLUParams },
{ "Dropout", TIDL_caffeMapDropoutParams },
{ "Softmax", TIDL_caffeMapSoftmaxParams },
{ "softmax", TIDL_caffeMapSoftmaxParams },
{ "Deconvolution", TIDL_caffeMapDeconvParams },
{ "Argmax", TIDL_caffeMapArgmaxParams },
{ "ArgMax", TIDL_caffeMapArgmaxParams },
{ "Bias", TIDL_caffeMapBiasParams },
{ "ShuffleChannel", TIDL_caffeMapShuffleParams },
{ "Eltwise", TIDL_caffeMapEltwiseParams },
{ "BatchNorm", TIDL_caffeMapBatchNormParams },
{ "Scale", TIDL_caffeMapScaleParams },
{ "InnerProduct", TIDL_caffeMapInnerProductParams },
{ "Split", TIDL_caffeMapSplitParams },
{ "Slice", TIDL_caffeMapSliceParams },
{ "Crop", TIDL_caffeMapCropParams },
{ "Flatten", TIDL_caffeMapFlattenParams },
{ "Permute", TIDL_caffeMapPermuteParams },
{ "PriorBox", TIDL_caffeMapPriorBoxParams },
{ "Reshape", TIDL_caffeMapReshapeParams },
{ "DetectionOutput", TIDL_caffeMapDetectionOutputParams }
};
Map function is used to translate layer parameters into internal structure.
int32_t TIDL_caffeMapEltwiseParams(sTIDL_OrgNetwork_t *pOrgTIDLNetStructure,
int32_t i,
int32_t& layerIndex,
int32_t& dataIndex,
NetParameter& netStructure,
NetParameter& netParams)
{
sTIDL_LayerPC_t &TIDLPCLayers = pOrgTIDLNetStructure->TIDLPCLayers[layerIndex];
TIDLPCLayers.layerType = TIDL_EltWiseLayer;
TIDLPCLayers.numOutBufs = 1;
TIDLPCLayers.numInBufs = netStructure.layer(i).bottom_size();
TIDLPCLayers.outData[0].dataId = dataIndex++;
if(netStructure.layer(i).eltwise_param().has_operation())
{
TIDLPCLayers.layerParams.eltWiseParams.eltWiseType = netStructure.layer(i).eltwise_param().operation();
}
else
{
TIDLPCLayers.layerParams.eltWiseParams.eltWiseType = TIDL_EltWiseSum;
}
return 0;
}
This function can only handle information inside current layer.

Reshape functions

Reshape functions aims to calculate the size, the element type, MACs.
int32_t TIDL_tfOutReshapeSoftmax(sTIDL_OrgNetwork_t *pOrgTIDLNetStructure, int32_t layerIndex)
{
sTIDL_LayerPC_t &TIDLPCLayers = pOrgTIDLNetStructure->TIDLPCLayers[layerIndex];
TIDLPCLayers.outData[0].elementType = TIDL_SinglePrecFloat;
TIDLPCLayers.outData[0].numDim = TIDLPCLayers.inData[0].numDim;
TIDLPCLayers.outData[0].dimValues[0] = TIDLPCLayers.inData[0].dimValues[0];
TIDLPCLayers.outData[0].dimValues[1] = 1;
TIDLPCLayers.outData[0].dimValues[2] = 1;
TIDLPCLayers.outData[0].dimValues[3] = TIDLPCLayers.inData[0].dimValues[3] * TIDLPCLayers.inData[0].dimValues[1] * TIDLPCLayers.inData[0].dimValues[2];
TIDLPCLayers.numMacs =
(int64_t)((int64_t)TIDLPCLayers.outData[0].dimValues[0] * TIDLPCLayers.outData[0].dimValues[1] *
TIDLPCLayers.outData[0].dimValues[2] * TIDLPCLayers.outData[0].dimValues[3]);
return 0;
}
For layers has the same output as input, use "TIDL_tfOutReshapeIdentity"
sTIDL_tfOutRehapeMap_t sTIDL_caffeOutRehapeTable[] =
{
{ TIDL_DataLayer , TIDL_tfOutReshapeDataLayer },
{ TIDL_ConvolutionLayer , TIDL_tfOutReshapeConvLayer },
{ TIDL_PoolingLayer , TIDL_tfOutReshapePoolingLayer },
{ TIDL_ReLULayer , TIDL_tfOutReshapeRelu },
{ TIDL_PReLULayer , TIDL_tfOutReshapePRelu },
{ TIDL_EltWiseLayer , TIDL_tfOutReshapeIdentity },
{ TIDL_InnerProductLayer , TIDL_tfOutReshapeIPLayer },
{ TIDL_SoftMaxLayer , TIDL_tfOutReshapeSoftmax },
{ TIDL_BatchNormLayer , TIDL_tfOutReshapeBN },
{ TIDL_BiasLayer , TIDL_tfOutReshapeIdentity },
{ TIDL_ScaleLayer , TIDL_tfOutReshapeIdentity },
{ TIDL_Deconv2DLayer , TIDL_tfOutReshapeDeConvLayer },
{ TIDL_ConcatLayer , TIDL_tfOutReshapeConcatLayer },
{ TIDL_SplitLayer , TIDL_tfOutReshapeSliceLayer },
{ TIDL_SliceLayer , TIDL_tfOutReshapeSliceLayer },
{ TIDL_CropLayer , TIDL_tfOutReshapeCropLayer },
{ TIDL_FlattenLayer , TIDL_tfOutReshapeFlattenLayer },
{ TIDL_DropOutLayer , TIDL_tfOutReshapeIdentity },
{ TIDL_ArgMaxLayer , TIDL_tfOutReshapeArgmaxLayer },
{ TIDL_DetectionOutputLayer , TIDL_tfOutReshapeDetOutLayer },
{ TIDL_ShuffleChannelLayer , TIDL_tfOutReshapeIdentity },
{ TIDL_ResizeLayer , TIDL_tfOutReshapeResize },
{ TIDL_RoiPoolingLayer , TIDL_tfOutReshapeRoiPoolingLayer},
{ TIDL_OdPostProcessingLayer , TIDL_tfOutReshapeOdPostProcessingLayer},
{ TIDL_CustomLayer , TIDL_tfOutReshapeIdentity },
{ TIDL_UnsupportedLayer , TIDL_tfOutReshapeIdentity },
{ TIDL_ConstDataLayer , TIDL_tfOutReshapeIdentity },
{ TIDL_PriorBoxLayer , TIDL_tfOutReshapeIdentity },
{ TIDL_PermuteLayer , TIDL_tfOutReshapeIdentity },
{ TIDL_ReshapeLayer , TIDL_tfOutReshapeIdentity },
{ TIDL_ShapeLayer , TIDL_tfOutReshapeIdentity },
{ TIDL_SqueezeLayer , TIDL_tfOutReshapeIdentity },
{ TIDL_PadLayer , TIDL_tfOutReshapePadLayer },
{ TIDL_TransposeLayer , TIDL_tfOutReshapeIdentity },
{ TIDL_ClipLayer , TIDL_tfOutReshapeClip }
};
It can utilize previous layer’s size and current layer’s parameters from internal structure.

Merge/convert layers

These functions are used by the import process.
These functions can search, remove, add, modify all the layers/operators imported.
These functions locates in tidl_import_common.cpp, so it cannot access prototxt/ONNX/PB.
int32_t tidl_mergePoolingLayer(sTIDL_OrgNetwork_t &pOrgTIDLNetStructure, int32_t layerIndex)
{
int32_t i1, i2, i3, i4;
int32_t status = 0;
int32_t merged;
for (i1 = 0; i1 < layerIndex; i1++)
{
if (pOrgTIDLNetStructure.TIDLPCLayers[i1].layerType == TIDL_PoolingLayer &&
pOrgTIDLNetStructure.TIDLPCLayers[i1].layerParams.poolParams.kernelH != 0 &&
pOrgTIDLNetStructure.TIDLPCLayers[i1].layerParams.poolParams.kernelW != 0)
{
merged = 0;
int32_t idx = tidl_getInLayer(pOrgTIDLNetStructure, layerIndex, pOrgTIDLNetStructure.TIDLPCLayers[i1].inData[0].dataId);
if (idx == -1)
{
continue;
}
sTIDL_LayerPC_t &TIDLPCLayers = pOrgTIDLNetStructure.TIDLPCLayers[idx];
if ((TIDLPCLayers.layerType == TIDL_ConvolutionLayer) &&
(TIDLPCLayers.outConsumerCnt[0] == 1) &&
(pOrgTIDLNetStructure.TIDLPCLayers[i1].layerParams.poolParams.kernelH == 2) &&
(pOrgTIDLNetStructure.TIDLPCLayers[i1].layerParams.poolParams.kernelW == 2))
{
merged = 1;
}
if (merged == 1)
{
int32_t idx = tidl_getInLayer(pOrgTIDLNetStructure, layerIndex, pOrgTIDLNetStructure.TIDLPCLayers[i1].inData[0].dataId);
if (idx == -1)
{
return -1;
}
sTIDL_LayerPC_t &TIDLPCLayers = pOrgTIDLNetStructure.TIDLPCLayers[idx];
TIDLPCLayers.numMacs += pOrgTIDLNetStructure.TIDLPCLayers[i1].numMacs;
TIDLPCLayers.outData[0] = pOrgTIDLNetStructure.TIDLPCLayers[i1].outData[0];
strcpy((char*)TIDLPCLayers.outDataNames[0], (char*)pOrgTIDLNetStructure.TIDLPCLayers[i1].outDataNames[0]);
TIDLPCLayers.outConsumerCnt[0] = pOrgTIDLNetStructure.TIDLPCLayers[i1].outConsumerCnt[0];
TIDLPCLayers.layerParams.convParams.poolParams = pOrgTIDLNetStructure.TIDLPCLayers[i1].layerParams.poolParams;
TIDLPCLayers.layerParams.convParams.enablePooling = 1;
pOrgTIDLNetStructure.TIDLPCLayers[i1].numInBufs = -1;
pOrgTIDLNetStructure.TIDLPCLayers[i1].numOutBufs = -1;
}
}
}
return 0;
}

Table of Contents