TIDL Importer
TIDL Translation (Import) tool can a accept a pre-trained floating point model trained using caffe-jacinto ,BVLC-caffe or tensorflow or models exported to ONNX format. The import step also generate output using the range collection tool. This output would be very similar to final expected output in the inference library. Validate the import step by comparing the output with expected result before trying the imported model on inference library
TIDL Import Tool Design
The tool available in "ti_dl/utils/tidlModelImport/out" can be used to import user trained models with below command
Usage:
< Import tool > < Configuration file >
Example:
./out/tidl_model_import.out ../../test/testvecs/config/import/tidl_import_j11.txt
Note : Run the Import tool from the "ti_dl/utils/tidlModelImport" directory only. We recommend to use the file path as mentioned in example import configuration file. If you face any issues in any file path, try with absolute paths
Model Format-Versions
- Proto file from below version are used for validating pre-trained models. In most cases new version models also shall work since the basic operations like convolution, Pooling etc don't change. Find more information on migrating to latest version if required in TIDL Import Tool Design
- Caffe - 0.17 (caffe-jacinto in gitHub)
- Tensorflow - 1.12
- ONNX - 1.3.0
- TFLite - Tensorflow 2.0-Alpha
- Since the tensor flow 2.0 is dropping the support for frozen buffer, we recommend to users to migrate to TFlite model format for Tensorflow 1.x.x as well. TFLite model format is supported in both TF 1.x.x and TF 2.x
The List of Import Configuration Parameters - Model Translation
These parameters are important for model translation/import.
Parameter | Default | Description |
modelType | 0: Caffe | 0: Caffe, 1: TensorFlow, 2: ONNX, 3: tfLite |
inputNetFile | MUST SET | Net definition from Training frames work. Example "deploy.prototxt" from caffe or frozen binary protobuf with parameters from tensorflow |
inputParamsFile | | Binary file for Model Parameters caffemodel from caffe. Not applicable for Tensorflow and ONNX |
outputNetFile | MUST SET | Output TIDL model with Net and Parameters |
outputParamsFile | MUST SET | Input and output buffer descriptor file for TIDL ivision interface |
numParamBits | 8bit | Bit depth for model parameters like Kernel, Bias etc. Max supported is 16. Any value > 8 is experimental in the current release. |
numFeatureBits | 8bit | Bit depth for Layer activation; Max supported is 16. Any value > 8 is experimental in the current release. |
quantizationStyle | 2: TIDL_QuantStyleNP2Fixed | Quantization method. 2: Linear Mode. 3: Power of 2 scales(EXPERIMENT) |
inQuantFactor | 1.0 | List: Scale factor of input feature, if the input range used in training is 0 to 1 and the same is passed as tensor range of 0 - 255 to TIDL then this parameters shall be 255. This can be ignored when inDataNorm is 1 |
inElementType | 1: 8bit Signed | List: Format for each input feature, 0 : 8bit Unsigned, 1 : 8bit Signed. |
inWidth | | List: each input tensors width. If set, this value will overwrite the input size. If not set, the import tool will use the original size of input net file. This value will pass on to Range Collection. |
inHeight | | List: each input tensors Height. If set, this value will overwrite the input size. If not set, the import tool will use the original size of input net file. This value will pass on to Range Collection. |
inNumChannels | | List: each input tensors Number of channels. If set, this value will overwrite the input size. If not set, the import tool will use the original size of input net file. This value will pass on to Range Collection. |
foldBnInConv2D | 1: Enable | Merge batch Norm layer weight and scale in previous convolution layer if possible. This will improve performance if set. |
metaArchType | -1: Not used | Meta Architecture used by the network TIDL_metaArchCaffeJacinto = 0, TIDL_metaArchTFSSD = 1. If metaArch is not used, please leave it default. |
metaLayersNamesList | | Configuration files describing the details of Meta Arch, Example pipeline.config for TF SSD. Set this value if metaArchType == 0 or 1. |
inDataNamesList | | List of input tensor names in network to be imported(use space/comma/tab to split). Will be useful to import portion of network. If not set, import tool will search the input net file for default inputs. |
outDataNamesList | | List of output tensor names in network to be imported(use space/comma/tab to split). Will be useful to import portion of network. If not set, import tool will search the input net file for default outputs. |
The List of Import Configuration Parameters - Input Pre-Processing
In most cases the input to network would be processed using some image transform functions. Example, Mean subtraction, Scaling or Standard deviation, cropping etc. The below parameters are embed in TIDL model file, so it can be used during inference.
Parameter | Default | Description |
inDataNorm | 0: Disable | Enable / Disable Normalization on input tensor. Scale and mean values are applicable only if this is enabled |
inMean | | List: Mean value needs to be subtracted for each channel of all input tensors |
inScale | | List: Scale value needs to be multiplied after means subtract for each channel of all input tensors |
inDataFormat | 1: RGB planar | List: Input tensor color format. 0: BGR planar, 1: RGB planar. Refer TIDL API guide for supported formats list |
inResizeType | 0: Normal | Image resize type . 0: normal resize, 1: keep aspect & ratio. Refer TIDL API guide for supported formats list |
resizeWidth | | Image cropping is performed if resizeWidth > imWidth. If not set, resize is disabled. Import tool will use inWidth & inHeight. |
resizeHeight | | Image cropping is performed if resizeHeight > inHeight. If not set, resize is disabled. Import tool will use inWidth & inHeight. |
The List of Import Configuration Parameters - Performance Simulation
Parameter | Default | Description |
perfSimTool | ../../utils/perfsim/ti_cnnperfsim.out(.exe) | Network Compiler executable file path. If not specified, uses from default location |
perfSimConfig | ../../test/testvecs/config/import/perfsim_base.cfg | Network Compiler Configuration file. Refer Network Compiler guide for configuration parameters |
graphVizTool | ../../utils/tidlModelGraphviz/out/tidl_graphVisualiser.out(.exe) | Graphic visualize tool executable file path. If not specified, uses from default location |
The List of Import Configuration Parameters - Range Collection
Examples
Configuration files used for validation of models are provided in the "ti_dl/test/testvecs/config/import/" folder for reference. Below are few example configuration file
Caffe Trained SSD Based Object Detection Net
- [modelType] This is a caffe model(prototxt + caffemodel).
- [inDataFormat] Input data format is BGR planar. The input size will get from original prototxt(no overwrite). By default, import tool will load image file using OpenCV and resize it and convert it to BGR planar as input.
- [inData] Images for range collection are listed in "detection_list.txt" text file. By default, the list contains JPEG/PNG compressed file list.
- [postProcType] The result is bounding box list, post process is to draw bounding box on original image.
modelType = 0
inputNetFile = "../../test/testvecs/models/public/caffe/jdetNet512x512/deploy.prototxt"
inputParamsFile = "../../test/testvecs/models/public/caffe/jdetNet512x512/voc0712_ssdJacintoNetV2_iter_106000.caffemodel"
outputNetFile = "../../test/testvecs/config/tidl_models/caffe/tidl_net_jdetNet_512x512.bin"
outputParamsFile = "../../test/testvecs/config/tidl_models/caffe/tidl_io_jdetNet_512x512_"
inDataFormat = 0
inData = "../../test/testvecs/config/detection_list.txt"
postProcType = 2
Caffe Trained Squeeze Net
- [modelType] This is a caffe model(prototxt + caffemodel).
- [inDataNorm] Input data need to be normalized, since the model is trained based on a specified range.
- [inMean & inScale] The import tool should use following value for input data normalization.
- [inDataFormat] Input data format is BGR planar. The input size will be set to resize width & height. By default, import tool will load image file using OpenCV, resize it and convert it to BGR planar as input.
- [inData] Images for range collection are listed in "detection_list.txt" text file. By default, the list contains JPEG/PNG compressed file list.
- [postProcType] The result is classification. The post process is to tell me the top-1 class name(by default, ImageNet is used).
modelType = 0
inputNetFile = "../../test/testvecs/models/public/caffe/squeezeNet1.1/deploy.prototxt"
inputParamsFile = "../../test/testvecs/models/public/caffe/squeezeNet1.1/squeezenet_v1.1.caffemodel"
outputNetFile = "../../test/testvecs/config/tidl_models/caffe/tidl_net_suqeezenet_1_1.bin"
outputParamsFile = "../../test/testvecs/config/tidl_models/caffe/tidl_io_suqeezenet_1_1_"
inDataNorm = 1
inMean = 104 117 123
inScale = 1 1 1
inDataFormat = 0
resizeWidth = 256
resizeHeight = 256
inData = ../../test/testvecs/config/imageNet_sample_val.txt
postProcType = 1
Tensorflow Trained InceptionNet V1
- [modelType] This is a tensorflow pb file.
- [inDataNorm] Input data need to be normalized, since the model is trained based on specified range.
- [inMean & inScale] The import tool should use following value for input data normalization.
- [resize] By default, input data format is RGB planar. The input size will be set to inWidth & inHeight. By default, import tool will load image file using OpenCV, resize it, crop it and convert it to RGB planar as input.
- [inData] Images for range collection are listed in "detection_list.txt" text file. By default, the list contains JPEG/PNG compressed file list.
- [postProcType] The result is classification. The post process is to tell me the top-1 class name(by default, ImageNet is used).
modelType = 1
inputNetFile = "../../test/testvecs/models/public/tensorflow/inceptionNetv1/inception_v1_fbn.pb"
outputNetFile = "../../test/testvecs/config/tidl_models/tensorflow/tidl_net_inception_v1.bin"
outputParamsFile = "../../test/testvecs/config/tidl_models/tensorflow/tidl_io_inception_v1_"
inDataNorm = 1
inMean = 128 128 128
inScale = 0.0078125 0.0078125 0.0078125
resizeWidth = 256
resizeHeight = 256
inWidth = 224
inHeight = 224
inNumChannels = 3
inData = ../../test/testvecs/config/imageNet_sample_val_bg.txt
postProcType = 1
ONNX Squeeze Net
- [modelType] This is a ONNX file.
- [inDataNorm] Input data need to be normalized, since the model is trained based on specified range.
- [inMean & inScale] The import tool should use following value for input data normalization.
- [resize] By default, input data format is RGB planar. The input size will be set to inWidth & inHeight. By default, import tool will load image file using OpenCV, resize it, crop it and convert it to RGB planar as input.
- [inData] Images for range collection are listed in "detection_list.txt" text file. By default, the list contains JPEG/PNG compressed file list.
- [postProcType] The result is classification. The post process is to tell me the top-1 class name(by default, ImageNet is used).
modelType = 2
inputNetFile = "../../test/testvecs/models/public/onnx/squeezenet1.1.onnx"
outputNetFile = "../../test/testvecs/config/tidl_models/onnx/tidl_net_squeezenet1.1.bin"
outputParamsFile = "../../test/testvecs/config/tidl_models/onnx/tidl_io_squeezenet1.1_"
inDataNorm = 1
inMean = 123.675 116.28 103.53
inScale = 0.017125 0.017507 0.017429
resizeWidth = 256
resizeHeight = 256
inWidth = 224
inHeight = 224
inNumChannels = 3
inData = ../../test/testvecs/config/imageNet_sample_val.txt
postProcType = 1
ONNX with metaArch Import Example
- [metaLayersNamesList] add SSD metaArch to the backbone.
modelType = 2
numParamBits = 8
numFeatureBits = 8
quantizationStyle = 2
inputNetFile = "../../test/testvecs/models/internal/onnx/tiad_ssd/softmax/ssd_lite_mobilenet_v1_tiod_epoch_6.onnx"
outputNetFile = "../../test/testvecs/config/tidl_models/onnx/tidl_net_tiad_ssd.bin"
outputParamsFile = "../../test/testvecs/config/tidl_models/onnx/tidl_io_tiad_ssd_"
inDataNorm = 1
inMean = 104 117 123
inScale = 1 1 1
inDataFormat = 1
inWidth = 768
inHeight = 384
inNumChannels = 3
metaArchType = 3
numFrames = 3
metaLayersNamesList = ../../test/testvecs/models/internal/onnx/tiad_ssd/softmax/tidl_meta_arch.prototxt
inData = "../../test/testvecs/config/tiad_bmp_list.txt"
postProcType = 2
perfSimConfig = ../../test/testvecs/config/import/perfsim_base.cfg
tfLite Import Example
modelType = 3
numParamBits = 15
quantizationStyle = 2
inputNetFile = ../../test/testvecs/models/public/tflite/mobilenet_v1_1.0_224.tflite
outputNetFile = "../../test/testvecs/config/tidl_models/tflite/tidl_net_tflite_mobilenet_v1_1.0_224.bin"
outputParamsFile = "../../test/testvecs/config/tidl_models/tflite/tidl_io_tflite_mobilenet_v1_1.0_224_"
inDataNorm = 1
inMean = 128 128 128
inScale = 0.0078125 0.0078125 0.0078125
resizeWidth = 256
resizeHeight = 256
inWidth = 224
inHeight = 224
inNumChannels = 3
inData = ../../test/testvecs/config/imageNet_sample_val_bg.txt
postProcType = 1