TI Deep Learning Library User Guide
|
This section provides steps to run imported model using TIDL library and sample test application. Please refer TIDL Model Import section for details on importing pre-trained models. User shall try this step only after validating the output of import step against expected results.
The Sample Application binaries available in "ti_dl/test/" can be use to run inference on imported models.
We have sample allocation built for Host PC (Emulation) and Target (J7ES EVM). The Host PC (Emulation) and Target will produce same results.
User shall first run the The Host Emulation on PC (Windows or Linux) using the below step. Output can be validated against the expected results. After validating on host emulation, the inference for model can be executed on EVM and output can be compared against host emulation run.
This binary by default would read the configuration list file "ti_dl/test/testvecs/config/config_list.txt" and run all the configuration files available in the list file.
Note
It is possible to run a different configuration list by running it as:
PC_dsp_test_dl_algo.out relative/path/to/some/other/config/list/txt
It is also possible to run a single inference configuration file by running it as:
PC_dsp_test_dl_algo.out s:reative/path/to/single/infer/config/txt
Parameter | Description |
---|---|
netBinFile | Input TIDL model with Net and Parameters - generated by Import tools |
ioConfigFile | Input and output buffer descriptor file for TIDL ivision interface - generated by Import tools |
inData | Input tensors File for Reading |
outData | Output tensors File for Writing |
inFileFormat | Input File Format - 0 : Compressed Image (JPEG/PNG), 1: RAW Image, 2: Compressed Image List. When inFileformat = 1 then type of raw data can be defined using rawDataInElementType. If rawDataInElementType is not set then raw data type is assumed to be same as inElementType |
numFrames | Number of input tensors to be processed from the input file |
postProcType | Post processing on output tensor. 0 : Disable, 1- Classification top 1 and 5 accuracy, 2 – Draw bounding box for OD, 3 - Pixel level color blending |
postProcDataId | Output Tensor ID on which the Post processing needs to be performed, 0 : Default |
quantRangeExpansionFactor | Margin that needs to be applied on Feature map Range . Example 1.2 would apply add 20% margin to range values |
quantRangeUpdateFactor | Rate at which the range values shall be updated after each process. Example 0.1 would apply add 10% to current process range and 90% running range. This parameter is only applicable for stats collection during import and is not used during inference. |
debugTraceLevel | Verbose debug trace level. Default 0 |
writeTraceLevel | Layer level tensor trace level. Default : 0, 0 - No Tensor Trace, 1- Fixed Point , 2- Padded Fixed Point, 3 - Floating point |
writeOutput | Write output tensors to file. Default : 1, 1 - All output tensors written to file in integer format, 2 - All output tensors written to file in float format, 0 - Each output tensor is compared against the file pointing to outData |
numItrPerf | Number of iterations to be averaged for reporting Performance - Default : 1 |
Note
It is possible to override a parameter in all the inference configuration files present in "ti_dl/test/testvecs/config/config_list.txt" by
providing the parameter as an argument to the application. For example, to run all inferences with "writeTraceLevel" = 1,
the appication can be run as
PC_dsp_test_dl_algo.out –writeTraceLevel 3
This feature is available when using a configuration file other then the default "ti_dl/test/testvecs/config/config_list.txt":
PC_dsp_test_dl_algo.out relative/path/to/some/other/config/list/txt –writeTraceLevel 3
Note
The global override can then be further tuned for a particular inference by editing its entry in the configuration list file.
For example to run jacintoNet11v2 with "writeTraceLevel" = 1 (after the global override of "writeTraceLevel" = 3), the entry can be:
1 testvecs/config/infer/public/caffe/tidl_infer_jacintonet11v2.txt –writrTraceLevel 1
Note
It is possible to override multiple parameters (both global overrides and test case specfic overrides), for example:
PC_dsp_test_dl_algo.out –writeTraceLevel 3 –debugTraceLevel 3
Configuration files used for validation of models are provided in the "ti_dl/test/testvecs/config/infer/" folder for reference
Below is one example configuration file for Inference
On Successful execution the output tensor will be generated in as binary raw file in the path specified by "outData".
The above inference test can be executed on Jacinto7 SoC using the TI_DEVICE_a72_test_dl_algo_host_rt.out
binary (distributed with the installation). This section will describe the steps required to run the imported models on target.
H/W requirements
Preparing the SD card
user@ubuntu-pc$ cd ${PSDKRA_PATH}/vision_apps user@ubuntu-pc$ make linux_fs_install_sd
Booting up the EVM
Insert the SD card in to the EVM and power on the EVM and wait for Linux to complet the boot. Log in as root and run below to execute the TIDL application
root@ j7-evm:~# cd /opt/vision_apps root@ j7-evm:~# source ./vision_apps_init.sh
root@ j7-evm:~# cd /opt/tidl_test root@ j7-evm:/opt/tidl_test# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib root@ j7-evm:/opt/tidl_test# ./TI_DEVICE_a72_test_dl_algo_host_rt.out
Decoding the output
The test output is printed on console which shows the number of megacycles taken for each test case. Assuming C7x is running at 1 GHz, the time-taken-per-frame and FPS for each test can be calculated as:
Time-taken-per-frame in milliseconds = (1000 / C7x CPU clock in MHz) x Number of mega cycles
FPS = 1 / Time-taken-per-frame in milliseconds
For example, From the output above :
Test | Mega cycles count | Time taken pe | |
---|---|---|---|
JSegNet21V2 | 8.58 | 8.58 | 116.55 |
PeleeNet | 9.77 | 9.77 | 102.3 |
MobileNetV2 | 6.33 | 6.33 | 157.98 |
Note : Above numbers are representative numbers, to get the real numbers for these networks, you should run the same on EVM
For image classification tests, the input class and the inferred class is also printed (e.g. in MobileNetV2 test).
After all the tests are complete, the postprocessed images for object detection and semantic segmentation are stored in testvecs/output
.
Validating test output
Take the SD card out of the EVM and plug it into PC. After the SD card is mounted in ${SDCARD_MOUNT_DIR}
, you can check the contents in ${SDCARD_MOUNT_DIR}/opt/tidl_test/testvecs/output
.
The postprocessed output files should be presnt in
${SDCARD_MOUNT_DIR}/opt/tidl_test/testvecs/output/pelee.bin_ti_lindau_000020.bmp_000000_tidl_post_proc2.bmp
${SDCARD_MOUNT_DIR}/opt/tidl_test/testvecs/output/jsegNet1024x512.bin_ti_lindau_I00000.bmp_000000_tidl_post_proc3.bmp