8.6. Developing deep learning applications with TIDL and OpenVX¶
8.6.1. Introduction¶
TI Deep Learning library is a deep learning inference engine to accelerate your deep learning networks on TI Jacinto 7 SoCs. TIDL runs on C7x/MMA. TI OpenVX allows to access TIDL from A72. Hence to use TIDL in system scenario ones need to understand the TIDL library and also how to use it from within OpenVX.
This developer note describes the high level steps one should follow to know more about TIDL and how to run custom networks on TI SoC using TIDL.
Note
- It is HIGHLY recommended to follow the steps listed below to run your networks on EVM.
- DO NOT skip the steps listed below, each step verifies a certain aspect of the custom network and would help better identify and isolate issues in network execution.
8.6.2. Documentation References¶
| SDK Component | Documentation | Description | Section |
|---|---|---|---|
| TIDL | LINK | TIDL library on C7x | Main Page |
| TIDL | LINK | TIDL Network Performance Analyzer | Main Page |
| TI OpenVX | LINK | TIDL OpenVX Node API on A72 | TIOVX User Guide > TIOVX Support Kernels > TI Extension Kernels > tivxTIDLNode |
| vision apps | LINK | TIDL OpenVX Demo Application (Simple non-pipelined) | Application / Demos > DL Demos > Image Classification Application |
| vision apps | LINK | TIDL OpenVX Demo Application (More complex, multi-ch, multi-network, pipelined) | Application / Demos > DL Demos > Auto Valet Parking Application |
| vision apps | LINK | Pre/post processing sample OpenVX nodes | APIs > TI defined OpenVX Kernels for Vision Apps > TIVX Kernels for Image Pre/Post Processing |
8.6.3. Source Code References¶
| SDK Component | File / Folder | Decription |
|---|---|---|
| TIDL | tidl_j7_01_01_00_10/ti_dl/inc/itidl_ti.h | TIDL library interface on C7x |
| TIDL | tidl_j7_01_01_00_10/ti_dl/test/PC_dsp_test_dl_algo.out | Standalone TIDL PC executable |
| TIDL | tidl_j7_01_01_00_10/ti_dl/test/TI_DEVICE_dsp_test_dl_algo.out | Standalone TIDL C7x executable |
| TI OpenVX | tiovx/kernels_j7/tidl | TIDL OpenVX Node implementation on C7x and A72 |
| TI OpenVX | tiovx/kernels_j7/include/TI/j7_tidl.h | TIDL OpenVX Node interface on A72 |
| vision apps | vision_apps/apps/dl_demos/app_tidl | TIDL OpenVX Demo application (simple) |
| vision apps | vision_apps/apps/dl_demos/app_avp | TIDL OpenVX Demo application (more complex) |
| vision apps | vision_apps/kernels/img_proc | Pre/post processing sample OpenVX nodes |
8.6.4. Steps to run your custom network with TIDL¶
8.6.4.1. Step 1: Import your network to TIDL network format¶
- See TIDL user guide [LINK], section Getting Started
Important
Make sure your network only uses the TIDL supported layers. See TIDL user guide [LINK], section Data Sheet > TIDL Supported Layers
- Proceed to next step, after network import is sucessful.
8.6.4.2. Step 2: Run the imported TIDL network in PC HOST emulation mode¶
- Run the newly imported network on PC to confirm that TIDL can execute this network.
- See TIDL user guide [LINK], section Build and Run Instructions > TIDL Sample Application
- Proceed to next step, after network execution on PC is successful.
Note
- Typically one can use the pre-built executable and no need to build the executable from source.
- It is enough to run test the network with 1-2 frames at this step
- Here the actual execution speed will be slow since its running in a emulated manner on PC
8.6.4.3. Step 3: Run the imported TIDL network on EVM in C7x baremetal standalone mode¶
- Run the newly imported network on EVM on C7x to confirm that TIDL on C7x can execute this network.
- See TIDL user guide [LINK], section Build and Run Instructions > TIDL Sample Application > Steps to run Sample Application on EVM
- Proceed to next step, after network execution on EVM is successful.
Note
- Typically one can use the pre-built executable and no need to build the executable from source.
- It is enough to test the network with 1-2 frames at this step
- Here the network execution speed will be optimal from C7x point of view.
- You will need CCS to load and run the TIDL sample application on EVM.
8.6.4.4. Step 4: Run the imported TIDL network on EVM with OpenVX (non-pipelined mode)¶
- Run the newly imported network on EVM with OpenVX. Here one can feed more number of frames. Use Linux to feed a large number of frames and/or visualize the output on display
- Customize the demo vision_apps/apps/dl_demos/app_tidl to feed input and optionally visualize the output
Important
By default this demo is written for the specific application of image classification using ImageNet dataset. So most likely you will need to modify this demo to run and visualize your DL network.
Note
- In this demo, pre-processing, TIDL node, post processing, visualization is all run sequentially.
- Here TIDL runs optimally from C7x point of view,
- However this is not the most optimal from system execution point of view, since in final system many of these steps can be pipelined at frame boundary to allow overall higher system FPS.
8.6.4.5. Step 5: Run the imported TIDL network on EVM with OpenVX (pipelined mode)¶
- At this stage you may want to run your network with rest of system processing like camera or display or other networks.
- Now you can make your own application according to your system requirements.
- To get maximum performance in system, one must pipeline TIDL network execution with other processing like camera, pre-processing, post-processing, visualization, display
Note
- TI provides a sample system level, high performance, pipelined TIDL demo called Auto Valet Parking (vision_apps/apps/dl_demos/app_tidl_avp)
- This can be used as reference. However you will most likely need to heavily customize this demo for your network and application.