3. Edge AI sample apps

There are 3 ways you can explore running a typical Edge AI usecase on TDA4VM EVM,

  • Trying the out-of-box Edge AI gallery

  • Develop Edge AI applications using Python and C++ reference applications

  • Run optimized end-to-end GStreamer pipelines using OpTIFlow

The SDK is packaged with networks which does 3 DL tasks as below,

  • Image Classification: Detects top N most approximate classes in the dataset for the given input frame

  • Object Detection: Detects and draws bounding boxes around the objects, also classifies the objects to one of the classes in dataset

  • Semantic Segmentation: Classifies each pixel into class in dataset

3.1. Out-of-box GUI app

When the TDA4VM EVM is powered on with SD card in place, the Edge AI Gallery comes up on boot as shown. One can connect a USB 2.0 mouse and click on the buttons in the left panel which starts the Edge AI application running the selected DL task. In the background, a GStremer pipeline is launched which reads a compressed video file and runs a DL network on the decoded content. The output of DL inference is overlayed on the image and sent to display.

../_images/tda4vm_oob_demo.jpg

Users can select different DL tasks to execute on the compressed video. There is also a “Custom” button, which when pressed can select the input source, which can a compressed video file (H.264/H.265), USB camera or IMX219 camera. One can also select from a list of pre-imported DL networks available in the filesystem and start the application. This will automatically construct a GStreamer pipeline with required elements and launch the application.

3.2. Python/C++ apps

Python based demos are simple executable scripts written for image classification, object detection and semantic segmentation. Demos are configured using a YAML file. Details on configuration file parameters can be found in Configuring applications

Sample configuration files for out of the box demos can be found in edgeai-gst-apps/configs this folder also contains a template config file which has brief info on each configurable parameter edgeai-gst-apps/configs/app_config_template.yaml

Here is how a Python based image classification demo can be run,

/opt/edgeai-gst-apps/apps_python# ./app_edgeai.py ../configs/image_classification.yaml

The demo captures the input frames from connected USB camera and passes through pre-processing, inference and post-processing before sent to display. Sample output for image classification and object detection demos are as below,

logo1

logo2

To exit the demo press Ctrl+C.

C++ apps are cross compiled while packaging, they can be directly tested as given below

/opt/edgeai-gst-apps/apps_cpp# ./bin/Release/app_edgeai ../configs/image_classification.yaml

To exit the demo press Ctrl+C.

C++ apps can be modified and built on the target as well using below steps

/opt/edgeai-gst-apps/apps_cpp# rm -rf build bin lib
/opt/edgeai-gst-apps/apps_cpp# mkdir build
/opt/edgeai-gst-apps/apps_cpp# cd build
/opt/edgeai-gst-apps/apps_cpp/build# cmake ..
/opt/edgeai-gst-apps/apps_cpp/build# make -j2

3.3. OpTIFlow

In Edge AI Python and C++ applications, post processing and DL inference are done between appsink and appsrc application boundaries. This makes the data flow sub-optimal because of unnecessary data format conversions to work with open source components.

This is solved by providing DL-inferer plugin which calls one of the supported DL runtime and a post-process plugin which works natively on NV12 format, avoiding unnecessary color formats conversions.

Users can write their own pipeline or use optiflow application to generate and run the end-to-end pipeline. Optiflow application shares the same config file as used by Python/C++ apps.

To directly run the end-to-end pipeline use the following command.

/opt/edgeai-gst-apps/optiflow# ./optiflow.py ../configs/object_detection.yaml

To exit the demo press Ctrl+C.

To just dump the end-to-end pipeline use the following command.

/opt/edgeai-gst-apps/optiflow# ./optiflow.py ../configs/object_detection.yaml -t

Note

Python, C++ and OpTIFlow applications are similar by construction and can accept the same config file