3. Edge AI sample apps¶
There are 3 ways you can explore running a typical Edge AI usecase on TDA4VM EVM,
Trying the out-of-box Edge AI gallery
Develop Edge AI applications using Python and C++ reference applications
Run optimized end-to-end GStreamer pipelines using OpTIFlow
Checkout Edge AI marketplace demos
The SDK is packaged with networks which does 3 DL tasks as below,
Image Classification: Detects top N most approximate classes in the dataset for the given input frame
Object Detection: Detects and draws bounding boxes around the objects, also classifies the objects to one of the classes in dataset
Semantic Segmentation: Classifies each pixel into class in dataset
3.1. Out-of-box Edge AI Gallery¶
When the TDA4VM EVM is powered on with SD card in place, the Edge AI Gallery comes up on boot as shown. One can connect a USB 2.0 mouse and click on the buttons in the left panel which starts the Edge AI application running the selected DL task. In the background, a GStremer pipeline is launched which reads a compressed video file and runs a DL network on the decoded content. The output of DL inference is overlayed on the image and sent to display.
Users can select different DL tasks to execute on the compressed video. There is also a “Custom” button, which when pressed can select the input source, which can a compressed video file (H.264/H.265), USB camera or IMX219 camera. One can also select from a list of pre-imported DL networks available in the filesystem and start the application. This will automatically construct a GStreamer pipeline with required elements and launch the application.
3.2. Python/C++ apps¶
Python based demos are simple executable scripts written for image classification, object detection and semantic segmentation. Demos are configured using a YAML file. Details on configuration file parameters can be found in Configuring applications
Sample configuration files for out of the box demos can be found in
edgeai-gst-apps/configs this folder also contains a template config file
which has brief info on each configurable parameter edgeai-gst-apps/configs/app_config_template.yaml
Here is how a Python based image classification demo can be run,
/opt/edgeai-gst-apps/apps_python# ./app_edgeai.py ../configs/image_classification.yaml
The demo captures the input frames from connected USB camera and passes through pre-processing, inference and post-processing before sent to display. Sample output for image classification and object detection demos are as below,
|
|
To exit the demo press Ctrl+C.
C++ apps are cross compiled while packaging, they can be directly tested as given below
/opt/edgeai-gst-apps/apps_cpp# ./bin/Release/app_edgeai ../configs/image_classification.yaml
To exit the demo press Ctrl+C.
C++ apps can be modified and built on the target as well using below steps
/opt/edgeai-gst-apps/apps_cpp# rm -rf build bin lib
/opt/edgeai-gst-apps/apps_cpp# mkdir build
/opt/edgeai-gst-apps/apps_cpp# cd build
/opt/edgeai-gst-apps/apps_cpp/build# cmake ..
/opt/edgeai-gst-apps/apps_cpp/build# make -j2
3.3. OpTIFlow¶
In Edge AI Python and C++ applications, post processing and DL inference are done between appsink and appsrc application boundaries. This makes the data flow sub-optimal because of unnecessary data format conversions to work with open source components.
This is solved by providing DL-inferer plugin which calls one of the supported DL runtime and a post-process plugin which works natively on NV12 format, avoiding unnecessary color formats conversions.
Users can write their own pipeline or use optiflow application to generate and run the end-to-end pipeline. Optiflow application shares the same config file as used by Python/C++ apps.
To directly run the end-to-end pipeline use the following command.
/opt/edgeai-gst-apps/optiflow# ./optiflow.py ../configs/object_detection.yaml
To exit the demo press Ctrl+C.
To just dump the end-to-end pipeline use the following command.
/opt/edgeai-gst-apps/optiflow# ./optiflow.py ../configs/object_detection.yaml -t
Note
Python, C++ and OpTIFlow applications are similar by construction and can accept the same config file. For more information on the config file format please refer to Configuring applications
3.4. Edge AI marketplace¶
Checkout more applications developed using edgeai-gst-apps to solve real world problems