DatasheetΒΆ
This chapter describes the performance measurements of the Edge AI Inference demos.
Performance data of the demos can be auto generated by running following command on target:
root@j7-evm:/opt/edge_ai_apps/tests# ./gen_data_sheet.sh
The performence mesurments includes the following
FPS : Effective framerate at which the application runs
Total time : Average time taken to process each frame, which includes pre-processing, inference and post-processing time
Inference time : Average time taken to infer each frame
CPU loading : Loading on different CPU cores present
DDR BW : DDR read and write BW used
HWA Loading : Loading on different Hardware accelerators present
Following are the latest performance numbers of the C++ demos:
Source : USB Camera Capture Framerate : 30 fps Resolution : 720p format : JPEG
Model |
FPS |
Total time (ms) |
Inference time (ms) |
A72 Load (%) |
DDR Read BW (MB/s) |
DDR Write BW (MB/s) |
DDR Total BW (MB/s) |
C71 Load (%) |
C66_1 Load (%) |
C66_2 Load (%) |
MCU2_0 Load (%) |
MCU2_1 Load (%) |
MSC_0 (%) |
MSC_1 (%) |
VISS (%) |
NF (%) |
LDC (%) |
SDE (%) |
DOF (%) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
ONR-CL-6150-mobileNetV2-1p4-qat |
31.13 |
33.24 |
3.02 |
15.18 |
1741 |
771 |
2512 |
8.0 |
66.0 |
33.0 |
5.0 |
1.0 |
14.92 |
0 |
0 |
0 |
0 |
0 |
0 |
TFL-CL-0000-mobileNetV1-mlperf |
31.15 |
33.20 |
1.90 |
23.50 |
1636 |
766 |
2402 |
5.0 |
65.0 |
32.0 |
4.0 |
1.0 |
14.55 |
0 |
0 |
0 |
0 |
0 |
0 |
TFL-OD-2020-ssdLite-mobDet-DSP-coco-320x320 |
30.76 |
33.67 |
5.21 |
17.97 |
1794 |
838 |
2632 |
15.0 |
69.0 |
32.0 |
4.0 |
1.0 |
15.13 |
0 |
0 |
0 |
0 |
0 |
0 |
TVM-CL-3410-gluoncv-mxnet-mobv2 |
31.07 |
33.21 |
2.02 |
21.41 |
1661 |
754 |
2415 |
6.0 |
67.0 |
32.0 |
4.0 |
1.0 |
14.93 |
0 |
0 |
0 |
0 |
0 |
0 |
Source : Video Video Framerate : 25 fps Resolution : 720p Encoding : h264
Model |
FPS |
Total time (ms) |
Inference time (ms) |
A72 Load (%) |
DDR Read BW (MB/s) |
DDR Write BW (MB/s) |
DDR Total BW (MB/s) |
C71 Load (%) |
C66_1 Load (%) |
C66_2 Load (%) |
MCU2_0 Load (%) |
MCU2_1 Load (%) |
MSC_0 (%) |
MSC_1 (%) |
VISS (%) |
NF (%) |
LDC (%) |
SDE (%) |
DOF (%) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
ONR-CL-6150-mobileNetV2-1p4-qat |
25.82 |
39.62 |
3.03 |
9.21 |
1722 |
1125 |
2847 |
2.0 |
16.0 |
10.0 |
1.0 |
1.0 |
11.20 |
0 |
0 |
0 |
0 |
0 |
0 |
TFL-CL-0000-mobileNetV1-mlperf |
25.94 |
39.58 |
2.02 |
6.47 |
684 |
67 |
751 |
1.0 |
1.0 |
1.0 |
1.0 |
1.0 |
11.97 |
0 |
0 |
0 |
0 |
0 |
0 |
TFL-OD-2020-ssdLite-mobDet-DSP-coco-320x320 |
25.68 |
39.65 |
5.06 |
7.48 |
691 |
75 |
766 |
2.0 |
1.0 |
1.0 |
1.0 |
0.0 |
7.35 |
0 |
0 |
0 |
0 |
0 |
0 |
TVM-CL-3410-gluoncv-mxnet-mobv2 |
25.92 |
39.56 |
2.02 |
8.16 |
699 |
109 |
808 |
1.0 |
1.0 |
1.0 |
1.0 |
1.0 |
6.40 |
0 |
0 |
0 |
0 |
0 |
0 |
Source : CSI Camera (ov5640) Capture Framerate : 30 fps Resolution : 720p format : YUYV
Model |
FPS |
Total time (ms) |
Inference time (ms) |
A72 Load (%) |
DDR Read BW (MB/s) |
DDR Write BW (MB/s) |
DDR Total BW (MB/s) |
C71 Load (%) |
C66_1 Load (%) |
C66_2 Load (%) |
MCU2_0 Load (%) |
MCU2_1 Load (%) |
MSC_0 (%) |
MSC_1 (%) |
VISS (%) |
NF (%) |
LDC (%) |
SDE (%) |
DOF (%) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
ONR-CL-6150-mobileNetV2-1p4-qat |
30.60 |
34.15 |
3.04 |
64.32 |
2731 |
1049 |
3780 |
9.0 |
75.0 |
33.0 |
4.0 |
1.0 |
14.29 |
0 |
0 |
0 |
0 |
0 |
0 |
TFL-CL-0000-mobileNetV1-mlperf |
29.84 |
34.20 |
1.75 |
10.83 |
1660 |
802 |
2462 |
5.0 |
72.0 |
32.0 |
4.0 |
1.0 |
14.41 |
0 |
0 |
0 |
0 |
0 |
0 |
TFL-OD-2020-ssdLite-mobDet-DSP-coco-320x320 |
29.95 |
34.13 |
5.40 |
13.85 |
1849 |
884 |
2733 |
15.0 |
76.0 |
32.0 |
4.0 |
1.0 |
14.32 |
0 |
0 |
0 |
0 |
0 |
0 |
TVM-CL-3410-gluoncv-mxnet-mobv2 |
29.89 |
34.19 |
2.00 |
10.85 |
1685 |
792 |
2477 |
6.0 |
73.0 |
32.0 |
4.0 |
1.0 |
14.34 |
0 |
0 |
0 |
0 |
0 |
0 |
Source : CSI Camera with VISS (imx219) Capture Framerate : 30 fps Resolution : 1080p format : SRGGB8
Model |
FPS |
Total time (ms) |
Inference time (ms) |
A72 Load (%) |
DDR Read BW (MB/s) |
DDR Write BW (MB/s) |
DDR Total BW (MB/s) |
C71 Load (%) |
C66_1 Load (%) |
C66_2 Load (%) |
MCU2_0 Load (%) |
MCU2_1 Load (%) |
MSC_0 (%) |
MSC_1 (%) |
VISS (%) |
NF (%) |
LDC (%) |
SDE (%) |
DOF (%) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
ONR-CL-6150-mobileNetV2-1p4-qat |
30.39 |
33.16 |
3.02 |
11.77 |
1778 |
842 |
2620 |
8.0 |
46.0 |
34.0 |
9.0 |
1.0 |
30.17 |
0 |
11.26 |
0 |
0 |
0 |
0 |
TFL-CL-0000-mobileNetV1-mlperf |
30.30 |
33.15 |
2.00 |
9.94 |
1669 |
840 |
2509 |
5.0 |
46.0 |
34.0 |
9.0 |
1.0 |
35.99 |
0 |
11.11 |
0 |
0 |
0 |
0 |
TFL-OD-2020-ssdLite-mobDet-DSP-coco-320x320 |
30.37 |
33.16 |
5.01 |
14.14 |
1843 |
913 |
2756 |
16.0 |
49.0 |
34.0 |
8.0 |
1.0 |
29.92 |
0 |
11.14 |
0 |
0 |
0 |
0 |
TVM-CL-3410-gluoncv-mxnet-mobv2 |
30.31 |
33.16 |
2.00 |
12.34 |
1695 |
828 |
2523 |
7.0 |
47.0 |
33.0 |
8.0 |
1.0 |
30.47 |
0 |
11.12 |
0 |
0 |
0 |
0 |