DatasheetΒΆ
This chapter describes the performance measurements of the Edge AI Inference demos.
Performance data of the demos can be auto generated by running following command on target:
root@j7-evm:/opt/edge_ai_apps/tests# ./gen_data_sheet.sh
The performence mesurments includes the following
FPS : Effective framerate at which the application runs
Total time : Average time taken to process each frame, which includes pre-processing, inference and post-processing time
Inference time : Average time taken to infer each frame
CPU loading : Loading on different CPU cores present
DDR BW : DDR read and write BW used
HWA Loading : Loading on different Hardware accelerators present
Following are the latest performance numbers of the C++ demos:
Source : USB Camera Capture Framerate : 30 fps Resolution : 720p format : JPEG
Model |
FPS |
Total time (ms) |
Inference time (ms) |
A72 Load (%) |
DDR Read BW (MB/s) |
DDR Write BW (MB/s) |
DDR Total BW (MB/s) |
C71 Load (%) |
C66_1 Load (%) |
C66_2 Load (%) |
MCU2_0 Load (%) |
MCU2_1 Load (%) |
MSC_0 (%) |
MSC_1 (%) |
VISS (%) |
NF (%) |
LDC (%) |
SDE (%) |
DOF (%) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
ONR-CL-6150-mobileNetV2-1p4-qat |
30.48 |
33.06 |
3.02 |
32.30 |
2146 |
1069 |
3215 |
8.16 |
32.86 |
35.28 |
1.0 |
0.0 |
7.15 |
0 |
0 |
0 |
0 |
0 |
0 |
TFL-CL-0000-mobileNetV1-mlperf |
30.46 |
33.05 |
1.51 |
32.92 |
2047 |
1069 |
3116 |
4.32 |
31.8 |
32.92 |
1.0 |
0.0 |
7.4 |
0 |
0 |
0 |
0 |
0 |
0 |
TFL-OD-2020-ssdLite-mobDet-DSP-coco-320x320 |
30.61 |
33.05 |
5.35 |
30.71 |
140 |
1129 |
1269 |
14.17 |
36.90 |
33.48 |
2.0 |
0.0 |
7.12 |
0 |
0 |
0 |
0 |
0 |
0 |
TVM-CL-3410-gluoncv-mxnet-mobv2 |
30.38 |
33.05 |
2.15 |
30.26 |
2084 |
1066 |
3150 |
6.0 |
34.21 |
32.93 |
2.0 |
0.0 |
7.14 |
0 |
0 |
0 |
0 |
0 |
0 |
Source : Video Video Framerate : 25 fps Resolution : 720p Encoding : h264
Model |
FPS |
Total time (ms) |
Inference time (ms) |
A72 Load (%) |
DDR Read BW (MB/s) |
DDR Write BW (MB/s) |
DDR Total BW (MB/s) |
C71 Load (%) |
C66_1 Load (%) |
C66_2 Load (%) |
MCU2_0 Load (%) |
MCU2_1 Load (%) |
MSC_0 (%) |
MSC_1 (%) |
VISS (%) |
NF (%) |
LDC (%) |
SDE (%) |
DOF (%) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
ONR-CL-6150-mobileNetV2-1p4-qat |
25.47 |
39.37 |
3.02 |
27.4 |
2101 |
897 |
2998 |
6.48 |
10.98 |
29.83 |
2.0 |
0.0 |
6.8 |
0 |
0 |
0 |
0 |
0 |
0 |
TFL-CL-0000-mobileNetV1-mlperf |
25.55 |
39.35 |
1.66 |
32.16 |
1999 |
885 |
2884 |
3.75 |
10.9 |
30.78 |
1.0 |
0.0 |
5.95 |
0 |
0 |
0 |
0 |
0 |
0 |
TFL-OD-2020-ssdLite-mobDet-DSP-coco-320x320 |
25.43 |
39.36 |
5.07 |
34.25 |
53 |
944 |
997 |
12.21 |
13.10 |
29.33 |
1.0 |
0.0 |
6.10 |
0 |
0 |
0 |
0 |
0 |
0 |
TVM-CL-3410-gluoncv-mxnet-mobv2 |
25.53 |
39.37 |
2.01 |
27.52 |
2041 |
893 |
2934 |
5.24 |
10.99 |
30.17 |
1.0 |
0.0 |
6.1 |
0 |
0 |
0 |
0 |
0 |
0 |
Source : CSI Camera (ov5640) Capture Framerate : 30 fps Resolution : 720p format : YUYV
Model |
FPS |
Total time (ms) |
Inference time (ms) |
A72 Load (%) |
DDR Read BW (MB/s) |
DDR Write BW (MB/s) |
DDR Total BW (MB/s) |
C71 Load (%) |
C66_1 Load (%) |
C66_2 Load (%) |
MCU2_0 Load (%) |
MCU2_1 Load (%) |
MSC_0 (%) |
MSC_1 (%) |
VISS (%) |
NF (%) |
LDC (%) |
SDE (%) |
DOF (%) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
ONR-CL-6150-mobileNetV2-1p4-qat |
29.41 |
34.01 |
2.04 |
21.25 |
16 |
1096 |
1112 |
7.16 |
43.52 |
30.44 |
1.0 |
0.0 |
6.89 |
0 |
0 |
0 |
0 |
0 |
0 |
TFL-CL-0000-mobileNetV1-mlperf |
29.37 |
34.08 |
1.01 |
21.50 |
2069 |
1095 |
3164 |
4.13 |
41.44 |
30.80 |
1.0 |
0.0 |
6.85 |
0 |
0 |
0 |
0 |
0 |
0 |
TFL-OD-2020-ssdLite-mobDet-DSP-coco-320x320 |
29.40 |
34.02 |
5.03 |
19.14 |
147 |
1151 |
1298 |
13.45 |
47.61 |
31.29 |
1.0 |
0.0 |
6.91 |
0 |
0 |
0 |
0 |
0 |
0 |
TVM-CL-3410-gluoncv-mxnet-mobv2 |
29.45 |
34.03 |
2.00 |
21.94 |
2104 |
1095 |
3199 |
5.74 |
42.91 |
30.73 |
1.0 |
0.0 |
6.85 |
0 |
0 |
0 |
0 |
0 |
0 |