TI Deep Learning Library User Guide
TIDL Performance Summary
  • 1 RGB image is used for Performance Bench-marking
  • 8-bit mode is used for Both Network Parameters and feature vectors
  • The Performance numbers are time (in milli seconds) required to process One Image with C7x-MMA @ 1000Mhz, DDR @ 3733 on J7ES EVM
  • All the layers in the current TIDL SW version is not fully optimized. Expected to improve with future optimizations and shall be close to performance generated with Performance simulation tool.

Notes

  1. 1024x1024 resolution cases performance are excepted the improve in the next release. The 1024x124 performance would around 2x of 1024x512 in the planned future optimization for the networks. The similar scaling would be expected for higher resolutions like 2MP as well. The current Performance simulator numbers does not include this optimization.
  2. The few layers in the shuffle net not optimized, performance is expected to improve in the next release.
  3. The SSD network performance includes the post processing as well (Bos decoding, score computation, NMS, sorting etc)

Classification / Feature Extraction Networks

224x224

Si No Net Net GMACS Time (ms) - in v1.0 Performance simulator
1 MobileNet-1.0 V1 0.569 1.46 0.91
2 MobileNet-1.0 V2 0.314 2.27 1.19
3 InceptionNet V1 1.518 3.39 1.29
4 SqueezeNet 0.391 1.36 0.52
5 resnet50 (TF) 3.505 6.3 3.44
6 resnet18 v1 (Onnx) 1.821 2.98 1.58

1024x512

Si No Net Net GMACS Time (ms) - in v1.0 Performance simulator
1 MobileNet-1.0 V1 6.091 5.95 5.43
2 MobileNet-1.0 V2 3.265 7.03 6.29
3 InceptionNet V1 15.857 14.52 9.48
4 SqueezeNet 3.990 7.97 6.3
5 resnet50 (TF) 36.611 37.76 33.36
6 resnet18 v1 (Onnx) 19.027 13.78 10.17
7 ShuffleNet v1 (Onnx) 1.394 9.36 2.97

1024x1024 (Refer Notes 1)

Si No Net Net GMACS Time (ms) - in v1.0 Performance simulator
1 MobileNet-1.0 V1 12.181 17.8 17.86
2 MobileNet-1.0 V2 6.530 19.94 19.95
3 InceptionNet V1 31.714 41.69 24.68
4 SqueezeNet 8.071 26.15 17.02
5 resnet50 (TF) 73.223 89.16 77.1
6 resnet18 v1 (Onnx) 38.053 34.03 22.74
7 ShuffleNet v1 (Onnx) 2.788 20.42 6.97

Application Networks - Used in SDK Demos

Si No Net Application Input Resolution Net GMACS Time (ms) - in v1.0 Performance simulator
1 MobileNet-1.0 V1 + SSD (tidl_net_jpsdNet ) Parking Spot 512x512 3.654 4.86 3.9
2 MobileNet-1.0 V1 + SSD (tidl_net_jvdNet ) Vehicle Detection 512x512 3.654 4.86 3.9
2 MobileNet-1.0 V1 + SSD (tidl_net_ssd ) Parking Spot + Vehicle Detection 768x384 3.6495 4.49 3.4
4 MobileNet-1.0 V2 + ASPP (tidl_net_onnx_tiadsegNet_v2) Semantic Segmentation 768x384 3.679 6.2 4.35