3.8. Machine Learning

Sitara Machine Learning toolkit brings machine learning to the edge by enabling machine learning inference on all Sitara devices (Arm-only, Arm + GPU, and Arm + specialized hardware accelerators). It is provided as part of TI’s Processor SDK Linux, free to download and use. Sitara machine learning today consists of ONNX Runtime, TensorFlow Lite, Arm NN, NNStreamer, and RNN library.

../_images/Sitara_machine_learning_stack_diagram.png

Fig. 3.3 Sitara Machine Learnining Offering

TensorFlow Lite

  • Open source deep learning runtime for on-device inference.

  • Runs on all Cortex-A ARM cores (AM3x, AM4x, AM5x, AM6x Sitara devices).

  • Imports Tensorflow Lite models.

  • Uses TIDL import tool to create TIDL offloadable Tensorflow Lite models, which can be executed via Tensorflow Lite runtime with TIDL acceleration.

ONNX Runtime

  • Open source inference engine available from Arm.

  • Runs on all Cortex-A ARM cores (AM3x, AM4x, AM5x, AM6x Sitara devices).

Arm NN

  • Open source inference engine available from Arm.

  • Runs on all Cortex-A ARM cores (AM3x, AM4x, AM5x, AM6x Sitara devices).

  • Imports ONNX and TensorFlow Lite models.

  • Provides TensorFlow Lite delegate.

RNN Library

  • Provides Long Short-Term Memory (LSTM) and fully connected layers in a standalone library to allow for rapid prototyping of inference applications that require Recurrent Neural Networks.

  • Runs on all Cortex-A ARM cores (AM3x, AM4x, AM5x, AM6x Sitara devices).

  • Integrated into TI’s Processor SDK Linux in an OOB demo for Predictive Maintenance.

TI Deep Learning (TIDL)

  • Accelerates deep learning inference on C66x DSP cores and/or on Embedded Vision Engine (EVE) subsystems.

  • Available on AM57x device only.

  • Supports CNN at the moment, and imports Caffe, ONNX, and Tensorflow models.

ML inference Library

Version

Delegate / Execution provider

Python API

C/C++ API

TensorFlow Lite

2.18.0

CPU, XNNPACK, ARMNN

YES

YES

ONNX Runtime

1.20.1

CPU, ACL

YES

YES

Arm NN

1.20.1

ACL

YES

YES