2. Compilation Explained

This section explains how to use the TI Neural Network Compiler for MCUs to compile neural networks.

2.1. Environment Setup

For a particular TI family of MCU devices, begin by downloading the corresponding SDK and setting environment variables to help with compilation.

Note

The TI Neural Network Compiler is supported on Windows and Linux platforms. On Windows, you can use the TI Neural Network Compiler for MCUs in PowerShell or Git BASH. To access Git BASH, download Git for Windows from https://gitforwindows.org/.

2.1.1. Setup for TI C28x Device Family

Follow these steps to set up your environment for using the TI Neural Network Compiler for MCUs for the TI C28x device family.

1. Download and install C2000WARE (version 5.03 or newer) from https://www.ti.com/tool/C2000WARE.

2. Define the following environment variables, specifying your installation path, device, and FPU type:

   Linux

   export C2000WARE_PATH=/path/to/C2000WARE
   export C2000_DEVICE=<your_device>        # e.g. f28p55x or f2837xd or f28p65x
   export C2000_DEVICE_FPU=<fpu32, fpu64>   # e.g. fpu32 for f28p55x and f2837xd, fpu64 for f28p65x
   

   Windows Git BASH

   export C2000WARE_PATH="\path\to\C2000WARE"
   export C2000_DEVICE=<your_device>          # e.g. f28p55x or f2837xd or f28p65x
   export C2000_DEVICE_FPU=<fpu32, fpu64>     # e.g. fpu32 for f28p55x and f2837xd, fpu64 for f28p65x
   

   Windows PowerShell

   $env:C2000WARE_PATH="\path\to\C2000WARE"
   $env:C2000_DEVICE="<your_device>"        # e.g. f28p55x or f2837xd or f28p65x
   $env:C2000_DEVICE_FPU="<fpu32, fpu64>"   # e.g. fpu32 for f28p55x and f2837xd, fpu64 for f28p65x
   

3. Download and install the C28/CLA Code Generation Tools from https://www.ti.com/tool/C2000-CGT.

4. Define the following environment variables, specifying your CGT installation path:

   Linux

   export C2000_CGT_PATH=/path/to/C2000-CGT
   export PATH=$C2000_CGT_PATH/bin:$PATH
   export CL2000_OPTIONS="--float_support=${C2000_DEVICE_FPU} --abi=eabi -O3 --opt_for_speed=5 --c99 -v28 -ml -mt --gen_func_subsections -I${C2000_CGT_PATH}/include -I${C2000WARE_PATH}/driverlib/${C2000_DEVICE}/driverlib -I${C2000WARE_PATH}/device_support/${C2000_DEVICE}/common/include -I."
   

   Windows Git BASH

   export C2000_CGT_PATH="\path\to\C2000-CGT"
   export PATH=$C2000_CGT_PATH\\bin:$PATH
   export CL2000_OPTIONS="--float_support=${C2000_DEVICE_FPU} --abi=eabi -O3 --opt_for_speed=5 --c99 -v28 -ml -mt --gen_func_subsections -I${C2000_CGT_PATH}\include -I${C2000WARE_PATH}\driverlib\\${C2000_DEVICE}\driverlib -I${C2000WARE_PATH}\device_support\\${C2000_DEVICE}\common\include -I."
   

   Windows PowerShell

   $env:C2000_CGT_PATH="\path\to\C2000-CGT"
   $env:PATH=$env:C2000_CGT_PATH + "\bin;" + $env:PATH
   

Note

When setting C2000WARE_PATH and C2000_CGT_PATH on Windows Git BASH or PowerShell, use Windows-style paths with quotations. For example, "C:\somefolder\".

2.2. Compilation Command

The following compilation command example compiles a neural network (model) into a library that can run on an NPU.

Linux / Windows Git BASH

tvmc compile --target="c, ti-npu" --target-c-mcpu=c28 ./int_model.onnx -o artifacts_c28/mod.a --cross-compiler="cl2000" --cross-compiler-options="$CL2000_OPTIONS"

Windows PowerShell

tvmc compile --target="c, ti-npu" .\int_model.onnx --target-c-mcpu=c28 -o artifacts_c28/mod.a --cross-compiler="cl2000" --cross-compiler-options="--float_support=$env:C2000_DEVICE_FPU --abi=eabi -O3 --opt_for_speed=5 --c99 -v28 -ml -mt --gen_func_subsections -I$env:C2000_CGT_PATH\include -I$env:C2000WARE_PATH\driverlib\$env:C2000_DEVICE\driverlib -I$env:C2000WARE_PATH\device_support\$env:C2000_DEVICE\common\include -I."

In this example:

• int_model.onnx is the input model.

• artifacts_c28/mod.a specifies artifacts_c28 as the compilation artifacts directory and mod.a as the generated library.

You can choose different names for the directory and the library.

2.2.1. Compiler Options

You can replace different portions of the example compilation command above using the following options. Note that options following ti-npu can be stacked together. For example, you could change --target="c, ti-npu" to --target="c, ti-npu type=soft skip_normalize=true output_int=true".

Host Processor Options	Description
--target-c-mcpu=c28	Host processor is C28x

Accelerator Options	Description
--target="c, ti-npu"	Run layers on an NPU.
--target="c, ti-npu type=soft"	Run layers with an optimized software implementation on a host processor.

Additional Options	Description
--target="c, ti-npu skip_normalize=true"	Skip float to integer input normalization sequence; see Performance Options for details
--target="c, ti-npu output_int=true"	Skip integer to float output casting; see Performance Options for details

2.3. Compilation Artifacts

Compilation artifacts are stored in the specified artifacts directory–for example, artifacts_c28 in the above compilation command example. This artifacts directory will contain:

• A header file (for example, tvmgen_default.h)

• Generated C code files (for example, lib0/lib1/lib2.c)

• A library file (for example, mod.a)

During compilation, the header file and generated C code files are compiled along with runtime C code files in the tinie-api directory to generate the mod.a library file. This makes the output from the compiler easier to integrate into a CCS project as described in the following section.

2.4. Integrating Compilation Artifacts into a CCS Project

2.4.1. Add to CCS Project

Follow these steps to integrate the output from the TI Neural Network Compiler for MCUs into a CCS project:

1. Copy the library and header file from the compilation artifacts directory into the CCS project for the user application.

2. In the CCS project’s linker command file, place the .rodata.tvm section in FLASH, and place the .bss.noinit.tvm section in SRAM.

3. Ensure that the compiler options used to compile the model are compatible with those in the CCS project properties and the device that will run the application. For example, set the float_support option (C2000_DEVICE_FPU) used to compile the model to fpu32 for F28P55x devices and set it to fpu64 for F28P65x devices.

2.4.2. Hardware NPU Specific

If the hardware NPU accelerator was specified as the target (--target="c, ti-npu") when compiling the neural network, then the CCS project needs the following settings:

1. Place .bss.noinit.tvm in global shared SRAM so that hardware NPU can access it. For example, RAMGS0, RAMGS1, RAMGS2, or RAMGS3.

2.5. Performance Options

2.5.1. Skip Input Feature Normalization (skip_normalize=true)

NPU-trained models have an input feature normalization sequence that converts float input data to quantized int8_t/uint8_t data. By default, TVM generates code for this float to integer conversion.

If the NPU option skip_normalize=true is specified, TVM prunes the model to skip the input feature normalization sequence; instead, it directly computes from integer data. If this option is specified, the user application should provide integer data–instead of float data–as input to the model. That is, the user application should perform input feature normalization outside of the TVM-generated code. The user application may be able to improve performance when processing time-series data in a sliding-window fashion, because the application can choose to normalize only new data in the window, while reusing already normalized results for old data in the window.

TVM logs the bias, scale, and shift parameters used for input feature normalization in a generated header file (for example, tvmgen_default.h). If no input feature normalization sequence is found in the model, the generated model library still uses the data type of the original model input. Please refer to the generated header file in the artifacts directory for details.

2.5.2. Skip Output Cast (output_int=true)

NPU-trained models produce float outputs by default. By default, TVM generates code to cast the model output from integer to float, if the original ONNX model output is in float.

When the option output_int=true is specified, TVM prunes the model to skip the final int to float cast and directly outputs the int, and user applications should interpret the inference results as int.

2.6. Mapping Between tvmc Command-Line Options and Tiny ML ModelMaker Arguments

If you are using TI’s EdgeAI Studio IDE, you will not use the TI Neural Network Compiler for MCUs directly. EdgeAI Studio interfaces with a command-line tool called Tiny ML ModelMaker, which in turn interfaces with the TI Neural Network Compiler.

Tiny ML ModelMaker directly uses parsed arguments instead of the tvmc command line, For example, the --target-c-mcpu=c28 tvmc command-line option is used as an entry {"target-c-mcpu" : "c28", ...} in the args dictionary (python) from the Tiny ML ModelMaker tool.