TI Deep Learning Product User Guide
|
This page lists layers/operators supported in current TIDL-RT version.
Note
In the current release, Only models/networks listed in the Sample Pre-trained CNN Models for TIDL are supported . Any Network which is not included in this list will NOT be supported
More networks (Including custom networks) will be supported in the future releases.
No | TIDL Layer Type | Caffe Layer Type | Tensorflow Ops | ONNX Ops | tflite Ops | Notes |
---|---|---|---|---|---|---|
1 | TIDL_ConvolutionLayer | Convolution ConvolutionDepthwise | Conv2D DepthwiseConv2dNative | Conv | CONV_2D DEPTHWISE_CONV_2D | Regular & depth-wise conv will be imported as conv. For TF and tflite DepthwiseConv2dNative, depth_multiplier shall be 1 in Number of input channels > 1. ReLU & BN layers will be merged into conv to get better performance. 1x1 conv will be converted to innerproduct. Validated kernel size: 1x1, 3x3, 5x5, 7x7,1x3,3x1,1x5,5x1,1x7,7x1. If stride == 4, only supported kernel == 11x11. if stride == 2, kernel should be less than 7. Even dimensions of kernel like 2x2, 4x4, 6x6 are not supported. Depthwise Separable Convolution only supports 3x3,5x5,7x7 with stride 1 and 3x3 with stride 2. Dilated Convolution is only supported for non-strided convolution Its recommended to have kernelH*kernelW*input channel/groupNum+enableBias % 64 == 0 whereever possible as it results into better utilization of hardware. Note : Please refer MMALIB release notes for all supported configuration. Note : Some of the kernel combination's are not optimized in current release, please refer MMALIB release notes for the same. |
2 | TIDL_BatchNormLayer | BatchNorm | FusedBatchNorm | BatchNormalization | ReLU & Scale & Bias & PReLU & Leaky Relu will be merged & imported as BN. All the channel-wise Broad cast operations are mapped to BN now. | |
3 | TIDL_PoolingLayer | Pooling | MaxPooling AvgPooling Mean | MaxPool AveragePool GlobalAveragePool | MAX_POOL_2D AVERAGE_POOL_2D MEAN | Validated pooling size: 1x1(MAX, stride 1x1/2x2), 2x2, 3x3. 4x4 pooling is not optimal. |
4 | TIDL_EltWiseLayer | EltWise | Add Mul | Add Mul | ADD MUL | Only support SUM/MAX/PRODUCT. Only support 2 inputs. |
5 | TIDL_InnerProductLayer | InnerProduct | MatMul | Gemm | FULLY_CONNECTED | Input shape must be 1x1x1xN. Please use global pooling/flatten before innerproduct. Feature size larger than 2048*2048 is not optimal. |
6 | TIDL_SoftMaxLayer | SoftMax | Softmax | Softmax | SOFTMAX | Input shape must be 1x1x1xN. Please use global pooling/flatten before softmax. |
7 | TIDL_Deconv2DLayer | Deconvolution | Conv2DTranspose | ConvTranspose | TRANSPOSE_CONV | Only 8x8, 4x4 and 2x2 kernel with 2x2 stride is supported. Recommend to use Resize/Upsample to get better performance. The output feature-map size shall be 2x the input |
8 | TIDL_ConcatLayer | Concat | ConcatV2 | Concat | CONCATENATION | Concat will do channel-wise combination by default. Concat will be width-wise if coming after a flatten layer. used in the context of SSD. Width/Height wise concat is supported with Caffe |
9 | TIDL_SliceLayer | Slice | Slice | Split | NA | Only support channel-wise slice. |
10 | TIDL_CropLayer | Crop | NA | NA | NA | |
11 | TIDL_FlattenLayer | Flatten | NA | Flatten | NA | 16bit is not optimal in current version. |
12 | TIDL_ArgMaxLayer | ArgMax | Argmax | ArgMax | ARG_MAX | Only support axis == 1, mainly for the last layer of sematic segmentation. |
13 | TIDL_DetectionOutputLayer | DetectionOutput | tensorflow Object Detection API | NA | NA | Please refer to comment 1. |
14 | TIDL_ShuffleChannelLayer | ShuffleChannel | NA | Reshape + Transpose + Reshape | NA | |
15 | TIDL_ResizeLayer | NA | ResizeNearestNeighbor ResizeBilinear | UpSample | RESIZE_NEAREST_NEIGHBOR RESIZE_BILINEAR | Only support Power of 2 and symmetric resize. Note that any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. As an example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize |
16 | TIDL_DepthToSpaceLayer | NA | NA | DepthToSpace | DEPTH_TO_SPACE | Supports non-strided convolution with upscale of 2, 4 and 8 |
17 | TIDL_SigmoidLayer | SIGMOID/LOGISTIC | Sigmoid/Logistic | Sigmoid/Logistic | SIGMOID/LOGISTIC | |
18 | TIDL_PadLayer | NA | Pad | Pad | PAD | |
19 | TIDL_ColorConversionLayer | NA | NA | NA | NA | Only YUV420 NV12 format conversion to RGB/BGR color format is supported |
20 | TIDL_BatchReshapeLayer | NA | NA | NA | NA | used to covert batch of images to format which suits TIDL-RT and then convert back, refer here for more details |
21 | TIDL_DataConvertLayer | NA | NA | NA | NA | NA |
No | Caffe Layer Type | Tensorflow Ops | ONNX Ops | tflite Ops | Notes |
---|---|---|---|---|---|
1 | Bias | BiasAdd | Bias will be imported as BN. | ||
2 | Scale | Scale will be imported as BN. | |||
3 | ReLU | Relu Relu6 | Relu | RELU RELU6 | ReLU will be imported as BN. |
4 | PReLU | Prelu | PReLU will be imported as BN. | ||
5 | Split | Split | Split layer will be removed after import. | ||
6 | Reshape | Reshape | Reshape | RESHAPE | Please refer to comment 1. |
7 | Permute | Please refer to comment 1. | |||
8 | Priorbox | Please refer to comment 1. | |||
9 | Pad | Pad | PAD | Padding will be taken care of during import process, and this layer will be automatically removed by import tool. | |
10 | MINIMUM | For relu6 / relu8 | |||
11 | DropOut | This layer is only used in training, and this layer will be automatically removed during import process. | |||
12 | Squeeze | Flatten | |||
13 | Shape | Resize | |||
14 | Transpose | For ShuffleChannelLayer | |||
15 | Clip | Parametric activation threshold PACT | |||
16 | LeakyRelu | LEAKY_RELU | Leaky Relu will be imported as BN |
Any unrecognized layers/operators will be converted to TIDL_UnsupportedLayer as a place-holder. The shape & parameters might not be correct. You may get the TIDL-RT importer result, but with such situation imported model will not work for inference on target/PC. |