TI Deep Learning Product User Guide
|
This page lists layers/operators supported in current TIDL-RT version.
Note
In the current release, Only models/networks listed in the Sample Pre-trained CNN Models for TIDL are supported . Any Network which is not included in this list will NOT be supported
More networks (Including custom networks) will be supported in the future releases.
No | TIDL Layer Type | Caffe Layer Type | Tensorflow Ops | ONNX Ops | tflite Ops | Notes |
---|---|---|---|---|---|---|
1 | TIDL_ConvolutionLayer | Convolution ConvolutionDepthwise | Conv2D DepthwiseConv2dNative | Conv | CONV_2D DEPTHWISE_CONV_2D | Regular & depth-wise conv will be imported as conv. For TF and tflite DepthwiseConv2dNative, depth_multiplier shall be 1 in Number of input channels > 1. ReLU & BN layers will be merged into conv to get better performance. 1x1 conv will be converted to innerproduct. Validated kernel size: 1x1, 3x3, 5x5, 7x7,1x3,3x1,1x5,5x1,1x7,7x1. If stride == 4x4, only supported kernel == 11x11. if stride == 2x2, kernel should be 1x1, 3x3, 5x5 or 7x7. Even dimensions of kernel like 2x2, 4x4, 6x6 are not supported. Depthwise Separable Convolution only supports 3x3,5x5,7x7 with stride 1x1 and 3x3 with stride 2x2. Assyemtric strides like 1x2 or 2x1 are NOT supported Dilated Convolution is only supported for non-strided convolution Its recommended to have kernelH*kernelW*input channel/groupNum+enableBias % 64 == 0 whereever possible as it results into better utilization of hardware. Note : Please refer MMALIB release notes for all supported configuration. Note : Some of the kernel combination's are not optimized in current release, please refer MMALIB release notes for the same. |
2 | TIDL_BatchNormLayer | BatchNorm | FusedBatchNorm | BatchNormalization | ReLU & Scale & Bias & PReLU & Leaky Relu will be merged & imported as BN. All the channel-wise Broad cast operations are mapped to BN now. | |
3 | TIDL_PoolingLayer | Pooling | MaxPooling AvgPooling Mean | MaxPool AveragePool GlobalAveragePool | MAX_POOL_2D AVERAGE_POOL_2D MEAN | Validated pooling size: 1x1(MAX, stride 1x1/2x2), 2x2, 3x3. 4x4 pooling is not optimal. |
4 | TIDL_EltWiseLayer | EltWise | Add Mul | Add Mul | ADD MUL | Only support SUM/MAX/PRODUCT. Only support 2 inputs Both the inputs shall be Variable tensors (Not a Constant or Initializer) |
5 | TIDL_InnerProductLayer | InnerProduct | MatMul | Gemm | FULLY_CONNECTED | Input shape must be 1x1x1xN. Please use global pooling/flatten before innerproduct. Feature size larger than 2048*2048 is not optimal. |
6 | TIDL_SoftMaxLayer | SoftMax | Softmax | Softmax | SOFTMAX | Input shape must be 1x1x1xN. Please use global pooling/flatten before softmax. |
7 | TIDL_Deconv2DLayer | Deconvolution | Conv2DTranspose | ConvTranspose | TRANSPOSE_CONV | Only 8x8, 4x4 and 2x2 kernel with 2x2 stride is supported. Recommend to use Resize/Upsample to get better performance. The output feature-map size shall be 2x the input |
8 | TIDL_ConcatLayer | Concat | ConcatV2 | Concat | CONCATENATION | Concat will do channel-wise combination by default. Concat will be width-wise if coming after a flatten layer. used in the context of SSD. Width/Height wise concat is supported with Caffe |
9 | TIDL_SliceLayer | Slice | Slice | Split | NA | Only support channel-wise slice. |
10 | TIDL_CropLayer | Crop | NA | NA | NA | |
11 | TIDL_FlattenLayer | Flatten | NA | Flatten | NA | 16bit is not optimal in current version. |
12 | TIDL_ArgMaxLayer | ArgMax | Argmax | ArgMax | ARG_MAX | Only support axis == 1, mainly for the last layer of sematic segmentation. |
13 | TIDL_DetectionOutputLayer | DetectionOutput | tensorflow Object Detection API | NA | NA | Please refer to comment 1. |
14 | TIDL_ShuffleChannelLayer | ShuffleChannel | NA | Reshape + Transpose + Reshape | NA | |
15 | TIDL_ResizeLayer | NA | ResizeNearestNeighbor ResizeBilinear | UpSample | RESIZE_NEAREST_NEIGHBOR RESIZE_BILINEAR | Only support Power of 2 and symmetric resize. Note that any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. As an example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize |
16 | TIDL_DepthToSpaceLayer | NA | NA | DepthToSpace | DEPTH_TO_SPACE | Supports non-strided convolution with upscale of 2, 4 and 8 |
17 | TIDL_SigmoidLayer | SIGMOID/LOGISTIC | Sigmoid/Logistic | Sigmoid/Logistic | SIGMOID/LOGISTIC | |
18 | TIDL_PadLayer | NA | Pad | Pad | PAD | |
19 | TIDL_ColorConversionLayer | NA | NA | NA | NA | Only YUV420 NV12 format conversion to RGB/BGR color format is supported |
20 | TIDL_BatchReshapeLayer | NA | NA | NA | NA | used to covert batch of images to format which suits TIDL-RT and then convert back, refer here for more details |
21 | TIDL_DataConvertLayer | NA | NA | NA | NA | NA |
22 | TIDL_ScatterElementsLayer | NA | NA | ScatterND | NA | It is supported with following constraints a) 'data' input is ignored and assumed as complete zero value buffer b) 'indices' with only int32 data type, even though operator interface is int64 c) 'updates' data type can be int8,int16,uint8,uint16 d) 'indices' last two dimension should be Nx4, where N is the total number of updates and 4 indicates the batch, channel, height, width location of each update e) 'update' last two dimension should be Nx1, where N represent the total number of updates. The datatype of the ‘update’ is same as the data type of the input ‘data’ f) ‘index_put_’ operator in pytorch can be used to generate this operator in the onnx model |
No | Caffe Layer Type | Tensorflow Ops | ONNX Ops | tflite Ops | Notes |
---|---|---|---|---|---|
1 | Bias | BiasAdd | Bias will be imported as BN. | ||
2 | Scale | Scale will be imported as BN. | |||
3 | ReLU | Relu Relu6 | Relu | RELU RELU6 | ReLU will be imported as BN. |
4 | PReLU | Prelu | PReLU will be imported as BN. | ||
5 | Split | Split | Split layer will be removed after import. | ||
6 | Reshape | Reshape | Reshape | RESHAPE | Please refer to comment 1. |
7 | Permute | Please refer to comment 1. | |||
8 | Priorbox | Please refer to comment 1. | |||
9 | Pad | Pad | PAD | Padding will be taken care of during import process, and this layer will be automatically removed by import tool. | |
10 | MINIMUM | For relu6 / relu8 | |||
11 | DropOut | This layer is only used in training, and this layer will be automatically removed during import process. | |||
12 | Squeeze | Flatten | |||
13 | Shape | Resize | |||
14 | Transpose | For ShuffleChannelLayer | |||
15 | Clip | Parametric activation threshold PACT | |||
16 | LeakyRelu | LEAKY_RELU | Leaky Relu will be imported as BN |
Any unrecognized layers/operators will be converted to TIDL_UnsupportedLayer as a place-holder. The shape & parameters might not be correct. You may get the TIDL-RT importer result, but with such situation imported model will not work for inference on target/PC. |