5. Layer Configurations Supported on the NPU¶

Layers in Neural Network Models trained for the NPU can be mapped to the NPU for accelerated execution. This page summarizes the list of supported layers that can be used to run certain layer configurations.

Note

The restrictions specified here do not apply to the software library, because it is a software implementation. Support for layer configs that can run on the hardware NPU accelerator will improve as we implement more microcode.

5.1. Overview of Layer Types¶

First Convolution layer (FCONV, not depth-wise, not point-wise, input feature map channel is 1)

Generic Convolution layer (GCONV, not depth-wise, not point-wise, input feature map channel is a multiple of 4)

Depth-Wise Convolution layer (DWCONV)

Point-Wise Convolution layer (PWCONV)

Point-Wise Convolution with Residual input layer (PWCONVRES)

Transposed Convolution layer (TCONV)

Fully-Connected layer (FC)

Average Pooling layer (AVGPOOL)

Max Pooling layer (MAXPOOL)

5.2. Terminology and Notation¶

On the NPU, a layer computation takes an input feature map, computes with weights such as a convolution kernel, and produces an output feature map.

In the tables below, the column headings indicate the following:

ifmap: input feature map, also known as input tensor

ofmap: output feature map, also known as output tensor

kernel: convolution weights matrix, also known as filter

iB, iH, iW, iC: input feature map bit-width, height, width, channels

oB, oH, oW, oC: output feature map bit-width, height, width, channels

kB, kH, kW: kernel (or pool) bit-width, height, width

sH, sW: stride on height, width

pL, pR, pT, pB: padding input feature map on left, right, top, bottom. In general, padding on the input feature map is supported. When non-zero values are specified in a layer configuration row, it means that the specified padding is handled by the layer implementation on NPU. Otherwise, it means that padding (if any) is supported separately outside of the layer implementation.

In the tables below, the values in the rows below the headings indicate the following:

any: any positive integer value

m4: multiples of 4, e.g., 4, 8, 12, …

m5: multiples of 5, e.g., 5, 10, 15, …

m8b16: multiples of 8, begin with 16 (inclusive), e.g. 16, 24, 32, …

m1b69e72: multiples of 1, begin with 69, end with 72 (inclusive on both ends)

NA: not applicable

5.3. FCONV¶

iB	oB	kB	kH	kW	sH	sW	iH	iW	iC	oH	oW	oC	pL	pR	pT	pB	comment
8	8	2	10	4	2	2	any	any	1	any	any	m16	1	1	5	5
8	8	8	any	1	any	1	any	1	1	m1b2	1	m4	0	0	0	0

5.4. GCONV¶

iB	oB	kB	kH	kW	sH	sW	iH	iW	iC	oH	oW	oC	pL	pR	pT	pB	comment
8	8	2	any	1	any	1	any	1	any	any	1	32	0	0	0	0	kH>2 or iC>4
8	8	8	any	1	any	1	any	1	any	any	1	m4	0	0	0	0	iC is padded to m4

5.5. DWCONV¶

iB	oB	kB	kH	kW	sH	sW	iH	iW	iC	oH	oW	oC	pL	pR
8	8	2	3	3	1	1	any	5	m4	any	5	m4	1	1
8	8	2	9	1	1,2	1	any	1	m4	m5	1	m4	0	0
8	8	2	any	1	any	any	any	any	m4	any	any	m4	0	0

5.6. PWCONV¶

iB	oB	kB	kH	kW	sH	sW	iH	iW	iC	oH	oW	oC	pL	pR	pT	pB	comment
8	8	2	1	1	1	1	any	any	m8b16	any	any	m16	0	0	0	0

5.7. PWCONVRES¶

iB	oB	kB	kH	kW	sH	sW	iH	iW	iC	oH	oW	oC
8	8	2	1	1	1	1	any	any	48	any	any	m24
8	8	2	1	1	1	1	any	any	64	any	any	m16
8	8	2	1	1	1	1	any	any	96	any	any	96

5.8. FC¶

iB	oB	kB	kH	kW	sH	sW	iH	iW	iC	oH	oW	oC	pL	pR	pT	pB	comment
8	8	8	NA	NA	NA	NA	NA	NA	m1b16	NA	NA	any	NA	NA	NA	NA

5.9. AVGPOOL (global)¶

iB	oB	kB	kH	kW	sH	sW	iH	iW	iC	oH	oW	oC	pL	pR	pT	pB	comment
8	8	NA	global	global	NA	NA	any	any	m4	1	1	m4	NA	NA	NA	NA	(iH*iW-5)%4==0

5.10. AVGPOOL (non-global)¶

Non-global AVGPOOL layers are converted to DWCONV layers during compilation. Please refer to DWCONV for supported configs.

5.11. MAXPOOL¶

iB	oB	kB	kH	kW	sH	sW	iH	iW	iC	oH	oW	oC
8	8	NA	any	1	any	any	any	any	m4	any	any	m4
8	8	NA	1	any	any	any	any	any	m4	any	any	m4
8	8	NA	any	2	any	any	any	any	m4	any	any	m4
8	8	NA	2	any	any	any	any	any	m4	any	any	m4
8	8	NA	2	1	any	1	any	1	m4	m1b2	1	m4