3.2.2.19. NAND

Introduction

TI infrastructure for NAND Flash devices

TI’s SoC interface with NAND Flash devices via on-chip GPMC (General Purpose Memory Controller) interface or via AEMIF depending on the SoC.

For devices that include GPMC: The ECC algorithms required by NAND devices to protect their data, are managed by two independent hardware engines:

  • GPMC ECC engine: used for calculating ECC checksum while writing and reading the NAND device.

  • ELM ECC engine: used for locating and decoding ECC errors while reading the NAND device.

Depending on the SoC, a different set of NAND IP and drivers might be in use. The following table summarizes the NAND IP and drivers used for each SoC family.

Soc Family

NAND IP (drivers)

OMAP2420, OMAP3430

GPMC (omap2.o)

OMAP4430, OMAP5430

GPMC (omap2.o), ELM (omap_elm.o)

AM335x, AM437x, AM57x, AM65x

GPMC (omap2.o), ELM (omap_elm.o)

DM814, DM816

GPMC (omap2.o), ELM (omap_elm.o)

J5 (DRA62x), J6 (DRA72x), J721e

GPMC (omap2.o), ELM (omap_elm.o)

K2G

GPMC (omap2.o), ELM (omap_elm.o)

K2E, K2HK, K2L

AEMIF (davinci_nand.o)

DA850

AEMIF (davinci_nand.o)

Supported Features

GPMC NAND driver supports:

  • NAND devices having:

    • bus-width = x8 | x16

    • page-size = 2048 | 4096

    • block-size = 128k | 256k

  • 1-bit Hamming, BCH4, BCH8 and BCH16 ECC schemes.

  • Various transfer modes for different use-cases and applications (like Polled, Polled Prefetch, IRQ and DMA).

  • NAND boot support for custom non-ONFI compatible NAND devices using NAND-I2C boot-mode (Refer Chapter on Initialization in processor’s TRM).

  • Sub-page write

Accessing NAND partitions

Linux

In Linux, NAND partitions are accessed via mtd devices through the Linux MTD subsystem. The mtd device names are named /dev/mtdX, where X is and integer.

Determine NAND Partition MTD Identifier

Within the kernel figuring out the mtd device number that is for a particular NAND partition is simple. A user simply needs to view the list of mtd devices along with its name. Below command will provide this information:

cat /proc/mtd

An example of this output performed on the DRA71x EVM can be seen below.

dev:    size   erasesize  name
mtd0: 00010000 00010000 "QSPI.SPL"
mtd1: 00010000 00010000 "QSPI.SPL.backup1"
mtd2: 00010000 00010000 "QSPI.SPL.backup2"
mtd3: 00010000 00010000 "QSPI.SPL.backup3"
mtd4: 00100000 00010000 "QSPI.u-boot"
mtd5: 00080000 00010000 "QSPI.u-boot-spl-os"
mtd6: 00010000 00010000 "QSPI.u-boot-env"
mtd7: 00010000 00010000 "QSPI.u-boot-env.backup1"
mtd8: 00800000 00010000 "QSPI.kernel"
mtd9: 01620000 00010000 "QSPI.file-system"
mtd10: 00020000 00020000 "NAND.SPL"
mtd11: 00020000 00020000 "NAND.SPL.backup1"
mtd12: 00020000 00020000 "NAND.SPL.backup2"
mtd13: 00020000 00020000 "NAND.SPL.backup3"
mtd14: 00040000 00020000 "NAND.u-boot-spl-os"
mtd15: 00100000 00020000 "NAND.u-boot"
mtd16: 00020000 00020000 "NAND.u-boot-env"
mtd17: 00020000 00020000 "NAND.u-boot-env.backup1"
mtd18: 00800000 00020000 "NAND.kernel"
mtd19: 0f600000 00020000 "NAND.file-system"

As you can see from the above list, mtd devices not only include NAND partitions but other MTD partitions as well (e.g. OSPI). e.g. if you want to access the NAND.file-system partition you need to use /dev/mtd19. Linux knows the partition names, offsets and sizes via the MTD partition entry specified in the boards device tree.

Erasing, Reading and Writing

For the below sections it is important to remember to replace mtdX with the appropriate mtd device that is associated with the particular NAND partition that you want to use.

Erasing
Erasing a NAND partition can be performed by using the below command:
flash_erase /dev/mtdX 0 0
Writing
Writing a NAND partition is usually a two step process. Writing to NAND at a bit level is only able to change a bit from 1 to 0. This is problematic since frequently when writing new data you will need to change many bits from 1 to 0 along with changing some bits from 0 to 1. The only way to get around this is erasing the NAND partition before writing. This is because erasing sets all the bits in a partition to 1.
The command to write to a NAND partition is below:
nandwrite -p /dev/mtdX <filename>
Reading
Reading NAND can be done by running the below command:
nanddump /dev/mtdX -f <filename>

Note: The above command by default will save the full partition contents to the file. If you are interested in only a certain amount of data, additional parameters can be passed to the nanddump utility.

U-boot

Information regarding NAND booting and booting the kernel and file system from NAND can be found in the U-boot User Guide NAND section.

NAND Based File system

The bootloader and u-boot partitions don’t use any filesystem. The images are flased directly to NAND flash.

The Filesystem though uses UBIFS filesystem. NAND flash is prone to bit-flips. UBI + UBIFS takes care of the bit-flips issue and as well as many other things like wear leveling, bad-block management, etc.

Required Software for UBI image creation

Building a UBI file system requires two applications, ubinize and mkfs.ubifs. Both are both provided by mtd-utils package. (sudo apt-get install mtd-utils).

Building a UBI File system image

When building a UBI file system you need to have a directory that contains the exact files and directories layout that you plan to use for your file system. This is similar to the files and directories layout you will use to copy a file system onto a SD card for booting purposes. It is important that your file system size is smaller than the file system partition in the NAND.

Next you need a file named ubinize.cfg. Below contains the exact contents of ubinize.cfg you should use. However, replace <name> with a name of your choosing. e.g. rootfs

ubinize.cfg contents:

[ubifs]
 mode=ubi
 image=<name>.ubifs
 vol_id=0
 vol_type=dynamic
 vol_name=rootfs
 vol_flags=autoresize

To build a UBI files system requires the below two commands. The symbol <directory path> should be replaced with the path to your directory that you want to include into a ubifs. The symbol <name> should be replaced with the same value you used in creating ubinize.cfg. Make sure you use the same value of <name> across the two commands and ubinize.cfg. The symbols <MKUBIFS ARGS> and <UBINIZE ARGS> are board specific. Replace these values with the values seen in the below table based on the TI EVM you are using.

Commands to execute:

~# mkfs.ubifs -r <directory path> -o <name>.ubifs <MKUBIFS ARGS>
~# ubinize -o <name>.ubi <UBINIZE ARGS> ubinize.cfg

Once these commands are executed <name>.ubi can then be flashed into the NAND’s file-system partition.

Board Name

MKUBIFS Args

UBINIZE Args

AM335X GP EVM

-F -m 2048 -e 126976 -c 5600

-m 2048 -p 128KiB -s 512 -O 2048

AM437x GP EVM

-F -m 4096 -e 253952 -c 2650

-m 4096 -p 256KiB -s 4096 -O 4096

K2E EVM

-F -m 2048 -e 126976 -c 3856

-m 2048 -p 128KiB -s 2048 -O 2048

K2L EVM

-F -m 4096 -e 253952 -c 1926

-m 4096 -p 256KiB -s 4096 -O 4096

K2G EVM

-F -m 4096 -e 253952 -c 1926

-m 4096 -p 256KiB -s 4096 -O 4096

DRA71x EVM

-F -m 2048 -e 126976 -c 8192

-m 2048 -p 128KiB -s 512 -O 2048

AM64 GP EVM

-F -m 4096 -e 258048 -c 3970

-m 4096 -p 256KiB -s 1024 -O 1024

AM62 LP SK

-F -m 4096 -e 258048 -c 3970

-m 4096 -p 256KiB -s 1024 -O 1024

Table: Table of Parameters to use for Building UBI filesystem image


Flashing the UBI File system image to NAND

Copy the UBI image created above to the EVM via SD card or Network. Use ubiformat from EVM Linux to flash the UBI filesystem to the NAND file-system partition.

~# ubiformat -f rootfs.ubi /dev/mtd<X>

Alternatively, you can use u-boot to download and flash the UBI image to the NAND flash over USB in DFU mode (Not Supported in AM64 in 8.2).

At EVM u-boot:

=> setenv dfu_alt_info ${dfu_alt_info_nand}
=> dfu 0 nand list
DFU alt settings list:
dev: NAND alt: 0 name: NAND.tiboot3 layout: RAW_ADDR
dev: NAND alt: 1 name: NAND.tispl layout: RAW_ADDR
dev: NAND alt: 2 name: NAND.tiboot3.backup layout: RAW_ADDR
dev: NAND alt: 3 name: NAND.u-boot layout: RAW_ADDR
dev: NAND alt: 4 name: NAND.u-boot-env layout: RAW_ADDR
dev: NAND alt: 5 name: NAND.u-boot-env.backup layout: RAW_ADDR
dev: NAND alt: 6 name: NAND.file-system layout: RAW_ADDR

=> dfu 0 nand 0

At Host PC: Attach a micro-USB cable between the PC and the EVMs USB port.

linux-pc$ sudo dfu-util -l
Found DFU: [0451:6165] ver=0224, devnum=7, cfg=1, intf=0, path="3-13.1", alt=6, name="NAND.file-system", serial="0000000000000280"
Found DFU: [0451:6165] ver=0224, devnum=7, cfg=1, intf=0, path="3-13.1", alt=5, name="NAND.u-boot-env.backup", serial="0000000000000280"
Found DFU: [0451:6165] ver=0224, devnum=7, cfg=1, intf=0, path="3-13.1", alt=4, name="NAND.u-boot-env", serial="0000000000000280"
Found DFU: [0451:6165] ver=0224, devnum=7, cfg=1, intf=0, path="3-13.1", alt=3, name="NAND.u-boot", serial="0000000000000280"
Found DFU: [0451:6165] ver=0224, devnum=7, cfg=1, intf=0, path="3-13.1", alt=2, name="NAND.tiboot3.backup", serial="0000000000000280"
Found DFU: [0451:6165] ver=0224, devnum=7, cfg=1, intf=0, path="3-13.1", alt=1, name="NAND.tispl", serial="0000000000000280"
Found DFU: [0451:6165] ver=0224, devnum=7, cfg=1, intf=0, path="3-13.1", alt=0, name="NAND.tiboot3", serial="0000000000000280"

linux-pc$ sudo dfu-util -D rootfs.ubi -a NAND.file-system -v
Opening DFU capable USB device...
ID 0451:6165
Run-time device DFU version 0110
Claiming USB DFU Interface...
Setting Alternate Setting #6 ...
Determining device status: state = dfuIDLE, status = 0
dfuIDLE, continuing
DFU mode device DFU version 0110
Device returned transfer size 4096
Copying data from PC to DFU device
Download        [=                        ]   5%      4980736 bytes
Download        [=========================] 100%     90177536 bytes
Download done.
Sent a total of 90177536 bytes
state(7) = dfuMANIFEST, status(0) = No error condition is present
state(2) = dfuIDLE, status(0) = No error condition is present
Done!

Board specific configurations

Following table gives details about NAND devices present on various EVM boards

EVM

NAND Part #

Size

Bus-Widt h

Block-Si ze (KB)

Page-Siz e (KB)

OOB-Size (bytes)

ECC Scheme

Hardware

AM335x GP

MT29F2G0 8AB

256 MB

8

128

2

64

BCH 8

GPMC

AM437x GP

MT29F4G0 8AB

512 MB

8

256

4

224

BCH 16

GPMC

AM437x EPOS

MT29F4G0 8AB

512 MB

8

256

4

224

BCH 16

GPMC

DRA71x

MT29F2G1 6AADWP:D

256 MB

16

128

2

64

BCH 8

GPMC

K2G

MT29F2G1 6ABAFAWP :F

512 MB

16

128

2

64

BCH 16

GPMC

K2E

MT29F4G0 8ABBDAH4 D

1 GB

8

128

2

64

TBD

AEMIF

K2L

MT29F16G 08ADBCAH 4:C

512 MB

8

256

4

224

TBD

AEMIF |

AM64

MT29F8G0 8ADAFAH4 :F

1024 MB

8

256

4

256

BCH 8

GPMC

AM62

MT29F8G0 8ADAFAH4 :F

1024 MB

8

256

4

256

BCH 8

GPMC

Table: NAND Flash Specification Summary

AM43xx GP EVM

On this board, NAND Flash data lines are muxed with eMMC, so either eMMC or NAND can be used enabled at a time. By default NAND is enabled.

AM43xx EPOS EVM

On this board, NAND Flash control lines are muxed with QSPI, Thus either NAND or QSPI-NOR can be used at a time. By default NAND is enabled.

DRA71x EVM

On the board, NAND Flash signals are muxed between NAND, NOR and Video Out signals. Therefore, to have the signals properly muxed for NAND to work Pin 1 (first pin on the left) must be turned on and Pin 2 must be turned off. Pin 1 and 2 must never be switched on at the same time. Doing so may cause damage to the board or SoC.

AM64 GP EVM

NAND flash is not present on the EVM but needs to be added via an Expansion card (TDMS64DC02EVM) that plugs into the High Speed Expansion (HSE) port.

The NAND flash and SoC supports BCH16 ECC Scheme but the BootROM does not support BCH16. So BCH8 ECC Scheme has been used on this board.

Note

Aside from setting the correct bootmode (SYSBOOT[5:0]) for NAND boot, make sure that The Bus width (SYSBOOT[13]) and Muxed-device (SYSBOOT[12:11]) are set as given in the TRM.

Configurations (GPMC Specific)

How to enable GPMC NAND driver in Linux Kernel ?

GPMC NAND driver can be enabled/disabled via Linux Kernel Configuration tool. Enable below Configs to enable MTD Support along with MTD NAND driver support

$ make menuconfig  ARCH=arm

Device Drivers  --->
  <*> Memory Technology Device (MTD) support  --->
    <*>   Caching block device access to MTD devices
    <*>   Enable UBI - Unsorted block images  --->
    NAND  --->
        <*> Raw/Parallel NAND Device Support  --->
            <*>   OMAP2, OMAP3, OMAP4 and Keystone NAND controller
            [*]     Support hardware based BCH error correction
    Partition parsers  --->
        [*]   Command line partition table parsing
        <*> OpenFirmware (device tree) partitioning parser

Transfer Modes

Choose correct bus transfer mode

The GPMC NAND driver support following different modes of transfers data to external NAND device.

  • “prefetch-polled” Prefetch polled mode (default)

  • “polled” Polled mode, without prefetch

  • “prefetch-dma” Prefetch enabled DMA mode

  • “prefetch-irq” Prefetch enabled IRQ mode

Transfer mode can be configured in linux-kernel via DT binding <ti,nand-xfer-type> Refer: Linux kernel_docs @ $LINUX/Documentation/devicetree/bindings/mtd/ti,gpmc-nand.yaml

DMA vs Non DMA Mode (PIO Mode)

The NAND interface is a low speed interface when compared to the main CPU. This means for most CPU frequencies
if the CPU is reading the NAND buffers via polling then its fully capable of reading the NAND at its maximum speed.
Of course the trade off being that the CPU while polling the NAND is not capable of doing anything else thus significantly
increasing the overall CPU load.
DMA performs best when it can read large amount of data at a time. This is necessary since the overhead in setting up, executing and returning from a DMA request is not insignificant so to compensate its best for the DMA to read/write as much data as possible. This provides a dual purpose of significant reduction in CPU load for an operation and also high performance.

The current NAND subsystem within Linux currently deals with reading a single page from the NAND at a time. Unfortunately, the page size is small enough that the overhead for using the DMA (including Linux DMA software stack) negatively impacts the performance. Based on nand performance tests done in early 2016 using the DMA reduced NAND read and write performance by 10-20% depending on SOC. However, cpu load when using polling via the same NAND test were around 99%. When using DMA mode the CPU load for reading was around 35%-54% and for writing was around 15%-30% depending on SOC.

Performance optimizations on NAND

Tweak NAND device signal timings

Much of the NAND throughput can be improved by matching GPMC signal timings with NAND device present on the board. Although GPMC signal timing configurations are not same as those given in NAND device datasheets, but they can be easily derived based on details given in GPMC Controller functional specification.

  • Details of GPMC Signal Timing configurations and how to use them can be found in TI’s Processor TRM

Chapter General Purpose Memory Controller Section Signal Control

  • In Linux, GPMC signal timing configurations are specified via DTB.

Refer kernel_docs $LINUX/Documentation/devicetree/bindings/memory-controllers/ti,gpmc.yaml

Some timing configurations like <gpmc,rd-cycle-ns>, <gpmc,wr-cycle-ns> have larger impact on NAND throughput than others.

Tweaking UBIFS

Additional Resources

Following links should help you better understand NAND Flash as technology.

http://www.linux-mtd.infradead.org/doc/nand.html http://www.linux-mtd.infradead.org/doc/ubi.html http://www.linux-mtd.infradead.org/doc/ubifs.html https://wiki.linaro.org/Flash%20memory https://lwn.net/Articles/428584/