3.2.2.17. NAND
Introduction
TI infrastructure for NAND Flash devices
TI’s SoC interface with NAND Flash devices via on-chip GPMC (General Purpose Memory Controller) interface or via AEMIF depending on the SoC.
For devices that include GPMC: The ECC algorithms required by NAND devices to protect their data, are managed by two independent hardware engines:
GPMC ECC engine: used for calculating ECC checksum while writing and reading the NAND device.
ELM ECC engine: used for locating and decoding ECC errors while reading the NAND device.
Depending on the SoC, a different set of NAND IP and drivers might be in use. The following table summarizes the NAND IP and drivers used for each SoC family.
Soc Family |
NAND IP (drivers) |
---|---|
OMAP2420, OMAP3430 |
GPMC (omap2.o) |
OMAP4430, OMAP5430 |
GPMC (omap2.o), ELM (omap_elm.o) |
AM335x, AM437x, AM57x, AM65x |
GPMC (omap2.o), ELM (omap_elm.o) |
DM814, DM816 |
GPMC (omap2.o), ELM (omap_elm.o) |
J5 (DRA62x), J6 (DRA72x), J721e |
GPMC (omap2.o), ELM (omap_elm.o) |
K2G |
GPMC (omap2.o), ELM (omap_elm.o) |
K2E, K2HK, K2L |
AEMIF (davinci_nand.o) |
DA850 |
AEMIF (davinci_nand.o) |
Supported Features
GPMC NAND driver supports:
NAND devices having:
bus-width = x8 | x16
page-size = 2048 | 4096
block-size = 128k | 256k
1-bit Hamming, BCH4, BCH8 and BCH16 ECC schemes.
Various transfer modes for different use-cases and applications (like Polled, Polled Prefetch, IRQ and DMA).
NAND boot support for custom non-ONFI compatible NAND devices using NAND-I2C boot-mode (Refer Chapter on Initialization in processor’s TRM).
Sub-page write
Accessing NAND partitions
Linux
In Linux, NAND partitions are accessed via mtd devices through the Linux MTD subsystem. The mtd device names are named /dev/mtdX, where X is and integer.
Determine NAND Partition MTD Identifier
Within the kernel figuring out the mtd device number that is for a particular NAND partition is simple. A user simply needs to view the list of mtd devices along with its name. Below command will provide this information:
cat /proc/mtdAn example of this output performed on the DRA71x EVM can be seen below.
dev: size erasesize name mtd0: 00010000 00010000 "QSPI.SPL" mtd1: 00010000 00010000 "QSPI.SPL.backup1" mtd2: 00010000 00010000 "QSPI.SPL.backup2" mtd3: 00010000 00010000 "QSPI.SPL.backup3" mtd4: 00100000 00010000 "QSPI.u-boot" mtd5: 00080000 00010000 "QSPI.u-boot-spl-os" mtd6: 00010000 00010000 "QSPI.u-boot-env" mtd7: 00010000 00010000 "QSPI.u-boot-env.backup1" mtd8: 00800000 00010000 "QSPI.kernel" mtd9: 01620000 00010000 "QSPI.file-system" mtd10: 00020000 00020000 "NAND.SPL" mtd11: 00020000 00020000 "NAND.SPL.backup1" mtd12: 00020000 00020000 "NAND.SPL.backup2" mtd13: 00020000 00020000 "NAND.SPL.backup3" mtd14: 00040000 00020000 "NAND.u-boot-spl-os" mtd15: 00100000 00020000 "NAND.u-boot" mtd16: 00020000 00020000 "NAND.u-boot-env" mtd17: 00020000 00020000 "NAND.u-boot-env.backup1" mtd18: 00800000 00020000 "NAND.kernel" mtd19: 0f600000 00020000 "NAND.file-system"As you can see from the above list, mtd devices not only include NAND partitions but other MTD partitions as well (e.g. OSPI). e.g. if you want to access the NAND.file-system partition you need to use /dev/mtd19. Linux knows the partition names, offsets and sizes via the MTD partition entry specified in the boards device tree.
Erasing, Reading and Writing
For the below sections it is important to remember to replace mtdX with the appropriate mtd device that is associated with the particular NAND partition that you want to use.
ErasingErasing a NAND partition can be performed by using the below command:flash_erase /dev/mtdX 0 0WritingWriting a NAND partition is usually a two step process. Writing to NAND at a bit level is only able to change a bit from 1 to 0. This is problematic since frequently when writing new data you will need to change many bits from 1 to 0 along with changing some bits from 0 to 1. The only way to get around this is erasing the NAND partition before writing. This is because erasing sets all the bits in a partition to 1.The command to write to a NAND partition is below:nandwrite -p /dev/mtdX <filename>ReadingReading NAND can be done by running the below command:nanddump /dev/mtdX -f <filename>Note: The above command by default will save the full partition contents to the file. If you are interested in only a certain amount of data, additional parameters can be passed to the nanddump utility.
U-boot
Information regarding NAND booting and booting the kernel and file system from NAND can be found in the U-boot User Guide NAND section.
NAND Based File system
The bootloader and u-boot partitions don’t use any filesystem. The images are flased directly to NAND flash.
The Filesystem though uses UBIFS filesystem. NAND flash is prone to bit-flips. UBI + UBIFS takes care of the bit-flips issue and as well as many other things like wear leveling, bad-block management, etc.
Required Software for UBI image creation
Building a UBI file system requires two applications, ubinize and mkfs.ubifs. Both are both provided by mtd-utils package. (sudo apt-get install mtd-utils).
Building a UBI File system image
When building a UBI file system you need to have a directory that contains the exact files and directories layout that you plan to use for your file system. This is similar to the files and directories layout you will use to copy a file system onto a SD card for booting purposes. It is important that your file system size is smaller than the file system partition in the NAND.
Next you need a file named ubinize.cfg. Below contains the exact contents of ubinize.cfg you should use. However, replace <name> with a name of your choosing. e.g. rootfs
ubinize.cfg contents:
[ubifs] mode=ubi image=<name>.ubifs vol_id=0 vol_type=dynamic vol_name=rootfs vol_flags=autoresizeTo build a UBI files system requires the below two commands. The symbol <directory path> should be replaced with the path to your directory that you want to include into a ubifs. The symbol <name> should be replaced with the same value you used in creating ubinize.cfg. Make sure you use the same value of <name> across the two commands and ubinize.cfg. The symbols <MKUBIFS ARGS> and <UBINIZE ARGS> are board specific. Replace these values with the values seen in the below table based on the TI EVM you are using.
Commands to execute:
~# mkfs.ubifs -r <directory path> -o <name>.ubifs <MKUBIFS ARGS> ~# ubinize -o <name>.ubi <UBINIZE ARGS> ubinize.cfgOnce these commands are executed <name>.ubi can then be flashed into the NAND’s file-system partition.
Board Name
MKUBIFS Args
UBINIZE Args
AM335X GP EVM
-F -m 2048 -e 126976 -c 5600
-m 2048 -p 128KiB -s 512 -O 2048
AM437x GP EVM
-F -m 4096 -e 253952 -c 2650
-m 4096 -p 256KiB -s 4096 -O 4096
K2E EVM
-F -m 2048 -e 126976 -c 3856
-m 2048 -p 128KiB -s 2048 -O 2048
K2L EVM
-F -m 4096 -e 253952 -c 1926
-m 4096 -p 256KiB -s 4096 -O 4096
K2G EVM
-F -m 4096 -e 253952 -c 1926
-m 4096 -p 256KiB -s 4096 -O 4096
DRA71x EVM
-F -m 2048 -e 126976 -c 8192
-m 2048 -p 128KiB -s 512 -O 2048
AM64 GP EVM
-F -m 4096 -e 258048 -c 3970
-m 4096 -p 256KiB -s 1024 -O 1024
AM62 LP SK
-F -m 4096 -e 258048 -c 3970
-m 4096 -p 256KiB -s 1024 -O 1024
Table: Table of Parameters to use for Building UBI filesystem image
Flashing the UBI File system image to NAND
Copy the UBI image created above to the EVM via SD card or Network. Use ubiformat from EVM Linux to flash the UBI filesystem to the NAND file-system partition.
~# ubiformat -f rootfs.ubi /dev/mtd<X>Alternatively, you can use u-boot to download and flash the UBI image to the NAND flash over USB in DFU mode (Not Supported in AM64 in 8.2).
At EVM u-boot:
=> setenv dfu_alt_info ${dfu_alt_info_nand} => dfu 0 nand list DFU alt settings list: dev: NAND alt: 0 name: NAND.tiboot3 layout: RAW_ADDR dev: NAND alt: 1 name: NAND.tispl layout: RAW_ADDR dev: NAND alt: 2 name: NAND.tiboot3.backup layout: RAW_ADDR dev: NAND alt: 3 name: NAND.u-boot layout: RAW_ADDR dev: NAND alt: 4 name: NAND.u-boot-env layout: RAW_ADDR dev: NAND alt: 5 name: NAND.u-boot-env.backup layout: RAW_ADDR dev: NAND alt: 6 name: NAND.file-system layout: RAW_ADDR => dfu 0 nand 0At Host PC: Attach a micro-USB cable between the PC and the EVMs USB port.
linux-pc$ sudo dfu-util -l Found DFU: [0451:6165] ver=0224, devnum=7, cfg=1, intf=0, path="3-13.1", alt=6, name="NAND.file-system", serial="0000000000000280" Found DFU: [0451:6165] ver=0224, devnum=7, cfg=1, intf=0, path="3-13.1", alt=5, name="NAND.u-boot-env.backup", serial="0000000000000280" Found DFU: [0451:6165] ver=0224, devnum=7, cfg=1, intf=0, path="3-13.1", alt=4, name="NAND.u-boot-env", serial="0000000000000280" Found DFU: [0451:6165] ver=0224, devnum=7, cfg=1, intf=0, path="3-13.1", alt=3, name="NAND.u-boot", serial="0000000000000280" Found DFU: [0451:6165] ver=0224, devnum=7, cfg=1, intf=0, path="3-13.1", alt=2, name="NAND.tiboot3.backup", serial="0000000000000280" Found DFU: [0451:6165] ver=0224, devnum=7, cfg=1, intf=0, path="3-13.1", alt=1, name="NAND.tispl", serial="0000000000000280" Found DFU: [0451:6165] ver=0224, devnum=7, cfg=1, intf=0, path="3-13.1", alt=0, name="NAND.tiboot3", serial="0000000000000280" linux-pc$ sudo dfu-util -D rootfs.ubi -a NAND.file-system -v Opening DFU capable USB device... ID 0451:6165 Run-time device DFU version 0110 Claiming USB DFU Interface... Setting Alternate Setting #6 ... Determining device status: state = dfuIDLE, status = 0 dfuIDLE, continuing DFU mode device DFU version 0110 Device returned transfer size 4096 Copying data from PC to DFU device Download [= ] 5% 4980736 bytes Download [=========================] 100% 90177536 bytes Download done. Sent a total of 90177536 bytes state(7) = dfuMANIFEST, status(0) = No error condition is present state(2) = dfuIDLE, status(0) = No error condition is present Done!
Board specific configurations
EVM |
NAND Part # |
Size |
Bus-Widt h |
Block-Si ze (KB) |
Page-Siz e (KB) |
OOB-Size (bytes) |
ECC Scheme |
Hardware |
---|---|---|---|---|---|---|---|---|
AM335x GP |
MT29F2G0 8AB |
256 MB |
8 |
128 |
2 |
64 |
BCH 8 |
GPMC |
AM437x GP |
MT29F4G0 8AB |
512 MB |
8 |
256 |
4 |
224 |
BCH 16 |
GPMC |
AM437x EPOS |
MT29F4G0 8AB |
512 MB |
8 |
256 |
4 |
224 |
BCH 16 |
GPMC |
DRA71x |
MT29F2G1 6AADWP:D |
256 MB |
16 |
128 |
2 |
64 |
BCH 8 |
GPMC |
K2G |
MT29F2G1 6ABAFAWP :F |
512 MB |
16 |
128 |
2 |
64 |
BCH 16 |
GPMC |
K2E |
MT29F4G0 8ABBDAH4 D |
1 GB |
8 |
128 |
2 |
64 |
TBD |
AEMIF |
K2L |
MT29F16G 08ADBCAH 4:C |
512 MB |
8 |
256 |
4 |
224 |
TBD |
AEMIF | |
AM64 |
MT29F8G0 8ADAFAH4 :F |
1024 MB |
8 |
256 |
4 |
256 |
BCH 8 |
GPMC |
AM62 |
MT29F8G0 8ADAFAH4 :F |
1024 MB |
8 |
256 |
4 |
256 |
BCH 8 |
GPMC |
Table: NAND Flash Specification Summary
AM43xx GP EVM
On this board, NAND Flash data lines are muxed with eMMC, so either eMMC or NAND can be used enabled at a time. By default NAND is enabled.
AM43xx EPOS EVM
On this board, NAND Flash control lines are muxed with QSPI, Thus either NAND or QSPI-NOR can be used at a time. By default NAND is enabled.
DRA71x EVM
On the board, NAND Flash signals are muxed between NAND, NOR and Video Out signals. Therefore, to have the signals properly muxed for NAND to work Pin 1 (first pin on the left) must be turned on and Pin 2 must be turned off. Pin 1 and 2 must never be switched on at the same time. Doing so may cause damage to the board or SoC.
AM64 GP EVM
NAND flash is not present on the EVM but needs to be added via an Expansion card (TDMS64DC02EVM) that plugs into the High Speed Expansion (HSE) port.
The NAND flash and SoC supports BCH16 ECC Scheme but the BootROM does not support BCH16. So BCH8 ECC Scheme has been used on this board.
Note
Aside from setting the correct bootmode (SYSBOOT[5:0]) for NAND boot, make sure that The Bus width (SYSBOOT[13]) and Muxed-device (SYSBOOT[12:11]) are set as given in the TRM.
Configurations (GPMC Specific)
How to enable GPMC NAND driver in Linux Kernel ?
GPMC NAND driver can be enabled/disabled via Linux Kernel Configuration tool. Enable below Configs to enable MTD Support along with MTD NAND driver support
$ make menuconfig ARCH=arm
Device Drivers --->
<*> Memory Technology Device (MTD) support --->
<*> Caching block device access to MTD devices
<*> Enable UBI - Unsorted block images --->
NAND --->
<*> Raw/Parallel NAND Device Support --->
<*> OMAP2, OMAP3, OMAP4 and Keystone NAND controller
[*] Support hardware based BCH error correction
Partition parsers --->
[*] Command line partition table parsing
<*> OpenFirmware (device tree) partitioning parser
Transfer Modes
Choose correct bus transfer mode
The GPMC NAND driver support following different modes of transfers data to external NAND device.
“prefetch-polled” Prefetch polled mode (default)
“polled” Polled mode, without prefetch
“prefetch-dma” Prefetch enabled DMA mode
“prefetch-irq” Prefetch enabled IRQ mode
Transfer mode can be configured in linux-kernel via DT binding <ti,nand-xfer-type> Refer: Linux kernel_docs @ $LINUX/Documentation/devicetree/bindings/mtd/ti,gpmc-nand.yaml
DMA vs Non DMA Mode (PIO Mode)
The current NAND subsystem within Linux currently deals with reading a single page from the NAND at a time. Unfortunately, the page size is small enough that the overhead for using the DMA (including Linux DMA software stack) negatively impacts the performance. Based on nand performance tests done in early 2016 using the DMA reduced NAND read and write performance by 10-20% depending on SOC. However, cpu load when using polling via the same NAND test were around 99%. When using DMA mode the CPU load for reading was around 35%-54% and for writing was around 15%-30% depending on SOC.
Performance optimizations on NAND
Tweak NAND device signal timings
Much of the NAND throughput can be improved by matching GPMC signal timings with NAND device present on the board. Although GPMC signal timing configurations are not same as those given in NAND device datasheets, but they can be easily derived based on details given in GPMC Controller functional specification.
Details of GPMC Signal Timing configurations and how to use them can be found in TI’s Processor TRM
Chapter General Purpose Memory Controller Section Signal Control
In Linux, GPMC signal timing configurations are specified via DTB.
Refer kernel_docs $LINUX/Documentation/devicetree/bindings/memory-controllers/ti,gpmc.yaml
Some timing configurations like <gpmc,rd-cycle-ns>, <gpmc,wr-cycle-ns> have larger impact on NAND throughput than others.
Tweaking UBIFS
Specify -o bulk_read while mounting UBIFS (read ahead)
Tweak Linux VM (kernel knobs for VM)
Additional Resources
Following links should help you better understand NAND Flash as technology.
http://www.linux-mtd.infradead.org/doc/nand.html http://www.linux-mtd.infradead.org/doc/ubi.html http://www.linux-mtd.infradead.org/doc/ubifs.html https://wiki.linaro.org/Flash%20memory https://lwn.net/Articles/428584/