3.2.3. LTP-DDT Validation¶
Document License
This work is licensed under the Creative Commons Attribution-Share Alike 3.0 United States License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/3.0/us/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA.
LTP-DDT Overview
LTP-DDT is a test application used by Texas Instruments to validate Linux releases.
LTP-DDT uses LTP’s test infrastructure, such as:
Test execution drivers (PAN)
Top-level test scripts (i.e. runltp)
Same Folder Hierarchy and test case definition format
LTP-DDT test cases are LTP test cases and vice-versa.
The main additions or ‘enhacements’ of LTP-DDT compared to LTP are:
PLATFORM files. LTP-DDT uses PLATFORM files to identify platform hardware and software features.
OVERRIDE mechanism. Default test case parameters are automatically overridden based on PLATFORM features.
ATOMIC scripts. Code reuse is foster by writing scripts that implement small well-defined actions. Test scripts rely on these atomic scripts to execute their actions.
AUTOMATIC FILTERING. Test cases are filtered based on the test requirements and the PLATFORM features.
TESTCASE ANNOTATIONS. Test scenario files are annotated with following annotations @name, @desc, @requires and @setup_requires. The @requires and @setup_requires are used to select test cases at run time based on the PLATFORM features.
All LTP-DDT test cases and test code reside in <testcases-root>/ddt/ and <testcode-root>/ddt/ folders respectively.
LTP-DDT Highlights
Easy to use (automatically filter test cases not applicable for platform)
Easy to support new platforms (just define the platform file)
Test cases can be easily wrap or imported to Test Management Systems (Use of testcase annotations facilitates this)
High Code Reuse (atomic scripts and test scripts are reused and parameters are adjusted on the fly)
Test Suites
alsa
cpu hotplug
crypto
timers
emmc
mmc/sd
ethernet
fbdev
gpio
gstreamer (multimedia)
hdmi
i2c
ipc
latency under different use cases (important for RT kernel)
lmbench
memory tests
mm (ltp’s memory management)
msata
nand
nor
pci
pipes (ltp)
power management
programmable real-time unit (PRU)
pwm
qspi
realtime (ltp)
rng
rtc
sata
scheduler (ltp)
sgx (graphics)
smp
spi
syscalls (ltp)
system (use-cases, e.g. multiple tests running in parallel)
thermal
timers (ltp)
touchscreen
uart
usb host (multiple tests with different classes)
usb device
v4l2
vlan
dwt
wlan
Device Under Tests Supported
LTP-DDT has been used on following devices:
LTP-DDT Devices
am170x-evm
am389x-evm
am57xx-beagle-x15
da850-omapl138-evm
dra72x-hsevm
k2e-evm
omap3evm
am180x-evm
am437x-idk
am57xx-evm
dm365-evm
dra76x-evm
k2e-hsevm
omap5-evm
am181x-evm
am437x-sk
am57xx-hsevm
dm368-evm
dra7xx-evm
k2g-evm
omapl138-lcdk
am335x-evm
am43xx-epos
am654x-evm
dm385-evm
dra7xx-hsevm
k2g-hsevm
pandaboard-es
am335x-hsevm
am43xx-gpevm
am654x-hsevm
dm6467-evm
dragonboard410c
k2g-ice
sdp3430
am335x-ice
am43xx-hsevm
am654x-idk
dm813x-evm
hikey
k2hk-evm
tci6614-evm
am335x-sk
am571x-idk
beagleboard
dra71x-evm
j721e-evm
k2hk-hsevm
ti811x-evm
am3517-evm
am572x-idk
beaglebone
dra71x-hsevm
j721e-evm-ivi
k2l-evm
ti813x-evm
am37x-evm
am574x-hsidk
beaglebone-black
dra71x-lcard
j721e-idk-gw
k2l-hsevm
am387x-evm
am574x-idk
da830-omapl137-evm
dra72x-evm
Host Platform Requirements
Linux host is required :
for compiling LTP-DDT.
to host the NFS server to boot the EVM with NFS as root filesystem
to run host utilities - e.g.iperf
Host Software Requirements
GCC Tool chain for ARM
Serial console terminal application
TFTP and NFS servers. NFS server is required only in case of NFS boot.
iperf utility on the host.
Filesystem Requirements
LTP-DDT relies on other open source test tools. The following test tools must be available in the target filesystem to run ltp-ddt:
alsa utilities
evtest
hdparm
iperf
lmbench
rt-tests (cyclictest)
There is an Arago/OE recipe here that builds a filesystem image w/ the above tools plus:
bonnie++
iozone3
ltp-ddt
Installation
Clone the project
git clone https://git.ti.com/git/test-automation/ltp-ddt.git
Running Tests
Run DDT tests the same way you run LTP tests. Use ltprun program and pass to
it the test scenario file in the runtest directory (option -f) to run and the platform (option -P) to use. For example:
./runltp -P am180x-evm -f ddt/lmbench
In addition to selecting test scenarios using -f option, users can also
The runltp script have lot of options. Some useful ones for stress tests are:
-t DURATION: Define duration of the test in s,m,h,d.
-x INSTANCES: Run multiple test instances in parallel.
-c <options>: Run test under additional background CPU load
-D <options>: Run test under additional background load on Secondary storage
-m <options>: Run test under additional background load on Main memory
-i <options>: Run test under additional background load on IO Bus
-n : Run test with network traffic in background.
Please refer to README-DDT file section 8) for more details.
Running NAND Sanity Tests
– Run all NAND sanity tests
Using below command to run NAND sanity tests.
./runltp -P <platform> -s "NAND_S_" -S skiplist
If there are more than one flash filesystem supported, say, jffs2 and ubifs and you don’t run jffs2 test cases. You need create a file called ‘skiplist’ (this filename could be anything) and put to-be-skipped test case tag in this file. Here is the content of skiplist to skip jffs2 test cases.
@ cat skiplist
_JFFS2
– Run NAND performance test
./runltp -P <platform> -s "NAND_L_PERF" -S skiplist
Join