Overview

Introduction

Testing is a very important part of embedded development. However, from project creation to using the debugger to verify expression values, testing quickly becomes a complicated and time-consuming task. Fortunately, Code Composer Studio(CCS) is packaged with many great tools that facilitates the creation of automated testing and ultimately increases the efficiency of the user. These tools are simple to use and offer APIs for popular scripting languages such as Javascript, Python and TCL. This document uses a concrete example to demonstrate how to create a basic automated test for CCS projects.

Example project overview

The SimpleLink Fundamentals Workshop for CCS offers an introductory lesson to new SimpleLink users of CCS. The workshop is based on a basic example project that is shipped with the device SDK. When the example project changes, the workshop would cease to be applicable, therefore it is important to regularly check if the workshop is up to date. To reduce the manual labour involved in testing the workshop, a simple automated test is created to perform the basic tasks in the workshop. It is impossible to remove 100% of the manual work because the workshop involves interacting with the physical device, so the goal of the automated test is to reduce the manual work as much as possible.

Setup and preparations

Required files

To automate the test flow, you need to know the path to the following executables and files:

1. eclipsec.exe(Windows)/eclipse(Linux): You need this for command-line project create/build. You can find this in the CCS folder tree.

   Example path: D:\Builds\CCS8.1.0.00007\ccsv8\eclipse\eclipsec.exe

2. dss.bat(Windows)/dss.sh(Linux): You need this for running Debug Server Script(DSS). You can also find this in the CCS folder tree.

   Example path: D:\Builds\CCS8.1.0.00007\ccsv8\ccs_base\scripting\bin\dss.bat

3. project_name.projectspec: You need this to create the example project. You can find this in the device specific SDK folder tree.

   A more in-depth guide for projectspec can be found here.

   Example path: C:\ti\simplelink_msp432p4_sdk_2_10_00_14\examples\rtos\MSP_EXP432P401R\drivers\empty\tirtos\ccs\ empty_MSP_EXP432P401R_tirtos_ccs.projectspec

   Note: There can be multiple devices in the SDK folder. Remember to specify the correct one!

To make the test independent of CCS and SDK locations, you need to gather their paths from the tester.

One way to do this is through command line arguments:

SET cur_dir=%~dp0

REM ccs_path should be the full windows path up to and including the ccsX folder ex:D:\Builds\CCS8.1.0.00007\ccsv8
SET ccs_path=%~1
IF "%ccs_path%"=="" GOTO BAD_ARG

REM sdk_path should be the root of the sdk folder ex:C:\ti\simplelink_msp432p4_sdk_2_10_00_14\
SET sdk_path=%~2
IF "%sdk_path%"=="" GOTO BAD_ARG

REM device_name should be the device name as shown in the sdk ex:MSP_EXP432P401R,CC1310_LAUNCHXL etc.
SET device_name=%~3
IF "%device_name%"=="" GOTO BAD_ARG

REM Set up eclipsec path
SET "eclipsec_path=%ccs_path%\eclipse\eclipsec.exe"
IF NOT EXIST %eclipsec_path% GOTO BAD_ECLIPSEC

REM Set up the projectspec path
SET "projectspec_path=%sdk_path%\examples\rtos\%device_name%\drivers\empty\tirtos\ccs\empty_%device_name%_tirtos_ccs.projectspec" 
IF NOT EXIST %projectspec_path% GOTO BAD_PROJECTSPEC

REM Set up the dss path
SET "dss_path=%ccs_path%\ccs_base\scripting\bin\dss.bat"
IF NOT EXIST %dss_path% GOTO BAD_DSS

cur_dir=$(pwd)
ccs_path=$1
if [ -z "$ccs_path" ]
then
    echo "Missing the ccs_path argument."
    exit 1
fi
sdk_path=$2
if [ -z "$sdk_path" ]
then
    echo "Missing the sdk_path argument."
    exit 1
fi
device_name=$3
if [ -z "$device_name" ]
then
    echo "Missing the device_name argument."
    exit 1
fi
eclipse_path="$ccs_path/eclipse/eclipse"
echo $eclipse_path
if [ ! -f "$eclipse_path" ]
then
    echo "Missing eclipse at $eclipse_path"
    exit 1
fi

projectspec_path="$sdk_path/examples/rtos/$device_name/drivers/empty/tirtos/ccs/empty_${device_name}_tirtos_ccs.projectspec"
if [ ! -f "$projectspec_path" ]
then
    echo "Missing projectspec at $projectspec_path"
    exit 1
fi
dss_path="$ccs_path/ccs_base/scripting/bin/dss.sh"
if [ ! -f "$dss_path" ]
then
    echo "Missing dss at $dss_path"
    exit 1
fi

Set up the workspace

You should generate a new folder as the working directory for each test you launch. This will keep the workspace organized.

One way to do this is by generating sequential directories:

REM Setting up testing workspace
SET test_num=0
:CREATE_TEST_DIR
SET "test_dir=%cur_dir%\workshopTest\workspace_%test_num%"
IF EXIST %test_dir% GOTO INC_TESTN
MKDIR %test_dir%

...

:INC_TESTN
SET /a test_num+=1
GOTO CREATE_TEST_DIR

test_num=0
test_dir="$cur_dir/workshopTest/workspace_$test_num"
while [ -d "$test_dir" ] 
do
    test_num=$((test_num+1))
    test_dir="$cur_dir/workshopTest/workspace_$test_num"
done
mkdir -p $test_dir

You can decide whether to keep or remove the generated directory once the test finishes running.

Create/build project from command-line

Most of the automation of the test is done through Debug Server Scripting(DSS), but DSS lacks project management APIs. So, the use of CCS terminal tools is required to automate project creation and build.

Create project

You can create the project from the *.projectspec file using the following command on Windows:

eclipsec -noSplash -data "<workspace_dir>" -application com.ti.ccstudio.apps.projectCreate -ccs.projectSpec <file>

In the batch file, it would look like this:

CALL "%eclipsec_path%" -noSplash -data "%test_dir%" -application com.ti.ccstudio.apps.projectCreate -ccs.projectSpec "%projectspec_path%" > "%test_dir%\p_create.log" || GOTO FAIL_CREATE

On Linux, just change eclipsec into eclipse:

eclipse -noSplash -data "<workspace_dir>" -application com.ti.ccstudio.apps.projectCreate -ccs.projectSpec <file>

cmd="$eclipse_path -noSplash -data $test_dir -application com.ti.ccstudio.apps.projectCreate -ccs.projectSpec $projectspec_path > $test_dir/p_create.log"
if ! eval "$cmd"
then
    echo "Project create has failed."
    exit 1
fi

Build project

You can build the project after a the project is successfully created using the following command:

eclipsec -noSplash -data "<workspace_dir>" -application com.ti.ccstudio.apps.projectBuild -ccs.projects <project_name>

In the batch file, it would look like this:

CALL "%eclipsec_path%" -noSplash -data "%test_dir%" -application com.ti.ccstudio.apps.projectBuild -ccs.projects empty_%device_name%_tirtos_ccs > "%test_dir%\p_build.log" || GOTO FAIL_BUILD

Again, just change eclipsec into eclipse when on Linux.

Important generated files

Two important files are generated by the end of project create and build.

1. device_name.ccxml : This is the chip specific target configuration file that is needed to establish a JTAG debug session. Details can be found here.
2. project_name.out : This is the standalone executable file that needs to be flashed onto the device.

You should locate and save their paths for later uses:

FOR /r "%test_dir%" %%a in (*.ccxml) do set CCXML_path=%%~dpnxa
IF "%CCXML_path%"=="" GOTO BAD_CCXML
ECHO Found .ccxml file %CCXML_path%
FOR /r "%test_dir%" %%a in (empty_*.out) do set OUT_path=%%~dpnxa
IF "%OUT_path%"=="" GOTO BAD_OUT
ECHO Found .out file %OUT_path%

ccxml_path=$(find $test_dir -name "*.ccxml")
if [ ! -f $ccxml_path ]
then
    echo "ccxml file not found: $ccxml_path"
fi
out_path=$(find $test_dir -name "empty_*.out")
if [ ! -f $out_path ]
then
    echo "out file not found: $out_path"
fi

Basic project automation using DSS

To automatically run/debug the program, you need use debug server scripting(DSS) to interact with the debug server using various scripting languages(Javascript will be used for this example). These scripts can be created on a per-test basis and are portable across different platforms. In this example, the scripts will be directly called from the basic .sh/.bat test framework, but they can be incorporated into any test framework.

To launch the script, use this command:

Windows:

CALL "%dss_path%" "script_name.js" [arguments] || GOTO FAIL

Linux:

cmd="$dss_path script_name.js $ccs_path $test_dir $ccxml_path $out_path"
if ! eval "$cmd"
then
    echo "Test failed."
    exit 1
fi

The first thing to test is whether the clean example program works correctly. The "empty" project's behavior should be a simple flashing LED on the device. Since the behavior is observed in the physical world, the tester needs to confirm the correct behavior. To do this, the script automatically flashes the device and runs the program. Then, it will prompt Is LED blinking? y/n and wait for a user input to confirm the correct behavior.

1. Setup the debug session

Start the script by obtaining all the path information from the caller(the batch file):

var ccsbase_ppath = arguments[0].replace(/\\/g,"/");
var workspace_path = arguments[1].replace(/\\/g,"/");
var ccxml_path = arguments[2].replace(/\\/g,"/");
var out_path = arguments[3].replace(/\\/g,"/");

Import all the necessary packages:

importPackage(Packages.com.ti.debug.engine.scripting);
importPackage(Packages.com.ti.ccstudio.scripting.environment);
importPackage(Packages.java.lang);
importPackage(Packages.java.io);

Setup the logger, prefferably in the testing workspace created earlier:

script.traceBegin(workspace_path + "/basic_test_out.xml",ccsbase_path + "/ccs_base/scripting/examples/DebugServerExamples/DefaultStylesheet.xsl");
script.traceSetConsoleLevel(TraceLevel.INFO);
script.traceSetFileLevel(TraceLevel.ALL);

Configure the debugger with the target configuration file:

var debugServer = script.getServer("DebugServer.1");
debugServer.setConfig(ccxml_path);

Open a debug session to the desired core:

var debugSession = debugServer.openSession(".*[Cc][Oo][Rr][Tt][Ee][Xx].*");

Finally connect to the target and load the program:

debugSession.target.connect();
debugSession.memory.loadProgram(out_path);

2. Run the program

You have two options when you run the program.

One is using debugSession.target.run(): Issue a run command to the target but don't return control to the host application/script until the target has been halted due to to hitting a breakpoint, hitting the end of a program or timing out.

The other is using debugSession.target.runAsynch(): Issue a run command to the target and return control immediately (even if target is still running) to the host application/script.

For the basic example here, use the runAsynch() function because you want the board to be blinking while you prompt the tester for a confirmation input at the same time.

debugSession.target.runAsynch();
// Verify the behavior is correct
var stdin = new BufferedReader(new InputStreamReader(System['in']));
while(true){
    print("Is LED blinking? y/n");
    var answer=stdin.readLine();
    if(answer.equals("y")){
        script.traceWrite("The example project runs correctly.");
        break;    
    }
    if(answer.equals("n")){
        script.traceWrite("The example project is broken, the LED is not blinking.");
        exitCode=1;
        break;    
    }
}

3. Exit with success or failure

You should terminate the debug script with a proper exit code so that the batch file calling the script can know whether the test has passed or failed.

debugSession.terminate();
debugServer.stop();
script.traceEnd(); 
java.lang.System.exit(exitCode);

var ccsbase_path = arguments[0].replace(/\\/g,"/");
var workspace_path = arguments[1].replace(/\\/g,"/");
var ccxml_path = arguments[2].replace(/\\/g,"/");
var out_path = arguments[3].replace(/\\/g,"/");
/* print(ccsbase_path);
print(workspace_path);
print(ccxml_path);
print(out_path); */
var exitCode=0;
// Import packages
importPackage(Packages.com.ti.debug.engine.scripting);
importPackage(Packages.com.ti.ccstudio.scripting.environment);
importPackage(Packages.java.lang);
importPackage(Packages.java.io);
try{
    // Create scripting environment object
    var script = ScriptingEnvironment.instance();
    // Configure the logger
    script.traceBegin(workspace_path + "/basic_test_out.xml", 
        ccsbase_path + "/ccs_base/scripting/examples/DebugServerExamples/DefaultStylesheet.xsl");
    script.traceSetConsoleLevel(TraceLevel.INFO);
    script.traceSetFileLevel(TraceLevel.ALL);
    // Get the debug server environment
    var debugServer = script.getServer("DebugServer.1");
    // Configure debugger with ccxml
    debugServer.setConfig(ccxml_path);
    // Open debug session
    var debugSession = debugServer.openSession(".*[Cc][Oo][Rr][Tt][Ee][Xx].*");
    // Connect to the target
    debugSession.target.connect();
    // Load Program
    debugSession.memory.loadProgram(out_path);
}catch(ex){
    script.traceWrite("Basic test script failed to run due to error:\n" + ex);
    java.lang.System.exit(1);
}

// Run actions
debugSession.target.runAsynch();
// Verify the behaviour is correct
var stdin = new BufferedReader(new InputStreamReader(System['in']));
while(true){
    print("Is LED blinking? y/n");
    var answer=stdin.readLine();
    if(answer.equals("y")){
        script.traceWrite("The example project runs correctly.");
        break;    
    }
    if(answer.equals("n")){
        script.traceWrite("The example project is broken, the LED is not blinking.");
        exitCode=1;
        break;    
    }
}
// Termination logic
debugSession.terminate();
debugServer.stop();
script.traceEnd(); 
java.lang.System.exit(exitCode);

Advanced project automation using DSS

Part of the CCS workshop is to modify the base example project by adding in buttons and interrupts:

Currently the program simply blinks LED0 on the LaunchPad at 0.5 Hz. Say that some functionality needs to be added to the program such that the blinking frequency of LED0 is controllable. The program will be modified as follows.

• When the program is run on the LaunchPad, LED0 will initially be blinking at 1 Hz.
• Whenever GPIO_Button0 is pressed, the frequency of LED0 will be doubled, up to a maximum of 32 Hz.
• Whenever GPIO_Button1 is pressed, the frequency of LED0 will be halved, down to a minimum of 1 Hz.
• Whenever either GPIO_Button0 or GPIO_Button1 is pressed, the state of LED1 will be toggled.

To do this in the test, make a back-up copy of the original empty.c file and overwrite it with a modified version:

REN "%test_dir%\empty_%device_name%_tirtos_ccs\empty.c" "empty.c.original"
COPY "%cur_dir%\empty.modified" "%test_dir%\empty_%device_name%_tirtos_ccs\empty.c"
CALL "%eclipsec_path%" -noSplash -data "%test_dir%" -application com.ti.ccstudio.apps.projectBuild -ccs.projects empty_%device_name%_tirtos_ccs > "%test_dir%\p_mod_build.log"|| GOTO FAIL_BUILD

mv "${test_dir}/empty_${device_name}_tirtos_ccs/empty.c" "${test_dir}/empty_${device_name}_tirtos_ccs/empty.c.original"
cp "$cur_dir/empty.modified" "${test_dir}/empty_${device_name}_tirtos_ccs/empty.c"
cmd="$eclipse_path -noSplash -data $test_dir -application com.ti.ccstudio.apps.projectBuild -ccs.projects empty_${device_name}_tirtos_ccs >> $test_dir/p_build.log"
if ! eval "$cmd"
then
    echo "Project build has failed."
    exit 1
fi

Counting physical input using breakpoints

One manual instruction from the CCS workshop is the following:

Click Run → Debug and then Resume to run the program on the target. Test the program by verifying that the pressing buttons on the LaunchPad controls the LED0 blink frequency and toggles the state of LED1.

This instruction makes sense for a person going through the workshop for learning but is too vague to be used in a test. Make the instruction more specific like the following:

Test the program by verifying that pressing button0 5 times on the LaunchPad makes LED0 blink faster, and pressing button1 5 times on the LaunchPad makes LED0 blink slower.

This ensures that while still following the original instruction, the test can be repeated, and requires only one binary observation to be made.

In the test script, prompt the tester to press the buttons a specific number of times by setting breakpoints in the button interrupt handler to count the presses. Breakpoints can be added using the function debugSession.breakpoint.add(). Pass in the symbol name of the interrupt handler function as a parameter to add a breakpoint to the start of the function:

var stdin = new BufferedReader(new InputStreamReader(System['in']));
var click_left = 5;
var btn0_bp = debugSession.breakpoint.add("gpioButtonFxn0");
do {
    print("Please press button 0. Presses remaining: "+click_left);    
    debugSession.target.run();
    click_left = click_left - 1;
} while(click_left>0);
debugSession.breakpoint.remove(btn0_bp);
while(debugSession.target.isHalted()){
    debugSession.target.runAsynch();
}
// Verify the behavior is correct
while(true){
    print("Is LED blinking at a fast rate? y/n");
    var answer=stdin.readLine();
    if(answer.equals("y")){
        script.traceWrite("The example project runs correctly.");
        break;    
    }
    if(answer.equals("n")){
        script.traceWrite("The example project is broken, the blink frequency does not increase with presses of button 0.");
        exitCode=1;
        break;    
    }
}

Reading memory value

In another part of the workshop, you need to:

Place your mouse over an instance of the led0_on_off_useconds variable in empty.c. A tool-tip will appear, showing the current value of the variable. The value is still 15625 (corresponding to a frequency of 32 Hz), because the breakpoint has suspended to program at the start of the function, before it has doubled the value of led0_on_off_useconds.

You can simulate this action in DSS by reading the memory value of led0_on_off_useconds. Just use the following function:

var led_rate = debugSession.expression.evaluate("led0_on_off_useconds");
if(led_rate == 15625){
    script.traceWrite("The led frequency is correct: " + led_rate);
}else{
    script.traceWrite("The led frequency is not correct: " + led_rate + ". Should be 15625.");
    exitCode=1;
}

Setting watchpoint with value trigger

To test the watchpoint section of the workshop, you need to follow the following instruction:

This configures the watchpoint to only trigger (suspend program execution) when the value 15625 is written to led0_on_off_useconds. Resume program execution and verify that the program only suspends on button presses that set the LED blinking frequency to 32 Hz.

You can add configure a breakpoint to work as a watchpoint by manipulating its properties. Instead of using breakpoint.add(symbol_name) to add a breakpoint, start by creating a breakpointProperties object.

var led_bp_properties = debugSession.breakpoint.createProperties(1);

Then you set the properties like this:

led_bp_properties.setString("Hardware Configuration.Type", "Watchpoint");
led_bp_properties.setString("Hardware Configuration.Type.Location", "&led0_on_off_useconds");
led_bp_properties.setString("Hardware Configuration.Type.Memory", "Write");
led_bp_properties.setString("Hardware Configuration.Type.With Data", "Yes");
led_bp_properties.setNumeric("Hardware Configuration.Type.With Data.Data Value", "0x3d09");

You can find the hierarchy of the properties by looking at breakpoint properties in the CCS gui:

Some properties are only unlocked when another is set to a particular value. Take Data Value for example, it is only settable once With Data is set to Yes.

Lastly, construct the breakpoint using the breakpointProperties like this:

var led_bp = debugSession.breakpoint.add(led_bp_properties);

Counting clock cycles using profile clock

Lastly the profile clock instruction from the workshop states:

• Press Button1 on the LaunchPad. This triggers the first breakpoint and suspends program execution. Note that the Profile Clock at the bottom-right of Code Composer Studio is displaying a very large number. This is the number of clock cycles that have passed on the target since the program began execution. Double-click on the Profile Clock to reset this count to zero.
• Click on Run → Resume. The program will resume execution until the second breakpoint, at which point the program will suspend again. The Profile Clock should show a clock cycle count of about 100 or 200, depending on your specific LaunchPad. This is the number of clock cycles that the target takes to run the program between the two breakpoints.

You can do the same in DSS by using debugSession.clock:

debugSession.clock.enable();
var btn0_start_bp = debugSession.breakpoint.add("empty.c",79);
var btn0_end_bp = debugSession.breakpoint.add("empty.c",90);

print("Please press button 1");    
debugSession.target.run();
//BP triggers
debugSession.clock.reset();
debugSession.target.run();
//BP triggers
var clock_num = debugSession.clock.read();

Example: Running the test on CC1310 LaunchPad (Windows)

Here is a step-by-step example of how a tester will run the test, demonstrated on a CC1310 LaunchPad.

1. Unzip the content of ccs_automated-testing-files.zip into a directory of choice. In this example, the folder will be D:\Temp.
2. Install and locate CCSv8. In this example, CCSv8 is installed at D:\Builds\CCS8.1.0.00007\ccsv8
3. Install and locate CC13x0 SDK. In this example, the SDK is installed at C:\ti\simplelink_cc13x0_sdk_2_10_00_36\
4. Connect the CC1310 launchpad to the computer.
5. Open a console and navigate to D:\Temp.

6. Run the following command to launch the test:

   Select text

    runWorkshopTest.bat "D:\Builds\CCS8.1.0.00007\ccsv8\" "C:\ti\simplelink_cc13x0_sdk_2_10_00_36\" "CC1310_LAUNCHXL"
   

   You should see the following lines slowly print to the console:

   Select text

    Creating project
    Project create successful
    Building project
    Project build successful
    Found .ccxml file D:\Temp\workshopTest\workspace_5\empty_CC1310_LAUNCHXL_tirtos_ccs\targetConfigs\CC1310F128.ccxml
    Found .out file D:\Temp\workshopTest\workspace_5\empty_CC1310_LAUNCHXL_tirtos_ccs\Debug\empty_CC1310_LAUNCHXL_tirtos_ccs.out
    Unchanged empty.c file found!
    Basic Test: begin
    Cortex_M3_0: GEL Output: Board Reset Complete.
    Is LED blinking? y/n
   

7. Observe the board led, it should be blinking. Input lowercase y to the console.

8. The test prints the following while moving to the next test:

   Select text

    The example project runs correctly.
    Basic Test: PASSED!
            1 file(s) copied.
    Project build successful
    Modification Test: begin
    Cortex_M3_0: GEL Output: Board Reset Complete.
    Please press button 0. Presses remaining: 5
   

   The test will print additional lines with each press of button0 to assist in the countdown. Do not worry if one press seems to decrease the number twice or more.

9. After pressing button0 5 times, you will be asked if the LED is blinking faster. The LED should be blinking at a fast rate that looks always on. Now, do the same with button 1.

10. After completion of the previous test, the breakpoint test will start. You should see the following output after following the instructions displayed:

    Select text

    Breakpoint Test: begin
    Cortex_M3_0: GEL Output: Board Reset Complete.
    Please press button 0. Presses remaining: 5
    Please press button 0. Presses remaining: 4
    Please press button 0. Presses remaining: 3
    Please press button 0. Presses remaining: 2
    Please press button 0. Presses remaining: 1
    Please press button 1.
    The led frequency is correct: 15625
    Breakpoint Test: PASSED!
    

11. For the next watchpoint test, keep pressing button0 until the watchpoint triggers:

    Select text

    Please press button 0 several times.
    The led frequency is correct: 15625
    Watchpoint Test: PASSED!
    

12. For the final profile clock test, just press button1 and it should pass if the number of clock cycle is reasonable.

    Select text

    Please press button 1
    The clock cycle count between breakpoint is 115
    The profile clock is working correctly
    Profile clock Test: PASSED!
    

Summary

With the functionalities provided by CCS command line utilities and DSS APIs, automated tests can be created with ease. The platform-independence of the DSS scripts ensures that they only need to be written once. Their portability also mean that they can be incorporated into different testing frameworks other than the basic on shown in this example. As demonstrated by the example, the automation of the project has eliminated the manual repetitive work of project create/build, setting up workspace and verifying values etc. thus greatly reducing the time taken to run the test.

Resources

TI Code Composer Studio Product Page
Related Technical Documents
TI E2E Technical Forums

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Last updated: May 17 2022 14:16 Central Time