19. Static Shared Objects in Multi-Core Applications

19.1. Multi-Core System Characteristics

The tiarmclang compiler tools support sharing code and data that is common to multiple applications that are running in a multi-threaded system on multiple TI Arm core processors. This feature helps to reduce the overall size of the code and data that must be loaded into local and shared memory on TI devices that have the following characteristics:

  • Multiple Identical TI Arm Core Processors

    TI device contains two or more identical TI Arm core processors that can execute code and access data from local core or shared system memory. For example, the TI AM263x devices contain 4 Cortex-R5 devices that are each equipped with an FPU.

  • Ability to Execute Code and Access Shared Read-Only (RO) Data from Shared System Memory

    Each TI Arm core processor on the TI device must be able to fetch instructions and access RO data directly from shared system memory.

  • Shared System to Local Core Memory Mapping for Shared Read/Write (RW) Data Objects

    Each of the identical TI Arm core processors are equipped with a mechanism that is able to map an address range from shared system memory to local memory that is only accessible to an individual TI Arm core processor.

    For example, if the multi-threaded, multi-core system places a global read/write (RW) data object in shared system memory, then each application must maintain its own copy of the global RW data object in local core memory that is referenced using its shared memory address.

    An example of such a mechanism might be a MMU or a Region Address Translation Unit or RAT (as featured in TI AM263x devices).

For more details about local core and shared system memory that is available on a TI device, please refer to device information that can be found within ti.com.

19.2. Multi-Threaded, Multi-Core System Development Flow

At a high-level general perspective, the development flow for this feature is as follows:

  1. Compile and link initial individual applications, generating a link information XML file for each.

  2. Identify functions, RO data objects, and RW data objects that are common among the individual applications.

  3. Collect common functions, RO data object, and RW data objects into a Shared Static Object (SSO) that will be loaded into shared system memory.

  4. Re-link each individual application against SSO from step 3, allocating code and data that is NOT defined in the SSO to local core memory.

Note

Load and Initialize SSO Content Before Attempting to Access

The contents of the SSO file must be loaded and initialized in system memory prior to executing any code that may access any function or data object defined in the SSO.

To further explore the creation of an SSO and its role in the multi-threaded, multi-core system development flow, consider the following example walk-through.

19.3. Example Walk-Through

For this tutorial, assume that there are two TI Arm Cortex-R5 core processors, each running a simple application on separate cores. The source for the first application is contained in sub-directory core1:

/* main.c */
extern void do_some_arm(void);

int main() {
  do_some_arm();
  return 0;
}

/* do_some_arm.c */
volatile int xyz = 10;

void do_some_arm(void) {
  xyz += 2;
}

For the sake of simplicity, assume that sub-directory core2 contains identical copies of the above source files.

19.3.2. Create a Static Shared Object (SSO) Linker Command File

The goal of the next step in the development flow is to identify the functions and data objects that are common to two or more of the applications that are participating in the multi-core system. To facilitate carrying out this task in a programmatic fashion, an opti-share sub-directory is included in the installation of the tiarmclang compiler tools.

The opti-share sub-directory contains the opti-share.js JavaScript that can process the collection of XML link information files that resulted from the initial application builds to identify the common functions and data objects that can be placed in shared system memory, and with the help of an additional --mem_spec=<file> input option, opti-share.js decides on the placement of the shared functions, read-only (RO) data objects, and read-write (RW) date objects, generating an output SSO linker command file.

19.3.2.1. XML Link Information Content

Before running the opti-share.js script to create the SSO linker command file for this example, it is worthwhile to examine the contents of the XML link information files, app1.xml and app2.xml, to understand what information is being used by the opti-share.js script to identify the functions and data objects that are common to both app1.out and app2.out.

In app1.xml, there are object_component XML tags that represent the sections where main(), do_some_arm(), and xyz are defined:

%> cat app1.xml
<?xml version="1.0" encoding="ISO-8859-1" ?>
<link_info>
   <banner>TI ARM Clang Linker Unix v5.1.0.LTS</banner>
   ...
   <object_component_list>
      ...
      <object_component id="oc-3a">
         <name>.text.main</name>
         <load_address>0x700410b0</load_address>
         <readonly>true</readonly>
         <executable>true</executable>
         <run_address>0x700410b0</run_address>
         <size>0x24</size>
         <alignment>0x10</alignment>
         <input_file_ref idref="fl-1"/>
         <refd_ro_sections>
            <object_component_ref idref="oc-4b"/>
         </refd_ro_sections>
         <value>3f6edba534d59127feedee0098469769</value>
      </object_component>
      ...
      <object_component id="oc-4b">
         <name>.text.do_some_arm</name>
         <load_address>0x700410f0</load_address>
         <readonly>true</readonly>
         <executable>true</executable>
         <run_address>0x700410f0</run_address>
         <size>0x18</size>
         <alignment>0x10</alignment>
         <input_file_ref idref="fl-2"/>
         <refd_rw_sections>
            <object_component_ref idref="oc-5d"/>
         </refd_rw_sections>
         <value>9702185a47f41f85174342f2b3507aa9</value>
      </object_component>
      ...
      <object_component id="oc-5d">
         <name>.data.xyz</name>
         <load_address>0x7004111c</load_address>
         <readwrite>true</readwrite>
         <run_address>0x7004111c</run_address>
         <size>0x4</size>
         <alignment>0x4</alignment>
         <input_file_ref idref="fl-2"/>
         <value>41883520c3071f5f4a4a4613fb005e0c</value>
      </object_component>
      ...
   </object_component_list>
   ...
   <title>Link successful</title>
</link_info>

Likewise, in app2.xml, the sections containng the definitions of main(), do_some_arm(), and xyz are also represented by object_component XML tags:

%> cat app2.xml
<?xml version="1.0" encoding="ISO-8859-1" ?>
<link_info>
   <banner>TI ARM Clang Linker Unix v5.1.0.LTS</banner>
   ...
   <object_component_list>
      ...
      <object_component id="oc-3a">
         <name>.text.main</name>
         <load_address>0x700810b0</load_address>
         <readonly>true</readonly>
         <executable>true</executable>
         <run_address>0x700810b0</run_address>
         <size>0x24</size>
         <alignment>0x10</alignment>
         <input_file_ref idref="fl-1"/>
         <refd_ro_sections>
            <object_component_ref idref="oc-4b"/>
         </refd_ro_sections>
         <value>3f6edba534d59127feedee0098469769</value>
      </object_component>
      ...
      <object_component id="oc-4b">
         <name>.text.do_some_arm</name>
         <load_address>0x700810f0</load_address>
         <readonly>true</readonly>
         <executable>true</executable>
         <run_address>0x700810f0</run_address>
         <size>0x18</size>
         <alignment>0x10</alignment>
         <input_file_ref idref="fl-2"/>
         <refd_rw_sections>
            <object_component_ref idref="oc-5d"/>
         </refd_rw_sections>
         <value>9702185a47f41f85174342f2b3507aa9</value>
      </object_component>
      ...
      <object_component id="oc-5d">
         <name>.data.xyz</name>
         <load_address>0x7008111c</load_address>
         <readwrite>true</readwrite>
         <run_address>0x7008111c</run_address>
         <size>0x4</size>
         <alignment>0x4</alignment>
         <input_file_ref idref="fl-2"/>
         <value>41883520c3071f5f4a4a4613fb005e0c</value>
      </object_component>
      ...
   <title>Link successful</title>
</link_info>

A few things to observe about the contents of app1.xml and app2.xml:

  • The hash value computed for a function or data object is found in the value member of the object_component that represents the section where the function or data object is defined

    • main() is defined in section .text.main

    • do_some_arm() is defined in section .text.do_some_arm

    • xyz is defined in section .data.xyz

  • The hash values for main(), do_some_arm(), and xyz in app1.xml match the hash values for main(), do_some_arm(), and xyz in app2.xml even though the address where they are loaded differ between app1.xml and app2.xml; this indicates that the implementations of main(), do_some_arm(), and xyz are identical between app1 and app2

  • The refd_ro_sections and refd_rw_sections lists that appear in object_component records facilitate a reconstruction of the application’s section reference graph. In this example specifically:

    • the <object_compenent> records for main() in both app1.xml and app2.xml contain a refd_ro_sections list that enumerates the sections (or object_component records) that are referenced from main(), namely the .text.do_some_arm section; representing the call from main() to do_some_arm()

    • the <object_compenent> records for do_some_arm() in both app1.xml and app2.xml contain a refd_rw_sections list that enumerates the sections (or object_component records) that are referenced from do_some_arm(), namely the .data.xyz section; representing the read and write of xyz from do_some_arm()

19.3.2.2. Invoking the opti-share.js Script

The opti-share.js script is invoked using Node.js. For this example, the command line is as follows:

%> node /path/to/install/opti-share/opti-share.js -o sso.cmd core1/app1.xml core2/app2.xml --mem_spec=ex_mem_spec.json

Where the arguments to the opti-share.js invocation are as follows:

  • -o sso.cmd - specifies the name of the linker command file to be written by the opti-share.js script

  • core1/app1.xml core2/app2.xml - the path to and name of the XML link information files considered by the opti-share.js to identify common functions and data objects

  • --mem_spec=ex_mem_spec.json - JSON file containing a specification of available shared memory areas and placement instructions for the collections of common functions, RO data objects, initialized RW data objects, and uninitialized RW data objects

This command results in an SSO linker comand file, sso.cmd, with the following content:

%> cat sso.cmd
--map_file=sso.map
--output_file=sso.out
--no_entry_point
--sso

MEMORY {
SHARED_RX: o=0x70070000 l=0x00006000
SHARED_RO: o=0x70076000 l=0x00004000
SHARED_RW: o=0x7007A000 l=0x00008000
}

SECTIONS {

    .shared.text {
        . = align(16); "/path/to/example/core1/do_some_arm.o"(.text.do_some_arm)
        . = align(16); "/path/to/example/core1/main.o"(.text.main)
        . = align(4); "/path/to/install/lib/armv7r-ti-none-eabihf/c/libc.a"<exit.c.obj>(.text:abort)
        . = align(4); "/path/to/install/lib/armv7r-ti-none-eabihf/c/libc.a"<mpu_init.c.obj>(.text.__mpu_init)
        . = align(4); "/path_to_install/lib/armv7r-ti-none-eabihf/c/libc.a"<pre_init.c.obj>(.text._system_pre_init)
        . = align(4); "/path/to/install/lib/armv7r-ti-none-eabihf/c/libsysbm.a"<hostexit.c.obj>(.text.HOSTexit)
    } > SHARED_RX, priority(4)

    .shared.data {
        . = align(4); "/path/to/example/core1/do_some_arm.o"(.data.xyz)
    } > SHARED_RW, priority(1)
}

19.3.2.3. SSO Linker Command File Content from ex_mem_spec.json

As mentioned above, the MEMORY directive is dictated by the ex_mem_spec.json file that describes the available shared memory regions. The ex_mem_spec.json file also contains placement instructions for the four potential output sections that the opti-share.js script can write into the SECTIONS directive of the generated SSO linker command file.

These include:

  • .shared.text - the collection of common functions placed in shared memory

  • .shared.rodata - the collection of common RO data objects placed in shared memory

  • .shared.data - the collection of initialized RW data objects placed in shared memory

  • .shared.bss - the collection of uninitialized RW data objects placed in shared memory

Note

Shared RW Data Objects Get Duplicated in Local Core Memory

If opti-share.js determines that a RW data object is eligible to go into shared memory, the subsequent link of the application against the SSO file allocates a local copy of the RW data object since each application must maintain its own copy of the shared RW data object. This can reduce code size savings realized via shared functions. However, the advantage of sharing a RW data object is it can enables a greater number of common functions to be placed in shared memory.

To avoid sharing of RW data objects, there are two additional opti-share.js options:

  • --nodata - when specified, opti-share.js does not share any initialized RW data objects

  • --nobss - when specified, opti-share.js does not share any uninitialized RW data objects

Note

Example ex_mem_spec.json Available in Compiler Tools Installation

The tiarmclang compiler tools installation includes an example memory specfication file, ex_mem_spec.json, in the opti-share sub-directory relative to where the tools are installed.

19.3.2.4. SSO Linker Command File Content from opti-share.js

The opti-share.js script is responsible for identifying common functions and data objects to be placed in shared memory. There are a few general rules that the opti-share.js implementation uses to determine which functions and data objects are eligible to be placed in shared memory:

  • In order for two function or data objects to be considered identical, their hash values must match exactly

  • There must be two or more identical instances of a function or data object among the individual applications participating in the multi-core system

  • Any function or data references from a candidate common function must be ruled a common function or data object

To emphasize the significance of the third bullet above, please note …

Note

Static Shared Object Defined

The essential characteristic of a static shared object is that all function or data object symbol references from functions that are defined in a static shared object must also be defined in the same static shared object.

For example, in the tutorial example under consideration, if do_some_arm() were implemented differently in app1.out vs. app2.out, then main() would be ruled ineligible to be placed in shared memory since a function that it references is not identical in both app1.out and app2.out.

Once all eligible common functions and data objects have been identified, opti-share.js populates the content of the output section specifications in the SSO linker command file with the full path name for each input section that contains the definition of a common function or data object.

Note

This Development Flow Assumes the File System Remains Consistent

Due to the use of full path names in the SSO linker command file, the multi-core development flow under discussion in this chapter asssumes that the state of the file system remains consistent between the initial compile and link of the individual applications and the relink of each application against the SSO.

19.3.3. Build the SSO File

Building an SSO file is as simple as invoking the linker with the SSO linker command file as the only argument:

%> tiarmclang -Wl,sso.cmd

In the above SSO linker command file content, the following linker options are already included:

--map_file=sso.map
--output_file=sso.out
--no_entry_point
--sso

That is, the linker is already instructed to generate a map file called sso.map and an SSO output file called sso.out. The --no_entry_point option instructs the linker to create an ELF executable object file that does not have a defined entry symbol. Finally, the --sso option informs that linker that it is in the process of generating an SSO output file.

19.4. Static Shared Object Options Summary

For quick reference, these are the options involved in the creation of the SSO linker command file and the build of the SSO file as described in the sections above:

--gen_xml_func_hash

When the --gen_xml_func_hash linker option is combined with the --xml_link_info linker option, the linker includes hash values for functions and data objects and section reference information in the --xml_link_info output.

--import_sso

The --import_sso option is used when relinking an application against a previously created SSO file. This enables an application to execute code and access RO data from shared memory, enabling the developer to realize overall system code size savings by sharing functions and RO data with other applications in the multi-core system.

--sso

Given an SSO linker command file that specifies the placemen of common functions and data objects in shared memory, the --sso linker option instructs the linker to produce an SSO file using only the SSO linker command file to enumerate the input sections that will be included in the link.

Generates a well-formed XML file containing detailed information about the result of a link.