13. Static Shared Objects in Multi-Core Applications

13.1. Multi-Core System Characteristics

The tiarmclang compiler tools support sharing code and data that are common to multiple applications running in a multi-threaded system on multiple TI core processors. This feature helps reduce the overall size of the code and data that must be loaded into local and shared memory on TI devices that have the following characteristics:

  • Multiple Identical TI Core Processors. A TI device may contain two or more identical core processors that can execute code and access data from local core or shared system memory. For example, the TI AM263x devices contain 4 Cortex-R5 devices that are each equipped with an FPU.

  • Ability to Execute Code and Access Shared Read-Only (RO) Data from Shared System Memory. Each core processor on the TI device must be able to fetch instructions and access RO data directly from shared system memory.

  • Shared System to Local Core Memory Mapping for Shared Read-Write (RW) Data Objects. Each of the identical TI core processors is equipped with a mechanism that can map an address range from shared system memory to local memory that is accessible only to an individual TI core processor. For example, if a multi-threaded, multi-core system places a global read-write (RW) data object in shared system memory, then each application must maintain its own copy of the global RW data object in local core memory that is referenced using its shared memory address. An example of such a mechanism might be an MMU or a Region Address Translation Unit or RAT (as featured in TI AM263x devices).

For more details about local core and shared system memory that is available on a TI device, please refer to device information that can be found within ti.com.

13.2. Multi-Threaded, Multi-Core System Development Flow

At a high-level general perspective, the development flow for this feature is as follows:

  1. Compile and link initial individual applications, generating an XML link information file for each.

  2. Identify functions, RO data objects, and RW data objects that are common among the individual applications.

  3. Collect common functions, RO data objects, and RW data objects into a Shared Static Object (SSO) to be loaded into shared system memory.

  4. Re-link each individual application against SSO from step 3, allocating code and data that is NOT defined in the SSO to local core memory.

Note

Load and Initialize SSO Content Before Execution

The contents of the SSO file must be loaded and initialized in system memory prior to executing any code that may access any function or data object defined in the SSO.

To further explore the creation of an SSO and its role in the multi-threaded, multi-core system development flow, consider the following example.

13.3. Example

For this example, assume there are two TI Arm Cortex-R5 processors, each running a simple application on separate cores. The source for the first application is contained in sub-directory core1:

/* main.c */
extern void do_some_work(void);

int main() {
  do_some_work();
  return 0;
}

/* do_some_work.c */
volatile int xyz = 10;

void do_some_work(void) {
  xyz += 2;
}

For the sake of simplicity, assume that the core2 sub-directory contains identical copies of the above source files.

13.3.2. Step 2: Identify Common Objects

The next step in the development flow is to identify the functions and data objects that are common to two or more of the applications that participate in the multi-core system.

An opti-share sub-directory is provided in the tiarmclang compiler tools installation as a location to perform this task. The opti-share sub-directory contains the opti-share.js JavaScript file, which can process the XML link information files generated by the initial application builds. This script identifies common functions and data objects that can be placed in shared system memory. The opti-share.js script uses an additional --mem_spec=<file> input option to determine the placement of the shared functions, read-only (RO) data objects, and read-write (RW) data objects. As output, it generates an SSO linker command file.

Before running the opti-share.js script to create the SSO linker command file for this example, it is useful to examine the contents of the XML link information files, app1.xml and app2.xml, to understand the information used by the opti-share.js script to identify functions and data objects that are common to both app1.out and app2.out.

In app1.xml, there are object_component XML tags that represent the sections where main(), do_some_work(), and xyz are defined:

%> cat app1.xml
<?xml version="1.0" encoding="ISO-8859-1" ?>
<link_info>
   ...
   <object_component_list>
      ...
      <object_component id="oc-3a">
         <name>.text.main</name>
         <load_address>0x700410b0</load_address>
         <readonly>true</readonly>
         <executable>true</executable>
         <run_address>0x700410b0</run_address>
         <size>0x24</size>
         <alignment>0x10</alignment>
         <input_file_ref idref="fl-1"/>
         <refd_ro_sections>
            <object_component_ref idref="oc-4b"/>
         </refd_ro_sections>
         <value>3f6edba534d59127feedee0098469769</value>
      </object_component>
      ...
      <object_component id="oc-4b">
         <name>.text.do_some_work</name>
         <load_address>0x700410f0</load_address>
         <readonly>true</readonly>
         <executable>true</executable>
         <run_address>0x700410f0</run_address>
         <size>0x18</size>
         <alignment>0x10</alignment>
         <input_file_ref idref="fl-2"/>
         <refd_rw_sections>
            <object_component_ref idref="oc-5d"/>
         </refd_rw_sections>
         <value>9702185a47f41f85174342f2b3507aa9</value>
      </object_component>
      ...
      <object_component id="oc-5d">
         <name>.data.xyz</name>
         <load_address>0x7004111c</load_address>
         <readwrite>true</readwrite>
         <run_address>0x7004111c</run_address>
         <size>0x4</size>
         <alignment>0x4</alignment>
         <input_file_ref idref="fl-2"/>
         <value>41883520c3071f5f4a4a4613fb005e0c</value>
      </object_component>
      ...
   </object_component_list>
   ...
   <title>Link successful</title>
</link_info>

Likewise, in app2.xml, the sections containing the definitions of main(), do_some_work(), and xyz are also represented by object_component XML tags:

%> cat app2.xml
<?xml version="1.0" encoding="ISO-8859-1" ?>
<link_info>
   <banner>TI Linker Unix v5.1.0.LTS</banner>
   ...
   <object_component_list>
      ...
      <object_component id="oc-3a">
         <name>.text.main</name>
         <load_address>0x700810b0</load_address>
         <readonly>true</readonly>
         <executable>true</executable>
         <run_address>0x700810b0</run_address>
         <size>0x24</size>
         <alignment>0x10</alignment>
         <input_file_ref idref="fl-1"/>
         <refd_ro_sections>
            <object_component_ref idref="oc-4b"/>
         </refd_ro_sections>
         <value>3f6edba534d59127feedee0098469769</value>
      </object_component>
      ...
      <object_component id="oc-4b">
         <name>.text.do_some_work</name>
         <load_address>0x700810f0</load_address>
         <readonly>true</readonly>
         <executable>true</executable>
         <run_address>0x700810f0</run_address>
         <size>0x18</size>
         <alignment>0x10</alignment>
         <input_file_ref idref="fl-2"/>
         <refd_rw_sections>
            <object_component_ref idref="oc-5d"/>
         </refd_rw_sections>
         <value>9702185a47f41f85174342f2b3507aa9</value>
      </object_component>
      ...
      <object_component id="oc-5d">
         <name>.data.xyz</name>
         <load_address>0x7008111c</load_address>
         <readwrite>true</readwrite>
         <run_address>0x7008111c</run_address>
         <size>0x4</size>
         <alignment>0x4</alignment>
         <input_file_ref idref="fl-2"/>
         <value>41883520c3071f5f4a4a4613fb005e0c</value>
      </object_component>
      ...
   <title>Link successful</title>
</link_info>

A few things to observe about the contents of app1.xml and app2.xml:

  • The hash value computed for a function or data object is found in the value member of the object_component that represents the section where the function or data object is defined

    • main() is defined in section .text.main

    • do_some_work() is defined in section .text.do_some_work

    • xyz is defined in section .data.xyz

  • The hash values for main(), do_some_work(), and xyz in app1.xml match the hash values for main(), do_some_work(), and xyz in app2.xml even though the addresses where they are loaded differ between app1.xml and app2.xml. Matching hash values indicate that the implementations of main(), do_some_work(), and xyz are identical between app1 and app2

  • The refd_ro_sections and refd_rw_sections lists that appear in object_component records facilitate a reconstruction of the application’s section reference graph. In this example specifically:

    • the <object_compenent> records for main() in both app1.xml and app2.xml contain a refd_ro_sections list that enumerates the sections (or object_component records) that are referenced from main(), namely the .text.do_some_work section; representing the call from main() to do_some_work()

    • the <object_compenent> records for do_some_work() in both app1.xml and app2.xml contain a refd_rw_sections list that enumerates the sections (or object_component records) that are referenced from do_some_work(), namely the .data.xyz section; representing the read and write of xyz from do_some_work()

13.3.3. Step 3: Create a Static Shared Object (SSO) File

Invoke the opti-share.js script using Node.js. For this example, the command line is as follows:

%> node /path/to/install/opti-share/opti-share.js -o sso.cmd core1/app1.xml core2/app2.xml --mem_spec=ex_mem_spec.json

The arguments to the opti-share.js invocation are as follows:

  • -o sso.cmd - specifies the name of the linker command file to be written by the opti-share.js script

  • core1/app1.xml core2/app2.xml - the path to and name of the XML link information files considered by the opti-share.js to identify common functions and data objects

  • --mem_spec=ex_mem_spec.json - JSON file containing a specification of available shared memory areas and placement instructions for the collections of common functions, RO data objects, initialized RW data objects, and uninitialized RW data objects. This file is provided as part of the tiarmclang compiler tools installation.

This command results in an SSO linker command file, sso.cmd, with the following content:

%> cat sso.cmd
--map_file=sso.map
--output_file=sso.out
--no_entry_point
--sso

MEMORY {
SHARED_RX: o=0x70070000 l=0x00006000
SHARED_RO: o=0x70076000 l=0x00004000
SHARED_RW: o=0x7007A000 l=0x00008000
}

SECTIONS {

    .shared.text {
        . = align(16); "/path/to/example/core1/do_some_work.o"(.text.do_some_work)
        . = align(16); "/path/to/example/core1/main.o"(.text.main)
        . = align(4); "/path/to/install/lib/armv7r-ti-none-eabihf/c/libc.a"<exit.c.obj>(.text:abort)
        . = align(4); "/path/to/install/lib/armv7r-ti-none-eabihf/c/libc.a"<mpu_init.c.obj>(.text.__mpu_init)
        . = align(4); "/path_to_install/lib/armv7r-ti-none-eabihf/c/libc.a"<pre_init.c.obj>(.text._system_pre_init)
        . = align(4); "/path/to/install/lib/armv7r-ti-none-eabihf/c/libsysbm.a"<hostexit.c.obj>(.text.HOSTexit)
    } > SHARED_RX, priority(4)

    .shared.data {
        . = align(4); "/path/to/example/core1/do_some_work.o"(.data.xyz)
    } > SHARED_RW, priority(1)
}

13.3.3.1. SSO Linker Command File Content from ex_mem_spec.json

As mentioned above, the MEMORY directive is dictated by the ex_mem_spec.json file, which describes the available shared memory regions. This file is installed in the opti-share sub-directory. The ex_mem_spec.json file also contains placement instructions for the four potential output sections that the opti-share.js script can write into the SECTIONS directive of the generated SSO linker command file.

These include:

  • .shared.text - the collection of common functions placed in shared memory

  • .shared.rodata - the collection of common RO data objects placed in shared memory

  • .shared.data - the collection of initialized RW data objects placed in shared memory

  • .shared.bss - the collection of uninitialized RW data objects placed in shared memory

Note

Shared RW Data Objects Get Duplicated in Local Core Memory

If opti-share.js determines that a RW data object is eligible to go into shared memory, the subsequent link of the application against the SSO file allocates a local copy of the RW data object, since each application must maintain its own copy of the shared RW data object. Such local copies can reduce code size savings realized via shared functions. However, the advantage of sharing a RW data object is that it can enable a greater number of common functions to be placed in shared memory.

To avoid sharing of RW data objects, there are two additional opti-share.js options:

  • --nodata - when specified, opti-share.js does not share any initialized RW data objects

  • --nobss - when specified, opti-share.js does not share any uninitialized RW data objects

13.3.3.2. SSO Linker Command File Content from opti-share.js

The opti-share.js script is responsible for identifying common functions and data objects to be placed in shared memory. There are a few general rules that the opti-share.js implementation uses to determine which functions and data objects are eligible to be placed in shared memory:

  • In order for two function or data objects to be considered identical, their hash values must match exactly.

  • There must be two or more identical instances of a function or data object among the individual applications participating in the multi-core system.

  • Any function or data references from a candidate common function must be ruled a common function or data object.

To emphasize the significance of the third bullet above, be aware of the following note.

Note

Static Shared Object Defined

The essential characteristic of a static shared object is that all function or data object symbol references from functions that are defined in a static shared object must also be defined in the same static shared object.

For example, in the example under consideration, if do_some_work() were implemented differently in app1.out vs. app2.out, then main() would be ruled ineligible to be placed in shared memory since a function that it references is not identical in both app1.out and app2.out.

Once all eligible common functions and data objects have been identified, opti-share.js populates the content of the output section specifications in the SSO linker command file with the full path name for each input section that contains the definition of a common function or data object.

Note

Development Flow Assumes the File System Remains Consistent

The full path name is referenced in the SSO linker command file. Therefor, the file system organization must remain consistent from the initial compilation to the link of the individual applications to the relink of each application against the SSO.

13.3.3.3. Build the SSO File

Building an SSO file is as simple as invoking the linker with the SSO linker command file as the only argument:

%> tiarmclang -Wl,sso.cmd

In the above SSO linker command file content, the following linker options are already specified:

--map_file=sso.map
--output_file=sso.out
--no_entry_point
--sso

That is, the linker is already instructed to generate a map file called sso.map and an SSO output file called sso.out. The --no_entry_point option instructs the linker to create an ELF executable object file that does not have a defined entry symbol. Finally, the --sso option informs the linker that it is in the process of generating an SSO output file.

13.4. Static Shared Object Options Summary

For quick reference, these are the command-line options used to create and build the SSO linker command file as described in the sections above:

--gen_xml_func_hash

When the --gen_xml_func_hash linker option is combined with the --xml_link_info linker option, the linker includes hash values for functions and data objects and section reference information in the --xml_link_info output.

--import_sso

The --import_sso option is used when relinking an application against a previously created SSO file. This option enables an application to execute code and access RO data from shared memory, enabling the developer to realize overall system code size savings by sharing functions and RO data with other applications in the multi-core system.

--sso

Given an SSO linker command file that specifies the placement of common functions and data objects in shared memory, the --sso linker option instructs the linker to produce an SSO file using only the SSO linker command file to enumerate the input sections to be included in the link.

Generates a well-formed XML file containing detailed information about the result of a link.