From RTSC-Pedia

[printable version]  [offline version]

offline version generated on 04-Aug-2010 21:08 UTC

Managing Compiler Toolchains

How to manage XDCtools use of your compiler toolchain

Background

The embedded C/C++ programmer's development environment often contains multiple C/C++ compiler toolchains from different vendors. It's not uncommon, for example, to use native C/C++ toolchains to develop and test large portions of an embedded code base before cross-compiling for execution on an embedded device. Native development is often used to enable software development to precede the availability of embedded hardware, allowing hardware and software development to proceed in parallel. But because each toolchain has unique project files, different compiler options, and unique tool usage requirements, native development is often supplanted by pure embedded development after hardware platforms become available. Maintaining two unique sets of project files is a luxury most teams can't afford and later regret when, late in a project, intermittent failures — easily debugged on native platforms — require heroic efforts by the best developers to find using the limited tools typically available on embedded devices.

XDCtools is designed to make the transition between different C/C++ toolchains as seamless as possible. This has several advantages for embedded development:

• testing and debugging - by making it easy to move back and forth between native and embedded development environments at any point in the development cycle, you can always quickly leverage the best debugging, profiling, and analysis tools of any toolchain as well as state of the art runtime error trapping provided by native platforms that allow you to trap NULL pointer use (via native platform MMU support) and even use of uninitialized data (via tools like valgrind);
• porting to new devices - support for new devices often requires no change to any source files including project build files that are usually tied to specific toolchains; and
• product-line reuse - projects can simultaneously support multiple devices with little or no effort, even devices with completely different instruction sets (e.g, a TI C6x DSP and an Arm with custom accelerators).

To enable these benefits, XDCtools "abstracts" cross-development toolchains by always referencing these toolchains through a single fixed interface (xdc.bld.ITarget). This means that any toolchain supporting this interface can be used with XDCtools. Toolchains support this interface by providing one or more targets; i.e., modules that implement the ITarget interface. Most developers never need to implement this interface, they simply use a target supplied by someone else: typically the first developer who needs to use a particular toolchain for a project. While creating a target often requires fairly intimate knowledge of the toolchain and the C language, once it's created, all other developers use it by simply passing its name to those tools requiring use of the toolchain.

Configuring targets for your development environment

In order for targets to work in your development environment, you must "configure" each target with information specific to your development host. You must, at a minimum, specify the installation directory of each compiler toolchain you plan to use. This information is specified in a special file named "config.bld" and, although this file is an XDCscript file, its contents are often quite simple.

var C64P = xdc.module("ti.targets.C64P");
C64P.rootDir = "/user/local/ti/c64/6.1.2";

Line 1 loads the specified target, returning an object whose configuration parameters can be set on subsequent lines. Line 2 of this script specifies the only required parameter to use a target: rootDir — the installation directory of the compiler. This simple two line file is sufficient to enable the full functionality of XDCtools with TI's C64P compiler tool chain. As you might expect, adding support for other toolchains amounts to adding another two lines per toolchain.

While using a hard coded path in a config.bld script is the simplest way to get started, when integrating with an existing development environment, the fact that config.bld is an XDCscript script means that it's also easy to seamlessly inter-operate with existing tools. It's possible to reference environment variables, read external files, load XML files, and even run external commands. Suppose, for example, that your development group maintains all cross-development tools on a shared network drive and that the location of these tools is specified by an environment variable named TOOLS. The following config.bld script uses TOOLS to configure the ti.targets.C64P target.

var tools = java.lang.System.getenv("TOOLS");
var C64P = xdc.module("ti.targets.C64P");
C64P.rootDir = tools + "/c64/6.1.2";

Line 1 gets the value of the TOOLS environment variable, line 2 loads the ti.targets.C64P target, and line 3 sets the rootDir configuration parameter to the value of TOOLS followed by "/c64/6.1.2". For more information about the capabilities of XDCscript scripts see The XDCscript Language.

In addition to bridging XDCtools with your existing development environment, config.bld is used to centralize the definitions of target-specific or development-group-specific options enabling you to avoid duplicating this information in every project's build files. Duplication of this information is generally a bad practice as it makes it difficult to change or fix bugs in options passed to your compiler(s). Suppose, for example, that your development group has a convention that all debug builds have the C macro definition DEBUG defined and no code should disable interrupts for more than 100 instruction cycles. Starting with the config.bld file above, the following example configures the ti.targets.C64P target to support both of these new requirements.

var tools = java.lang.System.getenv("TOOLS");
var C64P = xdc.module("ti.targets.C64P");
C64P.rootDir = tools + "/c64/6.1.2";
/* define group-specific macros for debug builds */
C64P.profiles["debug"].compileOpts.defs = "-DDEBUG";
/* never disable interrupts for more than 100 cycles */
C64P.ccOpts.prefix = "-mi=100";

Every project using this config.bld can rely on the DEBUG macro being defined for debug builds and, perhaps more importantly, you can be sure that all such projects are compiled in such a way that the compiler never allows loops to disable interrupts for more than 100 cycles. If, in the future, you decide that it would be better to allow 150 cycles (to give the compiler more optimization flexibility and get some performance gain), you can change this single file and simply rebuild all affected projects.

For more information about configuration options available, see the xdc.bld.ITarget reference for options supported by all targets and the reference documentation for individual targets for options specific to that target; for example, the ti.targets.C64P reference page lists configuration options specific to the C64P target.

As you can see, config.bld files for mature development efforts can easily grow to more than the two lines shown above. Fortunately, since config.bld files are XDCscript files, it is possible to factor common settings into a single file and then load and modify these setting on an as needed basis. The section Leveraging an existing config.bld file below illustrates this technique.

How and when config.bld is used

There are several points in the development process where information about your C/C++ toolchains and hardware platforms is required by XDCtools:

• as part of the configuration process, xdc.tools.configuro generates a C file which is compiled into an object file that must be linked into the final application
• during the creation of a package, the xdc command determines and records the versions of the compilers used to build the package, and
• if you opt to use xdc to build your C/C++ sources, the makefiles generated by xdc contain references to each of your toolchain's tools (compilers, linkers, and archivers).

During package creation

During package creation.  The xdc command is used to create one or more releases of a package — simple archives (.tar, .tar.gz, or .zip) containing all the files necessary to use a package. A single version of a package can have multiple releases. For example, you may produce two releases of a codec package from a common set of sources, one that runs on a C6x architecture and one that runs on an Arm.

The xdc command is capable of compiling your sources or simply assembling files created via some other tools. But, whether you use xdc to compile your sources or not, the xdc command must be used to create releases of a package. This ensures that each release archive has the appropriate meta-data properly collected and stored in the specially named files which enables other tools to "see" and operate on the package as a whole. It's the meta-data about the package that allows package display tools to report package version numbers and release labels, the configuration tool to validate compatibility of the packages used in an application, and the repository management tools to know which packages are supplied by product bundles and to validate the compatibility of the packages placed in a repository.

The xdc command distinguishes between packages that are "buildable" from those that are not by the existence of a file named package.bld. As you might have guessed from this file's suffix, package.bld is related to config.bld and is also an XDCscript file. The package.bld script specifies what each release of a package must contain. During the processing of package.bld, this script reads information from the xdc.bld.BuildEnvironment module and updates the information maintained by the xdc.bld.PackageContents module. The contents of each release of a package is entirely determined by the state of the xdc.bld.PackageContents module at the conclusion of package.bld. But, before processing package.bld, the xdc command first runs config.bld to "configure" your build environment and set options that are shared across multiple packages.

var Build = xdc.module("xdc.bld.BuildEnvironment");
var Pkg = xdc.module("xdc.bld.PackageContents");
 
var C64P = xdc.module("ti.targets.C64P");
C64P.rootDir = "«compiler_install_dir»";
Build.targets = [C64P];         /* only build using C64P target */
 
Pkg.attrs.archiver = "zip";     /* package archive format is .zip */
Pkg.attrs.profile = "debug";    /* default build profile is "debug" */

As a rule, config.bld contains settings that are specific to your development environment (installation directories of compiler tool chains, optional compiler flags, etc.) whereas package.bld only specifies what should be in each package release. By keeping development environment specific information out of package.bld, your packages become portable; any developer can re-build the package without making any changes to the files within the package itself. The config.bld file becomes a convenient place to specify "global" conventions shared by all packages; e.g., the package archive format, compiler-specific options that control the level of warnings, etc.

Since config.bld scripts are used to initialize xdc.bld.BuildEnvironment, which is the input to package.bld scripts, the xdc command requires a config.bld file and, unless it's explicitly specified on the command line or it exists in the current package being built, xdc looks for it along the package path. This allows each package to easily override global conventions set by a shared config.bld or to simply use a shared config.bld.

During configuration

During configuration.  The xdc.tools.configuro tool automatically generates a config.bld file based on its command line parameters. The configuro tool's command line parameters allow you to either explicitly name the installation directory of your compiler (convenient within makefiles that often already have this information) or specify the location of a hand-crafted config.bld file. If you pass a config.bld file to configuro on the command line, the config.bld file generated by configuro loads the hand-crafted file, chooses a single target, and sets the compiler's installation directory (if it was specified in the command line).

For example, command line "xs xdc.tools.configuro -t «target_name» -c «compiler_install_dir» ..." causes configuro to create and use a config.bld file whose contents are shown below.

var target  = xdc.module("«target_name»");
target.rootDir = "«compiler_install_dir»";
Build.targets = [target];

Line 1 loads the target specified on the command line, line 2 configures the target's installation directory, and line 3 tells the XDCtools build engine to only build for this one target.

Similarly, if you also specify a hand-crafted config.bld to be used (via -b «config_bld_file»), configuro will generate the following config.bld for its use.

xdc.loadCapsule("«config_bld_file»");
var target = xdc.module("«target_name»");
target.rootDir = "«compiler_install_dir»";
Build.targets = [target];

In this case, the user specified «config_bld_file» is loaded on line 1 and this file can configure the target «target_name» with additional compiler flags, a different default platform, new or modified profiles, etc.

Leveraging an existing config.bld file

Sometimes a project simply needs to augment or slightly modify a config.bld that is shared among a set of related projects. Since config.bld files are XDCscript files that can load and execute other scripts, it's easy to create a simple config.bld that loads a "common" config.bld and makes minor changes. This technique allows you to maintain central control with a single config.bld while still allowing for the rare (but inevitable) exception.

One technique for sharing a common config.bld, while still allowing minor changes, is to place the shared config.bld in a sub-directory, say ./common, of any repository named on your package path. Whenever you need to create a config.bld script that is a minor variation of the common config.bld, simply use xdc.loadCapsule() to find and load the common file.

var common = xdc.loadCapsule("common/config.bld");
    :

Since xdc.loadCapsule() searches along the package path for the specified file, line 1 only needs to name the sub-directory and file rather than an absolute or relative path to the shared file. This makes the new config.bld file both portable (to other workstations) and position independent. Even though xdc.loadCapsule() searches along the package path, the sub-directory ./common does not itself have to be a package.

For the sake of creating a complete example, suppose you just need to add the -mo compiler flag to all ti.targets.C64P builds within a project and that this flag should not be used in any other project. Assuming we use a shared config.bld in a ./common sub-directory of some repository named on the package path, the following config.bld will do the trick.

var common = xdc.loadCapsule("common/config.bld");
var C64P = xdc.module("ti.targets.C64P");
C64P.ccOpts.prefix = C64P.ccOpts.prefix + " -mo";

Managing multiple toolchains

All of the config.bld examples we've seen so far have configured a single target. After all, when you first start development on a project, you'll generally begin with a single C compiler toolchain. As development matures, however, it's not uncommon to port a code base to another target. Fortunately, adding support for multiple toolchains is quite easy: you simply name the new target and set its rootDir configuration parameter in the same config.bld file that you started with.

Starting with the config.bld above that was used by the xdc command to build content using the ti.targets.C64P target, suppose we want to re-build the same code base for native execution on a Linux platform.

var Build = xdc.module("xdc.bld.BuildEnvironment");
var Pkg = xdc.module("xdc.bld.PackageContents");
 
var C64P = xdc.module("ti.targets.C64P");
C64P.rootDir = "«c64_compiler_install_dir»";
var Linux86 = xdc.module("gnu.targets.Linux86");
Linux86.rootDir = "«linux_compiler_install_dir»";
Build.targets = [C64P, Linux86];  /* build using both C64P and Linux86 targets */
 
Pkg.attrs.archiver = "zip";       /* package archive format is .zip */
Pkg.attrs.profile = "debug";      /* default build profile is "debug" */

As before, line 1 loads the new target, line 2 configures this target's installation directory, and line 3 tells the XDCtools build engine to build for both targets. If your packages use xdc to compile your sources and these sources are portable C, these three lines are sufficient to build your packages for both native execution on a Linux platform and execution on an embedded C64P device; no changes to individual packages are required.