From RTSC-Pedia

[printable version]  [offline version]

offline version generated on 04-Aug-2010 21:08 UTC

Compatibility Keys

How inter-package dependencies are managed

Introduction

In order to manage different versions of packages being updated at different times by the end-user, it is important that there be some mechanism for ensuring compatibility of a collection of packages. RTSC packages have built-in support for managing dependencies among a set of packages which relies on the RTSC concept of "compatibility keys". Every package has a compatibility key consisting of an ordered tuple of three or more numbers.

Compatibility Keys are not version numbers; i.e., two different versions of a package may share a common compatibility key. Moreover, because RTSC package tooling interprets these numbers to determine if certain combinations of packages are permissible, it is important that compatibility key numbers be managed very carefully. This is in contrast to "Product Marketing" version numbers, which are often chosen on an ad hoc basis to convey subjective qualities of a product release. A package's compatibility key combined with its release date and "release name" (an arbitrary name specified by the package producer at the time the release is created) is used by RTSC tooling as a version identifier.

Compatibility among packages is not only affected by different versions of other packages. Use of different versions of code generation tools can also be the source of incompatibilities. For example, a client using compiler version 4.0 might get a package of binary content that was built with version 5.0 of the same compiler. Although compiler vendors are generally very good about maintaining backward compatibility (the ability of new linkers to use objects produced by older compliers), for obvious reasons, code generation tools are usually not forward compatible (the ability of old linkers to use objects produced by new compilers). As a result, it is important that each package specify a target compatibility key for each compiler used to generate objects it contains.

Package Compatibility Keys

Each package has a Compatibility Key, of the form [M,S,R,P], that expresses the compatibility of this version of the package with other versions of the same package. The semantics of each dimension of the compatibility key for a package pkg is described in the table below.

To ensure compatibility among a group of packages each package specifies its compatibility with other versions of itself. Each package also specifies the compatibility key of the packages that it was built and tested against. Given this information about each package, it is possible to determine if a specified set of packages are expected to work together.

Target Compatibility Keys

Each target has a Compatibility Key, also of the form [M,S,R,P], that expresses the compatibility of pre-compiled object files produced by this version of the target's underlying tool-chain with those produced using other versions of the same tool-chain. The semantics of each dimension of the compatibility key for a target T is described in the table below.

Target compatibility keys are typically computed automatically by a target from the version number reported by the underlying compiler tool-chain. Each compiler tool-chain has a unique way of communicating compatibility via its version numbers but, by first converting these version numbers to a compatibility key, XDCtools can uniformly manage some compiler compatibility checks without having to know about any particular tool-chain.

Although there can be exceptions, most targets map compiler version numbers into compatibility keys by simply using the compiler's major and minor version number as the API Radius. For example, version 3.4.6 of a compiler might be converted to the compatibility key [1, 0, 3.4, 6]. This simple mapping implies:

• it's always possible to use a version of the compiler with the same (or greater) major or minor version number than that used to build any content you are referencing; e.g., you can use a 4.1.0 compiler to consume content built with 3.4.6.
• it's possible to use an earlier version of a compiler provided the major and minor version number match; e.g., you can use 3.4.2 to consume content built with 3.4.6 of the compiler.
• a warning (or optionally an error) will be triggered if you try to use an older compiler to consume content built with a compiler that has later major or minor version number; e.g., a warning is generated if you try to use 3.4.6 to consume content built with a 3.6.0.

Sources of Compatibility Keys

There are three sources of compatibility key information:

1. All packages declare at least [M, S, R] in their package.xdc package specification file;
2. All packages specify the compatibility keys of all other packages used in its construction and this specification appears in an XML file named package/package.rel.xml which complies with the DTD release.dtd supplied in the xdc.bld package; and
3. All packages specify the target compatibility key for all of the objects it contains and this specification appears in an XML file named package/package.bld.xml which complies with the DTD build.dtd supplied in the xdc.bld package.

So, in addition to a package's declared compatibility key (item 1 above), each package carries with it a set of "built-with" compatibility keys - the keys of the packages and targets used to build the package (items 2 and 3 above). These "built-with" keys provide the connection to other packages that allows RTSC tooling to detect when a specific combination of packages should not be used together.

Both files containing "built-with" compatibility keys, package/package.rel.xml and package/package.bld.xml, are automatically generated by the xdc command during the package build process. The "built-with" keys named in the package/package.rel.xml file are determined from two sources: the optionally supplied compatibility keys named by requires statements in the package's package.xdc file and all packages determined by examining the makefile dependencies generated by the RTSC Build Model during the build of the package. If you do not use the RTSC Build Model to compile your C content, you must manually update the package's package.xdc requires statements to ensure the completeness of the package's "built-with" keys. The "built-with" target keys named in package/package.bld.xml are determined by the targets named by the target configuration file (typically config.bld) used to build the package.

We don't require packages to specify the patch number because this value is not needed by tooling to detect a compatibility failure; if the first three digits of a pair of compatibility keys match the two components are (as far as the key is concerned) 100% interchangeable. The reason for having a patch digit is to enable compatibility keys optionally to serve as version numbers.

In the rare case that a package does not provide any programmatic interface that can be referenced by another package, the compatibility key for such a package is technically unnecessary. For example, some products include example applications (in the form of a package) which are never referenced by another package. We still require a compatibility key to be specified because there is no way for "package checking" tools to reliably recognize this situation. For these cases the package should simply be given the compatibility key [1, 0, 0].

Compatibility Key Checks

Package and target compatibility keys are checked only at specific points in the development process:

• during package installation,
• as part of the configuration process when modules are assembled, and
• when the producer creates a release of a package.

All checks can be controlled on a per package or per target basis and can even be disabled altogether.

Why is it necessary to control or disable the compatibility checks? There are times when, strictly speaking, a package as a whole is incompatible with another package but it is still possible to use both packages safely in an application. Suppose for example, that a package provides a new version of a module in which a C-language macro is reimplemented as a function. The package's compatibility key should indicate that a re-compile is necessary (a change in the S digit). On the other hand, clients that never reference this particular macro do not need to be recompiled even though the package compatibility key suggests that it is required. Since packages have a single compatibility key - rather than a key for each constant, type, and method of each module - and it's not practical to precisely determine what parts of each package are actually used by others, there are times when the compatibility checks will falsely indicate a problem.

The following sub-sections describe what checks are performed.

Installation Time Checks

Before a package can be used it must be installed in a repository that appears along your project's package path. If two "incompatible" packages are installed in the same repository and your application references both of these packages, it is difficult (though not impossible) to build your application with compatible versions of these packages. So, tools that build repositories, such as xdc.tools.repoman, perform the following checks on each repository:

1. all versions of each target T used to build a package in the repository are compatible with the "latest" version of T used: if T[M, S, R] is the latest version of T used by any package, and T[M', S', R'] is also used, then we require that M == M', S == S', and R >= R'
2. for every package P in the repository, the versions of all packages B used to build P are compatible with the versions of B currently contained in the repository: for each "built-with" package B[M, S, R] we compare the version of B in the repository B[M', S', R'] (if it exists) and require that M' == M, S' == S and R' >= R

Packages may be "installed" in any repository either by hand or by using a repository management tool such as xdc.tools.repoman. Installation by hand simply means that you've copied the package contents into a directory. In this case, of course, no compatibility checks are performed. You may, however, manually check that the packages in a repository are mutually compatible using the xdc.tools.check utility.

Configuration Time Checks

During configuration, three compatibility checks occur

1. for all imported packages P, P's "built-with" package keys are compared with "actual" keys: for each "built-with" package Q[M, S, R] we compare the "actual" imported Q[M', S', R'] and require that M' == M, S' == S and R' >= R
2. for all imported packages P, the configuration's target key is compared to P's target key: each package's "built-with" target T[M, S, R] is compared to the configuration's imported target T[M', S', R'] and we require that M' == M, S' == S and R' >= R
3. for all imported packages P, P may perform custom compatibility checks when P is imported by any script (via P.init())

As an example of a package-specific compatibility check, consider the following fragment of a package's xdc.IPackage implementation file, package.xs. In this case, validate() examines the version of another package (ti.bios) and based on the version either succeeds or fails.

function validate() {
    var biosVers = xdc.om["ti.bios"].$vers;        /* get ti.bios key as an array */ 
    if (biosVers.join(",").indexOf("1,2") == 0) {  /* if its key is 1,2,* ... */
        switch (biosVers[2]) {
            case 3: { /* avoid version 1,2,3,x, where x < 10 or x == 15 */
                if (biosVers[3] < 10 || biosVers[3] == 15) {
		    throw new Error("incompatible version of ti.bios");
                }
            }
         :

Package Release Time Checks

During the build of a release of a package P, two compatibility checks occur:

1. for all "required" packages R, the specified requirement key is "source compatible" with R's actual key (M identical, >= R)
2. for all "directly required" packages R, if R's key is not prefix of the "built-with" R's key, a remark is displayed

Managing "Built-With" Dependencies

Before releasing a package for general use it is important to review the package's dependencies. These dependencies affect whether the package can be used without warnings from the compatibility checks described above.

How to view built-with dependencies

How to view built-with dependencies.  The following script can be used to display a released package's external references.

function main(args) {
    /* convert package name to release XML file name */
    var file = args[0].replace(/\./g, '/') + "/package/package.rel.xml";
 
    /* load the specified package's release XML file */
    var ext = xdc.loadXML(file);
 
    /* use E4X extensions to get and display information from this file */
    var references = ext.references["package"];
    print("package " + ext["package"].@name + " references the following packages:");
    for (var i in references) {
        print("    " + references[i].@name + " [" + references[i].@version + "]");
    }
}

This script can be run using the -c option of the xs command.

% xs -c refs.xs xdc.runtime
    package xdc.runtime references the following packages:
        xdc.utils.tconf [1, 0, 0, 0]
        xdc.services.global [1, 0, 0]
        xdc.services.spec [1, 0, 0, 0]
        xdc.corevers [16, 0, 1, 0]
        xdc.bld [1, 0, 0, 0]
        xdc.services.intern.cmd [1, 0, 0, 0]
        xdc.services.intern.xsr [1, 0, 0]
        xdc [1, 1, 0, 0]
        xdc.shelf [1, 0, 0, 0]
        xdc.services.intern.gen [1, 0, 0, 0]
%

How to control built-with dependencies

How to control built-with dependencies.  A package's built-with dependencies are automatically computed by the xdc tool when building the release goal; e.g., when running the command "xdc release" in a package's base directory. The algorithm used to compute these dependencies scans the package's generated makefiles for external pre-requisites, computes the packages containing these pre-requisites, and records both the package name and version information for each such package.

There is currently (7/11/2008) no way to add or remove built-with dependencies to this automatically generated list.

Controlling Compatibility Key Checks

The next few sub-sections describe how to control the compatibility checks described above.

Installation Time Checks

When extracting package into a repository, the xdc.tools.repoman tool first checks that the packages are compatibile with the other packages in the repository. If an incompatibility is detected, the repository is not modified. You can force the installation of these packages using the -F flag, however.

xs xdc.tools.repoman -x -F ...

Configuration Time Checks

Configuration time checks are controlled by two "environment" variables:

• xdc.cfg.check.exclude - a regular expression that is used to select packages that should be excluded from the set of packages checked during configuration, and
• xdc.cfg.check.fatal - if set to false, force any incompatibilities detected to be treated as warnings only; otherwise incompatibilities are fatal

To define these environment variables you must pass the appropriate -D options to the XDCscript shell (xs). For example, to ignore any incompatibilities triggered by packages whose names start with ti.targets.rts, set xdc.cfg.check.exclude to the regular expression 'ti.targets.rts.*':

xs -Dxdc.cfg.check.exclude='ti.targets.rts.*' xdc.tools.configuro ...

Similarly, to treat any detected package incompatibilities as warnings only, set xdc.cfg.check.fatal to false:

xs -Dxdc.cfg.check.fatal=false xdc.tools.configuro ...

If you are using the RTSC Build Model to build your application, you can pass these options to the the XDCscript shell on a per executable basis via the xdcopts attribute of an Executable.

Pkg.addExecutable("app", targ, targ.platform, {
    xsopts: "-Dxdc.cfg.check.exclude='ti.targets.rts.*'"
});

In addition, if you don't want to set xsopts for each executable, it is possible to set it for all executables within a package by setting the xsopts attribute of the xdc.bld.PackageContents (Pkg) object. For example, to turn all compatibility checks into warnings for all executables in a package, add the following line to the package's build script:

Pkg.attrs.xsopts = "-Dxdc.cfg.check.fatal=false";

Package Release Time Checks

To ensure the correctness of the information contained in each package's package.xdc file, the release time checks can not be overridden.

[printable version]  [offline version]

offline version generated on 04-Aug-2010 21:08 UTC