From RTSC-Pedia

[printable version]

offline version generated on 04-Aug-2010 21:08 UTC

RTSC Packaging Primer/Lesson 9

Product Bundles — producing charlie_mathsuite_2_71

While content suppliers can always deliver elementary packages like charlie.sqrtlib on an individual basis—installed and deployed following the pattern first outlined in Lesson 1 using acme.utils—package producers will frequently field product bundles like charlie_mathsuite_2_71 as a convenient vehicle for aggregating multiple packages into a single deliverable. Whereas Lesson 2 focused on installing and deploying product bundles from the consumer's vantage, we'll now examine the internal steps required to orchestrate their production within package.bld scripts.

But first, we owe you some explanation behind the name charlie_mathsuite_2_71 itself—which then leads to a broader discussion on how to express software versioning within constituent packages such as charlie.sqrtlib.

Conveying version information

While we'll often use the terms interchangeably, this lesson will clearly distinquish between different versions versus multiple releases of the same package. As Lesson 6 illustrated using charlie.sqrtlib, multiple releases of the same package (drawing from a common base of content) occur at the same point in time. By contrast, when we refer to different versions of the same package, the dimension of time enters the equation. To complicate matters a bit, we'll routinely use different RTSC idioms for conveying version information in product bundles versus elementary packages.

External product names.  Externally branded products—MathSuite v2.71 from Charlie & Co.—will often encode some level of software versioning in their very name; for sure, we'd all recognize MathSuite v3.14 as a more recent (and perhaps improved) version of the same underlying product. But other than the fact that "larger" numbers mean "later" products, these sorts of labeling schemes often say very little about technical changes to programmatic contracts within the product's constituent packages—an important topic we'll address shortly.

Mapping external products (MathSuite v2.71) onto RTSC product bundles (charlie_mathsuite_2_71) applies a simple naming convention for the corresponding package—ensuring its global-uniqueness by prefixing the producer's identity (charlie), but using a style that differentiates product "wrapper" packages from elementary "bundled" packages with multi-segment names like charlie.sqrtlib. Following the protocol laid out in Lesson 2 for deploying product bundles, you'd rarely find packages with mangled names like charlie_mathsuite_2_71 along the current package path; rather, you'd see these names within the package path itself, en route to some internal repository contained therein.

In practice, RTSC product bundles become "one-shot" packages:  Charlie & Co. would never deliver a later version of the same package named charlie_mathsuite_2_71; instead, they would manufacture a new wrapper package with a different name (say, charlie_mathsuite_3_14) that reflects the product's external positioning. This further confirms that top-level wrapper packages would typically contain artifacts specific to a particular product version (say, its README file); generic content found and consumed along the package path—modules, headers, libraries—would instead reside within elementary packages that populate the product's internal repository(s).

Package compatibility key.  Elementary packages like charlie.sqrtlib could presumably reach potential clients as part of many different product bundles, each uniquely named to reflect a particular product version and each released at a unique point in time. But unlike top-level wrapper packages, nothing in the name charlie.sqrtlib conveys any information about the particular version of this package bundled within a particular product deliverable.

And yet, we know (and even come to expect!) that any given package will evolve over the course of time—potentially changing from one version to the next in ways that impact compatibility from the client programmer's perspective:

full binary-compatibility

The supplier may repair defects or improve performance, requiring clients to re-link existing code.

backward binary-compatibility

The supplier may add new functions to existing libraries, also requiring clients to re-link existing code.

source-compatibility

The supplier may add new fields to public structures in headers, now requiring clients to re-compile their code.

in-compatibility

The supplier may elect to remove or alter previously available features, thus requiring clients to re-write their code.

To formally convey this sort of versioning information, suppliers can specify a package compatibility key within the XDCspec package.xdc file that remains inside the package throughout its natural life-cycle. Consider, then, a side-by-side comparison of the charlie/sqrtlib/package.xdc file originally shipped as part of the MathSuite v2.71 product with a newer version of the charlie.sqrtlib package spec that might accompany (say) MathSuite v3.14.

/*! Library of square-root functions */
package charlie.sqrtlib [1,0,2,6] {
    module Settings;
}

/*! Bigger and better library of square-root functions */
package charlie.sqrtlib [1,0,4,3] {
    module Settings;
}

Even without reading the commentary here, the package compatibility key spec'd at line 1b intuitively seems "more recent" than the one associated with the (earlier) version spec'd at line 1a. To the point, though, comparing keys of the form [M,S,R,P] precisely communicates what sort of compatibility the supplier has maintained between these two different versions of the charlie.sqrtlib package released at different points in time:

full binary-compatibility

same [M,S,R] but different P, where P designates the patch level of this package

backward binary-compatibility

same [M,S] but different R, where R designates the radius level of this package

source-compatibility

same [M] but different S, where S designates the source level of this package

in-compatibility

different M, where M designates the major level of this package

Stated in other terms, suppliers of target content who promise binary-compatibility to their clients—a very admirable objective, and one apparently maintained here by charlie.sqrtlib—would effectively step just the P or R values of the key from one package version to the next:

• changing internal function implementations while preserving all current programmatic contracts would result in a patch, stepping only the P value; but

• adding new client-visible functions conceptually widens the radius of any prior contract, reflected by stepping the R value while resetting P.

Put another way, packages whose compatibility keys change in their M or S values from one version to the next have apparently required their clients to minimally re-build—and possibly even re-write—their own content. While often unavoidable in practice, suppliers should nevertheless weigh the cumulative impact of these sorts of changes when their clients themselves become producers of other RTSC packages.

Populating internal repositories

Turning now to the general process of manufacturing RTSC product bundles, recall again from Lesson 2 how the charlie_mathlib_2_71 wrapper package itself houses a repository (named packages) which in turn contains released forms of the charlie.sqrtlib and charlie.sqrtlib.samples packages. As with elementary packages, the package.bld script within the wrapper package defines the inventory of the product bundle in its deliverable form; and once again, the xdc.bld.PackageContents meta-module plays a central role in capturing this information.

Importing named releases.  In its simplest form, the package.bld script for charlie_mathlib_2_71 would populate its internal packages repository by directly importing individual packages releases referenced through strings of the form "«package-name»:«release-name»". Consider our first variant of package.bld:

/* first variant */
var Pkg = xdc.useModule('xdc.bld.PackageContents');
 
var PKG_RELEASES = [
    "charlie.sqrtlib:charlie_sqrtlib",
    "charlie.sqrtlib.samples:charlie_sqrtlib_samples"
];
 
var repo = Pkg.addRepository("packages");
repo.addPackages(PKG_RELEASES);
 
Pkg.attrs.archiver = "zip";

After creating an XDCscript object representing the newly-named repository at line 2, the call to addPackages at line 3 defines the set of package release archives to effectively "install" into this particular repository when releasing charlie_mathlib_2_71 itself. While each element within the PKG_RELEASES array defined at line 1 happens to reference the default archive name associated with these packages, you can surely imagine generalizations of this idiom with multiple repositories in the wrapper package populated with distinct releases of other elementary packages.

Importing labeled releases.  To shield the package.bld script within the wrapper package from the actual release names chosen for each of the elementary packages, we can alternatively import the appropriate release by using its label; recall from Lesson 6 that you can assign a meaningful label to each archive file manufactured by a particular package. Consider our second variant of package.bld, which now features a nested pair of for loops each iterating over successive elements of some XDCscript array:

/* second variant */
var Build = xdc.useModule('xdc.bld.BuildEnvironment');
var Pkg = xdc.useModule('xdc.bld.PackageContents');
 
var REQUIRED_PKGS = ["charlie.sqrtlib", "charlie.sqrtlib.samples"];
 
var pkgReleases = [];
 
for each (var pname in REQUIRED_PKGS) {
    for each (var relDesc in Build.getReleaseDescs(pname)) {
        if (relDesc.label = "default") {
            pkgReleases.push(pname + ":" + relDesc.name);
        }
    }
}
 
var repo = Pkg.addRepository("packages");
repo.addPackages(pkgReleases);
 
Pkg.attrs.archiver = "zip";

Working backwards from line 6, notice that we've computed the array of "«package-name»:«release-name»" strings passed to addPackages here. First initialized with an empty array on line 3, the standard JavaScript function push invoked at line 5 repeatedly extends this array with strings formed by concatenating pname [«package-name»] with the relDesc.name property [«release-name»]. The local variable pname used here iterates over each package named in the REQUIRED_PKGS array, initialized back at line 2.

In the case of relDesc, the for loop at line 4 successively binds this local variable to each element of an array of xdc.bld.Release.Desc objects—whose label and name properties we've accessed in the lines that follow. As a final note, we've once again imported the xdc.bld.BuildEnvironment meta-module in the package.bld script and then bound the corresponding XDCscript object to the local variable Build at line 1.

Importing required packages.  As it turns out, the same XDCspec requires statement already seen in Lesson 8 can essentially obsolete the REQUIRED_PKGS array initialized above at line 2. Besides publicly specifying the inventory of other packages released as part of a particular product bundle, the requires statement allows suppliers to designate specific versions of these packages identified by their compatibility key.

requires charlie.sqrtlib[1,0,2,6];
requires charlie.sqrtlib.samples;
 
/*! Charlie & Co. MathSuite 2.71 product bundle */
package charlie_mathsuite_2_71 {
}

Returning again to the package.bld script for charlie_mathlib_2_71, our final variant will now programmatically retrieve the names of those dependent packages specified through the requires statements beginning at line 1 of the corresponding package.xdc file.

/* final variant */
var Build = xdc.useModule('xdc.bld.BuildEnvironment');
var Pkg = xdc.useModule('xdc.bld.PackageContents');
 
var pkgReleases = [];
 
for each (var pname in Pkg.imports) {
    print("requires: " + pname);
    pname = pname.substr(0, pname.indexOf("{"));
    for each (var relDesc in Build.getReleaseDescs(pname)) {
        if (relDesc.label = "default") {
            var imp = pname + ":" + relDesc.name;
            print("importing: " + imp);
            pkgReleases.push(imp);
        }
    }
}
 
var repo = Pkg.addRepository("packages");
repo.addPackages(pkgReleases);
 
Pkg.attrs.archiver = "zip";

The for loop at line 1 binds the local variable pname to successive elements of Pkg.imports—an array of strings of the form "«package-name»{«optional-compatibility-key»", with the text after the "{" character omitted if the corresponding package has no specified key. After printing its original value, the script extracts the «package-name» portion of pname at line 2 using some built-in string functions courtesy of JavaScript. The remainder of the script mimics our second variant of package.bld.

Building with the xdc command

And now comes the fun part.... Simply invoke the xdc release -PD . command from within the charlie_mathlib_2_71 package directory, and let's just see what happens:

%> xdc release -PD .
    ...
making package.mak (because of package.bld) ...
requires: charlie.sqrtlib{1, 0, 2, 6
importing: charlie.sqrtlib:charlie_sqrtlib
requires: charlie.sqrtlib.samples{
importing: charlie.sqrtlib.samples:charlie_sqrtlib_samples
    ...
 
======== release [«examples»/charlie/sqrtlib] ========
all files complete.
    ...
making release file charlie_sqrtlib.zip (because ...) ...
 
======== release [«examples»/charlie/sqrtlib/samples] ========
all files complete.
    ...
making release file charlie_sqrtlib_samples.zip (because ...) ...
 
======== release [«examples»/charlie_mathsuite_2_71] ========
all files complete.
 
installing charlie.sqrtlib [release charlie_sqrtlib.zip] ...
installing charlie.sqrtlib.samples [release charlie_sqrtlib_samples.zip] ...
    ...
making release file charlie_mathsuite_2_71.zip (because of ...) ...

Starting at the top, we've (re-)generated package.mak for charlie_mathlib_2_71 by first executing what became the final variant of its package.bld script; the messages following line 1 originate with print statements embedded within the script itself. Moving on, the output at lines 2 and 3 reflects the prior release of the charlie.* packages named in the requires statements within the package.xdc spec—(re-)building these dependent packages on an as-needed basis, thanks to their generated package.mak files.

With all requisite packages now up-to-date, the formal release of charlie_mathsuite_2_71 begins at line 4 by first ensuring the current package has built all of its artifacts (if relevant). As suggested by the output at line 5, released forms of the dependent charlie.* packages then find themselves installed into a newly-created packages repository rooted under the charlie_mathsuite_2_71 package directory. The final step at line 6 would automatically include the packages repository—plus the released forms of the charlie.* packages it now contains—within the generated charlie_mathsuite_2_71.zip package archive.

See also

TODO:  write it

[printable version]

offline version generated on 04-Aug-2010 21:08 UTC