From RTSC-Pedia

[printable version]

offline version generated on 04-Aug-2010 21:08 UTC

Extending xdc.runtime Memory/Example 1

A heap that tracks Memory allocations

A Heap Monitor

In this section we develop a simple IHeap that illustrates how to create custom heaps. Suppose, for example that you want to monitor your application's use of memory and look for memory leaks or buffer overflows. The heap manager developed in this section illustrates how you can optionally add these checks to any existing heap manager without making any modifications to that manager.

How to Use the Check Heap Monitor

We begin by showing how the Check module can be configured atop an existing IHeap to monitor allocated memory. In the example below we show a simple client application that uses the Memory module to allocate a free memory. This example, illustrates that adding the Check only requires a very small change to an existing configuration file and no change to existing code.

#include <xdc/std.h>
 
#include <xdc/runtime/Error.h>
#include <xdc/runtime/Memory.h>
#include <xdc/runtime/LoggerBuf.h>
 
/* ======== main ======== */
Int main(Int argc, String argv[])
{
    Int i;
    LoggerBuf_Handle logger;
    Error_Block eb;
 
    Error_init(&eb);
 
    for (i = 0; i < 10; i++) {
        logger = LoggerBuf_create(NULL, &eb);
        if (logger != NULL) {
            if (i) { /* create a memory leak! */
                LoggerBuf_delete(&logger);
            }
        }
    }
 
    return (0);
}

var Error = xdc.useModule("xdc.runtime.Error");
var Memory = xdc.useModule("xdc.runtime.Memory");
var LoggerBuf = xdc.useModule("xdc.runtime.LoggerBuf");
 
var Check = xdc.useModule("examples.runtime.heap.Check");
var HeapStd = xdc.useModule("xdc.runtime.HeapStd");
 
/* make Check the heap for all allocations */
Memory.defaultHeapInstance = Check.create();
 
/* give Check a heap that can allocate real memory */
Check.common$.instanceHeap = 
    HeapStd.create({size: 4096});

The output of this example is shown below.

examples.runtime.heap.Check: line 100: memory leak: 2 more allocations than frees (heap = @08050688)

How to Implement the Check Heap Monitor

In this section we look under the covers to see how Check is implemented. We begin by looking at the implementation of the Check module and end by showing the other files necessary to deliver this module within a package.

The Check Module

All heap managers must implement the xdc.runtime.IHeap interface. In addition, because heap instances are most often created during configuration, heap managers should also support configuration-time instance creation. Fortunately, both of these requirements are easily meet. The Check.xdc file declares that the Check module inherits xdc.runtime.IHeap and includes an internal section that defines the instance object's state structure. This definition, together with the initialization function instance$static$init(), defined in Check.xs, is sufficient to enable static instances to be created during configuration.

The Check module works by simply maintaining the number of allocations minus the number of frees for each heap instance. If this number is non-zero at the end of the application, a memory leak has occurred. In order to get control at the end of the application, Check uses the System_atexit() method to install the doCheck function, defined in Check.c, as a function to be executed when the application exits. But, in order to call System_atexit(), the Check.xdc module specification uses the @ModuleStartup attribute to declare that it has a "startup" method that must be called prior to the application's main() entry-point. This startup method, named Check_Module_startup, is defined in Check.c.

import xdc.runtime.Error;
 
/*!
 *  ======== Check ========
 *  Check number of allocations and frees
 */
@ModuleStartup  /* this module has a startup fxn to be called prior to main */
module Check inherits xdc.runtime.IHeap
{
    /*!
     *  ======== E_memory ========
     */
    config Error.Id E_memory = {
        msg: "memory leak: %d more allocations than frees (heap = %p)"
    };
 
internal:
    struct Instance_State {
        Int count;
    };
}

The implementation of this specification is shown below.

#include <xdc/std.h>
 
#include <xdc/runtime/Startup.h>
#include <xdc/runtime/System.h>
 
#include "package/internal/Check.xdc.h"
 
static Void doCheck(Void); /*checks all instances */
 
/* ======== Check_Instance_init ======== */
Void Check_Instance_init(Check_Object *obj, 
    const Check_Params *params)
{
    obj->count = 0;
}
 
/* ======== Check_Module_startup ======== */
Int Check_Module_startup(Int state)
{
    if (!Startup_rtsDone()) {
        return (Startup_NOTDONE);
    }
 
    System_atexit(doCheck);
 
    return (Startup_DONE);
}

/* ======== instance$static$init ======== */
function instance$static$init(obj, params)
{
    obj.count = 0;
}

The doCheck() function, shown below, loops over all statically defined instances of the Check module looking for a counter that is non-zero.

#include <xdc/runtime/Error.h>
 
/* ======== doCheck ======== */
static Void doCheck(Void)
{
    Int i;
    Error_Block eb;
 
    Error_init(&eb);
 
    /* loop over all static instances */
    for (i = 0; i < Check_Object_count(); i++) {
        Check_Object *obj = Check_Object_get(NULL, i);
        if (obj->count != 0) {
            Error_raise(&eb, Check_E_memory, obj->count, (IArg)obj);
        }
    }
}

The methods used to loop over the instances, Check_Object_count() 1 and Check_Object_get() 2 are automatically generated from the Check module's specification. For more information about these methods, see the C Language Binding reference for Object_count() and Object_get(), respectively.

To complete the C implementation it is, of course, necessary to define methods that allocate and free memory (as well as any other methods declared in the xdc.runtime.IHeap interface). Shown below, these methods simply increment or decrement a counter and call the Check module's heap manager to do the "real" memory management. The Check_Object_heap() method returns the heap instance configured for the Check module and, to support concurrent environments, the counter is modified atomically using the "system gate" via Gate_enterSystem().

Recall that the Check_Object_heap() method, like all *_Module_* methods, is declared in the headers automatically generated from the Check.xdc specification and defined in the *.c file generated by the configuration process. See the C Language Binding for a complete list of these automatically generated methods.

#include <xdc/runtime/Gate.h>
#include <xdc/runtime/Memory.h>
 
/* ======== Check_alloc ======== */
Ptr Check_alloc(Check_Object *obj, SizeT size, 
    SizeT align, Error_Block *eb)
{
    Ptr result = Memory_alloc(Check_Object_heap(), size, align, eb);
 
    if (result != NULL) {
        /* atomically increment count */
        IArg key = Gate_enterSystem(); 
        obj->count++;
        Gate_leaveSystem(key);
    }
 
    return (result);
}
 
/* ======== Check_free ======== */
Void Check_free(Check_Object *obj, Ptr block, SizeT size)
{
    /* atomically decrement count */
    IArg key = Gate_enterSystem();
    obj->count--;
    Gate_leaveSystem(key);
 
    Memory_free(Check_Object_heap(), block, size);
}
 
/* ======== Check_isBlocking ======== */
Bool Check_isBlocking(Check_Object *obj)
{
    return (Memory_query(Check_Object_heap(), Memory_Q_BLOCKING));
}
 
/* ======== Check_getStats ======== */
Void Check_getStats(Check_Object *obj, Memory_Stats *stats)
{
    Memory_getStats(Check_Object_heap(), stats);
}

Finally we need to add a meta-domain function to Check.xs: module$use.

/* ======== module$use======== */
function module$use()
{
    xdc.useModule("xdc.runtime.System");
    xdc.useModule("xdc.runtime.Error");
    xdc.useModule("xdc.runtime.Gate");
    xdc.useModule("xdc.runtime.Memory");
}

The module$use function ensures that all modules used in the implementation of Check are included in both the configuration process and in the link of the application using Check. By adding these statements, the user of Check does not need to add xdc.useModule statements their configuration script for modules that are not directly referenced by their application. After all, the implementation of Check may change to require other modules and we don't want the client of Check to have to change anything to use this new version.

The Package Files

Since all modules must live in a package we also need to create a two other files:

1. package.xdc - a file that define the package's name and identifies the modules it contains, and
2. package.bld - a file that specifies which targets should be used to build the sources contained in this package.

package examples.runtime.heap { 
    module Check; 
}

for (var i = 0; i < Build.targets.length; i++) {
    var targ = Build.targets[i];
 
    var lib = Pkg.addLibrary("lib/" + Pkg.name, targ);
    lib.addObjects(["Check.c"]);
}

The package.xdc file is largely self explanatory; it defines the name of the package examples.runtime.heap and it specifies the modules contained in the package, Check.

The package.bld file, on the other hand, requires some explanation. The existence of package.bld indicates that the package is (re)buildable - unless the end-user is expected to build the package, deployed packages omit this file. The contents of this package.bld file indicate that the package contains a library in the lib/ sub-directory containing the compiled version of Check.c for every target supported by the producer's build environment. Remarkably, this package.bld file is sufficient to compile Check.c for every target, archive it into a library, and create a binary release of this package that supports all targets supported by the producer's build environment.

Summary and Extensions

This example illustrated how to create and deploy a new IHeap memory manager. Along the way, it also illustrated several key concepts:

• how a module can scan all of its statically created instances; Check_Object_count() and Check_Object_get().
• how a module can perform module-specific initialization before main(); @ModuleStartup and Check_Module_startup()
• how to "stack" IHeap memory managers to add functionality to an existing memory manager without changing (or re-compiling) the application's sources.

Although simplistic, it is clear that this example can be extended to provide far more robust memory checking. Possible extensions include

• wrapping each allocation with a leading and a trailing "cookie" which can be checked on each free operation to detect buffer over/under flow problems;
• tagging each allocation with a "thread id" so that when a leak is detected it's possible to identify the culprit and even recover the memory;
• add methods to Check that can be called by the application to set thread-specific limits on the total outstanding memory allocated by each thread; and
• add methods to Check that direct it to return NULL after a specific number of allocations allowing one to reliably test the application's ability to handle out-of-memory faults.

[printable version]

offline version generated on 04-Aug-2010 21:08 UTC