Memory Management

Technical Notes

Overview
Terminology
Resolving "Duplicate Symbol" Errors
Basic Rules of Heap Operation
Near Heap Characteristics
Far Heap Characteristics
Automatic Near Heap Initialization
Managing the Near Heap
Automatic Far Heap Initialization
Managing the Far Heap
Initializing and Managing the Relative Heap
Location of Near and Far Heaps
Location of Relative Heaps
Using and Relocating Relative Heaps

Overview

This unit consists of functions which manage near, far, and relative heaps. Heaps are simply program or system memory areas which have been reserved for dynamic memory allocation. Relative heaps are similar to near heaps but they may reside anywhere in memory and are fully relocatable. One near heap, one far heap, and any number of relative heaps may be defined and used simultaneously, regardless of the memory model.

The near and far heap management systems in this unit are provided as replacements for the standard C near and far heaps. In addition, the relative heap management system may be used as a standalone heap management system or as a supplement to the Spontaneous Assembly or standard C near and far heaps.

Terminology

In keeping with traditional heap terminology, blocks of memory are described as being on a heap when they are allocated. Although the same terminology is used to describe stacks, heaps do not operate like stacks. Stack memory must be allocated and deallocated on a last-in, first-out basis; heap memory may be allocated and deallocated at random.

Advantages of Using Spontaneous Assembly Heap Management

With the Spontaneous Assembly heap management system, the memory for the heap is only allocated once-at heap initialization time-as opposed to the C philosophy of heap management which allocates more heap memory each time space is needed for a new heap block. The price of more efficient heap management is that the programmer must plan ahead when allocating heap space. Especially in C++, where the new and delete functions place heavy demands on the heap, the added efficiency can be well worth the required forethought in determining the heap size.

By following the easy steps listed below, the Spontaneous Assembly heap management functions will be called internally by the C/C++ startup code and library functions (including new and delete) instead of malloc, calloc and free.

The present version of Spontaneous Assembly does not include heap management support for Windows or OS/2.

Installing Spontaneous Assembly Heap Management

While the Spontaneous Assembly heaps are similar to the heaps provided in most standard C libraries, the two systems may not be used in the same program to manage the same block of memory. For example if Spontaneous Assembly heap management functions are being used and the malloc function from the compiler library is called (either internally or externally), memory blocks allocated by the Spontaneous Assembly heaps could be overwritten. Since a substantial number ofstandard C functions use malloc and free internally and some C compiler startup codes use malloc and free, a macro must be used to ensure the use of only one heap management system.

The macro that must be used depends on the startup code that is being linked into the program. If the startup code of the compiler will be linked into the program, the HEAP macro should be used. If the Spontaneous Assembly startup code for C will be used, the HEAP_SA macro should be used. The HEAP_TSR macro should be used for TSRs written with Spontaneous Assembly (malloc, calloc, and free are only available as resident functions). If only the default heap will be used (a near heap in small data models and a far heap in large data models), the above macros can be substituted with NHEAP, NHEAP_SA, or NHEAP_TSR to reduce the amount of heap initialization code linked into the final program.


#include 
...
HEAP        /* macro dtermines compiler and init functions to use for initializing
               Spontaneous Assembly Heap management. */
           ...
           void main(void)
           {
           ...
           }
Once the proper macro is used, malloc, calloc, free, _near_malloc, _near_calloc, _near_free, _far_malloc, _far_calloc, and _far_free will all use the Spontaneous Assembly memory management system instead of the C heap management system; no further initialization is required.

Note that because all calls to malloc or calloc (both direct and internal) use the Spontaneous Assembly memory management functions, it is NOT necessary to change calls to malloc, calloc, or free to use the standard Spontaneous Assembly heap management functions. In the TINY, SMALL, and MEDIUM models, the Spontaneous Assembly near heap functions are used if malloc or calloc is called. If malloc or calloc is called in COMPACT or LARGE, the Spontaneous Assembly far heap functions are used. This mimics the behavior of malloc and calloc in most standard C libraries.

The following table summarizes how and when the C default heap management system is replaced by the Spontaneous Assembly heap management system:

HEAP TYPE   SMALL DATA MODELS           LARGE DATA MODELS

Near
Heap        Replaces C's default heap   (Not available)
            management system:
            malloc = _near_malloc
            calloc = _near_calloc
            free   = _near_free

            Initialized on first call
            to malloc,
            calloc, _near_malloc,
            _near_calloc,
            _far_malloc, or
            _far_calloc.



Far         Initialized on first call   Replaces C's default heap
Heap        to _far_malloc              management
            or _far_calloc.             system:
                                        malloc  = _far_malloc
            The near heap is            calloc  = _far_calloc
            automatically               free    = _far_free
            initialized (if it is not
            already                     Initialized on first call to
            initialized) prior to       malloc, calloc,
            initializing the far        _far_malloc, or _far_calloc.
            heap.



Relative    Always available. May be    Always available. May be used
Heaps       used in                     in
            addition to any other heap. addition to any other heap.
            Must be                     Must be
            initialized with _rel_init. initialized with _rel_init.
            (Libraries                  (Libraries do
            do not need to be           not need to be modified.)
            modified.)

Resolving "Duplicate Symbol" Errors

Since the _SA? libraries contain symbols which may be duplicates of symbols in standard C libraries (malloc, calloc, etc.), some linkers may generate a "duplicate symbol..." error when the Spontaneous Assembly heap management functions are used. This error message can usually be prevented by specifying a command line option which limits how the linker searches for external symbols (for example, /NOE for Microsoft's LINK.EXE). If the error message persists, extract/remove the malloc, calloc, and free .OBJ files from the standard C libraries which are generating duplicate symbol errors. This can be accomplished with the following command lines:

Microsoft C 6.0A/7.0, Quick C:


lib slibce *-malloc *-calloc *-free;
Borland C:

tlib cs *-nearheap *-farheap
Zortech C:

zorlib zls *-alloc *-far
It is important to modify the libraries for each memory model in which the Spontaneous Assembly heaps may be used.

Basic Rules of Heap Operation

The fundamental rules of operation are the same for all three types of heaps: A singly-linked heap block management methodology is used because it provides maximum speed and memory efficiency. However, this scheme also makes the heap susceptible to corruption. Caution must be exercised to never write beyond the boundaries of a heap block or the entire heap may become corrupted. Limited block address validation is performed within many of the heap management functions to help prevent heap corruption.

Although near, far, and relative heaps are very similar, they differ in initialization methods, allowed initialization locations, maximum allowed heap sizes, maximum block size, block allocation granularity, etc. These characteristics are described below.

Near Heap Characteristics

Initialization Automatic initialization on first call to an allocation function. Heap size defaults to all remaining data segment memory. The default heap size may be specified using the nheap_len and nheap_comlen variables (see Automatic Near Heap Initialization, below).

Number of heaps supported: One.

Heap location: At the end of static data in DGROUP.

Assumptions: Initialization assumes DOS segment ordering (DOSSEG) (except for Turbo C and Borland C; see Location of Near and Far Heaps below). In the TINY model, the stack is assumed to be at the high end of the data segment.

Maximum heap size: All unused space in the default data segment (up to 64K including heap control variables).

Maximum block size: 32,764 bytes.

Minimum block size: 0 bytes.

Overhead per block: 2 bytes.

Granularity: 2 bytes.

Block alignment: Word.

Access method: Block pointers are offsets within DGROUP (near pointers).

Heap control variables: Stored in the default data segment (DGROUP).

Far Heap Characteristics

Initialization Automatic initialization on first call to a far allocation function. Heap size defaults to all remaining contiguous far memory. The default heap size may be specified using the fheap_len and fheap_dosmin variables (see Automatic Far Heap Initialization, below).

Number of heaps supported: One.

Heap location: At the end of static data in DGROUP, or at the end of the near heap if it is initialized, whichever is higher in memory.

Assumptions: DOS segment ordering (DOSSEG) (except Turbo C and Borland C; see Location of Near and Far Heaps, below).

Maximum heap size: All unused contiguous conventional system memory.

Maximum block size: 524,270 bytes.

Minimum block size: 14 bytes.

Overhead per block: 2 bytes.

Granularity: 16 bytes.

Block alignment: Paragraph.

Access method: Block pointers are absolute segment addresses; far pointers have a zero offset.

Heap control variables: Stored in the default data segment (DGROUP).

Relative Heap Characteristics

Initialization: Custom initialization only. Heap size and starting segment is user-specified.

Number of heaps supported: Unlimited.

Heap location: Anywhere in conventional memory. May be relocated at any time.

Assumptions: Heap memory must be allocated for use by the heap before the heap is initialized.

Maximum heap size: Up to 64K including heap control variables.

Maximum block size: 32,764 bytes.

Minimum block size: 0 bytes.

Overhead per block: 2 bytes.

Granularity: 2 bytes.

Block alignment: Word.

Access method: Block pointers can be anywhere in memory (far pointers).

Heap control variables: Stored in the first 16 bytes of the heap segment.

Automatic Near Heap Initialization

nheap_len
(Variable) Contains the requested size of the near heap (SMALL data models only).
nheap_comlen
(Variable) Contains the minimum number of bytes to leave available for the .COM stack when the near heap is initialized (TINY model only).
nheap_start
(Variable) Contains the starting offset of the near heap.
nheap_size
(Variable) Contains the current size of the near heap.
The near heap is only available if the heap management system has been installed as described earlier in this chapter (see Installing Spontaneous Assembly Heap Management, above). In small data models the Spontaneous Assembly near heap replaces the C heap management system so that calls to malloc, calloc, or free automatically call their Spontaneous Assembly near heap counterparts. In large data models the Spontaneous Assembly far heap functions are not available.

The near heap is automatically initialized on the first call to a near or far memory allocation function. This usually occurs in the C startup code. A specific near heap initialization function is not necessary.

nheap_len and nheap_comlen are global near heap variables which determine the size of the near heap when it is initialized. The nheap_len variable specifies the maximum size of the near heap, in bytes (it is ignored in the TINY model). The default nheap_len value is -1, which specifies that the maximum available near data space is to be allocated to the near heap. The nheap_comlen variable is used in TINY models to specify the number of bytes to leave for the .COM stack. The default nheap_comlen value is 1000 bytes. Setting nheap_len to zero prevents the near heap from being initialized. The values of these variables may be modified by declaring them in the same module as main, as follows:


...
unsigned nheap_len = 5000;        /* init near heap to 5000 bytes */
#if __TINY__
unsigned nheap_comlen = 2000;     /* leave the .COM stack 2000 bytes */
#endif
...
If the amount of memory specified in nheap_len is not available, all available near data space is allocated to the near heap. In the TINY model, the near heap is not initialized if the amount of stack space specified in nheap_comlen is not available.

nheap_start and nheap_size are global near heap variables which indicate the starting near address (offset within DGROUP) and total size of the near heap, in bytes. A value of zero in nheap_size indicates that the near heap is not initialized. Any of the near heap functions may access and/or modify these variables at any time. In addition, the internal far heap initialization functions access these variables to ensure that the far heap does not overlap an existing near heap.

Managing the Near Heap

_near_malloc
Allocates a specified number of bytes on the near heap.
_near_calloc
Allocates space for a specified number of objects on the near heap and sets all allocated bytes to zero.
_near_realloc
Alters the size of a near heap memory block.
_near_free
Frees a block on the near heap.
_near_avail
Returns the number of available bytes on the near heap.
_near_max
Returns the size of the largest available block on the near heap.
These functions manage and return information about the near heap.

_near_malloc and _near_calloc allocate near heap memory. Both of these functions accept a size and return a pointer to the newly allocated block.

_near_free and _near_realloc may be used to deallocate and resize blocks on the near heap. _near_realloc may need to move a block when it is resized. Therefore, a new block pointer may be returned.

_near_avail and _near_max returns the total available space on the near heap and the size of the largest available block on the heap. Sizes are reported in bytes.

Note that the standard C function names (e.g., malloc and calloc) may also be used to access the Spontaneous Assembly heap management system once it has been properly installed. The near heap is not available in large data models.

Automatic Far Heap Initialization

fheap_len
(Variable) Contains the desired size for far heap initialization.
fheap_dosmin
(Variable) Contains the minimum number of bytes to leave available for DOS when the far heap is initialized.
fheap_start
(Variable) Contains the starting segment of the far heap.
fheap_size
(Variable) Contains the current size of the far heap.
The Spontaneous Assembly far heap is available in all memory models if the heap management system has been installed as described earlier in this chapter (see Installing Spontaneous Assembly Heap Management, above). In large data models the Spontaneous Assembly far heap replaces the C heap management system so that calls to malloc, calloc, or free automatically call their Spontaneous Assembly far heap counterparts. In small data models, however, the Spontaneous Assembly far heap functions must be called directly since the C heap management system is replaced by the Spontaneous Assembly near heap management system.

The far heap is automatically initialized on the first call to a far memory allocation function. For most C compilers, this occurs in the startup code. A specific far heap initialization function is not necessary.

fheap_len and fheap_dosmin are global far heap variables which determine the size of the far heap when it is initialized. The fheap_len variable specifies the maximum size of the far heap, in paragraphs. The nheap_dosmin variable specifies the number of paragraphs to leave available to DOS when the far heap is initialized. By default, fheap_len is -1 (which specifies that all available far data space is to be allocated to the far heap). The default value of fheap_dosmin is 0. Setting fheap_len to zero prevents the far heap from being initialized. The default values of these variables may be modified by declaring them in the same module as main, as follows:


   ...
   unsigned fheap_len = 1000;                   /* init far heap to 1000 paragraphs (64K) */
   #if __TINY__
   unsigned fheap_dosmin = 1000;                /* leave DOS 1000 paragraphs (64K) */
   #endif
   ...
If fewer than fheap_len minus fheap_dosmin paragraphs of far data space are available, the maximum available far heap memory minus fheap_dosmin paragraphs is allocated to the far heap.

fheap_start and fheap_size are global far heap variables which indicate the starting address (offset within DGROUP) and total size of the far heap, in paragraphs. A value of zero in fheap_size indicates that the far heap is not initialized. Any of the far heap functions may access and/or modify these variables at any time. In addition, the internal near heap initialization functions access these variables to ensure that the near heap does not overlap an existing far heap.

Managing the Far Heap

_far_malloc
Allocates a specified number of bytes on the far heap.
_far_calloc
Allocates space for a specified number of objects on the far heap and sets all allocated bytes to zero.
_far_realloc
Alters the size of a far heap memory block.
_far_free
Frees a block on the far heap.
_far_avail
Returns the number of available bytes on the far heap.
_far_max
Returns the size of the largest available block on the far heap.
These functions manage and return information about the far heap.

_far_malloc and _far_calloc allocate far heap memory. Both of these functions accept a size in bytes and return a pointer to the newly allocated block.

_far_free and _far_realloc may be used to deallocate and resize blocks on the far heap. _far_realloc may need to move a block when it is resized. Therefore, a new block pointer may be returned.

_far_avail and _far_max return the total available space on the far heap and the size of the largest available block on the heap. Sizes are reported in bytes.

Note that the standard C function names (e.g., malloc and calloc) may also be used to access the Spontaneous Assembly far heap management system in large data models once it has been properly installed. In small data models, however, the Spontaneous Assembly far heap functions must be called directly (using the names shown in the table above) since the C heap management system is replaced by the Spontaneous Assembly near heap management system.

Initializing and Managing the Relative Heap

_rel_init
Initializes a relative heap.
_rel_malloc
Allocates a specified number of bytes on a relative heap.
_rel_calloc
Allocates space for a specified number of objects on a relative heap and sets all allocated bytes to zero.
_rel_realloc
Alters the size of a relative heap memory block.
_rel_free
Frees a block on a relative heap.
_rel_avail
Returns the number of available bytes on a relative heap.
_rel_max
Returns the size of the largest available block on a relative heap.
These functions manage and return information about relative heaps.

_rel_init initializes a previously allocated region of memory for use as a relative heap. Any number of relative heaps may be initialized and used simultaneously. The address and size of the heap must be specified when any relative heap is initialized. Relative heap sizes are specified in bytes, and they must begin at segment boundaries.

_rel_malloc and _rel_calloc allocate relative heap memory. Both of these functions accept a size and return a pointer to the newly allocated block.

_rel_free and _rel_realloc may be used to deallocate and resize blocks on a relative heap. _rel_realloc may need to move a block when it is resized. Therefore, a new block pointer may be returned.

_rel_avail and _rel_max return the total available space on a relative heap and the size of the largest available block on a relative heap. Sizes are reported in bytes.

Location of Near and Far Heaps

The near heap starts at the end of static data in DGROUP and the far heap starts just after the near heap. (In the TINY model the far heap starts after the .COM stack, which follows the near heap; this is also true in the SMALL and MEDIUM models for Turbo and Borland C compilers.) Note that the near heap is part of DGROUP once it is initialized.

The internal near and far heap initialization functions use public variables and/or labels found in each compiler's startup code to identify the default starting address of each heap. The addresses found in the nheap_start and fheap_start variables are the same (or very close to the same) as the starting locations used by the standard C heap management routines.

The following memory maps show the locations of the near and far heaps after they are initialized. (Note the difference for Turbo and Borland C compilers in the SMALL and MEDIUM models.)