Internal SORT crashes after upgrading to ACUCOBOL-GT Version 10



After upgrading to version 10 some report programs are crashing.  Troubleshooting revealed that it is the SORT statement which is crashing the Runtime.


Version 10 introduced major changes in the SORT verb.  These changes eliminated the SORT_FILES Runtime configuration and added new ones.  Setting A_SORT_REGIONS to half the value of the previous setting of SORT_FILES will achieve the same effect with the new SORT.  We recommend testing with A_SORT_REGIONS=0 to see if this new mode of operation yields better performance in a given system.

Following is the full description of the changes to SORT.  Please study this information carefully and test to determine the best settings for your system and application:

The internals of the Runtime SORT verb have been updated to provide a new mode of operation for SORT that avoids a significant amount of file I/O and may provide better performance for large SORT operations.  A new method of managing the temporary files used by SORT means that fewer files are used in most cases.

The SORT-FILES configuration variable is no longer used by Runtime version 10 and above.  Similar configuration may be applied with the new configuration variables A_SORT_REGIONS and A_SORT_REGIONS_FINAL.  In order to understand how to use these variables, read the following overview of the SORT function.

In order to efficiently sort records, the records must be imported into memory.  The size of the memory area used for this phase is controlled by the SORT_MEMORY configuration variable.  If all the records fit into this memory area, the records are simply sorted and then provided to the output phase of the SORT verb.

Due to the logarithmic nature of the in-memory sorting operation, you will get the best performance by specifying as large a SORT_MEMORY setting as possible.  It is more efficient to sort a large batch of records at once than multiple smaller batches.

(Of course, this memory should be allocated within reason -- starving your system of memory may lead to slower performance for the whole system. Your operating system may also not return freed memory in the runtime process back to the operating system once the SORT operation is done. Because of this, you may want to isolate a large SORT operation into its own program, run by a temporary runtime process.  Once the process exits, the memory will be returned to the operating system.)

If there are more records than will fit into the SORT_MEMORY memory area, the records are written to a temporary file in sorted regions, where all the records in each region are in sorted order.  The next task is to merge together all of these sorted regions so that all the records are placed into the proper order.

One method for doing this is to perform a large merge operation across all the sorted regions, picking the smallest record from each region and sending it to the output phase.  This method is selected by setting A_SORT_REGIONS to 0.  This method does less file I/O because it only writes the records to the temporary file once.  However, the file I/O is more random in nature and more CPU time is used in order to manage this file access.  This method will tend to produce better performance when the SORT temporary file is stored on a relatively slower storage device such as a mechanical hard drive.

Another method is to make multiple passes through the sorted regions, merging together a subset of regions into a larger region.  (This method is similar to the pre-10.0 runtime SORT implementation.)  These new, larger regions are written to a second temporary file which may be subjected to further region merges until only one or more regions are left.  The number of regions merged in each pass is controlled by setting A_SORT_REGIONS to a value from 2 to 64.  The default value is 4.  This method does more file I/O than the previous method due to the multiple passes, but the file I/O is more sequential in nature and uses less CPU time.  This method tends to produce better performance when the SORT temporary file is stored on a relatively faster storage device such as a solid-state drive.

A_SORT_REGIONS_FINAL controls the number of sorted regions left in the temporary file before moving to the output phase.  If A_SORT_REGIONS_FINAL is set to 1, then all the records will be merged into a single sorted region before moving to the output phase.  When set to a non-zero value, there will be up to that many sorted regions left in the temporary file before the output phase.  If there is more than one region left, the remaining regions will be merged during the output phase, similar to the way the A_SORT_REGIONS=0 case is handled.  If you are having acceptable performance with a non-zero A_SORT_REGIONS setting, then you can safely leave A_SORT_REGIONS_FINAL at its default setting of 1.  This variable is mainly for experimentation in case performing one less sorted region merge pass provides better performance.  A_SORT_REGIONS_FINAL may be set to a minimum value of 1 and a maximum value of A_SORT_REGIONS. A_SORT_REGIONS_FINAL has no effect if A_SORT_REGIONS is set to 0.

A new configuration variable, A_SORT_FILE_MEMORY, may be used to specify an amount of memory to be used for buffering file operations on the temporary files.  This buffering is used when A_SORT_REGIONS=0 or A_SORT_REGIONS_FINAL is greater than 1 due to the random access nature of the final merge pass in those cases.  The amount of memory is specified in increments of 64KB, the same as for SORT_MEMORY.  The minimum value is 0, which disables the file buffer and is not recommended except for troubleshooting.  The maximum value is 65535.  The default value is 16. If SORT_MEMORY is larger than A_SORT_FILE_MEMORY and A_SORT_FILE_MEMORY is non-zero, the memory previously used for SORT_MEMORY will be used to improve file buffering.

The 6 byte internal minimum record size for SORT, documented in ECN-1514, has been removed for objects compiled with object semantics >= 10.0.


Comment List