MemCalc parses a NetWare 6.5 SEGSTATS.TXT file to generate a set of recomended tuning parameters to help the servers stability and performance.

In recent NetWare 6.x Service Packs, the memory management with NetWare has undergone a rather radical overhaul to help address limitations NetWare was starting to experience under intensive loads - e.g., Running large databases, multi gigabyte eDirectory trees with millions of objects, multiple Java applications etc.

These changes to memory management were initially quite problematic, but have become much more reliable in the current service packs - except, in my opinion, for the "auto tuning" feature that is enabled by default.

The AutoTuning feature operates by lowering the "File Cache Maximum Size" (FCMS) setting, which controls how much memory is available to the server for use as NSS and/or TFS cache. If the FCMS setting reaches its minimum possible value of 1GB then the auto tuning will then start recommending that the "-u" setting is reduced. The "-u" setting controls how much space is available for the "User Address Space". This is a logical memory region that is reserved for running protected mode and Java applications.

In my opinion, the memory tuning algorithm is too aggressive, and too simplistic:

• It wants to keep too much memory in the VM pool.

• It's too keen to drive the FCMS setting down.
  

• It will only "tune" in one direction, and it "tunes" the server to accommodate one off memory allocations - an NLM accidentally requesting a 1GB allocation today will mean a server "tuned" for that size memory footprint a year from now. If I remove a large memory footprint NLM from the server, memory will not be "tuned" back towards a larger cache- the memory will be forever reserved for the VM pool.

• The "tuning" can result in even more memory fragmentation than the tuning is designed to prevent. When the FCMS setting is reduced, the complete NSS cache is thrown away (flushed), then its starts growing again. I've seen servers with 2-3% of fragmented memory suddenly have 15% or more after being "tuned".

• The tuning can cause server abends. I've seen it cause Poison Pill abends on Cluster nodes, and other abends on standard servers.

This aggressive auto tuning in addition to some "interesting" default settings bias the server towards giving excessive memory to the VM pool at the expense of the FS Cache pool.

By "interesting" default values, I mean for example:

Set VM cache pool percentage=80

(Allow the VM Pool to have 80% of available memory)

VM Cache Pool Maximum Pages=4294967295

(Allow the VM Cache Pool to grow to 16TB)

FS Cache Pool Minimum Pages=1000

(The FS Cache Pool needs 4MB of memory)

FS Cache Pool Desired Pages=1250

(The FS Cache Pool wants 5MB of memory)

So by default, the VM cache pool wants 80% of all available memory, up to a maximum of 16TB, whereas the FS Cache pool needs at least 4MB and is happy with 5MB. Now, while a NetWare server with 5MB of cache can run surprisingly well (I saw one that had accidentally been configured with 4MB of NSS cache for over 6 months - there had been apparently no complaints about performance) I'd much rather have the bias in a file server to be towards memory being available for caching, not being reserved for when and if more memory is required for NLMs. Most fileservers tend to have a fairly static memory footprint, not an ever increasing one, and even with tuning biased towards the FS Cache, memory can be borrowed from that pool to accommodate NLM requests.

It has not been uncommon in the Novell Support Forums (news://support-forums.novell.com or http://support-forums.novell.com) to see people reporting servers with 4GB memory, where auto tuning has reduced the FCMS setting to its minimum 1GB value, and auto tuning still wants to lower the "-u" value to free up more space. These servers didn't need the massive amounts of memory in the VM pool - they were file servers that want memory for file caching.

Based on my own belief that there had to be a "better way" and the number of issues I'd seen reported in the Support Forums, I spent a lot of time researching how different memory settings affect the memory management and stability of the server. Based on that research I've made memory tuning recommendations to a large number of forum posters who were having memory tuning issues, and most of them have found their servers to be significantly more stable since applying the changes I recommended.

What follows are the formulas I developed for recommending memory tuning changes to a server. The formulas take a number of the values available from SEG.NLM. To get the required values, load SEG.NLM, then from the main screen do '/', then 'Info', then 'Write SEGSTATS.TXT'. The SEGSTATS.TXT file will be created in SYS:SYSTEM.

SEG monitors the server and records a number of key memory statistics, my formulae take those statistics and recommend manual memory tuning parameters.

Note that as these are manual settings, auto tuning is disabled, and if the memory usage of the server changes significantly, then the server will need to be retuned to reflect the change in memory usage.

Also, after making the changes to use manual rather than auto tuning, the server may still recommend that the FCMS and "-u" memory settings be changed. These recommendations can be ignored. Following them will have the same effect as auto tuning, except you're doing it rather than the server doing it automatically - the same problems will still occur.

Rather than calculate the values individually, I have a memory calculator (DOS and Linux versions) available that will automatically calculate the values for you from a SEGSTATS.TXT file.

The calculators can be downloaded from: http://www.caledonia.net/hamish.html

There is also an NLM version in development that will automatically make all the required changes to the server. Check back at the above page for updates.

The Formulas:

The inputs in the formulas are (note: all values in BYTES):

P       =   Physical memory below 4GB

NC   =   NLM Current footprint

NH   =   NLM Highwater mark (If NH > 2 * NC - find out why)

DS    =   Size of the eDirectory cache (DS NLM)

U      =   UAS setting

All of these values can be obtained from the SEGSTATS.TXT file.

The P value is available under the "Memory Summary Screen (.ms)" as the Known Memory - Server: value. E.g.,. in the following extract: it's the 3958861872 value.

Memory Summary Screen (.ms)

========================================================================

Note that for P, it's specifically the memory under 4GB. A server could have 4GB installed, but due to PCI address space requirements it could be split as 3GB below 4GB and 1GB as extended memory above 4GB. We're only interested in the memory below 4GB. The Known Memory - Server: value will be correct, so don't be surprised if it is less than the memory installed in the server.

The NC and NH Values are from the "Virtual Memory Cache Pool (L!=P) as the NLM/VM Memory and the High Water Mark values respectively. E.g., in the following extract: NC is 828,645,376 and NH is 850,239,488.

| Virtual Memory Cache Pool   (L!=P)                         |
|                                                            |
| VM Pool Size      :    2,155,872,256 bytes  (2.01 GB)      |
. . .
| VM Pages In Use   :       21,929,984 bytes  (20.9 MB)      |
| NLM Memory In Use :      829,440,000 bytes  (791.0 MB)     |
| NLM/VM Memory     :      828,645,376 bytes  (790.3 MB)     |
| Largest Segment   :        2,924,544 bytes  (2.8 MB)       |
| High Water Mark   :      850,239,488 bytes  (810.9 MB)     |

The DS value may be in the SEGSTATS.TXT file if DS.NLM is one of the "top six" NLM's on the server. If DS is listed it will be in the "Top 6 Memory Consuming NLMs". E.g., in the following extract: DS is 55,826,190.

If it is not listed, go to Monitor on the server, Loaded NLMs, and get the memory size from there.

                         Top 6 Memory Consuming NLMs
     NLM Name       Version       Date           Total NLM Memory
========================================================================
1.  CPFILTER.NLM   6.00     Nov 14, 2003   537,148,312  bytes (512.3 MB)
2.  NSS.NLM      3.23.04    Sep 26, 2005    76,325,680 bytes  (72.8 MB)
3.  SERVER.NLM   5.70.04    Aug 15, 2005    61,538,586 bytes  (58.7 MB)
4.  DS.NLM       10552.79   Aug 9 2005      55,826,190 bytes  (53.2 MB)

When getting the UAS, take the "User Space High Water Mark" under "Server High/Low Water Mark Values" right at the end of the SEGSTATS.TXT. The other values for UAS in the file have other meanings/values. E.g., in the following extract: UAS is 297,218,048

Server High/Low Water Mark Values
======================================================================

NLM Memory High Water Mark        = 850,239,488 bytes
File System High Water Mark       = 447,685 bytes

User Space Information:
  User Space High Water Mark      = 297,218,048 bytes

The U value is added to the server.exe line in autoexec.bat. eg if the User Address Space High Water Mark from SEGSTATS.TXT is "297,218,048 bytes", the server.exe commandline in autoexec.bat would be "server -u297218048"

Note: Setting -u too low can cause problems if at some later stage Java/Tomcat apps (e.g., iManager) will be loaded. Generally if iManager or other Java/Tomcat apps will be run, I recommend setting the -u value to at least 512MB (-u536870912). The same recommendation applies for cluster nodes where other protected mode applications may be failed onto that node - look for the worst case load on the server and tune around that.

The equations output the following values:

FCMS - The File Cache Maximum Size Setting (take the value from the equation and round to nearest multiple of 1MB (i.e., 1,048,576 not 1,000,000 )).

FSMP - The FS Cache Pool Minimum Pages

EC - eDir cache size.

All values other than EC go into the STARTUP.NCF - not AUTOEXEC.NCF. Make sure none of the values are also being set in AUTOEXEC.NCF, as we do not to revert from our tuned parameters to whatever values may have previously been set.. This is because?.. . When setting the values, put them in STARTUP.NCF and also set them from the server console, then reboot the server. Depending on the previous values in the registry, the new values are not always completely effective, so get them set to the "good" values before rebooting.

For servers with less than 2GB:

Generally leave alone, autotuning works, or more accurately, autotuning won't normally come into effect - do the DS preallocation though.

EC = (DS DIV 16777216 1) * 16777216

For servers with 2GB - 3.25GB:

FCMS = (P - (P DIV 10) - NC)

FSMP = (P - (P DIV 10) - NH) DIV 4096

EC = (DS DIV 16777216 1) * 16777216

For servers with more than 3.25GB:

FCMS = (P - (P DIV 10) - NC - U)

FSMP = (P - (P DIV 10) - NH - U) DIV 4096

EC = (DS DIV 16777216 1) * 16777216

The following constraints are then applied to the values:

If FSMP > 655360 then FSMP = 655360

If EC > 805306368 then EC = 805306368

The manual tuning settings we'll make are:

set auto tune server memory=off

set file cache maximum size=FCMS

set fs cache pool minimum pages=FSMP

set fs cache pool desired pages=524288

(Unless FSMP > 524288 in which case set to the FSMP value)

set fs cache pool lots of pages=524288

(Unless FSMP > 524288 in which case set to the FSMP value)

set vm cache pool percentage=5

set vm cache pool maximum pages percentage=5

For EC, we want to create a hard limited, preallocated eDir cache of that size - this prevents memory fragmentation due to the constant growing/shrinking of the eDir cache. Create a SYS:_NETWARE\_NDSDB.INI file with the following contents:

cache=EC

cacheadjustinterval=15

cachecleanupinterval=15

blockcachepercent=50

preallocatecache=true

EXAMPLE:

So for example, from a SEGSTATS.TXT I had lying around:

P = 3757543552

NC = 565,481,472

NH = 947,179,520

U = 559,448,064

DS = 22,259,470

we get:

FCMS = 2,256,535,552

FSMP = 457,803

EC = 33,554,432

So in STARTUP.NCF we add:

set auto tune server memory=off

set file cache maximum size=2256535552

set fs cache pool minimum pages=457803

set fs cache pool desired pages=524288

set fs cache pool lots of pages=524288

set vm cache pool percentage=5

set vm cache pool maximum pages percentage=5

And in _NDSDB.INI we add:

cache=33554432

cacheadjustinterval=15

cachecleanupinterval=15

blockcachepercent=50

preallocatecache=true

Conclusion:

I hope this document has helped in the understanding of the memory tuning changes I recommend, and why I make those recommendations. If you are having memory issues that you want assistance with, please post in the NetWare 6.x Support Forums, including the SEGSTAS.TXT and CONFIG.TXT files for the server that is causing problems.

download url:
http://www.caledonia.net/hamish.html