How are Silk Performer"s Percentile measurements calculated?

How are Silk Performer"s Percentile measurements calculated?

The percentile calculation method in Silk Performer is based on an approximation algorithm, using a series of "markers" which form an interpolated curve over the real measure points. This is because we do not store all single values in order to calculate the exact percentile graph due to storage and performance considerations.  Due to this you will not see the same results as if you extracted the measures and performed the percentile calculation in a 3rd party tool, e.g. excel, which does exact rather than approximation calculations.  When comparing such results it is important to consider that those presented in Silk Performer are conservative numbers, i.e. when interpreting the results it is important to realize that 50% <= 4.5 means that  at least 50% of the measurements are below 4.5 but it could be more.


The default configuration of the percentile algorithm delivers good results through the whole value range, however under certain circumstances it makes sense to adjust the parameters of the algorithm. For instance if you want the algorithm to be more exact within a particular range, you can adapt the fPQuantile parameter of the MeasureCalculatePercentiles function to that range.

MeasureCalculatePercentiles(in sName : string allownull,in nClass : number,                                           in bDisable : boolean optional,                                          in fPQuantile : float optional,                                          in nMarkerCount : number optional): boolean;fPQuantile: Specifies a value between 0 and 1 indicating the position round which the markers are grouped (optional, default is 0.5).nMarkerCount: Specifies the number of markers, which the percentile graph is based on.nMarkerCount must be an odd value <= 67 (optional, default is 21).

By default the fPQuantile parameter is set to 0.5, so will be more accurate around the 50% percentile, to make it more accurate at the 90% percentile set this parameter to 0.9. However it is recommended to leave this as default, it is only when you have a single outlier value that this is more of an approximation.


Additionally to increase the accuracy of the percentiles measurements you can increase the number of "marker" counts taken. By default nMarkerCount is set to 21 which means the percentile algorithm uses 21 markers which form an interpolated curve over the real measure points. However this can be increased to a maximum of 67, provided sufficient measurements are being taken; if there are fewer measure points than markers the algorithm sets some markers to fictive values along the interpolated curve.


It is possible that the values obtained from the MeasureGetPercentile function are slightly different from the Overview report values, this is because the Overview is based on the numbers of each single virtual user, fed into a merge algorithm, which merges markers lying very close to each other into one single marker.
NB The raw Overview marker data can be seen in the baselineReport.brp file in the Load Test folder, open with a text editor and view the following section:

BASELINE REPORT | USERTYPE LIST | USERTYPE | MEASURELIST | MEASURE| PERCENTILEDATA

Since the release of Silk Performer 9.5, a new precise means to determine the percentile data has been introduced, using the MeasureCalculateRawPercentiles() function, which includes the option to specify a desired granularity. The default granularity of 1 percent now delivers extremely precise percentile charts for your overview reports, even in very large load tests with many agent computers. Lower granularity enables a more precise percentile calculation, but consumes more memory on each agent computer.

The BDL function MeasureCalculatePercentiles() has been deprecated. If you use this function in a script, it will internally call the new MeasureCalculateRawPercentiles() function with default parameters instead.

DISCLAIMER:

Some content on Community Tips & Information pages is not officially supported by Micro Focus. Please refer to our Terms of Use for more detail.
Top Contributors
Version history
Revision #:
1 of 1
Last update:
‎2013-02-15 18:49
Updated by:
 
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.