How are Silk Performer"s Percentile measurements calculated?
The percentile calculation method in Silk Performer is based on an approximation algorithm, using a series of "markers" which form an interpolated curve over the real measure points. This is because we do not store all single values in order to calculate the exact percentile graph due to storage and performance considerations. Due to this you will not see the same results as if you extracted the measures and performed the percentile calculation in a 3rd party tool, e.g. excel, which does exact rather than approximation calculations. When comparing such results it is important to consider that those presented in Silk Performer are conservative numbers, i.e. when interpreting the results it is important to realize that 50% <= 4.5 means that at least 50% of the measurements are below 4.5 but it could be more.
The default configuration of the percentile algorithm delivers good results through the whole value range, however under certain circumstances it makes sense to adjust the parameters of the algorithm. For instance if you want the algorithm to be more exact within a particular range, you can adapt the fPQuantile parameter of the MeasureCalculatePercentiles function to that range.
MeasureCalculatePercentiles(in sName : string allownull,in nClass : number, in bDisable : boolean optional, in fPQuantile : float optional, in nMarkerCount : number optional): boolean;fPQuantile: Specifies a value between 0 and 1 indicating the position round which the markers are grouped (optional, default is 0.5).nMarkerCount: Specifies the number of markers, which the percentile graph is based on.nMarkerCount must be an odd value <= 67 (optional, default is 21).
By default the fPQuantile parameter is set to 0.5, so will be more accurate around the 50% percentile, to make it more accurate at the 90% percentile set this parameter to 0.9. However it is recommended to leave this as default, it is only when you have a single outlier value that this is more of an approximation.
Additionally to increase the accuracy of the percentiles measurements you can increase the number of "marker" counts taken. By default nMarkerCount is set to 21 which means the percentile algorithm uses 21 markers which form an interpolated curve over the real measure points. However this can be increased to a maximum of 67, provided sufficient measurements are being taken; if there are fewer measure points than markers the algorithm sets some markers to fictive values along the interpolated curve.
It is possible that the values obtained from the MeasureGetPercentile function are slightly different from the Overview report values, this is because the Overview is based on the numbers of each single virtual user, fed into a merge algorithm, which merges markers lying very close to each other into one single marker.
NB The raw Overview marker data can be seen in the baselineReport.brp file in the Load Test folder, open with a text editor and view the following section:
BASELINE REPORT | USERTYPE LIST | USERTYPE | MEASURELIST | MEASURE| PERCENTILEDATA
Since the release of Silk Performer 9.5, a new precise means to determine the percentile data has been introduced, using the MeasureCalculateRawPercentiles() function, which includes the option to specify a desired granularity. The default granularity of 1 percent now delivers extremely precise percentile charts for your overview reports, even in very large load tests with many agent computers. Lower granularity enables a more precise percentile calculation, but consumes more memory on each agent computer.
The BDL function MeasureCalculatePercentiles() has been deprecated. If you use this function in a script, it will internally call the new MeasureCalculateRawPercentiles() function with default parameters instead.