Highlighted
Valued Contributor.
Valued Contributor.
464 views

LoadRunner shows lower Avg response time for a higher load compared to lower load

I am testing a web application with http/html script. Compared to 100 user test my 300 user test shows a lower Avg response time even though hits/sec and throughput is high compared to 100 user test.

In both the tests we used the same script with same settings, except for # of users. Has anyone got some pointers on this?  Appreciate all your help

0 Likes
4 Replies
Highlighted
Outstanding Contributor.. Outstanding Contributor..
Outstanding Contributor..

Re: LoadRunner shows lower Avg response time for a higher load compared to lower load

Hi RR45, 

in my personal opinion, I don't think that "Hits Per Second" and "Throughput" graphs helps you about the test's performance results. "Hits Per Seconds" measures the number of "hits" or calls made to a web server, so the number of web server is being accessed; "Throughput" represents the net behavior through the communication between client and server. 

If more data is being sent through the server, then the server is able to process requests faster, and consequently the application's response time would be relatively lower. If throughput is low, then there is some bottleneck which is causing the response time to increase. So, response time and throughput are inversely proportional. In another way, we can say that response time increases might happen even when the server is less utilized.

Tell me if all is clear now, 

Lorenzo

0 Likes
Highlighted
Acclaimed Contributor.. Acclaimed Contributor..
Acclaimed Contributor..

Re: LoadRunner shows lower Avg response time for a higher load compared to lower load

Whenever you run tests and compare results realize that there are various type of metrics (in relation to number of users):

  1. Constant metrics, are metrics that do (should not) change when you play with the number of users in your test, like response times.
  2. User related metrics are metrics that are related to the amount of user that participate in a test. Examples are Throughput, Hits per second, CPU usage, Disk / Network bytes/second, etc.

So when you load test try to use a series of users, like 10, 50, 100, 200, 300, ... Draw graphs of the various metrics. When system resources are enough available 'user related metrics' will grow linear. Response times might be stable, but most of time they follow the normilzed formula (1/(1-u)) with u is system utilization [0,..1>.

 

In your case you triple number of users, so number of metrics type 2 should also triple.This depends also on how you setup your load test: free running or a constant iteration time per script. The latter should give a constant load with increasing response times unless those times get too large. The metrics are easier to compare with this way of working.

You should expect that response times stay more or less constant. When they go down (you did not specify how much) you see impact of external changes in your environment. You might think of:

  1. Network load was different between tests.
  2. Servers in test are used by others as well (always a bad thing).
  3. Warming up of database has impact. Note that you should exclude the warming up of your load test from your results.
  4. When you test against virtualized servers and do not collect physical system metrics all your tests might be useless.
  5. Power settings on Servers, ESX and BIOS are not High-performance, so you might suffer from micro sleeps when your CPU load is low.

Hope this helps.

Signature:
Reward community members who take time to respond and help.
0 Likes
Highlighted
Super Contributor.
Super Contributor.

Re: LoadRunner shows lower Avg response time for a higher load compared to lower load

Apart from the above suggestions, try once to compare the 90% response time. May be with 100 user test, you got few samples having very high values that increased your overall avg time.

Highlighted
Micro Focus Expert
Micro Focus Expert

Re: LoadRunner shows lower Avg response time for a higher load compared to lower load

Exactly, a single outlier would have a higher impact on smaller test samples, because their bad time would be over-represented.

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.