Highlighted
TerryK Absent Member.
Absent Member.
3388 views

Memory requirements?

Jump to solution

Hello.

I've been trying out an application served through IIS and was trying to resolve all those 'UI Element not found' cases. That was on a Browser Driven script since I couldn't get anything recorded in HTTP.

I've been through the forum and the KB, found various solutions that worked on and off, but only increasing think times gave more stable runs. So I left it at that until later.

Then I had a look at the memory used by the Silk Performer browsers and sort of freaked out. I did another scan of the forum/KB and verified that the each browser has a footprint around 100MB.

The question:

Is this specific to browser driven load tests or is it the same for any kind of recording? Did I miss some setting that uses the same browser instance for multiple virtual users?

I used to run (with other tools) almost 800 virtual users on a laptop (around 4-5MB per each, but for HTTP recording). To do the same with SP, I'd need about 80 similar laptops (assuming HTTP recording behaves the same). It sounds a bit off and hopefully i have it all wrong.

Anybody could please throw some light on it.

Thank you.

0 Likes
3 Solutions

Accepted Solutions
Micro Focus Contributor
Micro Focus Contributor

RE: Memory requirements?

Jump to solution

Hi Terry

Running scripts at protocol level (http recording) you would be able to get many hundreds or sometimes thousands of users per machine, as with other tools.  However browser driven load testing is more involved that http and involves more processes which use more memory.  

In a nutshell, there will be a perfrun process which runs with small amount of memory - although this can vary depending on your use case.  There will be a perfbrowserhost/app process and this is where the action happens, within this process is an IE control and within that control is the rendered state of your application under test.  So you need to consider the size of your application as it sits in IE normally, add to that the memory used within our perfbrowser process and you can see it will start to mount up.

Comparing BDLT approach to http approach is not a valid comparison, BDLT does so much more on the client side which http level does not (rendering, client side javascript execution, ajax requests, re-rendering etc) so it is correct that it uses more memory.  The size of the IE portion and the size of your application is also out of our control.

The extra memory used by BDLT versus the additional client side experience that is simulates is the tradeoff to consider when determining your testing approach.

On a side note, 100MB would be unusually large for the footprint. If you want to open a support incident we can verify if this is legitimate or if there is a problem.

0 Likes
shippee Frequent Contributor.
Frequent Contributor.

RE: Memory requirements?

Jump to solution

Because there are so many differences between machines with respect to CPU, memory, etc. I've had some success with running the "Evaluate Agent VUser Capacity" built into Silk Performer.  Open a project you want to work with, and within the PROJECT pane expand your Agents -> right click on the agent you want to check and select EVALUATE AGENT CAPACITY - this should give you a rough idea.  Given Silk Performer runs as x32 - notwithstanding we run it here on x64 windows systems, we get anywhere from 25 to 35 virtual users per agent.  We ended up setting up 20 virtual machines (made one and cloned it 19 times) and this seems to serve us well - however we only have a 500 VU license, so when you get much above that I imagine it might get a bit dicey.  I used to push the limits and go up to 45 - 50 VU's per agent, but got many, many time outs and other weirdness that was hard to explain, to this is probably one area one needs to be conservative with.

0 Likes
PhilipL Absent Member.
Absent Member.

RE: Memory requirements?

Jump to solution

Hi Terry,

If you are really so much resource constrained (in hardware and money) you have 2 options.

1.) Silk Performer Cloud Burst

With Cloud Burst you can leverage the cloud using Silk Performer in an easy way and can reuse script you ran locally.

Check this out: https://cloud.borland.com/

2.) Use protocol level project (Web Async)

Even if the web app is a Web 2.0 application in many cases it is possible to still do load testing with a "protocol level" project.

If you have things like SignalR / socket.io / WebSockets etc. in use it is recommended to create a "Web Async" project.

It is bit more work to get it running than using Browser Driven Load Testing but what you gain is the low resource consumption.

I hope I could help,

Philip

0 Likes
6 Replies
PhilipL Absent Member.
Absent Member.

RE: Memory requirements?

Jump to solution

I would highly recommend to read this arcticle:

When to use protocol / BDLT for web 2.0
community.microfocus.com/.../521.when-to-use-protocol-bdlt-for-web-2-0.aspx

In short:

When using Browser Driven Load Testing for each virtual user a single Internet Explorer instance is started.
With this comes some resource consumption.

Philip

Tags (1)
0 Likes
Micro Focus Contributor
Micro Focus Contributor

RE: Memory requirements?

Jump to solution

Hi Terry

Running scripts at protocol level (http recording) you would be able to get many hundreds or sometimes thousands of users per machine, as with other tools.  However browser driven load testing is more involved that http and involves more processes which use more memory.  

In a nutshell, there will be a perfrun process which runs with small amount of memory - although this can vary depending on your use case.  There will be a perfbrowserhost/app process and this is where the action happens, within this process is an IE control and within that control is the rendered state of your application under test.  So you need to consider the size of your application as it sits in IE normally, add to that the memory used within our perfbrowser process and you can see it will start to mount up.

Comparing BDLT approach to http approach is not a valid comparison, BDLT does so much more on the client side which http level does not (rendering, client side javascript execution, ajax requests, re-rendering etc) so it is correct that it uses more memory.  The size of the IE portion and the size of your application is also out of our control.

The extra memory used by BDLT versus the additional client side experience that is simulates is the tradeoff to consider when determining your testing approach.

On a side note, 100MB would be unusually large for the footprint. If you want to open a support incident we can verify if this is legitimate or if there is a problem.

0 Likes
shippee Frequent Contributor.
Frequent Contributor.

RE: Memory requirements?

Jump to solution

Because there are so many differences between machines with respect to CPU, memory, etc. I've had some success with running the "Evaluate Agent VUser Capacity" built into Silk Performer.  Open a project you want to work with, and within the PROJECT pane expand your Agents -> right click on the agent you want to check and select EVALUATE AGENT CAPACITY - this should give you a rough idea.  Given Silk Performer runs as x32 - notwithstanding we run it here on x64 windows systems, we get anywhere from 25 to 35 virtual users per agent.  We ended up setting up 20 virtual machines (made one and cloned it 19 times) and this seems to serve us well - however we only have a 500 VU license, so when you get much above that I imagine it might get a bit dicey.  I used to push the limits and go up to 45 - 50 VU's per agent, but got many, many time outs and other weirdness that was hard to explain, to this is probably one area one needs to be conservative with.

0 Likes
TerryK Absent Member.
Absent Member.

RE: Memory requirements?

Jump to solution

Thank you all.

I can understand the reasoning/need behind the large memory footprint in the browser driven test case.

It's just that I can't see how some medium-sized company that happens to develop ASP applications that use something like signalR, could afford to make available the number of agent machines required (the ones I'm evaluating for, unfortunately cannot).

I'll go through the KB in case there's a possibility for some other type of scripting that can handle the case I'm trying out.

Much appreciated for the pointers.

0 Likes
PhilipL Absent Member.
Absent Member.

RE: Memory requirements?

Jump to solution

Hi Terry,

If you are really so much resource constrained (in hardware and money) you have 2 options.

1.) Silk Performer Cloud Burst

With Cloud Burst you can leverage the cloud using Silk Performer in an easy way and can reuse script you ran locally.

Check this out: https://cloud.borland.com/

2.) Use protocol level project (Web Async)

Even if the web app is a Web 2.0 application in many cases it is possible to still do load testing with a "protocol level" project.

If you have things like SignalR / socket.io / WebSockets etc. in use it is recommended to create a "Web Async" project.

It is bit more work to get it running than using Browser Driven Load Testing but what you gain is the low resource consumption.

I hope I could help,

Philip

0 Likes
TerryK Absent Member.
Absent Member.

RE: Memory requirements?

Jump to solution

Thank you Philip.

Cloud is in the plans but not in the immediate future.

I'll try Web Async and see how it handles this case.

I thank you all once more.

All the answers have been more than useful but it seems I can only select one of them as the one that answers the question.

I'd say Jonny answered it, whereas shippee  and Philip gave me a good shove to the proper direction for solving this problem.

I'll mark it as answered later, just in case somebody knows of a way to mark multiple answers.

Cheers and have a nice weekend.

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.