Async Callbacks - Memory LEak for push conversation
Looks like we are dealing with a memory leak for Async Callbacks , a very bad one at that.
Start a test with LR 11.52 on Windows 2003 with a push conversation. Running as threads seems to kill mmdrv as we see memory profile growing and then once it reaches ~ 1.3 GB it fails. We have done every thing possible in terms of cutting down operations in the ResponsebodyBuffer method and params- no luck so far.
The App sends around 300-1000 bytes 3-4 times in a second.
Running as a process gives us a bit longer test ,but nowhere near as needed. We need to also run 5k-8k , I dont believe with all the hardware we got we can do as a process( a mix of 4 chasis/Blades and 8 VMS ) as we are limited with around 400-500 users per physical hostwhen u run as a process on the chasis servers.
12 seems to be a bit better but we cant move to 2008 on the physical servers yet 😞
Raising a ticket with Support , in the meanwhile is there any tips or tricks to adjust this behaviour ? under the hood options ? Any ideas ?
Hope you are fine!
What is the protocol(s) you are using for that test?
How many Load generators are you using to test the script(s)?
Please provide us the above information.
Thanks and Regards,
Regular HTTP protocol with Async callbacks implemented. Load generators as i mentioned earlier - a mix of 4 chasis/Blades and 8 VMS . But we were playing around in a couple of physical machines as they have more horse power.
We distinctly saw the mmdrv process memory used grow and once they reached beyond 1.2 GB all users will error out with Exception access memory violation erros.
I see that you are currently using two physical Server with more horsepower than the mix of 4 Blad Server Chassis, it is an example of a Cisco Blade Server Chassis component and display architecturesand how it looks like I believe : http://www.cisco.com/c/dam/en/us/products/collateral/servers-unified-computing/ucs-b-series-blade-servers/spec_sheet_c17-644224.pdf.
I believe you need to remember that all the resources are imporntant you we are planning a heavy Performance Test. It is also depends on what type of Peformance Test you are trying to execute. There are some types such as:
another processes being used, we need to discard or end the process that we do not need during the Load Test( for the 2 Host machines).
May you please let us know how many procotols are you using? Also What are the protocols being involved during the test, you can create a list of them?
Knowing the protocol we can find the possible causes that are producing this. Another recomendation is to redesign the script or scripts to ensure that it is run as separate iterations rather than repeatedly looping in a single iteration. This will ensure that memory allocated during the script iteration is released and thus prevent a memory leak occuring.