In the previous article, you have seen that by not limiting the number of worker threads in the Thread Pool, you can improve the invocation throughput. But can you think of any potential drawbacks and risks to the application with such configuration in practice, especially under high load?
In this article, you will learn how to use the Thread Pool Dispatcher “vbroker.se.<se_name>.scm.<scm_name>.dispatcher.threadMax” property to mitigate such risks.
The "echo_service_cpp" example is used in this demonstration.
Configure the Client by modifying c.sh:
Configure the Server by modifying s.sh:
Make sure you have set up the necessary VisiBroker environment and build the example before running this demonstration.
MEMORY(KB) THREADS SOCKETS
14304 205 1
Total Time taken for 200 threads in PID 26356 to complete all invocations is 10.1585 seconds
MEMORY(KB) THREADS SOCKETS
12944 54 1
Total Time taken for 200 threads in PID 28395 to complete all invocations is 40.0776 seconds
Compare the resource consumption and invocation performance measurement before and after tuning “vbroker.se.iiop_tp.scm.iiop_tp.dispatcher.threadMax=50” property at the Server side.
|Server Memory (KB)||Server Threads||Server Sockets||Total Time Taken (sec)|
Key observations after tuning:
When you do not limit the number of worker threads that can be created in the Thread Pool, a Server experiencing high concurrent request rate may try to create all the threads it deems necessary to service all these requests concurrently. Can you imagine the Server's resource consumption if 100000 requests arrives concurrently? If the resource limit of the process is exceeded, it may destabilize or even crash the whole application.
To mitigate this risk, you can limit the maximum number of worker threads that can be created by tuning the “threadMax” property. As you can see in the demonstration, limiting the “threadMax” may reduce the invocation throughput, but it can improve the stability of the application.
So what is the best value for "theadMax"? The optimal value for “threadMax” depends on the application performance requirements and the Operating System resource limitations. You must perform benchmark tests on your application under peak load conditions to find the optimal value. Sometimes, in order to achieve a very high invocation throughput, you may need to adjust the "ulimit" of the process, or even upgrade the CPU or memory specification of the machine if it is within your budget.
Although this demonstration is based on a VBC example, the same concept is also applicable to VBJ too.
Please note that "threadMax=0" does not have the same meaning for VBC and VBJ. For VBC , it means that the ORB will control the thread generation using an internal algorithm based on heuristics. But for VBJ, it means that the ORB does not limit the number of threads it can create. Furthermore, the "unlimitedConcurrency" property is not applicable to VBJ.
In this article, you have learnt that in order to achieve a stable application under high load, you may need to make a trade-off between resource consumption of the application and the invocation throughput, given a finite system resources.
In the next article, you will lean how to tune the Thread Pool Dispatcher "threadMaxIdle" property to conserve system thread resources.