Thread Pool Dispatcher threadMax

Thread Pool Dispatcher threadMax

[[Thread Pool Dispatcher unlimitedConcurrency|Back]]

In the previous article, you have seen that by not limiting the number of worker threads in the Thread Pool, you can improve the invocation throughput. But can you think of any potential drawbacks and risks to the application with such configuration in practice, especially under high load?

In this article, you will learn how to use the Thread Pool Dispatcher “vbroker.se.<se_name>.scm.<scm_name>.dispatcher.threadMax” property to mitigate such risks.

The "echo_service_cpp" [[Explanation of Example|example]] is used in this demonstration.

Scenario

  • The Client creates 200 threads to call 1 Server concurrently.
  • Each client thread calls the Server once.
  • Each invocation blocks for 10 seconds at the Server side.
  • Monitor the Server's resource consumption and invocation performance with “unlimitedConcurrency=true” set.
  • Observe the effect of setting "threadMax=50".

Preparation

Configure the Client by modifying c.sh:

  • Set the following properties:
    • server_sleep_time 10
    • vbroker.agent.enableLocator=false
  • Disable all the other properties.

Configure the Server by modifying s.sh:

  • Set the following properties:
    • vbroker.se.iiop_tp.scm.iiop_tp.dispatcher.unlimitedConcurrency=true
    • vbroker.agent.enableLocator=false
    • vbroker.se.default.local.manager.enabled=false
  • Disable all the other properties.

Execution

Make sure you have set up the necessary VisiBroker environment and build the example before running this demonstration.

  • Start the Server Monitor script:
    • mon_s.sh
  • Start the Server:
    • s.sh 1
  • Start the Client:
    • c.sh 1 1 200
  • Monitor the Server's resource consumption (printed by Server Monitor). E.g.:

MEMORY(KB) THREADS SOCKETS
14304      205     1

  • Note the total time to complete all invocations (printed by Client). E.g.:

Total Time taken for 200 threads in PID 26356 to complete all invocations is 10.1585 seconds

  • Stop the Client and Server.
  • Disable “vbroker.se.iiop_tp.scm.iiop_tp.dispatcher.unlimitedConcurrency=true” and set the following property at the Server side by modifying s.sh:

vbroker.se.iiop_tp.scm.iiop_tp.dispatcher.threadMax=50

  • Re-start the Server:
    • s.sh 1
  • Re-start the Client:
    • c.sh 1 1 200
  • Monitor the Server's resource consumption (printed by Server Monitor). E.g.:

MEMORY(KB) THREADS SOCKETS
12944      54      1

  • Note the total time to complete all invocations (printed by Client). E.g.:

Total Time taken for 200 threads in PID 28395 to complete all invocations is 40.0776 seconds

Observations

Compare the resource consumption and invocation performance measurement before and after  tuning “vbroker.se.iiop_tp.scm.iiop_tp.dispatcher.threadMax=50” property at the Server side.

Server Memory (KB)  Server Threads Server Sockets Total Time Taken (sec)
Before Tuning 14304 205 1 10.1585
After Tuning 12944 54 1 40.0776

Key observations after tuning:

  • The memory usage by the Server is lesser.
  • The number of threads created by the Server is lesser.
  • Total time taken for all the threads to complete the invocations is longer.

Explanation

When you do not limit the number of worker threads that can be created in the Thread Pool, a Server experiencing high concurrent request rate may try to create all the threads it deems necessary to service all these requests concurrently. Can you imagine the Server's resource consumption if 100000 requests arrives concurrently? If the resource limit of the process is exceeded, it may destabilize or even crash the whole application. 

To mitigate this risk, you can limit the maximum number of worker threads that can be created by tuning the “threadMax” property. As you can see in the demonstration, limiting the “threadMax” may reduce the invocation throughput, but it can improve the stability of the application.

So what is the best value for "theadMax"? The optimal value for “threadMax” depends on the application performance requirements and the Operating System resource limitations. You must perform benchmark tests on your application under peak load conditions to find the optimal value. Sometimes, in order to achieve a very high invocation throughput, you may need to adjust the "ulimit" of the process, or even upgrade the CPU or memory specification of the machine if it is within your budget.

Although this demonstration is based on a VBC++ example, the same concept is also applicable to VBJ too.

Please note that "threadMax=0" does not have the same meaning for VBC++ and VBJ. For VBC++, it means that the ORB will control the thread generation using an internal algorithm based on heuristics. But for VBJ, it means that the ORB does not limit the number of threads it can create. Furthermore, the "unlimitedConcurrency" property is not applicable to VBJ.

In this article, you have learnt that in order to achieve a stable application under high load, you may need to make a trade-off between resource consumption of the application and the invocation throughput, given a finite system resources.

In the next article, you will lean how to tune the Thread Pool Dispatcher "threadMaxIdle" property to conserve system thread resources.

[[Thread Pool Dispatcher unlimitedConcurrency|Back]]  |  [[Thread Pool Dispatcher threadMaxIdle|Next]]

Labels (1)

DISCLAIMER:

Some content on Community Tips & Information pages is not officially supported by Micro Focus. Please refer to our Terms of Use for more detail.
Top Contributors
Version history
Revision #:
2 of 2
Last update:
‎2020-03-13 21:06
Updated by:
 
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.