The previous articles focused on the tuning of thread management properties to optimize performance and improve stability. However, another important factor that you must always consider during performance tuning is the connection management configuration. In the following articles, you will learn about VisiBroker's connection management properties.
First, let us see how the "vbroker.se.<se_name>.scm.<scm_name>.manager.connectionMax" property can be tuned to improve the stability of the application.
The "echo_service_cpp" example is used in this demonstration.
How many sockets are established concurrently at the Server side to service the concurrent invocations from the 10 Clients?
Configure the Client by modifying c.sh:
Configure the Server by modifying s.sh:
Make sure you have set up the necessary VisiBroker environment and build the example before running this demonstration.
MEMORY(KB) THREADS SOCKETS
12544 34 10
. . . . . .
Total Time taken for 2 threads in PID 7193 to complete all invocations is 10.1308 seconds
Total Time taken for 2 threads in PID 7195 to complete all invocations is 10.0235 seconds
Total Time taken for 2 threads in PID 7196 to complete all invocations is 10.0265 seconds
MEMORY(KB) THREADS SOCKETS
12296 15 5
. . . . . .
Total Time taken for 2 threads in PID 7455 to complete all invocations is 19.0479 seconds
Total Time taken for 2 threads in PID 7456 to complete all invocations is 19.0853 seconds
Total Time taken for 2 threads in PID 7458 to complete all invocations is 19.1963 seconds
Compare the resource consumption and invocation performance measurement before and after tuning “vbroker.se.iiop_tp.scm.iiop_tp.manager.connectionMax=5” property at the Server side.
|Server Memory (KB)||Server Threads||Server Sockets||Avg Total Time Taken (sec)|
Key observations after tuning:
Contrast the Server socket resource usage with an earlier scenario. What is the main factor in the scenario contributing to the difference?
By default, the Server does not limit the number of incoming connections from the Clients (i.e. vbroker.se.<se_name>.scm.<scm_name>.manager.connectionMax=0). That's why all the 10 Clients can establish connections to the Server successfully, and get their requests serviced in parallel. But after "connectionMax" is set to 5, only 5 Clients managed to connect to the Server in parallel. The remaining 5 Clients have to wait for the first 5 Clients to complete their invocations before they can connect to the Server successfully, thus resulting in a delay in servicing their requests. This also means that the inactive connections from the first 5 Clients must be closed to maintain the maximum connection limit of 5. What side effect does this has on the first 5 Clients making subsequent invocations to the Server? They must re-establish connections to the Server. The TCP connection establishment handshake protocol may cause some noticeable delay, depending on the network bandwidth and traffic. This re-connection overhead can increase the latency of the Clients' invocations.
In this simple demonstration, you may not fully appreciate the benefits of limiting the number of connections at the Server side since you only see it's downside instead of it's advantage. However, this feature is very important in the situation where a large number of Clients attempt to connect to the Server simultaneously. Without any connection limit, the Server may accept too many Client connections, and this may exceed the resource limit of the process. This may destabilize the whole application or even crash the Server process.
A process's socket resources (or file descriptors) are shared by other non-VisiBroker modules, such as application specific database query, non-CORBA inter-process communication and flat file access. If the number of file descriptors used by the process has reached it's limit, then VisiBroker will not be able to accept any more new connections. Furthermore, some application specific modules that need file descriptor resources may encounter errors too. To workaround this issue, you can try increasing the process's file descriptor soft limit using the "ulimit" system utility. If this is not sufficient, most operating systems allow you to tune the file descriptor hard limit too.
To find the optimal "connectionMax" value for your application, you must estimate the potential Client load and know your system resource limitations. Setting it too small will cause excessive re-connections and reduce invocation performance, but less resources are needed. Setting it too large may improve invocation performance, but more resources are required. You should perform benchmark and stability tests to know the overall application behavior under peak load condition. It is inevitable that you have to make some trade-off between resource usage (i.e. sockets and memory) and performance (i.e. throughput and stability) given a finite system resource.
The "connectionMax" property is also applicable to VBJ too.
In the next article, you will lean how to tune the Server Engine "connectionMaxIdle" property to conserve system socket resources.