Idea ID: 1654601

Enhance current Jgroups on TCP implementation to reduce traffic for large environments

Status : Declined
over 2 years ago

Key points:

1. It is a common requirement from customer's infrastructure teams to discard using UDP traffic for security reasons. Also multicast/broadcast messages are not allowed for the network of some customer, for example , renting Amazon cloud.

2. There is a limitation in current SM design to locate all App servers in 1 subnet. Main reason is low response time is required for SM nodes to communicate to each other, however this can be tuned by traffic prioritization on inbetween node. The other reason, - you dont want to allow multicast UDP traffic between subnets.

3. Current implementation of Jgroups on TCP will generate extremely heavy TCP traffic for a large number of nodes which makes in unusable for large environemnts.  If we take 10 app servers with 10 servlets per server as an example, assuming they are connected in one LAN switch, if deployed in different switch, accross the router, will be more complicated and worse. As we know for multicase message, the sender just need to send 1 package to the switch, then switch will search the group list, and copy the data to the corresponding ports.
While for TCP, if one servlet need to send messages to all the other servlets, this servlet need to setup 99 TCP socket connections, and need to send 90 packets to the LAN switch (there are 9 servlets located in the same host, no need to send out), and also for TCP connections , there are extra 3 way establishing and 4 way disconnect. This is really a huge resource cosumption, for both the host and network.

This request is raised for R&D to evaluate possibilities of other approaches for Jgroups communication.
I understand we have very limited set of options there, but I think we can try one idea:

Enhance Jgroups on TCP to make it possible to use 2 step model, - Jgroups traffic for all servlets on 1 host can be combined into 1 virtual Jgroups node which will communicate then to gossip router. This way the number of virtual nodes whould be similar to the the number of hosts and traffic will be reduced as each servlet wouldn't need anymore to open connections to each other remote servlet. This is just one idea, will appreciate if R&D will help to evaluate it or share other potential approaches which will help to move from UDP multicast to stable Jgroups on TCP for large environments.


  • The idea is suggesting to communicate with TCP between hosts as virtual group and the virtual group will communicate with the servlet in hosts. The idea seems reduced the traffic between hosts but it also introduce a latency between servlets.  Besides, JGroups seems does not support the idea of communicate with virtual groups. In such case, this idea will not be considered in SM roadmap.