Tomcat Cluster in the Cloud

The platform multicast and redirecting requests to each server with load balancer provides session replication between the pairs of server nodes. This guarantees session exchange between the nodes through the local net and eliminates the need of additional software or Memcached. With this approach you can use a big clustered app hosting.

This instruction shows the clustering technology used in the platform by the example of the Tomcat app server.

Tomcat cluster

The given scheme presents a Tomcat cluster with two servers and one load balancer. All requests are handled and distributed by the balancer between different nodes due to the availability and server load.

If one of the servers fails, the users from that node will be automatically switched to the other instance of this Tomcat cluster. Thanks to the replication, the other instance already has all the sessions of the failed node, so end-users never notice any change.

To use the Tomcat clustering in the platform, you have to perform the next steps:

1. Log into your platform dashboard.

2. Click New Environment.

new environment button

3. Pick Tomcat as the application server you want to use, specify the cloudlet limits and turn on High-availability as shown in the picture below. State the name of the environment and click Create.

Tomcat cluster in topology wizard

Note: Horizontal scaling and High Availability mode are two different functionalities in the platform. With the first one you have multiple servers and the load is distributed evenly between the chosen number of instances with the load balancer. High Availability mode sets replication between pair or few pairs of servers using multicast membership.

When you enable High Availability a special Tomcat config file (iNET.elastic-ha.xml) is generated for each node by our system. Here is an example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
      channelSendOptions="4">

<Manager className="org.apache.catalina.ha.session.DeltaManager"
      expireSessionsOnShutdown="false"
      notifyListenersOnReplication="true"/>

<Channel className="org.apache.catalina.tribes.group.GroupChannel">

 <Membership className="org.apache.catalina.tribes.membership.McastService"
       address="228.0.0.4"
       port="${MagicPort}"
       frequency="500"
       dropTime="3000"/>

 <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
       address="${ReceiverIp}"
       port="4000"
       autoBind="100"
       selectorTimeout="5000"
       maxThreads="6"/>

 <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
 <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
 </Sender>

 <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
 <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
</Channel>

 <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
     filter=""/>
 <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

 <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
 <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>

Let’s look through this file in details:

1. This is the major element, inside which all the other cluster elements are configured.

1
2
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
       channelSendOptions="4">

The channelSendOptions flag is attached to every message sent by SimpleTcpCluster class or any objects that are calling the SimpleTcpCluster.send method.

2. The DeltaManager uses the SimpleTcpCluster.send method to send information though the channel.

1
2
3
<Manager className="org.apache.catalina.ha.session.DeltaManager"
     expireSessionsOnShutdown="false"
     notifyListenersOnReplication="true"/>;

3. The group communication framework inside Tomcat is called Tribes. It is used as the channel element here. It encapsulates everything that relates the membership, logic and communication.

1
<Channel className="org.apache.catalina.tribes.group.GroupChannel">  

4. Membership is made using multicast. Tomcat cluster separation consists of a multicast address and port number. Communication between nodes is realized over TCP.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
<Membership className="org.apache.catalina.tribes.membership.McastService"
    address="228.0.0.4"
    port="${MagicPort}"
    frequency="500"
    dropTime="3000"/>
    <Membership className="org.apache.catalina.tribes.membership.McastService"
        address="228.0.0.4"
        port="${MagicPort}"
        frequency="500"
        dropTime="3000"/>

{MagicPort} is a unique port number for the cluster, which is generated on the fly from the Java arguments.

5. Tribes' logic of sending and receiving data includes two components: sender and receiver. The Receiver is responsible for data receiving. There is a thread pool in this element which has a maxThreads and minThreads setting. The address attribute is the host address that will be broadcasted by the membership component to the other nodes.

1
2
3
4
5
6
<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
    address="${ReceiverIp}"
    port="4000"
    autoBind="100"
    selectorTimeout="5000"
    maxThreads="6"/>

6. The Sender component is responsible for sending messages to other nodes. The Sender includes the ReplicationTransmitter, a special shell component, and Transport sub component. Messages can be sent concurrently with the NIO sender and parallel with pool of senders.

1
2
3
<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>

7. The elements of the Tribes stack interceptors are:

TcpFailureDetector - verifies crashed members via TCP
MessageDispatch15Interceptor - dispatches messages to a thread pool to send message asynchronously

1
2
3
<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
</Channel>

8. The cluster uses valves to track requests to web applications:

ReplicationValve finds out when the request has been completed and initiates the replication
JvmRouteBinderValve is responsible for backing up your data

1
2
3
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve" 
     filter="">
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

9. The SimpleTcpCluster is a sender and receiver of the Channel object, so components are registered as listeners to this cluster.

1
2
<ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>

High Availability configuration is automated so you can easily set it up for every Java app server supported by the platform.

Tomcat Cluster in the Cloud

What’s next?