Tag: Cluster

How to get All Cluster Node Details using JMX in JBossAS7.1.2

Hi,

In JBoss EAP5 or AS6 or previous releases there was a concept of Partition Name which could be used to get the list of active cluster members, However in JBoss AS7 there is no concept of PartitionName rather in UDP mode the lustering happens based on the multicast asddress (Clustering can be achieved using TCP mode of communication as well). However many developers want to list the number of active members running as part of a JBoss As7 cluster programatically.

NOTE:

Point-1) . In JBoss AS7 until you deploy an application which requires clustering the JBoss will now start the clustering services. So in case of web Applications you should deploy at least one application on your Cluster members which has <distributable/> tag specified in it’s “WEB-INF/web.xml” file.

Point-2) . If you are using EJBs (and not web application to deploy on your cluster) make sure that the EJBs are clustered in order to start Cluster services. EJBs can be annotated using the following annotation in order to be clustered. @org.jboss.ejb3.annotation.Clustered

Point-3) . In Case of WebBased cluster the ObjectName of the Clustered instance will be “jgroups:type=channel,cluster=web” by default which can be used in the JMX Code in order to look query the MBean.

Point-3) . In Case of EJBBased cluster the ObjectName of the Clustered instance will be “jgroups:type=channel,cluster=ejb” by default which can be used in the JMX Code in order to look query the MBean.

Point-5) . If you are running your Server in Domain Mode then you should not use the Native Management Port 9999 in order to query the MBeans rather you should use the individual server’s Remoting Port 4447 in your JMX Code. by adding the following configuration on your JBoss Profile inside domain.xml you can allow accessing the MBeans via Remoting port 4447 rather than (native management port 9999)

<subsystem xmlns="urn:jboss:domain:jmx:1.1">
    <show-model value="true"/>
     <remoting-connector  use-management-endpoint="false" />
</subsystem>

Point-6) . If you are using Native Management Interface port 9999 in order to query your Cluster MBean (in standalone mode) then you should use the Credentials (username/password) for a user belonging to ManagementRealm, But in Domain mode as we need to connect to server using Remoting Port (4447) so you should create a ApplicationRealm user. in all the cluster Hosts.

Writing the JMX codce to query Cluster MBean

import java.util.Hashtable;
import javax.management.MBeanServerConnection;
import javax.management.ObjectName;
import javax.management.remote.JMXConnector;
import javax.management.remote.JMXConnectorFactory;
import javax.management.remote.JMXServiceURL;

public class AS7ClusterView {
	public static void main(String[] args) throws Exception {

		String host = "10.10.10.10"; // Your JBoss Native Interface Bin Address default is localhost
		int port = 9999;             // management port     // In Domain Mode you should use  4447 port of individual server
		String urlString = "service:jmx:remoting-jmx://" + host + ":" + port;
		System.out.println(" nnt**** urlString: " + urlString);
                String webClusterObjectName="jgroups:type=channel,cluster="web"";
                //String ejbClusterObjectName="jgroups:type=channel,cluster="ejb"";


		JMXServiceURL serviceURL = new JMXServiceURL(urlString);


		Hashtable h = new Hashtable();
		String[] credentials = new String[] { "admin", "admin123" };
		h.put("jmx.remote.credentials", credentials);

		JMXConnector jmxConnector = JMXConnectorFactory.connect(serviceURL,null);
		MBeanServerConnection connection = jmxConnector.getMBeanServerConnection();
		ObjectName objectName = new ObjectName(webClusterObjectName);
		String clusterView = (String) connection.getAttribute(objectName,"View");
		Long receivedMessages = (Long) connection.getAttribute(objectName,"ReceivedMessages");
		String name = (String) connection.getAttribute(objectName, "Name");
		String clusterName = (String) connection.getAttribute(objectName,"ClusterName");

		System.out.println(" clusterView = " + clusterView);
		System.out.println(" receivedMessages = " + receivedMessages);
		System.out.println(" name = " + name);
		System.out.println(" clusterName = " + clusterName);
		jmxConnector.close();
	}

}

Now open a Terminal where we need to set the PATh and CLASSPATH like following and then compile and run the program:

For UNIX Based Operating System:

export PATH=/home/userone/jdk1.6.0_21/bin:$PATH
export CLASSPATH=/home/userone/jboss-as-7.1.2.Final/bin/client/jboss-client.jar:$CLASSPATH:.:

++++++++++++++++++++++++++++++++
For Windows Based Operating System

set PATH=C:jdk1.6.0_21bin;%PATH%
set CLASSPATH=c:jboss-as-7.1.2.Finalbinclientjboss-client.jar;%CLASSPATH%;.;

.
.
Thanks 🙂
MiddlewareMagic Team


Using mod_cluster with JBoss AS 7.1 cluster

We have seen how to create clusters in standalone, domain with multicast and unicast protocols also which have been listed below. However in this article we would be seeing how to use mod_cluster with our created clusters in both standalone and domain mode.

Following are the list of article we have created till now for creating a cluster in JBoss AS 7.1.1 Final till now

  1. How to create cluster in JBoss AS 7.1 in standalone mode?
  2. How to create cluster in JBoss AS 7.1 in domain mode?
  3. Creating TCP cluster in JBoss AS 7.1 in standalone mode?

Again we would be using JBoss AS 7.1.1.Final for this article as well so that we are all on the same page, as few things have been change from JBoss AS 7.0 to JBoss AS 7.1.

Changes taken place:

  1. AJP connector is enabled by default in standalone-ha.xml , standalone-full-ha.xml and domain.xml for ha and full-ha profile.
  2. Instead of JVMRoute, now we have to use instance-id

Using mod_cluster with JBoss AS 7.1 cluster

For this we would have to modify from both side Apache as well as JBoss AS7 side, hence lets see the configuration one at a time

Apache side configuration

Lets see what all configuration has to be made from Apache side

  1. You can configure mod_cluster from the below article, just follow the “Apache side configuration” part only
  2. How to configure mod_cluster with JBoss ?

  3. In the above article you can just add the IP_ADDRESS of the box on which Apache is running which is in the Step-3 something as shown below
  4. ############### mod_cluster Setting - STARTED ###############
    LoadModule slotmem_module modules/mod_slotmem.so
    LoadModule manager_module modules/mod_manager.so
    LoadModule proxy_cluster_module modules/mod_proxy_cluster.so
    LoadModule advertise_module modules/mod_advertise.so
    
    Listen 1.1.1.1:80
    
    <VirtualHost 1.1.1.1:80>
    	<Directory />
    		Order deny,allow
    		Allow from all
    	</Directory>
    	<Location /mod_cluster-manager>
    		SetHandler mod_cluster-manager
    		Order deny,allow
    		Allow from all
    	</Location>
              KeepAliveTimeout 60
              ManagerBalancerName mycluster
              ServerAdvertise On
    </VirtualHost>
    ############### mod_cluster Setting - ENDED ###############
    

JBoss side configuration

Now we would see what all changes we have to be done from JBoss end in standalone and domain files

Standalone

  1. First you would have to follow all the steps given in the link – How to create cluster in JBoss AS 7.1 in standalone mode? and then make the below changes in all the standalone servers.
  2. Give a unique name in the server element, as shown below .
  3. standalone-node1

    <server name="standalone-node1" xmlns="urn:jboss:domain:1.2">
    

    standalone-node2

    <server name="standalone-node2" xmlns="urn:jboss:domain:1.2">
    
  4. As told earlier you would have to add the instance-id attribute in web subsystem as shown below in both the standalone nodes.
  5.         <subsystem xmlns="urn:jboss:domain:web:1.1" default-virtual-server="default-host" instance-id="${jboss.node.name}" native="false">
                <connector name="http" protocol="HTTP/1.1" scheme="http" socket-binding="http"/>
                <connector name="ajp" protocol="AJP/1.3" scheme="http" socket-binding="ajp"/>
                .
                .
                .
            </subsystem>
    
  6. Last you just have to add the proxy-list in the attribute in mod-cluster-config of modcluster subsystem, which would be having IP Address and Port on which your Apache server is running so that JBoss server can communicate with it, as shown below in both the standalone nodes.
  7.         <subsystem xmlns="urn:jboss:domain:modcluster:1.0">
                <mod-cluster-config advertise-socket="modcluster" proxy-list="1.1.1.1:80">
                .
                .
                .
                </mod-cluster-config>
            </subsystem>
    

If everything is configured properly you can hit the URL = http://1.1.1.1/mod_cluster-manager which would show similar below screen. Below image is showing when cluster is been made in on the same box using standalone mode

mod_cluster for Standalone on the same box

Domain

  1. First you would have to follow all the steps given in the link – How to create cluster in JBoss AS 7.1 in domain mode? and then make the below changes in your domain file.
  2. We would have to add the instance-id attribute in web subsystem as shown below in domain.xml for the respective profile ha and full-ha which is been used.
  3.             <subsystem xmlns="urn:jboss:domain:web:1.1" default-virtual-server="default-host" instance-id="${jboss.node.name}" native="false">
                    <connector name="http" protocol="HTTP/1.1" scheme="http" socket-binding="http"/>
                    <connector name="ajp" protocol="AJP/1.3" scheme="http" socket-binding="ajp"/>
                     .
                     .
                     .
                </subsystem>
    
  4. Last you just have to add the proxy-list in the attribute in mod-cluster-config of modcluster subsystem, which would be having IP Address and Port on which your Apache server is running so that JBoss server can communicate with it, as shown below in domain.xml for the respective profile ha and full-ha which is been used.
  5.         <subsystem xmlns="urn:jboss:domain:modcluster:1.0">
                <mod-cluster-config advertise-socket="modcluster" proxy-list="1.1.1.1:80">
                .
                .
                .
                </mod-cluster-config>
            </subsystem>
    

If everything is configured properly you can hit the URL = http://1.1.1.1/mod_cluster-manager which would show similar below screen. Below image is showing when cluster is been made in on the same box as well as in remote box using domain mode.

Same Box

mod_cluster for Domain mode on the same box

Remote Box

mod_cluster for Domain mode on Remote box

Testing

In all our previous article we had asked you to use one of our application called ClusterWebAp, this is the time when we would be testing if our cluster is working properly with mod_cluster configuration.

  1. Once everything thing is been configured and have been started up properly you can hit the application using Apache URL as shown below and you would see something like in the snap-shot
  2. http://1.1.1.1/ClusterWebApp
    

    ClusterWebApp

  3. Suppose the first request goes to standalone-node2 , you can see there is a session-id which is been generated. Now you just have to shutdown your standalone-node2 and then click on the link Click Here to move to nextPage sessionCheck.jsp on the same page.
  4. Once you click on the link, you will notice that the request has been served by the standalone-node1 this time as node2 had been brought down, but the session-id remains the same which was created earlier having the same timestamp as shown below
  5. Session Replication

With this test it is clear that our cluster is working fine as the session replication is happening properly on JBoss using mod_cluster.

Issues you might face

If you have not given the instance-id in the domain.xml file for ha or full-ha and have a step up a cluster which is having a mod_cluster in front of them you might get the following error in the logs.

[Server:ha-server-1] 20:29:36,429 ERROR [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Error [MEM: MEM: Old node still exist: {4}] sending command CONFIG to proxy 10.10.10.10:80, configuration will be reset
[Server:ha-server-2] 20:29:37,426 ERROR [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Error [MEM: MEM: Can't read node: {4}] sending command STATUS to proxy 10.10.10.10:80, configuration will be reset
[Server:ha-server-2] 20:29:47,432 ERROR [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Error [MEM: MEM: Old node still exist: {4}] sending command CONFIG to proxy 10.10.10.10:80, configuration will be reset
[Server:ha-server-1] 20:29:56,441 ERROR [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Error [MEM: MEM: Can't read node: {4}] sending command STATUS to proxy 10.10.10.10:80, configuration will be reset

For this you just have to follow the step-2 of Domain configuration, which would fix this issue.


Creating TCP cluster in JBoss AS 7.1 in standalone mode?

In my previous article we had seen How to create cluster in JBoss AS 7.1 in standalone mode? which was using UDP protocol.

By default JBoss AS7 uses UDP protocol, but for security constrains some companies do not open/allow to use UDP protocol or multicast addresses, hence during that time TCP protocol or unicast address can be used to create a cluster which is also supported by JBoss AS 7.

In this article we would be showing you how you use standalone mode to create a cluster using TCP protocol. However as we all know in standalone mode we have different xml files under the configuration folder from which cluster is enabled in standalone-ha.xml and standalone-full-ha.xml, thus make sure you would be using them and not other xml files.

In this article we would be using JBoss AS 7.1.1 Final which is the latest version of JBoss in community version.

Steps to create a TCP cluster in JBoss AS 7.1 in standalone mode

We would be seeing two scenarios here one would be creating a cluster on the same box and second when creating a cluster between different boxes.

Scenario 1: Cluster on same box

  1. Once you have unzipped jboss-as-7.1.1.Final.zip , you would have to create two copies of standalone folder and rename them as standalone-node1 and standalone-node2 as shown below
  2. /home/user/jboss-as-7.1.1.Final/standalone-node1
    /home/user/jboss-as-7.1.1.Final/standalone-node2
    

    Note: Make sure you keep the original copy for standalone folder as it is for future usage.

  3. In both the standalone-nodex you would have to make the following changes in standalone-ha.xml files to tell JBoss to start the cluster in TCP protocol
  4. From:

    <subsystem xmlns="urn:jboss:domain:jgroups:1.1" default-stack="udp">
                <stack name="udp">
               .
               .
               .
                </stack>
                <stack name="tcp">
                    <transport type="TCP" socket-binding="jgroups-tcp" diagnostics-socket-binding="jgroups-diagnostics"/>
                    <protocol type="MPING" socket-binding="jgroups-mping"/>
                    <protocol type="MERGE2"/>
                    <protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd"/>
                    <protocol type="FD"/>
                    <protocol type="VERIFY_SUSPECT"/>
                    <protocol type="BARRIER"/>
                    <protocol type="pbcast.NAKACK"/>
                    <protocol type="UNICAST2"/>
                    <protocol type="pbcast.STABLE"/>
                    <protocol type="pbcast.GMS"/>
                    <protocol type="UFC"/>
                    <protocol type="MFC"/>
                    <protocol type="FRAG2"/>
                </stack>
            </subsystem>
    

    To:

    <subsystem xmlns="urn:jboss:domain:jgroups:1.1" default-stack="tcp">
                <stack name="udp">
               .
               .
               .
                </stack>
                <stack name="tcp">
                    <transport type="TCP" socket-binding="jgroups-tcp" diagnostics-socket-binding="jgroups-diagnostics"/>
                    <protocol type="TCPPING">
                        <property name="initial_hosts">10.10.10.10[7600],10.10.10.10[7600]</property>
                        <property name="num_initial_members">2</property>
                        <property name="port_range">0</property>
                        <property name="timeout">2000</property>
                    </protocol>
                    <protocol type="MERGE2"/>
                    <protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd"/>
                    <protocol type="FD"/>
                    <protocol type="VERIFY_SUSPECT"/>
                    <protocol type="BARRIER"/>
                    <protocol type="pbcast.NAKACK"/>
                    <protocol type="UNICAST2"/>
                    <protocol type="pbcast.STABLE"/>
                    <protocol type="pbcast.GMS"/>
                    <protocol type="UFC"/>
                    <protocol type="MFC"/>
                    <protocol type="FRAG2"/>
                </stack>
            </subsystem>
    
      Note: Following are the changes made

    1. We have replaced default-stack=”udp” to default-stack=”tcp” in the subsystem element.
    2. And added “TCPPING” protocol element with its sub-elements

    Where:
    initial_hosts = is a list of comma-seperated combo of IP_ADDRESS and PORT for pinging.
    num_initial_members = specifies the maximum number of responses to wait for.
    port_range = specifies the range of ports to ping on each host in the initial_hosts list.
    timeout = specifies the maximum number of milliseconds to wait for any responses.

  5. Now you would have to run the below command to start both the JBoss node in a cluster
  6. Node1

    ./standalone.sh -c standalone-ha.xml -b 10.10.10.10 -Djboss.server.base.dir=../standalone-1 -Djboss.node.name=node1 -Djboss.socket.binding.port-offset=100
    

    Node2

    ./standalone.sh -c standalone-ha.xml -b 10.10.10.10 -Djboss.server.base.dir=../standalone-2 -Djboss.node.name=node2 -Djboss.socket.binding.port-offset=200
    

    Where:
    -c = is for server configuration file to be used
    -b = is for binding address
    -Djboss.server.base.dir = is for the path from where node is present
    -Djboss.node.name = is for the name of the node
    -Djboss.socket.binding.port-offset = is for the port offset on which node would be running

      Note: However we need to keep in mind the following things

    1. Both the nodes should have unique node names
    2. Both the nodes should have unique socket binding port-offsets as they are running on the same box
  7. Once both the node comes up properly you would not see them in cluster, hence to make sure if both of the nodes are in a cluster then you would need to deploy the an application which has the distributable tag in web.xml . You can download one of our sample clustered application by : clicking here
  8. After downloading the ClusterWebApp.war you just have to keep it in (/home/user/jboss-as-7.1.1.Final/standalone-nodeX/deployments) both nodes deployments folder, just after that you would see similar below messages in both the nodes prompt, having both node names in there cluster view.
  9. 21:49:11,988 INFO  [stdout] (pool-14-thread-1) -------------------------------------------------------------------
    21:49:11,989 INFO  [stdout] (pool-14-thread-1) GMS: address=node2/web, cluster=web, physical address=10.10.10.10:7800
    21:49:11,989 INFO  [stdout] (pool-14-thread-1) -------------------------------------------------------------------
    .
    .
    21:49:15,954 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (pool-15-thread-1) ISPN000094: Received new cluster view: [node1/web|1] [node1/web, node2/web]
    .
    

Scenario 2: Cluster on different boxes

  1. After unzipping JBoss AS 7 in both the boxes [i.e. box-1=10.10.10.10 and box-2=20.20.20.20 ] then you can create just a single copies of standalone folder in respective boxes
  2. Box-1 : 10.10.10.10

    /home/user/jboss-as-7.1.1.Final/standalone-node1
    

    Box-2 : 20.20.20.20

    /home/user/jboss-as-7.1.1.Final/standalone-node2
    
  3. In both the standalone-nodex you would have to follow the same Step-2 of Scenario 1, however only one change would be there in “initial_hosts” you would have to give both boxes IP_ADDRESS as shown below
  4. <protocol type="TCPPING">
    	<property name="initial_hosts">10.10.10.10[7600],20.20.20.20[7600]</property>
    	.
    	.
    	.
    </protocol>
    
  5. Now you would have to run the below command to start both the JBoss node in a cluster
  6. Node1 on Box-1 [10.10.10.10]

    ./standalone.sh -c standalone-ha.xml -b 10.10.10.10 -Djboss.server.base.dir=../standalone-node1 -Djboss.node.name=node1 
    

    Node2 on Box-2 [20.20.20.20]

    ./standalone.sh -c standalone-ha.xml -b 20.20.20.20 -Djboss.server.base.dir=../standalone-node2 -Djboss.node.name=node2
    
      Note: However we need to keep in mind the following things 

    1. Both the nodes should have unique node names
    2. Both the nodes should be running on the IP_ADDRESS or HOST_NAME of the box

    Here we would not have to worry about the port conflicts as we are running both the nodes on different boxes having different binding address.

  7. Repeat the same step-4 and step-5 of Scenario-1 and you would then see the same cluster view in each running nodes prompts.

Copyright © 2010-2012 Middleware Magic. All rights reserved. |