In the vain of its creator (Red Hat): “… we gain the most when we share. It is core to our business and our development models.” As a result, we are bringing you a free course on JBoss AS 7 (EAP 6). In the course you will learn the following:

• Introduction
  • JBoss Application Server architecture (standalone, process controller, host controller, domain)
  • JBoss Application Server internal architecture (listen threads, threads pools, containers)
  • Standalone configuration
  • Command-line utility
  • Admin console
• JVM Tuning
  • Code generation
  • Memory management
  • Garbage collection performance
  • JConsole and JVisualVM
• Deployment
  • Packaging applications recommendations
  • Deployment descriptors
  • Server logging
  • Front-end configuration using the Apache HTTP Server and mod_jk
• Class Loading
  • Modules
• Security
  • JAAS introduction
  • Role-based security
  • Secure communications (SSL and TLS)
• Configure Resources
  • Domain configuration
  • Configure data sources
  • Monitoring resources
  • Configure JMS environment
• Clustering
  • Unicast or multicast?
  • Denial of service configuration
  • Vertical and horizontal scaling
  • Caching
  • Apache HTTP Server and mod_cluster
  • Load balancing
  • Load testing

Note that this is a course you can do at your own pace. The exercises lead you step-by-step through the process of installing, configuring, deploying etcetera. The material consists of the following:

• installation, the necessary software to set-up the environment:
  • JBoss Application Server
  • Java SE Development Kit
  • Apache HTTP Server
  • mod_jk
  • mod_cluster
  • The Grinder
• software, contains the deployments (.ear and .war files) that are used in the exercises.
• voorbeelden.zip (means examples.zip in Dutch), contains the code that is used to build the applications and an example configuration that you are going to set-up in the exercises.
• jboss-opgaven.pdf, the exercises.
• jboss-sheets.pdf, the sheets.

Have fun!

In this post we put Spring and Hibernate 4 together. We start out by creating a Java client to see what is needed to make things work. Next, we create a Web application (by adding a servlet) and deploy the application to JBoss AS7 (and use the native Hibernate API). Here, we show what needs to be done in order to make Spring and the native Hibernate API work together on the JBoss Application Server, i.e., we create an extra module (org.springframework) and make sure this is loaded by the deployed Web module. Finally, we show how to include messaging as well. More information on how the JBoss environment is set-up can be found in the post Building a Coherence Cluster with Multiple Application Servers. We also show how the environment can be migrated from JBoss to WebLogic (or vice versa). More information about the WebLogic environment can be found in the post Deploy WebLogic12c to Multiple Machines.

Client

Let us start by creating a Java client, just to get a feeling of what needs to be done and not have the complexity of a Java EE Server (such as, for example, JBoss AS7) along with it. To this end we can use the following Spring configuration

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:tx="http://www.springframework.org/schema/tx"
       xmlns:jee="http://www.springframework.org/schema/jee"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
       http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
       http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-3.0.xsd">
    <bean id="company" class="model.logic.CompanyBean">
        <property name="sessionFactory" ref="sessionfactory"/>
    </bean>
    <bean id="sessionfactory" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
        <property name="dataSource" ref="datasource"/>
        <!--property name="jtaTransactionManager" ref="transactionManager"/-->
        <property name="mappingResources">
            <list>
                <value>model/entities/person.hbm.xml</value>
            </list>
        </property>
        <property name="hibernateProperties">
            <props>
                <prop key="hibernate.dialect">${hibernate.dialect}</prop>
                <!--prop key="hibernate.current_session_context_class">${hibernate.current_session_context_class}</prop-->
                <prop key="hibernate.listeners.envers.autoRegister">${hibernate.listeners.envers.autoRegister}</prop>
                <prop key="hibernate.show_sql">${hibernate.show_sql}</prop>
                <prop key="hibernate.format_sql">${hibernate.format_sql}</prop>
            </props>
        </property>
    </bean>
    <!--jee:jndi-lookup id="datasource" jndi-name="java:/jdbc/OracleDS" resource-ref="false"/-->
    <bean id="datasource" class="oracle.jdbc.pool.OracleDataSource" destroy-method="close">
        <property name="driverType" value="${jdbc.driverClassName}"/>
        <property name="URL" value="${jdbc.url}"/>
        <property name="user" value="${jdbc.username}"/>
        <property name="password" value="${jdbc.password}"/>
    </bean>
    <bean id="propertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="locations">
            <list>
                <value>classpath:spring.properties</value>
            </list>
        </property>
    </bean>
    <!--bean id="transactionManager" class="org.springframework.transaction.jta.JtaTransactionManager"/-->
    <bean id="transactionManager" class="org.springframework.orm.hibernate4.HibernateTransactionManager">
        <property name="sessionFactory" ref="sessionfactory"/>
    </bean>
    <tx:annotation-driven transaction-manager="transactionManager"/>
</beans>

Note that we did not set the hibernate.current_session_context_class property. We should never use that property when combining Hibernate with Spring as it destroys proper session and transaction management when using the HibernateTransactionManager. On the other hand, when on a Java EE Server and using the JtaTransactionManager, set the property to the value thread. Another thing to note is that the EnversIntegrator is always detected by default while initializing the ServiceRegistry using the Native Hibernate API causing a MappingException even when Envers is not used when deployed on JBoss AS7. That is why we have set the value of the hibernate.listeners.envers.autoRegister property to false.

The contents of the referred spring.properties look as follows

jdbc.driverClassName=oracle.jdbc.OracleDriver
jdbc.url=jdbc:oracle:thin:@192.168.1.60:1521:orcl11
jdbc.username=example
jdbc.password=example

hibernate.dialect=org.hibernate.dialect.Oracle10gDialect
hibernate.current_session_context_class=thread
hibernate.listeners.envers.autoRegister=false
hibernate.show_sql=false
hibernate.format_sql=false

The CompanyBean class looks as follows

package model.logic;

import model.entities.Person;
import org.hibernate.Session;
import org.hibernate.SessionFactory;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;

@Transactional(propagation = Propagation.REQUIRED, readOnly = false)
public class CompanyBean implements Company {

    private SessionFactory sessionFactory;

    public SessionFactory getSessionFactory() {
        return sessionFactory;
    }

    public void setSessionFactory(SessionFactory sessionFactory) {
        this.sessionFactory = sessionFactory;
    }

    public void insertPerson(Person person) {
        Person temporaryPerson = findPerson(person.getSofinummer());
        if (temporaryPerson == null) {
            System.out.println("INSERT " + person);
            getCurrentSession().save(person);
        } else {
            updatePerson(person);
        }
    }

    public void removePerson(Integer sofinummer) {
        Person temporaryPerson = findPerson(sofinummer);
        if (temporaryPerson != null) {
            System.out.println("REMOVE " + temporaryPerson);
            getCurrentSession().delete(temporaryPerson);
        }
    }

    public void updatePerson(Person person) {
        System.out.println("UPDATE " + person);
        getCurrentSession().merge(person);
    }

    private Person findPerson(Integer sofinummer) {
        return (Person) getCurrentSession().get(Person.class, sofinummer);
    }

    private final Session getCurrentSession(){
      return getSessionFactory().getCurrentSession();
   }
}

To test the set-up, we can use something like

package model.test;

import model.entities.Person;
import model.logic.Company;
import model.utils.SpringUtilities;

import java.util.Random;

public class Test {

    private Random generator = new Random();

    public static void main(String[] args) {
        Test test = new Test();

        Company company = SpringUtilities.getCompany();

        for (int i = 0; i < 100; i++) {
            Person person = test.createPerson();
            if (test.generator.nextDouble() < 0.01) {
                company.removePerson(person.getSofinummer());
            } else {
                company.insertPerson(person);
            }
        }
    }

    private Person createPerson() {
        Person person = new Person();
        person.setNaam(Long.toString(Math.abs(generator.nextLong()), 36));
        person.setSofinummer(generator.nextInt(10000));
        return person;
    }
}

When the test is run something like the following output is observed

Aug 14, 2012 2:59:35 PM org.springframework.context.support.AbstractApplicationContext prepareRefresh
INFO: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@145d068: startup date [Tue Aug 14 14:59:35 CEST 2012]; root of context hierarchy
Aug 14, 2012 2:59:35 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [spring-config.xml]
Aug 14, 2012 2:59:36 PM org.springframework.core.io.support.PropertiesLoaderSupport loadProperties
INFO: Loading properties file from class path resource [spring.properties]
Aug 14, 2012 2:59:36 PM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@1faba46: defining beans [company,sessionfactory,datasource,propertyConfigurer,transactionManager,org.springframework.aop.config.internalAutoProxyCreator,org.springframework.transaction.annotation.AnnotationTransactionAttributeSource#0,org.springframework.transaction.interceptor.TransactionInterceptor#0,org.springframework.transaction.config.internalTransactionAdvisor]; root of factory hierarchy
Aug 14, 2012 2:59:36 PM org.hibernate.annotations.common.Version <clinit>
INFO: HCANN000001: Hibernate Commons Annotations {4.0.1.Final}
Aug 14, 2012 2:59:36 PM org.hibernate.Version logVersion
INFO: HHH000412: Hibernate Core {4.1.2.Final}
Aug 14, 2012 2:59:36 PM org.hibernate.cfg.Environment <clinit>
INFO: HHH000206: hibernate.properties not found
Aug 14, 2012 2:59:36 PM org.hibernate.cfg.Environment buildBytecodeProvider
INFO: HHH000021: Bytecode provider name : javassist
Aug 14, 2012 2:59:36 PM org.hibernate.internal.util.xml.DTDEntityResolver resolveEntity
WARN: HHH000223: Recognized obsolete hibernate namespace http://hibernate.sourceforge.net/. Use namespace http://www.hibernate.org/dtd/ instead. Refer to Hibernate 3.6 Migration Guide!
Aug 14, 2012 2:59:36 PM org.hibernate.dialect.Dialect <init>
INFO: HHH000400: Using dialect: org.hibernate.dialect.Oracle10gDialect
Aug 14, 2012 2:59:36 PM org.hibernate.engine.transaction.internal.TransactionFactoryInitiator initiateService
INFO: HHH000399: Using default transaction strategy (direct JDBC transactions)
Aug 14, 2012 2:59:36 PM org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory <init>
INFO: HHH000397: Using ASTQueryTranslatorFactory
Aug 14, 2012 2:59:36 PM org.hibernate.validator.internal.util.Version <clinit>
INFO: HV000001: Hibernate Validator 4.3.0.Final
Aug 14, 2012 2:59:36 PM org.springframework.orm.hibernate4.HibernateTransactionManager afterPropertiesSet
INFO: Using DataSource [oracle.jdbc.pool.OracleDataSource@161f39e] of Hibernate SessionFactory for HibernateTransactionManager
UPDATE 179arxyy6tshk 6176
INSERT 1sf4e5oz89uhi 2402
UPDATE t9pjpfnqxc4k 8413
UPDATE 1ckym7ttz46mf 5246
UPDATE 1u7litwyaigu4 1211
INSERT 74ra63w7aql6 2238
INSERT 5v48tzzk75qo 5902
INSERT mh5zpr30rjkd 2082
UPDATE 13xzu6r3g0ruz 5877
UPDATE 1jeqjhlfjwv81 9332
INSERT 19zb01tagisqq 6022
INSERT q5v1re6g6svp 3071
INSERT 12t5tsoyld5nl 9991
INSERT m7dhwiwvx7oi 9089
INSERT 1hz8k3rq1cngf 5596
UPDATE 14hhycu8lxban 4169
INSERT 4hh6749zjr67 2970
INSERT 143ofpk3kupmn 8769
UPDATE 16cfhq1shjzu7 8439
UPDATE scijv0dgh6m4 4234

Server

When we want to run the application on a Java EE Server, such as JBoss AS7, we can use the JtaTransactionManager instead of the HibernateTransactionManager. We also get a reference to a datasource by looking it up in JNDI, i.e.,

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:tx="http://www.springframework.org/schema/tx"
       xmlns:jee="http://www.springframework.org/schema/jee"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
       http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
       http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-3.0.xsd">
    <bean id="company" class="model.logic.CompanyBean">
        <property name="sessionFactory" ref="sessionfactory"/>
    </bean>
    <bean id="sessionfactory" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
        <property name="dataSource" ref="datasource"/>
        <property name="jtaTransactionManager" ref="transactionManager"/>
        <property name="mappingResources">
            <list>
                <value>model/entities/person.hbm.xml</value>
            </list>
        </property>
        <property name="hibernateProperties">
            <props>
                <prop key="hibernate.dialect">${hibernate.dialect}</prop>
                <prop key="hibernate.current_session_context_class">${hibernate.current_session_context_class}</prop>
                <prop key="hibernate.listeners.envers.autoRegister">${hibernate.listeners.envers.autoRegister}</prop>
                <prop key="hibernate.show_sql">${hibernate.show_sql}</prop>
                <prop key="hibernate.format_sql">${hibernate.format_sql}</prop>
            </props>
        </property>
    </bean>
    <jee:jndi-lookup id="datasource" jndi-name="java:/jdbc/OracleDS" resource-ref="false"/>
    <bean id="propertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="locations">
            <list>
                <value>classpath:spring.properties</value>
            </list>
        </property>
    </bean>
    <bean id="transactionManager" class="org.springframework.transaction.jta.JtaTransactionManager"/>
    <tx:annotation-driven transaction-manager="transactionManager"/>
</beans>

To test the configuration we can use the following servlet

package userinterface.servlets;

import model.entities.Person;
import model.logic.Company;
import model.utils.SpringUtilities;

import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.Random;

@WebServlet(name = "TestServlet", urlPatterns = "/testservlet")
public class TestServlet extends HttpServlet {

    private Random generator = null;
    private Company company = null;

    @Override
    public void init() throws ServletException {
        generator = new Random();
        company = SpringUtilities.getCompany();
    }

    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        Person person = createPerson();

        if (generator.nextDouble() < 0.01) {
            company.removePerson(person.getSofinummer());
        } else {
            company.insertPerson(person);
        }
    }

    @Override
    public void destroy() {
        generator = null;
        company = null;
    }

    private Person createPerson() {
        Person person = new Person();
        person.setNaam(Long.toString(Math.abs(generator.nextLong()), 36));
        person.setSofinummer(generator.nextInt(10000));
        return person;
    }
}

One thing to note is that when we need to develop a servlet (by using an IDE) we have to make sure the right classes are on the classpath. Usually, when developing a Java EE application the javaee-api-6.0.jar would suffice. One thing to note is that if we were to run our Test class again we would run into the following: Caused by: java.lang.ClassFormatError: Absent Code attribute in method that is not native or abstract in class file javax/validation/Validation. To resolve this we need the full implementation of the bean-validator.jar on the classpath and make sure that it is loaded before the javaee-api-6.0.jar.

Before, we can deploy the application we have to make sure the Spring classes are picked up by the Web module. To this end, we create the following module on JBoss AS7. First, create a directory structure that includes the necessary jars and a module.xml file as follows

${JBOSS_HOME}/modules
	/org/springframework
		/main
			aopalliance-1.0.jar
			cglib-2.2.2.jar
			commons-logging-1.0.4.jar
			module.xml
			org.springframework.aop-3.1.2.RELEASE.jar
			org.springframework.asm-3.1.2.RELEASE.jar
			org.springframework.beans-3.1.2.RELEASE.jar
			org.springframework.context-3.1.2.RELEASE.jar
			org.springframework.core-3.1.2.RELEASE.jar
			org.springframework.expression-3.1.2.RELEASE.jar
			org.springframework.jdbc-3.1.2.RELEASE.jar
			org.springframework.jms-3.1.2.RELEASE.jar
			org.springframework.orm-3.1.2.RELEASE.jar
			org.springframework.transaction-3.1.2.RELEASE.jar
			org.springframework.web-3.1.2.RELEASE.jar

where module.xml has the following contents

<module xmlns="urn:jboss:module:1.1" name="org.springframework">
    <resources>
		<resource-root path="aopalliance-1.0.jar"/>
		<resource-root path="cglib-2.2.2.jar"/>
		<resource-root path="commons-logging-1.0.4.jar"/>
		<resource-root path="org.springframework.aop-3.1.2.RELEASE.jar"/>
		<resource-root path="org.springframework.asm-3.1.2.RELEASE.jar"/>
		<resource-root path="org.springframework.beans-3.1.2.RELEASE.jar"/>
		<resource-root path="org.springframework.context-3.1.2.RELEASE.jar"/>
		<resource-root path="org.springframework.core-3.1.2.RELEASE.jar"/>
		<resource-root path="org.springframework.expression-3.1.2.RELEASE.jar"/>
		<resource-root path="org.springframework.jdbc-3.1.2.RELEASE.jar"/>
		<resource-root path="org.springframework.jms-3.1.2.RELEASE.jar"/>
		<resource-root path="org.springframework.orm-3.1.2.RELEASE.jar"/>
		<resource-root path="org.springframework.transaction-3.1.2.RELEASE.jar"/>
		<resource-root path="org.springframework.web-3.1.2.RELEASE.jar"/>
    </resources>
    <dependencies>
        <module name="javax.api" export="true"/>
        <module name="org.apache.log4j" export="true"/>
        <module name="org.antlr" export="true"/>
        <module name="org.dom4j" export="true"/>
        <module name="org.hibernate" export="true"/>
        <module name="javax.persistence.api" export="true"/>
        <module name="org.javassist" export="true"/>
        <module name="org.jboss.logging" export="true"/>
        <module name="javax.transaction.api" export="true"/>
    </dependencies>
</module>

Note that we also have included a Hibernate dependency (as this is needed by Spring when using Hibernate with it). To make sure we have all the required files, we can check a Hibernate 4 distribution and look in the ${HIBERNATE_HOME}/lib/required directory. That is, we need the following jars:

antlr-2.7.7.jar (loaded by adding the module org.antlr)
dom4j-1.6.1.jar (loaded by adding the module org.dom4j)
hibernate-commons-annotations-4.0.1.Final.jar (loaded by adding the module org.hibernate)
hibernate-core-4.0.1.Final.jar (loaded by adding the module org.hibernate)
hibernate-jpa-2.0-api-1.0.1.Final.jar (loaded by adding the module javax.persistence.api)
javassist-3.15.0-GA.jar (loaded by adding the module org.javassist)
jboss-logging-3.1.0.GA.jar (loaded by adding the module org.jboss.logging)
jboss-transaction-api_1.1_spec-1.0.0.Final.jar (loaded by adding the module javax.transaction.api)

Another important module that Spring depends on is the javax.api, as Spring needs XML related classes, such as parsers. Note that JBoss AS7 does not make javax packages, that are present in rt.jar, available. To make this happen we need to include the javax.api module. One important thing to note is that the javax.api module does not include a reference to javax.xml.parsers. To this end, edit the module.xml (located in the ${JBOSS_HOME}/modules/javax/api/main directory) and make sure that it does, i.e.,

<module xmlns="urn:jboss:module:1.1" name="javax.api">
    <dependencies>
        <system export="true">
            <paths>
				...
				<!-- To resolve java.lang.ClassNotFoundException: javax.xml.parsers.ParserConfigurationException from [Module "org.springframework:main" from local module loader @2393385d (roots: /home/jboss/jboss-as-7.1.0.Final/modules)] we have to add -->
                <path name="javax/xml/parsers"/>
				...
				<!-- java.lang.ClassNotFoundException: org.xml.sax.EntityResolver from [Module "org.springframework:main" from local module loader @2393385d (roots: /home/jboss/jboss-as-7.1.0.Final/modules)] is the first ClassNotFoundException we run into if the javax.api module is not added as a dependency to the org.springframework module -->
                <path name="org/xml/sax"/>
                <path name="org/xml/sax/ext"/>
                <path name="org/xml/sax/helpers"/>
            </paths>
        </system>
    </dependencies>
</module>

With the modules in place, restart the servers to make sure the modules are loaded

[jboss@axis-into-ict ~]$ cd jboss-as-7.1.0.Final/bin/

[jboss@axis-into-ict bin]$ ./jboss-cli.sh
You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.

[disconnected /] connect 192.168.1.150:9999

[domain@192.168.1.150:9999 /] /host=jboss/server-config=cluster-server1:restart
{
    "outcome" => "success",
    "result" => "STARTING"
}
[domain@192.168.1.150:9999 /] /host=jboss/server-config=cluster-server2:restart
{
    "outcome" => "success",
    "result" => "STARTING"
}

As we are dealing with a .war deployment, we have to add the jboss-deployment-structure.xml file to the WEB-INF directory, such that the org.springframework module can be accessed by the Web module. The contents of jboss-deployment-structure.xml look as follows

<jboss-deployment-structure>
    <deployment>
        <dependencies>
            <module name="org.springframework">
				<imports>
					<include path="META-INF**"/>
					<include path="org**"/>
				</imports>
			</module>
        </dependencies>
    </deployment>
</jboss-deployment-structure>

The imports element above is critical for the proper functioning of the Spring custom name-space capabilities. Applications that use custom name-spaces and wish to use a shared Spring module must use the JBoss deployment structure descriptors with the imports above. To deploy the application we can use

[domain@192.168.1.150:9999 /] deploy /home/jboss/jboss-as-7.1.0.Final/deploy/springhibernate/SpringHibernate.war --server-groups=cluster-group
[domain@192.168.1.150:9999 /] undeploy SpringHibernate.war --server-groups=cluster-group

which also shows how to undeploy it. The application can be reached at http://192.168.1.150:8888/SpringHibernate/testservlet. After doing a few requests the following is observed in the logging:

# LOGGING CLUSTER_SERVER1
12:52:50,289 INFO  [org.springframework.context.support.ClassPathXmlApplicationContext] (ajp--192.168.1.150-9080-3) Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@876e4b8: startup date [Tue Aug 14 12:52:50 CEST 2012]; root of context hierarchy
12:52:50,294 INFO  [org.springframework.beans.factory.xml.XmlBeanDefinitionReader] (ajp--192.168.1.150-9080-3) Loading XML bean definitions from class path resource [spring-config.xml]
12:52:50,345 INFO  [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer] (ajp--192.168.1.150-9080-3) Loading properties file from class path resource [spring.properties]
12:52:50,348 INFO  [org.springframework.beans.factory.support.DefaultListableBeanFactory] (ajp--192.168.1.150-9080-3) Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@6ddb6b07: defining beans [company,sessionfactory,datasource,propertyConfigurer,transactionManager,org.springframework.aop.config.internalAutoProxyCreator,org.springframework.transaction.annotation.AnnotationTransactionAttributeSource#0,org.springframework.transaction.interceptor.TransactionInterceptor#0,org.springframework.transaction.config.internalTransactionAdvisor]; root of factory hierarchy
12:52:50,366 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp--192.168.1.150-9080-3) Using JTA UserTransaction: org.jboss.tm.usertx.client.ServerVMClientUserTransaction@2cc821ef
12:52:50,367 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp--192.168.1.150-9080-3) Using JTA TransactionManager: com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate@500b3f4c
12:52:50,368 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp--192.168.1.150-9080-3) Using JTA TransactionSynchronizationRegistry: com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionSynchronizationRegistryImple@26b58796
12:52:50,374 WARN  [org.hibernate.internal.util.xml.DTDEntityResolver] (ajp--192.168.1.150-9080-3) HHH000223: Recognized obsolete hibernate namespace http://hibernate.sourceforge.net/. Use namespace http://www.hibernate.org/dtd/ instead. Refer to Hibernate 3.6 Migration Guide!
12:52:50,425 INFO  [org.hibernate.dialect.Dialect] (ajp--192.168.1.150-9080-3) HHH000400: Using dialect: org.hibernate.dialect.Oracle10gDialect
12:52:50,428 INFO  [org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory] (ajp--192.168.1.150-9080-3) HHH000397: Using ASTQueryTranslatorFactory
12:52:50,605 INFO  [stdout] (ajp--192.168.1.150-9080-3) UPDATE meh9qfr1uqch 5938
13:03:16,102 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE wegp27iyxw0j 2846
13:03:16,648 INFO  [stdout] (ajp--192.168.1.150-9080-4) INSERT 1xddpcvtis10e 5966
13:03:16,754 INFO  [stdout] (ajp--192.168.1.150-9080-7) INSERT mhvagbqaxgsd 7337
13:03:16,792 INFO  [stdout] (ajp--192.168.1.150-9080-7) UPDATE 34fxu4haa4q5 9804
13:03:16,866 INFO  [stdout] (ajp--192.168.1.150-9080-3) UPDATE 5amkxlyuzyp8 8927
13:03:16,942 INFO  [stdout] (ajp--192.168.1.150-9080-3) UPDATE 11ykc13m0r10b 4393
13:03:17,611 INFO  [stdout] (ajp--192.168.1.150-9080-3) INSERT 1fkms210dthwr 488
13:03:18,198 INFO  [stdout] (ajp--192.168.1.150-9080-3) INSERT cfp1i5sy4bpt 3455
13:03:18,223 INFO  [stdout] (ajp--192.168.1.150-9080-7) UPDATE p9jqdlnpv5d8 1074
13:03:18,246 INFO  [stdout] (ajp--192.168.1.150-9080-4) INSERT fnu0r03mgas0 2027
13:03:18,263 INFO  [stdout] (ajp--192.168.1.150-9080-3) UPDATE dgh5ib56gze7 1215
13:03:18,296 INFO  [stdout] (ajp--192.168.1.150-9080-7) INSERT 1n4x67w9aufr 7767
13:03:18,297 INFO  [stdout] (ajp--192.168.1.150-9080-9) UPDATE 8xy8prwm00yy 8807
13:03:18,326 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE 1jkvwulhwqksx 6280
13:03:18,334 INFO  [stdout] (ajp--192.168.1.150-9080-8) INSERT 1lrmb4dyxurp2 5249
13:03:18,339 INFO  [stdout] (ajp--192.168.1.150-9080-3) UPDATE mo1vpdb5v0sm 5048
13:03:18,345 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE qb6e3wq3ok05 7163
13:03:18,357 INFO  [stdout] (ajp--192.168.1.150-9080-3) INSERT wlxcjpediiml 961
13:03:18,358 INFO  [stdout] (ajp--192.168.1.150-9080-2) INSERT fr6ji76ohzsi 842
13:03:18,365 INFO  [stdout] (ajp--192.168.1.150-9080-7) INSERT 1otburj6mb9er 7645
13:03:18,372 INFO  [stdout] (ajp--192.168.1.150-9080-9) INSERT 198eulbfrkd0b 6503

# LOGGING CLUSTER_SERVER2
13:03:16,786 INFO  [org.springframework.context.support.ClassPathXmlApplicationContext] (ajp--192.168.1.150-9081-2) Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@652130d1: startup date [Tue Aug 14 13:03:16 CEST 2012]; root of context hierarchy
13:03:16,830 INFO  [org.springframework.beans.factory.xml.XmlBeanDefinitionReader] (ajp--192.168.1.150-9081-2) Loading XML bean definitions from class path resource [spring-config.xml]
13:03:17,107 INFO  [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer] (ajp--192.168.1.150-9081-2) Loading properties file from class path resource [spring.properties]
13:03:17,127 INFO  [org.springframework.beans.factory.support.DefaultListableBeanFactory] (ajp--192.168.1.150-9081-2) Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@5c594008: defining beans [company,sessionfactory,datasource,propertyConfigurer,transactionManager,org.springframework.aop.config.internalAutoProxyCreator,org.springframework.transaction.annotation.AnnotationTransactionAttributeSource#0,org.springframework.transaction.interceptor.TransactionInterceptor#0,org.springframework.transaction.config.internalTransactionAdvisor]; root of factory hierarchy
13:03:17,191 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp--192.168.1.150-9081-2) Using JTA UserTransaction: org.jboss.tm.usertx.client.ServerVMClientUserTransaction@6ebfc8d0
13:03:17,192 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp--192.168.1.150-9081-2) Using JTA TransactionManager: com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate@3839029b
13:03:17,193 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp--192.168.1.150-9081-2) Using JTA TransactionSynchronizationRegistry: com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionSynchronizationRegistryImple@33ec79b6
13:03:17,286 INFO  [org.hibernate.annotations.common.Version] (ajp--192.168.1.150-9081-2) HCANN000001: Hibernate Commons Annotations {4.0.1.Final}
13:03:17,291 INFO  [org.hibernate.Version] (ajp--192.168.1.150-9081-2) HHH000412: Hibernate Core {4.0.1.Final}
13:03:17,293 INFO  [org.hibernate.cfg.Environment] (ajp--192.168.1.150-9081-2) HHH000206: hibernate.properties not found
13:03:17,295 INFO  [org.hibernate.cfg.Environment] (ajp--192.168.1.150-9081-2) HHH000021: Bytecode provider name : javassist
13:03:17,330 WARN  [org.hibernate.internal.util.xml.DTDEntityResolver] (ajp--192.168.1.150-9081-2) HHH000223: Recognized obsolete hibernate namespace http://hibernate.sourceforge.net/. Use namespace http://www.hibernate.org/dtd/ instead. Refer to Hibernate 3.6 Migration Guide!
13:03:17,523 INFO  [org.hibernate.dialect.Dialect] (ajp--192.168.1.150-9081-2) HHH000400: Using dialect: org.hibernate.dialect.Oracle10gDialect
13:03:17,543 INFO  [org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory] (ajp--192.168.1.150-9081-2) HHH000397: Using ASTQueryTranslatorFactory
13:03:17,581 INFO  [org.hibernate.validator.util.Version] (ajp--192.168.1.150-9081-2) Hibernate Validator 4.2.0.Final
13:03:18,134 INFO  [stdout] (ajp--192.168.1.150-9081-4) INSERT 1fhk6fys4sd6j 9363
13:03:18,135 INFO  [stdout] (ajp--192.168.1.150-9081-8) INSERT re87jkorw73z 9832
13:03:18,141 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 1ndgkrn77u2it 9820
13:03:18,142 INFO  [stdout] (ajp--192.168.1.150-9081-5) INSERT 1w2cp0u6lk89x 3356
13:03:18,143 INFO  [stdout] (ajp--192.168.1.150-9081-3) INSERT 17rjkpgd9eez9 5957
13:03:18,151 INFO  [stdout] (ajp--192.168.1.150-9081-9) UPDATE 1ke3x8flbzh2l 2468
13:03:18,152 INFO  [stdout] (ajp--192.168.1.150-9081-2) UPDATE b0hsnirgzegu 3428
13:03:18,208 INFO  [stdout] (ajp--192.168.1.150-9081-11) UPDATE 346ubsid1bcj 5107
13:03:18,217 INFO  [stdout] (ajp--192.168.1.150-9081-10) INSERT hhq72mspnlm6 3525
13:03:18,226 INFO  [stdout] (ajp--192.168.1.150-9081-3) INSERT 18z5vn487oq68 1145
13:03:18,229 INFO  [stdout] (ajp--192.168.1.150-9081-8) UPDATE 1g2v0qag7awbe 9292
13:03:18,241 INFO  [stdout] (ajp--192.168.1.150-9081-10) UPDATE 1iedrjwclb888 5015
13:03:18,244 INFO  [stdout] (ajp--192.168.1.150-9081-3) UPDATE h4nv7n4u4imi 431
13:03:18,256 INFO  [stdout] (ajp--192.168.1.150-9081-10) INSERT 146qoll1n1jpt 1994
13:03:18,265 INFO  [stdout] (ajp--192.168.1.150-9081-10) UPDATE sca0ze1e6a55 4618
13:03:18,273 INFO  [stdout] (ajp--192.168.1.150-9081-10) UPDATE neyi37tgskek 7683
13:03:18,331 INFO  [stdout] (ajp--192.168.1.150-9081-10) UPDATE 1itpm85ux9vu8 8101
13:03:18,344 INFO  [stdout] (ajp--192.168.1.150-9081-3) UPDATE 1hzxrlkqyxwvz 2642
13:03:18,350 INFO  [stdout] (ajp--192.168.1.150-9081-11) INSERT 1qsg68xingq39 5017
13:03:18,365 INFO  [stdout] (ajp--192.168.1.150-9081-8) INSERT 1a42lp62sxldw 6196
13:03:18,365 INFO  [stdout] (ajp--192.168.1.150-9081-10) UPDATE 1sd2tjmkikrwd 3633
13:03:18,368 INFO  [stdout] (ajp--192.168.1.150-9081-11) UPDATE 12326fhs1lgs6 1466
13:03:18,369 INFO  [stdout] (ajp--192.168.1.150-9081-3) INSERT 8z47zy6uo0lr 9042

JMS

Let us add some messaging as well. To this end, we add the following to the Spring configuration

<beans ...>
    <bean id="company" class="model.logic.CompanyBean">
        ...
        <property name="jmsTemplate" ref="jmstemplate"/>
    </bean>
	...
    <jee:jndi-lookup id="connectionfactory" jndi-name="java:/ConnectionFactory" resource-ref="false"/>
    <jee:jndi-lookup id="destination" jndi-name="java:/queue/test" resource-ref="false"/>
    <bean id="jmstemplate" class="org.springframework.jms.core.JmsTemplate">
        <property name="connectionFactory" ref="connectionfactory"/>
        <property name="defaultDestination" ref="destination"/>
    </bean>
    <bean class="org.springframework.jms.listener.SimpleMessageListenerContainer">
        <property name="connectionFactory" ref="connectionfactory"/>
        <property name="destination" ref="destination"/>
        <property name="messageListener" ref="companymdp"/>
    </bean>
    <bean id="companymdp" class="model.logic.CompanyMDP"/>
</beans>

To send messages we edit the CompanyBean class as follows, i.e., add a property called jmsTemplate, add a method sendMessage that is called from the insertPerson and removePerson methods

package model.logic;

import model.entities.Person;
import org.hibernate.Session;
import org.hibernate.SessionFactory;
import org.springframework.jms.core.JmsTemplate;
import org.springframework.jms.core.MessageCreator;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;

import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.ObjectMessage;

@Transactional(propagation = Propagation.REQUIRED, readOnly = false)
public class CompanyBean implements Company {

    private SessionFactory sessionFactory;
	private JmsTemplate jmsTemplate;

    public SessionFactory getSessionFactory() {
        return sessionFactory;
    }

    public void setSessionFactory(SessionFactory sessionFactory) {
        this.sessionFactory = sessionFactory;
    }

    public JmsTemplate getJmsTemplate() {
        return jmsTemplate;
    }

    public void setJmsTemplate(JmsTemplate jmsTemplate) {
        this.jmsTemplate = jmsTemplate;
    }

    public void insertPerson(Person person) {
        Person temporaryPerson = findPerson(person.getSofinummer());
        if (temporaryPerson == null) {
            System.out.println("INSERT " + person);
            getCurrentSession().save(person);
            sendMessage(person);
        } else {
            updatePerson(person);
        }
    }

    public void removePerson(Integer sofinummer) {
        Person temporaryPerson = findPerson(sofinummer);
        if (temporaryPerson != null) {
            System.out.println("REMOVE " + temporaryPerson);
            getCurrentSession().delete(temporaryPerson);
            sendMessage(temporaryPerson);
        }
    }

    public void updatePerson(Person person) {
        System.out.println("UPDATE " + person);
        getCurrentSession().merge(person);
    }

    private Person findPerson(Integer sofinummer) {
        return (Person) getCurrentSession().get(Person.class, sofinummer);
    }

    private final Session getCurrentSession() {
        return getSessionFactory().getCurrentSession();
    }

    private void sendMessage(final Person person) {
        getJmsTemplate().send(new MessageCreator(){
            public Message createMessage(javax.jms.Session session) throws JMSException {
                ObjectMessage message = session.createObjectMessage();
                message.setObject(person);
                return message;
            }
        });
    }
}

Before we can deploy the application, we have to make sure the javax.jms.api module is added to the org.springframework module, i.e.,

<module xmlns="urn:jboss:module:1.1" name="org.springframework">
    <resources>
	<resource-root path="aopalliance-1.0.jar"/>
	<resource-root path="cglib-2.2.2.jar"/>
	<resource-root path="commons-logging-1.0.4.jar"/>
	<resource-root path="org.springframework.aop-3.1.2.RELEASE.jar"/>
	<resource-root path="org.springframework.asm-3.1.2.RELEASE.jar"/>
	<resource-root path="org.springframework.beans-3.1.2.RELEASE.jar"/>
	<resource-root path="org.springframework.context-3.1.2.RELEASE.jar"/>
	<resource-root path="org.springframework.core-3.1.2.RELEASE.jar"/>
	<resource-root path="org.springframework.expression-3.1.2.RELEASE.jar"/>
	<resource-root path="org.springframework.jdbc-3.1.2.RELEASE.jar"/>
	<resource-root path="org.springframework.jms-3.1.2.RELEASE.jar"/>
	<resource-root path="org.springframework.orm-3.1.2.RELEASE.jar"/>
	<resource-root path="org.springframework.transaction-3.1.2.RELEASE.jar"/>
	<resource-root path="org.springframework.web-3.1.2.RELEASE.jar"/>
    </resources>
    <dependencies>
        <module name="javax.api" export="true"/>
        <module name="org.apache.log4j" export="true"/>
        <module name="org.antlr" export="true"/>
        <module name="org.dom4j" export="true"/>
        <module name="org.hibernate" export="true"/>
        <module name="javax.persistence.api" export="true"/>
        <module name="org.javassist" export="true"/>
        <module name="org.jboss.logging" export="true"/>
        <module name="javax.transaction.api" export="true"/>
        <module name="javax.jms.api" export="true"/>
    </dependencies>
</module>

With the module in place, restart the servers and deploy the application, i.e.,

[domain@192.168.1.150:9999 /] undeploy SpringHibernate.war --server-groups=cluster-group
[domain@192.168.1.150:9999 /] /host=jboss/server-config=cluster-server2:restart
{
    "outcome" => "success",
    "result" => "STARTING"
}
[domain@192.168.1.150:9999 /] /host=jboss/server-config=cluster-server1:restart
{
    "outcome" => "success",
    "result" => "STARTING"
}
[domain@192.168.1.150:9999 /] deploy /home/jboss/jboss-as-7.1.0.Final/deploy/springhibernate/SpringHibernate.war --server-groups=cluster-group

The application can be reached at http://192.168.1.150:8888/SpringHibernate/testservlet. After doing a few requests the following is observed in the logging:

# LOGGING CLUSTER-SERVER1
15:45:13,723 INFO  [org.springframework.context.support.ClassPathXmlApplicationContext] (ajp--192.168.1.150-9080-5) Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@1b7fc253: startup date [Tue Aug 14 15:45:13 CEST 2012]; root of context hierarchy
15:45:13,760 INFO  [org.springframework.beans.factory.xml.XmlBeanDefinitionReader] (ajp--192.168.1.150-9080-5) Loading XML bean definitions from class path resource [spring-config.xml]
15:45:14,119 INFO  [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer] (ajp--192.168.1.150-9080-5) Loading properties file from class path resource [spring.properties]
15:45:14,141 INFO  [org.springframework.beans.factory.support.DefaultListableBeanFactory] (ajp--192.168.1.150-9080-5) Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@411167f3: defining beans [company,sessionfactory,datasource,propertyConfigurer,transactionManager,org.springframework.aop.config.internalAutoProxyCreator,org.springframework.transaction.annotation.AnnotationTransactionAttributeSource#0,org.springframework.transaction.interceptor.TransactionInterceptor#0,org.springframework.transaction.config.internalTransactionAdvisor,connectionfactory,destination,jmstemplate,org.springframework.jms.listener.SimpleMessageListenerContainer#0,companymdp]; root of factory hierarchy
15:45:14,204 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp--192.168.1.150-9080-5) Using JTA UserTransaction: org.jboss.tm.usertx.client.ServerVMClientUserTransaction@539546ee
15:45:14,204 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp--192.168.1.150-9080-5) Using JTA TransactionManager: com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate@3839029b
15:45:14,205 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp--192.168.1.150-9080-5) Using JTA TransactionSynchronizationRegistry: com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionSynchronizationRegistryImple@2818de48
15:45:14,291 INFO  [org.hibernate.annotations.common.Version] (ajp--192.168.1.150-9080-5) HCANN000001: Hibernate Commons Annotations {4.0.1.Final}
15:45:14,296 INFO  [org.hibernate.Version] (ajp--192.168.1.150-9080-5) HHH000412: Hibernate Core {4.0.1.Final}
15:45:14,297 INFO  [org.hibernate.cfg.Environment] (ajp--192.168.1.150-9080-5) HHH000206: hibernate.properties not found
15:45:14,299 INFO  [org.hibernate.cfg.Environment] (ajp--192.168.1.150-9080-5) HHH000021: Bytecode provider name : javassist
15:45:14,332 WARN  [org.hibernate.internal.util.xml.DTDEntityResolver] (ajp--192.168.1.150-9080-5) HHH000223: Recognized obsolete hibernate namespace http://hibernate.sourceforge.net/. Use namespace http://www.hibernate.org/dtd/ instead. Refer to Hibernate 3.6 Migration Guide!
15:45:14,549 INFO  [org.hibernate.dialect.Dialect] (ajp--192.168.1.150-9080-5) HHH000400: Using dialect: org.hibernate.dialect.Oracle10gDialect
15:45:14,567 INFO  [org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory] (ajp--192.168.1.150-9080-5) HHH000397: Using ASTQueryTranslatorFactory
15:45:14,603 INFO  [org.hibernate.validator.util.Version] (ajp--192.168.1.150-9080-5) Hibernate Validator 4.2.0.Final
15:45:14,957 INFO  [org.springframework.context.support.DefaultLifecycleProcessor] (ajp--192.168.1.150-9080-5) Starting beans in phase 2147483647
15:45:15,124 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT fe1g8llttlid 9594
15:45:15,185 INFO  [stdout] (Thread-1 (HornetQ-client-global-threads-1978193102)) RECEIVED OBJECT MESSAGE fe1g8llttlid 9594
15:47:17,962 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE 1ruzaxp3ovrza 5434
15:47:18,507 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 13uem5a3s0tr0 4176
15:47:18,555 INFO  [stdout] (Thread-2 (HornetQ-client-global-threads-1978193102)) RECEIVED OBJECT MESSAGE 13uem5a3s0tr0 4176
15:47:18,573 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT o0on52y2q6rc 7154
15:47:18,581 INFO  [stdout] (Thread-2 (HornetQ-client-global-threads-1978193102)) RECEIVED OBJECT MESSAGE o0on52y2q6rc 7154
15:47:18,686 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE ji9vigyqm6tn 1580
15:47:18,721 INFO  [stdout] (ajp--192.168.1.150-9080-2) INSERT 501w5lrllrl1 6984
15:47:18,734 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-1978193102)) RECEIVED OBJECT MESSAGE 501w5lrllrl1 6984
15:47:18,795 INFO  [stdout] (ajp--192.168.1.150-9080-2) INSERT q8i3p3v69ta9 9696
15:47:18,808 INFO  [stdout] (Thread-1 (HornetQ-client-global-threads-1978193102)) RECEIVED OBJECT MESSAGE q8i3p3v69ta9 9696
15:47:18,870 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE 187qlr278gvk2 7989
15:47:18,944 INFO  [stdout] (ajp--192.168.1.150-9080-2) INSERT af0n7nhmhtn4 7354
15:47:18,954 INFO  [stdout] (Thread-1 (HornetQ-client-global-threads-1978193102)) RECEIVED OBJECT MESSAGE af0n7nhmhtn4 7354
15:47:19,773 INFO  [stdout] (ajp--192.168.1.150-9080-2) INSERT 1rki73vjyxick 4429
15:47:19,797 INFO  [stdout] (Thread-2 (HornetQ-client-global-threads-1978193102)) RECEIVED OBJECT MESSAGE 1rki73vjyxick 4429
15:47:19,805 INFO  [stdout] (ajp--192.168.1.150-9080-2) INSERT 7t977akfo2gv 6383
15:47:19,813 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-1978193102)) RECEIVED OBJECT MESSAGE 7t977akfo2gv 6383
15:47:20,265 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 18nupcmjbywyz 9416
15:47:20,272 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-1978193102)) RECEIVED OBJECT MESSAGE 18nupcmjbywyz 9416
15:47:20,288 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE ul6rda8yg61u 1891
15:47:20,298 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE 1fotcalk9shsq 1717
15:47:20,306 INFO  [stdout] (ajp--192.168.1.150-9080-2) INSERT fz2zkhr4bw9g 4969
15:47:20,330 INFO  [stdout] (Thread-2 (HornetQ-client-global-threads-1978193102)) RECEIVED OBJECT MESSAGE fz2zkhr4bw9g 4969
15:47:20,332 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT nqmx27r3vcoz 7686
15:47:20,357 INFO  [stdout] (ajp--192.168.1.150-9080-4) INSERT 14znb00a9jipx 363
15:47:20,374 INFO  [stdout] (Thread-1 (HornetQ-client-global-threads-1978193102)) RECEIVED OBJECT MESSAGE nqmx27r3vcoz 7686
15:47:20,389 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-1978193102)) RECEIVED OBJECT MESSAGE 14znb00a9jipx 363
15:47:20,426 INFO  [stdout] (ajp--192.168.1.150-9080-9) INSERT 8ptga7h7jpeb 6888
15:47:20,428 INFO  [stdout] (ajp--192.168.1.150-9080-7) UPDATE cuy0x72tv27s 4669
15:47:20,447 INFO  [stdout] (Thread-1 (HornetQ-client-global-threads-1978193102)) RECEIVED OBJECT MESSAGE 8ptga7h7jpeb 6888
15:47:20,465 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 3mws48gl5lz8 4333
15:47:20,467 INFO  [stdout] (ajp--192.168.1.150-9080-4) INSERT tfcnb3v0zt7p 7984
15:47:20,477 INFO  [stdout] (ajp--192.168.1.150-9080-8) INSERT 1ahyu3xkspnxr 3201
15:47:20,496 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-1978193102)) RECEIVED OBJECT MESSAGE 3mws48gl5lz8 4333
15:47:20,497 INFO  [stdout] (ajp--192.168.1.150-9080-6) UPDATE 137a6srifr1gz 8905
15:47:20,501 INFO  [stdout] (ajp--192.168.1.150-9080-10) UPDATE 16epgzgkpuhyl 3052
15:47:20,528 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE x59b33fx74ec 2535
15:47:20,528 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-1978193102)) RECEIVED OBJECT MESSAGE 1ahyu3xkspnxr 3201
15:47:20,531 INFO  [stdout] (ajp--192.168.1.150-9080-7) INSERT wkl0ag7vz56k 8274
15:47:20,537 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-1978193102)) RECEIVED OBJECT MESSAGE tfcnb3v0zt7p 7984
15:47:20,543 INFO  [stdout] (Thread-4 (HornetQ-client-global-threads-1978193102)) RECEIVED OBJECT MESSAGE wkl0ag7vz56k 8274

# LOGGING CLUSTER-SERVER2
15:47:18,658 INFO  [org.springframework.context.support.ClassPathXmlApplicationContext] (ajp-axis-into-ict.nl-192.168.1.150-9081-2) Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@28b3cf4a: startup date [Tue Aug 14 15:47:18 CEST 2012]; root of context hierarchy
15:47:18,696 INFO  [org.springframework.beans.factory.xml.XmlBeanDefinitionReader] (ajp-axis-into-ict.nl-192.168.1.150-9081-2) Loading XML bean definitions from class path resource [spring-config.xml]
15:47:19,005 INFO  [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer] (ajp-axis-into-ict.nl-192.168.1.150-9081-2) Loading properties file from class path resource [spring.properties]
15:47:19,025 INFO  [org.springframework.beans.factory.support.DefaultListableBeanFactory] (ajp-axis-into-ict.nl-192.168.1.150-9081-2) Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@3ff39e73: defining beans [company,sessionfactory,datasource,propertyConfigurer,transactionManager,org.springframework.aop.config.internalAutoProxyCreator,org.springframework.transaction.annotation.AnnotationTransactionAttributeSource#0,org.springframework.transaction.interceptor.TransactionInterceptor#0,org.springframework.transaction.config.internalTransactionAdvisor,connectionfactory,destination,jmstemplate,org.springframework.jms.listener.SimpleMessageListenerContainer#0,companymdp]; root of factory hierarchy
15:47:19,087 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp-axis-into-ict.nl-192.168.1.150-9081-2) Using JTA UserTransaction: org.jboss.tm.usertx.client.ServerVMClientUserTransaction@5935b50c
15:47:19,088 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp-axis-into-ict.nl-192.168.1.150-9081-2) Using JTA TransactionManager: com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate@5b33a7af
15:47:19,090 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp-axis-into-ict.nl-192.168.1.150-9081-2) Using JTA TransactionSynchronizationRegistry: com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionSynchronizationRegistryImple@54cf76c8
15:47:19,177 INFO  [org.hibernate.annotations.common.Version] (ajp-axis-into-ict.nl-192.168.1.150-9081-2) HCANN000001: Hibernate Commons Annotations {4.0.1.Final}
15:47:19,182 INFO  [org.hibernate.Version] (ajp-axis-into-ict.nl-192.168.1.150-9081-2) HHH000412: Hibernate Core {4.0.1.Final}
15:47:19,184 INFO  [org.hibernate.cfg.Environment] (ajp-axis-into-ict.nl-192.168.1.150-9081-2) HHH000206: hibernate.properties not found
15:47:19,186 INFO  [org.hibernate.cfg.Environment] (ajp-axis-into-ict.nl-192.168.1.150-9081-2) HHH000021: Bytecode provider name : javassist
15:47:19,221 WARN  [org.hibernate.internal.util.xml.DTDEntityResolver] (ajp-axis-into-ict.nl-192.168.1.150-9081-2) HHH000223: Recognized obsolete hibernate namespace http://hibernate.sourceforge.net/. Use namespace http://www.hibernate.org/dtd/ instead. Refer to Hibernate 3.6 Migration Guide!
15:47:19,433 INFO  [org.hibernate.dialect.Dialect] (ajp-axis-into-ict.nl-192.168.1.150-9081-2) HHH000400: Using dialect: org.hibernate.dialect.Oracle10gDialect
15:47:19,454 INFO  [org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory] (ajp-axis-into-ict.nl-192.168.1.150-9081-2) HHH000397: Using ASTQueryTranslatorFactory
15:47:19,486 INFO  [org.hibernate.validator.util.Version] (ajp-axis-into-ict.nl-192.168.1.150-9081-2) Hibernate Validator 4.2.0.Final
15:47:19,893 INFO  [org.springframework.context.support.DefaultLifecycleProcessor] (ajp-axis-into-ict.nl-192.168.1.150-9081-2) Starting beans in phase 2147483647
15:47:20,126 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-5) INSERT 1uygcxcowtu9a 331
15:47:20,127 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-6) INSERT mow4srfubly0 6092
15:47:20,180 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-4) INSERT 45ctomltbfnj 7350
15:47:20,182 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-3) INSERT 2bfcfctogwas 9945
15:47:20,186 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-2) INSERT 1ghndct2k38fq 7215
15:47:20,190 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-8) UPDATE 1q6wykxdwe57w 8885
15:47:20,191 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-9) UPDATE edte8od9xrou 4704
15:47:20,192 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-10) UPDATE 1adm2wi7sosp0 3801
15:47:20,194 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-11) INSERT 1cof2wpo05qux 8205
15:47:20,195 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-7) INSERT 1jq47i7b26tvp 609
15:47:20,307 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-7) INSERT 1qxeatjtlqzx5 3722
15:47:20,313 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-12) UPDATE cztialmryr6t 2306
15:47:20,323 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-699447810)) RECEIVED OBJECT MESSAGE 45ctomltbfnj 7350
15:47:20,335 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-3) UPDATE s964r48d3sen 6153
15:47:20,343 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-8) UPDATE g9jng4dhqtha 8360
15:47:20,359 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-12) INSERT 1fs7d2l62ags4 9334
15:47:20,363 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-699447810)) RECEIVED OBJECT MESSAGE 1ghndct2k38fq 7215
15:47:20,372 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-8) UPDATE 1fac6z0zdhxfo 4796
15:47:20,377 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-699447810)) RECEIVED OBJECT MESSAGE 1jq47i7b26tvp 609
15:47:20,378 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-5) UPDATE 1vkso90xvg1kf 3038
15:47:20,381 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-3) INSERT b5jtmtrok3ay 9950
15:47:20,385 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-12) INSERT 1eob8au4ivjaq 7646
15:47:20,394 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-699447810)) RECEIVED OBJECT MESSAGE 1uygcxcowtu9a 331
15:47:20,400 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-5) INSERT g3tpmc15n1pw 7940
15:47:20,403 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-699447810)) RECEIVED OBJECT MESSAGE 1cof2wpo05qux 8205
15:47:20,412 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-699447810)) RECEIVED OBJECT MESSAGE mow4srfubly0 6092
15:47:20,419 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-699447810)) RECEIVED OBJECT MESSAGE 2bfcfctogwas 9945
15:47:20,421 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-5) INSERT ub7uqkgxgvkn 7389
15:47:20,423 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-699447810)) RECEIVED OBJECT MESSAGE 1qxeatjtlqzx5 3722
15:47:20,433 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-699447810)) RECEIVED OBJECT MESSAGE 1fs7d2l62ags4 9334
15:47:20,437 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-12) INSERT 1o9uvk6v2q3yh 6427
15:47:20,445 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-699447810)) RECEIVED OBJECT MESSAGE b5jtmtrok3ay 9950
15:47:20,454 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-699447810)) RECEIVED OBJECT MESSAGE 1eob8au4ivjaq 7646
15:47:20,459 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-5) UPDATE ztxczcv9lczr 6350
15:47:20,467 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-699447810)) RECEIVED OBJECT MESSAGE g3tpmc15n1pw 7940
15:47:20,472 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-3) INSERT 1e50cqtnnx877 4627
15:47:20,492 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9081-12) INSERT gzgnffn3j4jv 7794
15:47:20,495 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-699447810)) RECEIVED OBJECT MESSAGE ub7uqkgxgvkn 7389
15:47:20,507 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-699447810)) RECEIVED OBJECT MESSAGE 1o9uvk6v2q3yh 6427
15:47:20,511 INFO  [stdout] (Thread-3 (HornetQ-client-global-threads-699447810)) RECEIVED OBJECT MESSAGE 1e50cqtnnx877 4627
15:47:20,520 INFO  [stdout] (Thread-1 (HornetQ-client-global-threads-699447810)) RECEIVED OBJECT MESSAGE gzgnffn3j4jv 7794

Transactions

In the example above we have used a SimpleMessageListenerContainer. A message listener container is used to receive messages from a JMS message queue and drive the MessageListener that is injected into it. The listener container is responsible for all threading of message reception and dispatches into the listener for processing. A message listener container is the intermediary between an MDP and a messaging provider, and takes care of registering to receive messages, participating in transactions, resource acquisition and release, exception conversion and suchlike. There are two standard JMS message listener containers packaged with Spring, each with its specialised feature set:

• SimpleMessageListenerContainer – This message listener container is the simpler of the two standard flavors. It creates a fixed number of JMS sessions and consumers at startup, registers the listener using the standard JMS MessageConsumer.setMessageListener method, and leaves it up the JMS provider to perform listener callbacks. This variant does not allow for dynamic adaption to runtime demands or for participation in externally managed transactions. Compatibility-wise, it stays very close to the spirit of the standalone JMS specification – but is generally not compatible with Java EE’s JMS restrictions.
• DefaultMessageListenerContainer – This message listener container is the one used in most cases. In contrast to SimpleMessageListenerContainer, this container variant does allow for dynamic adaption to runtime demands and is able to participate in externally managed transactions. Each received message is registered with an XA transaction when configured with a JtaTransactionManager; so processing may take advantage of XA transaction semantics. This listener container strikes a good balance between low requirements on the JMS provider, advanced functionality such as transaction participation, and compatibility with Java EE environments.

To use the second flavor in our set-up, we can change the Spring configuration to

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:tx="http://www.springframework.org/schema/tx"
       xmlns:jee="http://www.springframework.org/schema/jee"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
       http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
       http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-3.0.xsd">
    <bean id="company" class="model.logic.CompanyBean">
        <property name="sessionFactory" ref="sessionfactory"/>
        <property name="jmsTemplate" ref="jmstemplate"/>
    </bean>
	<!-- Hibernate SessionFactory config -->
    <bean id="sessionfactory" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
        <property name="dataSource" ref="datasource"/>
        <property name="jtaTransactionManager" ref="transactionManager"/>
        <property name="mappingResources">
            <list>
                <value>model/entities/person.hbm.xml</value>
            </list>
        </property>
        <property name="hibernateProperties">
            <props>
                <prop key="hibernate.dialect">${hibernate.dialect}</prop>
                <prop key="hibernate.current_session_context_class">${hibernate.current_session_context_class}</prop>
                <prop key="hibernate.listeners.envers.autoRegister">${hibernate.listeners.envers.autoRegister}</prop>
                <prop key="hibernate.show_sql">${hibernate.show_sql}</prop>
                <prop key="hibernate.format_sql">${hibernate.format_sql}</prop>
            </props>
        </property>
    </bean>
	<!-- Transaction config -->
	<bean id="transactionManager" class="org.springframework.transaction.jta.JtaTransactionManager"/>
	<tx:annotation-driven transaction-manager="transactionManager"/>
	<!-- Resource look-ups -->
    <jee:jndi-lookup id="datasource" jndi-name="java:/jdbc/OracleDS" resource-ref="false"/>
    <jee:jndi-lookup id="connectionfactory" jndi-name="java:/JmsXA" resource-ref="false"/>
    <jee:jndi-lookup id="destination" jndi-name="java:/queue/test" resource-ref="false"/>
    <!-- JMS config -->
	<bean id="jmstemplate" class="org.springframework.jms.core.JmsTemplate">
        <property name="connectionFactory" ref="connectionfactory"/>
        <property name="defaultDestination" ref="destination"/>
    </bean>
    <bean class="org.springframework.jms.listener.DefaultMessageListenerContainer">
        <property name="connectionFactory" ref="connectionfactory"/>
        <property name="destination" ref="destination"/>
        <property name="transactionManager" ref="transactionManager"/>
        <property name="messageListener" ref="companymdp"/>
    </bean>
    <bean id="companymdp" class="model.logic.CompanyMDP"/>
	<!-- Extra -->
    <bean id="propertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="locations">
            <list>
                <value>classpath:spring.properties</value>
            </list>
        </property>
    </bean>
</beans>

Note that code remains as it was. We have used the following JBoss configuration that shows how the used resources are configured

<domain xmlns="urn:jboss:domain:1.1">
	...
    <profiles>
        <profile name="cluster">
            <subsystem xmlns="urn:jboss:domain:datasources:1.0">
                <datasources>
                    <datasource jta="true" jndi-name="java:/jdbc/OracleDS" pool-name="OracleDS" enabled="true" use-java-context="true" use-ccm="true">
                        <connection-url>jdbc:oracle:thin:@192.168.1.60:1521:orcl11</connection-url>
                        <driver>oracle</driver>
                        <pool>
                            <min-pool-size>1</min-pool-size>
                            <max-pool-size>15</max-pool-size>
                            <prefill>true</prefill>
                            <use-strict-min>true</use-strict-min>
                        </pool>
                        <security>
                            <user-name>example</user-name>
                            <password>example</password>
                        </security>
                        <timeout>
                            <idle-timeout-minutes>0</idle-timeout-minutes>
                            <query-timeout>600</query-timeout>
                        </timeout>
                        <statement>
                            <prepared-statement-cache-size>10</prepared-statement-cache-size>
                        </statement>
                    </datasource>
                    <drivers>
                        <driver name="oracle" module="com.oracle.database">
                            <driver-class>oracle.jdbc.OracleDriver</driver-class>
                            <xa-datasource-class>oracle.jdbc.xa.client.OracleXADataSource</xa-datasource-class>
                        </driver>
                    </drivers>
                </datasources>
            </subsystem>
            <subsystem xmlns="urn:jboss:domain:messaging:1.1">
                <hornetq-server>
					...
                    <jms-connection-factories>
						...
                        <pooled-connection-factory name="hornetq-ra">
                            <transaction mode="xa"/>
                            <connectors>
                                <connector-ref connector-name="in-vm"/>
                            </connectors>
                            <entries>
                                <entry name="java:/JmsXA"/>
                            </entries>
                        </pooled-connection-factory>
                    </jms-connection-factories>
                    <jms-destinations>
                        <jms-queue name="testQueue">
                            <entry name="java:/queue/test"/>
                            <entry name="java:jboss/exported/jms/queue/test"/>
                        </jms-queue>
						...
                    </jms-destinations>
                </hornetq-server>
            </subsystem>
			...
        </profile>
    </profiles>
	...
    <deployments>
        <deployment name="LoadTest6.ear" runtime-name="LoadTest6.ear">
            <content sha1="161f51dde7f085c822cc4c68b306d57f1bee902d"/>
        </deployment>
        <deployment name="SpringHibernate.war" runtime-name="SpringHibernate.war">
            <content sha1="574d0617d92d8439844633557937be754a134370"/>
        </deployment>
    </deployments>
    <server-groups>
		...
        <server-group name="cluster-group" profile="cluster">
            <jvm name="default"/>
            <socket-binding-group ref="cluster-sockets"/>
            <deployments>
                <deployment name="LoadTest6.ear" runtime-name="LoadTest6.ear"/>
                <deployment name="SpringHibernate.war" runtime-name="SpringHibernate.war" enabled="false"/>
            </deployments>
        </server-group>
    </server-groups>
</domain>

Note that we have set the mode of the connection factory to XA (as is required when using the JtaTransactionManager and the DefaultMessageListenerContainer). One important thing to remember when using the DefaultMessageListenerContainer is that the default sessionAcknowledgeMode setting is AUTO_ACKNOWLEDGE (see here), so it not smart to use message.acknowledge in the MDP, otherwise we will run into

09:33:20,815 INFO  [org.springframework.context.support.ClassPathXmlApplicationContext] (ajp-axis-into-ict.nl-192.168.1.150-9080-7) Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@74359b24: startup date [Fri Aug 17 09:33:20 CEST 2012]; root of context hierarchy
09:33:20,945 INFO  [org.springframework.beans.factory.xml.XmlBeanDefinitionReader] (ajp-axis-into-ict.nl-192.168.1.150-9080-7) Loading XML bean definitions from class path resource [spring-config.xml]
09:33:21,415 INFO  [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer] (ajp-axis-into-ict.nl-192.168.1.150-9080-7) Loading properties file from class path resource [spring.properties]
09:33:21,441 INFO  [org.springframework.beans.factory.support.DefaultListableBeanFactory] (ajp-axis-into-ict.nl-192.168.1.150-9080-7) Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@6c11e58: defining beans [company,sessionfactory,datasource,propertyConfigurer,transactionManager,org.springframework.aop.config.internalAutoProxyCreator,org.springframework.transaction.annotation.AnnotationTransactionAttributeSource#0,org.springframework.transaction.interceptor.TransactionInterceptor#0,org.springframework.transaction.config.internalTransactionAdvisor,connectionfactory,destination,jmstemplate,org.springframework.jms.listener.DefaultMessageListenerContainer#0,companymdp]; root of factory hierarchy
09:33:21,535 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp-axis-into-ict.nl-192.168.1.150-9080-7) Using JTA UserTransaction: org.jboss.tm.usertx.client.ServerVMClientUserTransaction@7e0cf590
09:33:21,536 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp-axis-into-ict.nl-192.168.1.150-9080-7) Using JTA TransactionManager: com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate@4d50b06b
09:33:21,537 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp-axis-into-ict.nl-192.168.1.150-9080-7) Using JTA TransactionSynchronizationRegistry: com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionSynchronizationRegistryImple@38d060ac
09:33:21,784 INFO  [org.hibernate.annotations.common.Version] (ajp-axis-into-ict.nl-192.168.1.150-9080-7) HCANN000001: Hibernate Commons Annotations {4.0.1.Final}
09:33:21,792 INFO  [org.hibernate.Version] (ajp-axis-into-ict.nl-192.168.1.150-9080-7) HHH000412: Hibernate Core {4.0.1.Final}
09:33:21,794 INFO  [org.hibernate.cfg.Environment] (ajp-axis-into-ict.nl-192.168.1.150-9080-7) HHH000206: hibernate.properties not found
09:33:21,796 INFO  [org.hibernate.cfg.Environment] (ajp-axis-into-ict.nl-192.168.1.150-9080-7) HHH000021: Bytecode provider name : javassist
09:33:21,907 WARN  [org.hibernate.internal.util.xml.DTDEntityResolver] (ajp-axis-into-ict.nl-192.168.1.150-9080-7) HHH000223: Recognized obsolete hibernate namespace http://hibernate.sourceforge.net/. Use namespace http://www.hibernate.org/dtd/ instead. Refer to Hibernate 3.6 Migration Guide!
09:33:22,517 INFO  [org.hibernate.dialect.Dialect] (ajp-axis-into-ict.nl-192.168.1.150-9080-7) HHH000400: Using dialect: org.hibernate.dialect.Oracle10gDialect
09:33:22,539 INFO  [org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory] (ajp-axis-into-ict.nl-192.168.1.150-9080-7) HHH000397: Using ASTQueryTranslatorFactory
09:33:22,652 INFO  [org.hibernate.validator.util.Version] (ajp-axis-into-ict.nl-192.168.1.150-9080-7) Hibernate Validator 4.2.0.Final
09:33:23,152 INFO  [org.springframework.context.support.DefaultLifecycleProcessor] (ajp-axis-into-ict.nl-192.168.1.150-9080-7) Starting beans in phase 2147483647
09:33:23,552 INFO  [stdout] (ajp-axis-into-ict.nl-192.168.1.150-9080-7) INSERT 1chrw6b78f2w9 9719
09:33:23,671 WARN  [org.springframework.jms.listener.DefaultMessageListenerContainer] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) Execution of JMS message listener failed, and no ErrorHandler has been set.: java.lang.RuntimeException: javax.jms.IllegalStateException: Non XA connection
    at model.logic.CompanyMDP.onMessage(CompanyMDP.java:16) [classes:]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:562) [org.springframework.jms-3.1.2.RELEASE.jar:3.1.2.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:500) [org.springframework.jms-3.1.2.RELEASE.jar:3.1.2.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:468) [org.springframework.jms-3.1.2.RELEASE.jar:3.1.2.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:326) [org.springframework.jms-3.1.2.RELEASE.jar:3.1.2.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:244) [org.springframework.jms-3.1.2.RELEASE.jar:3.1.2.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1071) [org.springframework.jms-3.1.2.RELEASE.jar:3.1.2.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1063) [org.springframework.jms-3.1.2.RELEASE.jar:3.1.2.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:960) [org.springframework.jms-3.1.2.RELEASE.jar:3.1.2.RELEASE]
    at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_31]
Caused by: javax.jms.IllegalStateException: Non XA connection
    at org.hornetq.ra.HornetQRASession.getSession(HornetQRASession.java:1246)
    at org.hornetq.ra.HornetQRAMessage.acknowledge(HornetQRAMessage.java:71)
    at model.logic.CompanyMDP.onMessage(CompanyMDP.java:13) [classes:]
    ... 9 more

Another thing worth mentioning is that within a JTA transaction, the parameters passed to create(Queue/Topic)Session(boolean transacted, int acknowledgeMode) method are not taken into account. Depending on the JavaEE transaction context, the container makes its own decisions on these values.

Let us deploy the application again and see what happens:

# LOGGING CLUSTER SERVER 1
14:03:01,180 INFO  [org.jboss.as.webservices] (ServerService Thread Pool -- 32) JBAS015537: Activating WebServices Extension
14:03:01,242 INFO  [org.jboss.as.security] (ServerService Thread Pool -- 36) JBAS013101: Activating Security Subsystem
14:03:01,245 INFO  [org.jboss.as.security] (MSC service thread 1-1) JBAS013100: Current PicketBox version=4.0.6.final
14:03:01,359 INFO  [org.jboss.as.naming] (ServerService Thread Pool -- 42) JBAS011800: Activating Naming Subsystem
14:03:01,420 INFO  [org.jboss.as.osgi] (ServerService Thread Pool -- 41) JBAS011940: Activating OSGi Subsystem
14:03:01,491 INFO  [org.jboss.as.naming] (MSC service thread 1-3) JBAS011802: Starting Naming Service
14:03:01,508 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-3) JBAS015400: Bound mail session [java:jboss/mail/Default]
14:03:01,628 INFO  [org.jboss.as.clustering.jgroups] (ServerService Thread Pool -- 49) JBAS010260: Activating JGroups subsystem.
14:03:01,792 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 55) JBAS010280: Activating Infinispan subsystem.
14:03:01,813 INFO  [org.jboss.as.jacorb] (ServerService Thread Pool -- 54) JBAS016300: Activating JacORB Subsystem
14:03:01,883 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 58) JBAS010403: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
14:03:01,894 INFO  [org.jboss.as.configadmin] (ServerService Thread Pool -- 59) JBAS016200: Activating ConfigAdmin Subsystem
14:03:01,913 INFO  [org.jboss.ws.common.management.AbstractServerConfig] (MSC service thread 1-3) JBoss Web Services - Stack CXF Server 4.0.1.GA
14:03:02,259 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 58) JBAS010403: Deploying JDBC-compliant driver class oracle.jdbc.OracleDriver (version 11.2)
14:03:02,275 INFO  [org.jboss.as.modcluster] (MSC service thread 1-2) JBAS011704: Mod_cluster uses default load balancer provider
14:03:02,389 INFO  [org.apache.coyote.http11.Http11AprProtocol] (MSC service thread 1-3) Starting Coyote HTTP/1.1 on http--192.168.1.150-8080
14:03:02,413 INFO  [org.jboss.modcluster.ModClusterService] (MSC service thread 1-2) Initializing mod_cluster 1.2.0.Final
14:03:02,471 INFO  [org.apache.coyote.ajp.AjpAprProtocol] (MSC service thread 1-4) Starting Coyote AJP/1.3 on ajp--192.168.1.150-9080
14:03:02,515 INFO  [org.jboss.as.connector] (MSC service thread 1-3) JBAS010408: Starting JCA Subsystem (JBoss IronJacamar 1.0.7.Final)
14:03:02,551 INFO  [org.jboss.modcluster.advertise.impl.AdvertiseListenerImpl] (MSC service thread 1-2) Listening to proxy advertisements on 224.0.1.105:23364
14:03:02,700 INFO  [org.jboss.as.jaxr] (MSC service thread 1-1) Binding JAXR ConnectionFactory: java:jboss/jaxr/ConnectionFactory
14:03:03,010 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-1) live server is starting with configuration HornetQ Configuration (clustered=false,backup=false,sharedStore=true,journalDirectory=/home/jboss/jboss-as-7.1.0.Final/domain/servers/cluster-server1/data/messagingjournal,bindingsDirectory=/home/jboss/jboss-as-7.1.0.Final/domain/servers/cluster-server1/data/messagingbindings,largeMessagesDirectory=/home/jboss/jboss-as-7.1.0.Final/domain/servers/cluster-server1/data/messaginglargemessages,pagingDirectory=/home/jboss/jboss-as-7.1.0.Final/domain/servers/cluster-server1/data/messagingpaging)
14:03:03,012 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-1) Waiting to obtain live lock
14:03:03,064 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 55) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be pasivated.
14:03:03,244 INFO  [org.hornetq.core.persistence.impl.journal.JournalStorageManager] (MSC service thread 1-1) Using AIO Journal
14:03:03,256 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 55) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be pasivated.
14:03:03,673 INFO  [org.jboss.as.jacorb] (MSC service thread 1-3) JBAS016330: CORBA ORB Service started
14:03:03,741 INFO  [org.hornetq.core.server.impl.AIOFileLockNodeManager] (MSC service thread 1-1) Waiting to obtain live lock
14:03:03,742 INFO  [org.hornetq.core.server.impl.AIOFileLockNodeManager] (MSC service thread 1-1) Live Server Obtained live lock
14:03:04,099 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-4) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]
14:03:04,107 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-2) JBAS010400: Bound data source [java:/jdbc/OracleDS]
14:03:04,133 INFO  [org.jboss.as.jacorb] (MSC service thread 1-3) JBAS016328: CORBA Naming Service started
14:03:04,698 INFO  [org.jboss.as.remoting] (MSC service thread 1-3) JBAS017100: Listening on axis-into-ict.nl/192.168.1.150:4447
14:03:05,157 INFO  [org.hornetq.core.remoting.impl.netty.NettyAcceptor] (MSC service thread 1-1) Started Netty Acceptor version 3.2.5.Final-a96d88c axis-into-ict.nl:5445 for CORE protocol
14:03:05,158 INFO  [org.hornetq.core.remoting.impl.netty.NettyAcceptor] (MSC service thread 1-1) Started Netty Acceptor version 3.2.5.Final-a96d88c axis-into-ict.nl:5455 for CORE protocol
14:03:05,159 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-1) Server is now live
14:03:05,160 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-1) HornetQ Server version 2.2.11.Final (HQ_2_2_11_FINAL_AS7, 122) [2e926cad-78e4-11e1-b1c7-000c2976c82d]) started
14:03:05,166 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-1) trying to deploy queue jms.queue.testQueue
14:03:05,180 INFO  [org.jboss.as.messaging] (MSC service thread 1-1) JBAS011601: Bound messaging object to jndi name java:jboss/exported/jms/queue/test
14:03:05,181 INFO  [org.jboss.as.messaging] (MSC service thread 1-1) JBAS011601: Bound messaging object to jndi name java:/queue/test
14:03:05,210 INFO  [org.jboss.as.messaging] (MSC service thread 1-4) JBAS011601: Bound messaging object to jndi name java:/ConnectionFactory
14:03:05,211 INFO  [org.jboss.as.messaging] (MSC service thread 1-3) JBAS011601: Bound messaging object to jndi name java:jboss/exported/jms/RemoteConnectionFactory
14:03:05,212 INFO  [org.jboss.as.messaging] (MSC service thread 1-3) JBAS011601: Bound messaging object to jndi name java:/RemoteConnectionFactory
14:03:05,213 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-2) trying to deploy queue jms.topic.testTopic
14:03:05,278 INFO  [org.jboss.as.messaging] (MSC service thread 1-2) JBAS011601: Bound messaging object to jndi name java:/topic/test
14:03:05,281 INFO  [org.jboss.as.messaging] (MSC service thread 1-2) JBAS011601: Bound messaging object to jndi name java:jboss/exported/jms/topic/test
14:03:05,288 INFO  [org.jboss.as.deployment.connector] (MSC service thread 1-3) JBAS010406: Registered connection factory java:/JmsXA
14:03:05,297 INFO  [org.hornetq.ra.HornetQResourceAdapter] (MSC service thread 1-3) HornetQ resource adaptor started
14:03:05,298 INFO  [org.jboss.as.connector.services.ResourceAdapterActivatorService$ResourceAdapterActivator] (MSC service thread 1-3) IJ020002: Deployed: file://RaActivatorhornetq-ra
14:03:05,309 INFO  [org.jboss.as.deployment.connector] (MSC service thread 1-4) JBAS010401: Bound JCA ConnectionFactory [java:/JmsXA]
14:03:05,397 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss AS 7.1.0.Final "Thunder" started in 7781ms - Started 169 of 292 services (121 services are passive or on-demand)
14:03:11,925 INFO  [org.jboss.modcluster.ModClusterService] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Engine [jboss.web] will use jvmRoute: 2c78c6e8-7325-3857-8bfc-d957f56114dc
14:04:34,838 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-2) JBAS015876: Starting deployment of "SpringHibernate.war"
14:04:35,410 INFO  [org.jboss.web] (MSC service thread 1-3) JBAS018210: Registering web context: /SpringHibernate
14:04:35,621 INFO  [org.jboss.as.server] (host-controller-connection-threads - 1) JBAS018559: Deployed "SpringHibernate.war"
14:05:11,613 INFO  [org.springframework.context.support.ClassPathXmlApplicationContext] (ajp--192.168.1.150-9080-7) Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@4e24faf1: startup date [Wed Aug 22 14:05:11 CEST 2012]; root of context hierarchy
14:05:11,651 INFO  [org.springframework.beans.factory.xml.XmlBeanDefinitionReader] (ajp--192.168.1.150-9080-7) Loading XML bean definitions from class path resource [spring-config.xml]
14:05:11,912 INFO  [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer] (ajp--192.168.1.150-9080-7) Loading properties file from class path resource [spring.properties]
14:05:11,932 INFO  [org.springframework.beans.factory.support.DefaultListableBeanFactory] (ajp--192.168.1.150-9080-7) Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@215a730d: defining beans [company,sessionfactory,transactionManager,org.springframework.aop.config.internalAutoProxyCreator,org.springframework.transaction.annotation.AnnotationTransactionAttributeSource#0,org.springframework.transaction.interceptor.TransactionInterceptor#0,org.springframework.transaction.config.internalTransactionAdvisor,datasource,connectionfactory,destination,jmstemplate,org.springframework.jms.listener.DefaultMessageListenerContainer#0,companymdp,propertyConfigurer]; root of factory hierarchy
14:05:11,990 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp--192.168.1.150-9080-7) Using JTA UserTransaction: org.jboss.tm.usertx.client.ServerVMClientUserTransaction@16360771
14:05:11,991 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp--192.168.1.150-9080-7) Using JTA TransactionManager: com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate@76efb90
14:05:11,992 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp--192.168.1.150-9080-7) Using JTA TransactionSynchronizationRegistry: com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionSynchronizationRegistryImple@500c8efc
14:05:12,080 INFO  [org.hibernate.annotations.common.Version] (ajp--192.168.1.150-9080-7) HCANN000001: Hibernate Commons Annotations {4.0.1.Final}
14:05:12,084 INFO  [org.hibernate.Version] (ajp--192.168.1.150-9080-7) HHH000412: Hibernate Core {4.0.1.Final}
14:05:12,086 INFO  [org.hibernate.cfg.Environment] (ajp--192.168.1.150-9080-7) HHH000206: hibernate.properties not found
14:05:12,088 INFO  [org.hibernate.cfg.Environment] (ajp--192.168.1.150-9080-7) HHH000021: Bytecode provider name : javassist
14:05:12,124 WARN  [org.hibernate.internal.util.xml.DTDEntityResolver] (ajp--192.168.1.150-9080-7) HHH000223: Recognized obsolete hibernate namespace http://hibernate.sourceforge.net/. Use namespace http://www.hibernate.org/dtd/ instead. Refer to Hibernate 3.6 Migration Guide!
14:05:12,355 INFO  [org.hibernate.dialect.Dialect] (ajp--192.168.1.150-9080-7) HHH000400: Using dialect: org.hibernate.dialect.Oracle10gDialect
14:05:12,376 INFO  [org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory] (ajp--192.168.1.150-9080-7) HHH000397: Using ASTQueryTranslatorFactory
14:05:12,414 INFO  [org.hibernate.validator.util.Version] (ajp--192.168.1.150-9080-7) Hibernate Validator 4.2.0.Final
14:05:12,790 INFO  [org.springframework.context.support.DefaultLifecycleProcessor] (ajp--192.168.1.150-9080-7) Starting beans in phase 2147483647
14:05:12,921 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE tdq5p6nrua78 881
14:05:13,111 INFO  [stdout] (ajp--192.168.1.150-9080-7) INSERT 3zbulkvppaht 1587
14:05:13,192 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 3zbulkvppaht 1587
14:05:19,838 INFO  [stdout] (ajp--192.168.1.150-9080-7) INSERT 3677kx0gg6p8 3335
14:05:19,878 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 3677kx0gg6p8 3335
14:05:20,597 INFO  [stdout] (ajp--192.168.1.150-9080-7) UPDATE qd3nfb51iltt 5472
14:05:21,157 INFO  [stdout] (ajp--192.168.1.150-9080-7) INSERT 170gdo1t8e5ko 3391
14:05:21,225 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 170gdo1t8e5ko 3391
14:05:21,242 INFO  [stdout] (ajp--192.168.1.150-9080-6) INSERT 1vaezfdq5d887 5839
14:05:21,294 INFO  [stdout] (ajp--192.168.1.150-9080-3) UPDATE jlha13ox3hwq 4651
14:05:21,321 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1vaezfdq5d887 5839
14:05:21,368 INFO  [stdout] (ajp--192.168.1.150-9080-4) UPDATE s84z6ya9j2w5 65
14:05:21,443 INFO  [stdout] (ajp--192.168.1.150-9080-4) INSERT zsnamkv7f31q 4
14:05:21,505 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE zsnamkv7f31q 4
14:05:21,515 INFO  [stdout] (ajp--192.168.1.150-9080-2) INSERT 1f7y1xru7z3ol 1814
14:05:21,530 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1f7y1xru7z3ol 1814
14:05:21,589 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE 35fccgwsvlo0 2309
14:05:21,663 INFO  [stdout] (ajp--192.168.1.150-9080-2) INSERT u0sdivuxanqf 1574
14:05:21,680 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE u0sdivuxanqf 1574
14:05:21,740 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE h93lz8lmamwm 7188
14:05:21,811 INFO  [stdout] (ajp--192.168.1.150-9080-2) INSERT z03kievlq3ec 9039
14:05:21,866 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE z03kievlq3ec 9039
14:05:21,893 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE 1xfiowipw18ie 6334
14:05:21,960 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT t12jyo6mydxp 8628
14:05:21,995 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE t12jyo6mydxp 8628
14:05:22,035 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 1b7mtaj9ux4cf 6651
14:05:22,070 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1b7mtaj9ux4cf 6651
14:05:22,109 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 1497gjpwnx06e 5835
14:05:22,124 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1497gjpwnx06e 5835
14:05:22,182 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE jusago26clrr 470
14:05:22,257 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE cl3qbhtfb7ul 7990
14:05:22,330 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 1nvbqqb80m410 2000
14:05:22,377 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1nvbqqb80m410 2000
14:05:22,405 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT hfpwva5k7dgp 6255
14:05:22,439 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE hfpwva5k7dgp 6255
14:05:22,479 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE awkvfp0doliq 6581
14:05:22,553 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE 1xbh2guvpaeal 2005
14:05:22,627 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT bq62t89yksnl 7832
14:05:22,645 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE bq62t89yksnl 7832
14:05:22,701 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE 1v9cegaooyl7l 8801
14:05:22,778 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 5afihmfmz78m 8493
14:05:22,817 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 5afihmfmz78m 8493
14:05:22,886 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE 1u06cy6ymizsc 8529
14:05:22,961 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE cnj2vloagv7n 105
14:05:23,034 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 7knlbcrs35tv 4975
14:05:23,070 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 7knlbcrs35tv 4975
14:05:23,109 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE icaxkpdgtqix 8605
14:05:23,183 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT ib776bswgvgs 1069
14:05:23,199 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE ib776bswgvgs 1069
14:05:23,258 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE pfbjj6qrx0lk 4247
14:05:23,337 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 1flpyivctw575 7035
14:05:23,354 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1flpyivctw575 7035
14:05:23,408 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE hlot1fnb068s 1172
14:05:23,479 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT sz4k5gzq3jop 5815
14:05:23,495 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE sz4k5gzq3jop 5815
14:05:23,560 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT ay53hxwhydvt 1006
14:05:23,577 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE ay53hxwhydvt 1006
14:05:23,628 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE p8izdhg3kqwl 20
14:05:23,702 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE 1cbvcvwojd9r1 4816
14:05:23,776 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT bdeoggzsp10b 3061
14:05:23,834 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE bdeoggzsp10b 3061
14:05:23,851 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE tpkz5b3l2qxg 3116
14:05:23,925 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE 9egiwbb1zqtk 9439
14:05:24,002 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE pebum5s3v56r 7314
14:05:24,072 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 5tvfzpub30ea 1084
14:05:24,091 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 5tvfzpub30ea 1084
14:05:24,147 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE 1pf8pteh26ykp 3161
14:05:24,225 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 3ze6zl91gz7b 916
14:05:24,276 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 3ze6zl91gz7b 916
14:05:24,295 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE zu9klw0bi6wd 1375
14:05:24,369 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE 1wf30ovze2jtf 1643
14:05:24,443 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 3evnoqlt6oag 4807
14:05:24,464 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 3evnoqlt6oag 4807
14:05:24,517 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT jixj7dxjn3bp 6387
14:05:24,532 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE jixj7dxjn3bp 6387
14:05:24,592 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT a01r5mb9ldf4 6501
14:05:24,620 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE a01r5mb9ldf4 6501
14:05:24,665 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE 18w9pp8mdk461 2853
14:05:24,740 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE devujxahhtzj 4128
14:05:24,814 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 1i88goa3n83oh 4485
14:05:24,837 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1i88goa3n83oh 4485
14:05:24,888 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE 43shzvb5wlak 4168
14:05:24,963 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT t6gc6fxquz4m 1388
14:05:24,986 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE t6gc6fxquz4m 1388
14:05:25,036 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE qklpn5ve0ozp 2928
14:05:25,111 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE 1nrm8lmgk26hp 7968
14:05:25,184 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE pdby3n7ae41s 8360
14:05:25,259 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT leqaa72xl4gm 810
14:05:25,274 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE leqaa72xl4gm 810
14:05:25,332 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 1xyro8op0a0f3 4288
14:05:25,349 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1xyro8op0a0f3 4288
14:05:25,407 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 1c12hea4x2acw 7731
14:05:25,451 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1c12hea4x2acw 7731
14:05:25,481 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT rzlgra4tco9e 8689
14:05:25,541 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE rzlgra4tco9e 8689
14:05:25,555 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 1j7f10xaqfmv5 298
14:05:25,598 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1j7f10xaqfmv5 298
14:05:25,629 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 1nah1yqhlz161 9179
14:05:25,662 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1nah1yqhlz161 9179
14:05:25,703 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE r8txbotr8vav 8849
14:05:25,778 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 1g6cybvqakdi9 9726
14:05:25,818 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1g6cybvqakdi9 9726
14:05:25,852 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE 14plps4rq06yt 7683
14:05:25,926 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE 203bvunibuxk 8672
14:05:26,000 INFO  [stdout] (ajp--192.168.1.150-9080-5) REMOVE 1r9iozlb7qu9r 2409
14:05:26,018 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1r9iozlb7qu9r 2409
14:05:26,074 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 1ppcfzih1b7n1 2650
14:05:26,089 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1ppcfzih1b7n1 2650
14:05:26,149 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 1txlmv2g46ejn 7653
14:05:26,165 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1txlmv2g46ejn 7653
14:05:26,223 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE 14htpvro4ua8b 6724
14:05:26,297 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT gko8x4cupkow 2930
14:05:26,312 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE gko8x4cupkow 2930
14:05:26,370 INFO  [stdout] (ajp--192.168.1.150-9080-5) UPDATE v1umqwkx1k66 2260
14:05:26,445 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT zgqo2m35w2rf 1036
14:05:26,462 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE zgqo2m35w2rf 1036
14:05:26,519 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 13wg0dr2yz95m 5299
14:05:26,573 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 13wg0dr2yz95m 5299
14:05:26,601 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT m0u74puq89vw 9140
14:05:26,668 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE 1o73cgsnhxe1l 7199
14:05:26,690 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE m0u74puq89vw 9140
14:05:26,742 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE g3gvziu30w7f 2402
14:05:26,815 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE zjf2pdu9snm1 1587
14:05:26,889 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE 615ovr9vnkzc 2066
14:05:26,964 INFO  [stdout] (ajp--192.168.1.150-9080-2) INSERT jpwd49sr1cly 5427
14:05:26,992 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE jpwd49sr1cly 5427
14:05:27,038 INFO  [stdout] (ajp--192.168.1.150-9080-2) INSERT 1s0qu81j9u4q 2307
14:05:27,057 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1s0qu81j9u4q 2307
14:05:27,111 INFO  [stdout] (ajp--192.168.1.150-9080-2) INSERT 1s0apr8p9juh4 1542
14:05:27,146 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1s0apr8p9juh4 1542
14:05:27,186 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE gjs31ymrin87 4821
14:05:27,260 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE 11vot9hvntfs8 35
14:05:27,334 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE 1xo7mg9v65stg 1049
14:05:27,408 INFO  [stdout] (ajp--192.168.1.150-9080-2) INSERT wktkiqe9pqns 4409
14:05:27,424 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE wktkiqe9pqns 4409
14:05:27,482 INFO  [stdout] (ajp--192.168.1.150-9080-2) INSERT tm5zdpk92har 7091
14:05:27,556 INFO  [stdout] (ajp--192.168.1.150-9080-5) INSERT 1bc5firf5fmpp 4603
14:05:27,581 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE tm5zdpk92har 7091
14:05:27,673 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE 7wamgve30owb 4823
14:05:27,678 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1bc5firf5fmpp 4603
14:05:27,705 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE gdphs99168dv 3373
14:05:27,780 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE 1g9af1sb4cg8a 8509
14:05:27,853 INFO  [stdout] (ajp--192.168.1.150-9080-2) UPDATE nz1m577xzqs0 9061

# LOGGING CLUSTER SERVER 2
14:03:01,204 INFO  [org.jboss.as.webservices] (ServerService Thread Pool -- 32) JBAS015537: Activating WebServices Extension
14:03:01,288 INFO  [org.jboss.as.security] (ServerService Thread Pool -- 36) JBAS013101: Activating Security Subsystem
14:03:01,312 INFO  [org.jboss.as.security] (MSC service thread 1-2) JBAS013100: Current PicketBox version=4.0.6.final
14:03:01,367 INFO  [org.jboss.as.osgi] (ServerService Thread Pool -- 41) JBAS011940: Activating OSGi Subsystem
14:03:01,398 INFO  [org.jboss.as.naming] (ServerService Thread Pool -- 42) JBAS011800: Activating Naming Subsystem
14:03:01,538 INFO  [org.jboss.as.naming] (MSC service thread 1-2) JBAS011802: Starting Naming Service
14:03:01,575 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-2) JBAS015400: Bound mail session [java:jboss/mail/Default]
14:03:01,635 INFO  [org.jboss.as.clustering.jgroups] (ServerService Thread Pool -- 49) JBAS010260: Activating JGroups subsystem.
14:03:01,706 INFO  [org.jboss.as.jacorb] (ServerService Thread Pool -- 54) JBAS016300: Activating JacORB Subsystem
14:03:01,716 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 55) JBAS010280: Activating Infinispan subsystem.
14:03:01,809 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 58) JBAS010403: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
14:03:01,819 INFO  [org.jboss.as.configadmin] (ServerService Thread Pool -- 59) JBAS016200: Activating ConfigAdmin Subsystem
14:03:01,832 INFO  [org.jboss.ws.common.management.AbstractServerConfig] (MSC service thread 1-2) JBoss Web Services - Stack CXF Server 4.0.1.GA
14:03:02,005 INFO  [org.jboss.as.connector] (MSC service thread 1-4) JBAS010408: Starting JCA Subsystem (JBoss IronJacamar 1.0.7.Final)
14:03:02,190 INFO  [org.jboss.as.modcluster] (MSC service thread 1-2) JBAS011704: Mod_cluster uses default load balancer provider
14:03:02,253 INFO  [org.jboss.as.jaxr] (MSC service thread 1-1) Binding JAXR ConnectionFactory: java:jboss/jaxr/ConnectionFactory
14:03:02,340 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 58) JBAS010403: Deploying JDBC-compliant driver class oracle.jdbc.OracleDriver (version 11.2)
14:03:02,366 INFO  [org.jboss.modcluster.ModClusterService] (MSC service thread 1-2) Initializing mod_cluster 1.2.0.Final
14:03:02,417 INFO  [org.apache.coyote.ajp.AjpAprProtocol] (MSC service thread 1-4) Starting Coyote AJP/1.3 on ajp-axis-into-ict.nl-192.168.1.150-9081
14:03:02,475 INFO  [org.apache.coyote.http11.Http11AprProtocol] (MSC service thread 1-3) Starting Coyote HTTP/1.1 on http-axis-into-ict.nl-192.168.1.150-8081
14:03:02,479 INFO  [org.jboss.modcluster.advertise.impl.AdvertiseListenerImpl] (MSC service thread 1-2) Listening to proxy advertisements on 224.0.1.105:23364
14:03:02,647 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-1) live server is starting with configuration HornetQ Configuration (clustered=false,backup=false,sharedStore=true,journalDirectory=/home/jboss/jboss-as-7.1.0.Final/domain/servers/cluster-server2/data/messagingjournal,bindingsDirectory=/home/jboss/jboss-as-7.1.0.Final/domain/servers/cluster-server2/data/messagingbindings,largeMessagesDirectory=/home/jboss/jboss-as-7.1.0.Final/domain/servers/cluster-server2/data/messaginglargemessages,pagingDirectory=/home/jboss/jboss-as-7.1.0.Final/domain/servers/cluster-server2/data/messagingpaging)
14:03:02,651 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-1) Waiting to obtain live lock
14:03:02,858 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 55) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be pasivated.
14:03:02,885 INFO  [org.hornetq.core.persistence.impl.journal.JournalStorageManager] (MSC service thread 1-1) Using AIO Journal
14:03:03,042 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 55) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be pasivated.
14:03:03,486 INFO  [org.jboss.as.jacorb] (MSC service thread 1-3) JBAS016330: CORBA ORB Service started
14:03:03,584 INFO  [org.hornetq.core.server.impl.AIOFileLockNodeManager] (MSC service thread 1-1) Waiting to obtain live lock
14:03:03,586 INFO  [org.hornetq.core.server.impl.AIOFileLockNodeManager] (MSC service thread 1-1) Live Server Obtained live lock
14:03:03,859 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-3) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]
14:03:03,943 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-4) JBAS010400: Bound data source [java:/jdbc/OracleDS]
14:03:04,191 INFO  [org.jboss.as.jacorb] (MSC service thread 1-3) JBAS016328: CORBA Naming Service started
14:03:04,825 INFO  [org.jboss.as.remoting] (MSC service thread 1-4) JBAS017100: Listening on axis-into-ict.nl/192.168.1.150:4448
14:03:05,139 INFO  [org.hornetq.core.remoting.impl.netty.NettyAcceptor] (MSC service thread 1-1) Started Netty Acceptor version 3.2.5.Final-a96d88c axis-into-ict.nl:5446 for CORE protocol
14:03:05,141 INFO  [org.hornetq.core.remoting.impl.netty.NettyAcceptor] (MSC service thread 1-1) Started Netty Acceptor version 3.2.5.Final-a96d88c axis-into-ict.nl:5456 for CORE protocol
14:03:05,142 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-1) Server is now live
14:03:05,143 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-1) HornetQ Server version 2.2.11.Final (HQ_2_2_11_FINAL_AS7, 122) [3f00f7a1-78e4-11e1-800d-000c2976c82d]) started
14:03:05,163 INFO  [org.jboss.as.messaging] (MSC service thread 1-1) JBAS011601: Bound messaging object to jndi name java:/ConnectionFactory
14:03:05,165 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-2) trying to deploy queue jms.topic.testTopic
14:03:05,203 INFO  [org.jboss.as.messaging] (MSC service thread 1-2) JBAS011601: Bound messaging object to jndi name java:/topic/test
14:03:05,210 INFO  [org.jboss.as.messaging] (MSC service thread 1-2) JBAS011601: Bound messaging object to jndi name java:jboss/exported/jms/topic/test
14:03:05,211 INFO  [org.jboss.as.messaging] (MSC service thread 1-4) JBAS011601: Bound messaging object to jndi name java:jboss/exported/jms/RemoteConnectionFactory
14:03:05,212 INFO  [org.jboss.as.messaging] (MSC service thread 1-4) JBAS011601: Bound messaging object to jndi name java:/RemoteConnectionFactory
14:03:05,214 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-3) trying to deploy queue jms.queue.testQueue
14:03:05,224 INFO  [org.jboss.as.messaging] (MSC service thread 1-3) JBAS011601: Bound messaging object to jndi name java:jboss/exported/jms/queue/test
14:03:05,232 INFO  [org.jboss.as.messaging] (MSC service thread 1-3) JBAS011601: Bound messaging object to jndi name java:/queue/test
14:03:05,261 INFO  [org.jboss.as.deployment.connector] (MSC service thread 1-1) JBAS010406: Registered connection factory java:/JmsXA
14:03:05,270 INFO  [org.hornetq.ra.HornetQResourceAdapter] (MSC service thread 1-1) HornetQ resource adaptor started
14:03:05,271 INFO  [org.jboss.as.connector.services.ResourceAdapterActivatorService$ResourceAdapterActivator] (MSC service thread 1-1) IJ020002: Deployed: file://RaActivatorhornetq-ra
14:03:05,274 INFO  [org.jboss.as.deployment.connector] (MSC service thread 1-2) JBAS010401: Bound JCA ConnectionFactory [java:/JmsXA]
14:03:05,359 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss AS 7.1.0.Final "Thunder" started in 7666ms - Started 169 of 292 services (121 services are passive or on-demand)
14:03:11,983 INFO  [org.jboss.modcluster.ModClusterService] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Engine [jboss.web] will use jvmRoute: 4d87f106-10ff-342f-a6e6-eb2377d7c23f
14:04:34,835 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-3) JBAS015876: Starting deployment of "SpringHibernate.war"
14:04:35,550 INFO  [org.jboss.web] (MSC service thread 1-2) JBAS018210: Registering web context: /SpringHibernate
14:04:35,621 INFO  [org.jboss.as.server] (host-controller-connection-threads - 2) JBAS018559: Deployed "SpringHibernate.war"
14:05:16,923 INFO  [org.springframework.context.support.ClassPathXmlApplicationContext] (ajp--192.168.1.150-9081-8) Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@a58d300: startup date [Wed Aug 22 14:05:16 CEST 2012]; root of context hierarchy
14:05:16,960 INFO  [org.springframework.beans.factory.xml.XmlBeanDefinitionReader] (ajp--192.168.1.150-9081-8) Loading XML bean definitions from class path resource [spring-config.xml]
14:05:17,216 INFO  [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer] (ajp--192.168.1.150-9081-8) Loading properties file from class path resource [spring.properties]
14:05:17,234 INFO  [org.springframework.beans.factory.support.DefaultListableBeanFactory] (ajp--192.168.1.150-9081-8) Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@164163e1: defining beans [company,sessionfactory,transactionManager,org.springframework.aop.config.internalAutoProxyCreator,org.springframework.transaction.annotation.AnnotationTransactionAttributeSource#0,org.springframework.transaction.interceptor.TransactionInterceptor#0,org.springframework.transaction.config.internalTransactionAdvisor,datasource,connectionfactory,destination,jmstemplate,org.springframework.jms.listener.DefaultMessageListenerContainer#0,companymdp,propertyConfigurer]; root of factory hierarchy
14:05:17,301 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp--192.168.1.150-9081-8) Using JTA UserTransaction: org.jboss.tm.usertx.client.ServerVMClientUserTransaction@575887db
14:05:17,302 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp--192.168.1.150-9081-8) Using JTA TransactionManager: com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate@445aa997
14:05:17,303 INFO  [org.springframework.transaction.jta.JtaTransactionManager] (ajp--192.168.1.150-9081-8) Using JTA TransactionSynchronizationRegistry: com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionSynchronizationRegistryImple@1b1dd242
14:05:17,389 INFO  [org.hibernate.annotations.common.Version] (ajp--192.168.1.150-9081-8) HCANN000001: Hibernate Commons Annotations {4.0.1.Final}
14:05:17,394 INFO  [org.hibernate.Version] (ajp--192.168.1.150-9081-8) HHH000412: Hibernate Core {4.0.1.Final}
14:05:17,395 INFO  [org.hibernate.cfg.Environment] (ajp--192.168.1.150-9081-8) HHH000206: hibernate.properties not found
14:05:17,397 INFO  [org.hibernate.cfg.Environment] (ajp--192.168.1.150-9081-8) HHH000021: Bytecode provider name : javassist
14:05:17,435 WARN  [org.hibernate.internal.util.xml.DTDEntityResolver] (ajp--192.168.1.150-9081-8) HHH000223: Recognized obsolete hibernate namespace http://hibernate.sourceforge.net/. Use namespace http://www.hibernate.org/dtd/ instead. Refer to Hibernate 3.6 Migration Guide!
14:05:17,659 INFO  [org.hibernate.dialect.Dialect] (ajp--192.168.1.150-9081-8) HHH000400: Using dialect: org.hibernate.dialect.Oracle10gDialect
14:05:17,678 INFO  [org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory] (ajp--192.168.1.150-9081-8) HHH000397: Using ASTQueryTranslatorFactory
14:05:17,711 INFO  [org.hibernate.validator.util.Version] (ajp--192.168.1.150-9081-8) Hibernate Validator 4.2.0.Final
14:05:17,993 INFO  [org.springframework.context.support.DefaultLifecycleProcessor] (ajp--192.168.1.150-9081-8) Starting beans in phase 2147483647
14:05:18,212 INFO  [stdout] (ajp--192.168.1.150-9081-8) UPDATE 1nuxg993ownem 3052
14:05:21,115 INFO  [stdout] (ajp--192.168.1.150-9081-8) UPDATE afhvji2k8ft7 2716
14:05:21,185 INFO  [stdout] (ajp--192.168.1.150-9081-5) INSERT 1ihgf2p7aavvv 6429
14:05:21,328 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1ihgf2p7aavvv 6429
14:05:21,329 INFO  [stdout] (ajp--192.168.1.150-9081-3) UPDATE 1om511m6nzi22 4774
14:05:21,343 INFO  [stdout] (ajp--192.168.1.150-9081-4) INSERT 1hwkf2797ujf6 5149
14:05:21,404 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1hwkf2797ujf6 5149
14:05:21,441 INFO  [stdout] (ajp--192.168.1.150-9081-6) INSERT 1ezyhvfha3q2t 3836
14:05:21,478 INFO  [stdout] (ajp--192.168.1.150-9081-4) UPDATE 1ixbel89n7f5w 9298
14:05:21,502 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1ezyhvfha3q2t 3836
14:05:21,551 INFO  [stdout] (ajp--192.168.1.150-9081-6) INSERT 1us8n7k0bnqpo 5426
14:05:21,593 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1us8n7k0bnqpo 5426
14:05:21,627 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 1ppw7krne465v 6792
14:05:21,701 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 1prt2df94i5mk 2092
14:05:21,776 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 10gvhg82ztew9 2783
14:05:21,793 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 10gvhg82ztew9 2783
14:05:21,858 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 1bd9yo40vzizt 1911
14:05:21,923 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 5n9h8rwq7q60 1343
14:05:21,997 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 8m9k1pr28gg 5547
14:05:22,013 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 8m9k1pr28gg 5547
14:05:22,071 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 17r9ee4bt0yxs 3649
14:05:22,150 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 1gn2tjij0f1ob 1035
14:05:22,219 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT m6xox6axoz3b 3157
14:05:22,235 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE m6xox6axoz3b 3157
14:05:22,294 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 110r9484rvt73 294
14:05:22,310 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 110r9484rvt73 294
14:05:22,368 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 1c9c6tb2agswh 9932
14:05:22,407 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1c9c6tb2agswh 9932
14:05:22,444 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 50ipwt1zajdx 2131
14:05:22,515 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 29h8tcc65j3 4548
14:05:22,533 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 29h8tcc65j3 4548
14:05:22,594 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT bhunhawsm27p 1700
14:05:22,611 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE bhunhawsm27p 1700
14:05:22,664 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 9bh7fa74xnrm 1225
14:05:22,738 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 19f4hj421s3fe 5128
14:05:22,815 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT ydgfqj2u2aax 8844
14:05:22,833 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE ydgfqj2u2aax 8844
14:05:22,849 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT mmw7kqjojuyq 1966
14:05:22,865 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE mmw7kqjojuyq 1966
14:05:22,923 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE qpbhdyvgftko 9965
14:05:22,997 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT m22f1xogzs2w 5940
14:05:23,013 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE m22f1xogzs2w 5940
14:05:23,075 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE q9lquj1fxj4m 4504
14:05:23,147 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE belkm98whjwh 7385
14:05:23,222 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 1dq640hgxrqoe 496
14:05:23,239 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1dq640hgxrqoe 496
14:05:23,294 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 1pevcvw6wa91q 8567
14:05:23,323 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1pevcvw6wa91q 8567
14:05:23,370 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 15bm9suyux8ui 1530
14:05:23,389 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 15bm9suyux8ui 1530
14:05:23,443 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 1kmqno89copfp 7009
14:05:23,516 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 1l8beowesem93 8780
14:05:23,547 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1l8beowesem93 8780
14:05:23,595 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 1k9yrkaxqeizz 7066
14:05:23,665 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE p93n8ojg5gaf 8632
14:05:23,739 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 1v9ix94fvw3s2 5428
14:05:23,813 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT eo8kvhum40m3 4855
14:05:23,849 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE eo8kvhum40m3 4855
14:05:23,887 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT ehuvh8mnrdor 7519
14:05:23,901 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE ehuvh8mnrdor 7519
14:05:23,962 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 42395k914sap 9967
14:05:23,988 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 42395k914sap 9967
14:05:24,035 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT y3s6i0iy5bxc 2600
14:05:24,051 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE y3s6i0iy5bxc 2600
14:05:24,111 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 3us32d2krjvq 4008
14:05:24,186 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 9rg1q23jbt1h 5890
14:05:24,208 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 9rg1q23jbt1h 5890
14:05:24,332 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 1ayxc04tdei19 9978
14:05:24,348 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1ayxc04tdei19 9978
14:05:24,406 INFO  [stdout] (ajp--192.168.1.150-9081-7) REMOVE 1aczxa7rq66gc 7734
14:05:24,422 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1aczxa7rq66gc 7734
14:05:24,480 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 1qxbi4c03ptl3 2096
14:05:24,494 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1qxbi4c03ptl3 2096
14:05:24,554 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE k2zwhkup05z6 8428
14:05:24,633 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT as7552dnufu 899
14:05:24,648 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE as7552dnufu 899
14:05:24,703 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT l5zi29x6gxno 5344
14:05:24,717 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE l5zi29x6gxno 5344
14:05:24,777 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE r311dyu91dym 1859
14:05:24,852 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 1fxxn3gl91h8m 3280
14:05:24,925 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 1xqnw0zd807wd 2425
14:05:24,975 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1xqnw0zd807wd 2425
14:05:25,001 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 1md19po93vtqw 6630
14:05:25,073 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 2c9cplr913cz 1431
14:05:25,108 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 2c9cplr913cz 1431
14:05:25,147 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT v43yhet3xd1g 66
14:05:25,162 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE v43yhet3xd1g 66
14:05:25,221 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 1f7m1o552i1ap 5103
14:05:25,296 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 1vso63da2hxsz 4821
14:05:25,369 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT sz5f4p22ihxm 8460
14:05:25,418 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE sz5f4p22ihxm 8460
14:05:25,445 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 2a7fe1kbyn05 3485
14:05:25,496 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 2a7fe1kbyn05 3485
14:05:25,518 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT ydkzaajaai23 8793
14:05:25,585 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE ydkzaajaai23 8793
14:05:25,593 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 1nazvx44fsryy 2224
14:05:25,610 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1nazvx44fsryy 2224
14:05:25,667 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT ckpdo13cbu3v 8647
14:05:25,681 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE ckpdo13cbu3v 8647
14:05:25,740 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT bnydn2bxk8wd 1831
14:05:25,755 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE bnydn2bxk8wd 1831
14:05:25,815 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE fl1lk4bscdoj 7360
14:05:25,889 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE r8s5rxlisovh 3498
14:05:25,963 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 1o12rg3cdx2vk 1350
14:05:26,037 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 1gr04agkc8l6 4563
14:05:26,111 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 1c3ksedbtpaoq 3054
14:05:26,150 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1c3ksedbtpaoq 3054
14:05:26,185 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 17s438pq34yde 3952
14:05:26,219 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 17s438pq34yde 3952
14:05:26,259 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 14grcfgntxm7g 6624
14:05:26,274 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 14grcfgntxm7g 6624
14:05:26,333 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 9rlyy5ku91wv 257
14:05:26,348 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 9rlyy5ku91wv 257
14:05:26,407 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 5ek3d1123mwp 3383
14:05:26,451 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 5ek3d1123mwp 3383
14:05:26,481 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT xucmrjfo2tkr 3652
14:05:26,526 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE xucmrjfo2tkr 3652
14:05:26,556 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 13st8rbs16ek0 4776
14:05:26,586 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 13st8rbs16ek0 4776
14:05:26,631 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 21xvhp0ohbc0 6680
14:05:26,705 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT yyqn8d3edsvp 340
14:05:26,751 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE yyqn8d3edsvp 340
14:05:26,779 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 19qwovr0pstvu 3878
14:05:26,852 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 10lz3rhlhi1mn 8828
14:05:26,926 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 5m3i1gvcm0jx 4490
14:05:26,971 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 5m3i1gvcm0jx 4490
14:05:27,007 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 10x2dlavqagr7 1772
14:05:27,053 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 10x2dlavqagr7 1772
14:05:27,074 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 1irlznr87cp7v 6876
14:05:27,149 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT jh9pj68vmhla 7165
14:05:27,196 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE jh9pj68vmhla 7165
14:05:27,222 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 2eibuwgmnyai 1734
14:05:27,297 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE cmh5uq6jfdh1 1425
14:05:27,371 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 1by141cbe2jw2 3937
14:05:27,415 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1by141cbe2jw2 3937
14:05:27,446 INFO  [stdout] (ajp--192.168.1.150-9081-7) INSERT 1t1gansg3mryj 540
14:05:27,520 INFO  [stdout] (ajp--192.168.1.150-9081-6) INSERT b8uhwlgalst0 7532
14:05:27,544 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE b8uhwlgalst0 7532
14:05:27,584 INFO  [stdout] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) RECEIVED OBJECT MESSAGE 1t1gansg3mryj 540
14:05:27,594 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 1ox069v1p4zue 7684
14:05:27,686 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 12obz6cvdizpk 4250
14:05:27,741 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE z7hc2gm1tko7 9967
14:05:27,819 INFO  [stdout] (ajp--192.168.1.150-9081-7) UPDATE 1sn0ranavpi7w 4442

Migration

Let us just deploy the application on WebLogic Server and see if we can get the Spring Hibernate application to work here as well. To this end, we create two shared libraries one for Spring 3.1.2 and one for Hibernate 4.1.2. The hibernate-4.1.war shared library has the following contents:

hibernate-4.1
	META-INF
		MANIFEST.MF
	WEB-INF
		lib
			antlr-2.7.7.jar
			dom4j-1.6.1.jar
			hibernate-commons-annotations-4.0.1.Final.jar
			hibernate-core-4.1.2.Final.jar
			hibernate-jpa-2.0-api-1.0.1.Final.jar
			javassist-3.15.0-GA.jar
			jboss-logging-3.1.0.GA.jar
			jboss-transaction-api_1.1_spec-1.0.0.Final.jar

in which the MANIFEST.MF contains:

Manifest-Version: 1.0
Created-By: 1.6.0_05 (BEA Systems, Inc.)
Extension-Name: hibernate
Specification-Title: Hibernate Library
Specification-Version: 4.1
Specification-Vendor: Middleware Magic
Implementation-Title: Hibernate Library
Implementation-Version: 4.1.2
Implementation-Vendor: Middleware Magic

The spring-3.1.war shared library has the following contents:

spring-3.1
	META-INF
		MANIFEST.MF
	WEB-INF
		lib
			aopalliance-1.0.jar
			cglib-2.2.2.jar
			commons-logging-1.0.4.jar
			org.springframework.aop-3.1.2.RELEASE.jar
			org.springframework.asm-3.1.2.RELEASE.jar
			org.springframework.beans-3.1.2.RELEASE.jar
			org.springframework.context-3.1.2.RELEASE.jar
			org.springframework.core-3.1.2.RELEASE.jar
			org.springframework.expression-3.1.2.RELEASE.jar
			org.springframework.jdbc-3.1.2.RELEASE.jar
			org.springframework.jms-3.1.2.RELEASE.jar
			org.springframework.orm-3.1.2.RELEASE.jar
			org.springframework.transaction-3.1.2.RELEASE.jar
			org.springframework.web-3.1.2.RELEASE.jar

in which the MANIFEST.MF contains:

Manifest-Version: 1.0
Created-By: 1.6.0_05 (BEA Systems, Inc.)
Extension-Name: spring
Specification-Title: Spring Library
Specification-Version: 3.1
Specification-Vendor: Middleware Magic
Implementation-Title: Spring Library
Implementation-Version: 3.1.2
Implementation-Vendor: Middleware Magic

We deploy the shared libraries to the WebLogic environment that is set-up as in the post Deploy WebLogic12c to Multiple Machines. The used Spring configuration looks as follows:

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:tx="http://www.springframework.org/schema/tx"
       xmlns:jee="http://www.springframework.org/schema/jee"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
       http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
       http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-3.0.xsd">
    <bean id="company" class="model.logic.CompanyBean">
        <property name="sessionFactory" ref="sessionfactory"/>
        <property name="jmsTemplate" ref="jmstemplate"/>
    </bean>
	<!-- Hibernate SessionFactory config -->
    <bean id="sessionfactory" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
        <property name="dataSource" ref="datasource"/>
        <property name="jtaTransactionManager" ref="transactionManager"/>
        <property name="mappingResources">
            <list>
                <value>model/entities/person.hbm.xml</value>
            </list>
        </property>
        <property name="hibernateProperties">
            <props>
                <prop key="hibernate.dialect">${hibernate.dialect}</prop>
                <prop key="hibernate.current_session_context_class">${hibernate.current_session_context_class}</prop>
                <prop key="hibernate.listeners.envers.autoRegister">${hibernate.listeners.envers.autoRegister}</prop>
                <prop key="hibernate.show_sql">${hibernate.show_sql}</prop>
                <prop key="hibernate.format_sql">${hibernate.format_sql}</prop>
            </props>
        </property>
    </bean>
	<!-- Transaction config -->
	<bean id="transactionManager" class="org.springframework.transaction.jta.JtaTransactionManager"/>
	<tx:annotation-driven transaction-manager="transactionManager"/>
	<!-- Resource look-ups -->
    <jee:jndi-lookup id="datasource" jndi-name="jdbc/exampleDS" resource-ref="false"/>
    <jee:jndi-lookup id="connectionfactory" jndi-name="jms/ConnectionFactory" resource-ref="false"/>
	<jee:jndi-lookup id="destination" jndi-name="jms/CompanyQueue" resource-ref="false"/>
    <!-- JMS config -->
	<bean id="jmstemplate" class="org.springframework.jms.core.JmsTemplate">
        <property name="connectionFactory" ref="connectionfactory"/>
        <property name="defaultDestination" ref="destination"/>
    </bean>
    <bean class="org.springframework.jms.listener.DefaultMessageListenerContainer">
        <property name="connectionFactory" ref="connectionfactory"/>
        <property name="destination" ref="destination"/>
        <property name="transactionManager" ref="transactionManager"/>
        <property name="messageListener" ref="companymdp"/>
    </bean>
    <bean id="companymdp" class="model.logic.CompanyMDP"/>
	<!-- Extra -->
    <bean id="propertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="locations">
            <list>
                <value>classpath:spring.properties</value>
            </list>
        </property>
    </bean>
</beans>

Note that the only things that are changed are the JNDI-names of the used resources (data source, connection factory and destination). In order to deploy the application to WebLogic we can use the following directory structure

/springhibernate
	/app
		SpringHibernate.war
	/plan
		/WEB-INF
			weblogic.xml
		plan.xml

in which weblogic.xml has the following contents

<weblogic-web-app xmlns="http://xmlns.oracle.com/weblogic/weblogic-web-app" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/weblogic/weblogic-web-app http://xmlns.oracle.com/weblogic/weblogic-web-app/1.4/weblogic-web-app.xsd">
	<container-descriptor>
		<prefer-application-packages>
			<package-name>antlr.*</package-name>
		</prefer-application-packages>
	</container-descriptor>
	<library-ref>
		<library-name>hibernate</library-name>
		<specification-version>4.1</specification-version>
		<implementation-version>4.1.2</implementation-version>
		<exact-match>true</exact-match>
	</library-ref>
	<library-ref>
		<library-name>spring</library-name>
		<specification-version>3.1</specification-version>
		<implementation-version>3.1.2</implementation-version>
		<exact-match>true</exact-match>
	</library-ref>
</weblogic-web-app>

When the application is deployed and the testservlet is accessed the following is observed in the logging

# LOGGING CLUSTER-SERVER1
Aug 17, 2012 11:17:18 AM org.springframework.context.support.AbstractApplicationContext prepareRefresh
INFO: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@1e3741a5: startup date [Fri Aug 17 11:17:18 CEST 2012]; root of context hierarchy
Aug 17, 2012 11:17:18 AM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [spring-config.xml]
Aug 17, 2012 11:17:18 AM org.springframework.core.io.support.PropertiesLoaderSupport loadProperties
INFO: Loading properties file from class path resource [spring.properties]
Aug 17, 2012 11:17:18 AM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@1e57db48: defining beans [company,sessionfactory,datasource,propertyConfigurer,transactionManager,org.springframework.aop.config.internalAutoProxyCreator,org.springframework.transaction.annotation.AnnotationTransactionAttributeSource#0,org.springframework.transaction.interceptor.TransactionInterceptor#0,org.springframework.transaction.config.internalTransactionAdvisor,connectionfactory,destination,jmstemplate,org.springframework.jms.listener.DefaultMessageListenerContainer#0,companymdp]; root of factory hierarchy
Aug 17, 2012 11:17:18 AM org.springframework.transaction.jta.JtaTransactionManager checkUserTransactionAndTransactionManager
INFO: Using JTA UserTransaction: ClientTM[cluster-server1+axis-into-ict.nl:9001+base_domain+t3+]
Aug 17, 2012 11:17:18 AM org.springframework.transaction.jta.JtaTransactionManager checkUserTransactionAndTransactionManager
INFO: Using JTA TransactionManager: ClientTM[cluster-server1+axis-into-ict.nl:9001+base_domain+t3+]
Aug 17, 2012 11:17:18 AM org.springframework.transaction.jta.JtaTransactionManager initTransactionSynchronizationRegistry
INFO: Using JTA TransactionSynchronizationRegistry: ClientTM[cluster-server1+axis-into-ict.nl:9001+base_domain+t3+]
Aug 17, 2012 11:17:19 AM org.hibernate.annotations.common.Version <clinit>
INFO: HCANN000001: Hibernate Commons Annotations {4.0.1.Final}
Aug 17, 2012 11:17:19 AM org.hibernate.Version logVersion
INFO: HHH000412: Hibernate Core {4.1.2.Final}
Aug 17, 2012 11:17:19 AM org.hibernate.cfg.Environment <clinit>
INFO: HHH000206: hibernate.properties not found
Aug 17, 2012 11:17:19 AM org.hibernate.cfg.Environment buildBytecodeProvider
INFO: HHH000021: Bytecode provider name : javassist
Aug 17, 2012 11:17:19 AM org.hibernate.internal.util.xml.DTDEntityResolver resolveEntity
WARN: HHH000223: Recognized obsolete hibernate namespace http://hibernate.sourceforge.net/. Use namespace http://www.hibernate.org/dtd/ instead. Refer to Hibernate 3.6 Migration Guide!
Aug 17, 2012 11:17:19 AM org.hibernate.dialect.Dialect <init>
INFO: HHH000400: Using dialect: org.hibernate.dialect.Oracle10gDialect
Aug 17, 2012 11:17:19 AM org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory <init>
INFO: HHH000397: Using ASTQueryTranslatorFactory
Aug 17, 2012 11:17:20 AM org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup start
INFO: Starting beans in phase 2147483647
INSERT 1vfaxqqm2868k 9170
RECEIVED OBJECT MESSAGE 1vfaxqqm2868k 9170
INSERT 1azwiw7lwh4y7 6205
RECEIVED OBJECT MESSAGE 1azwiw7lwh4y7 6205
UPDATE 1t4a3l9tlt0i1 7843
UPDATE 1gpie3s1fmk7a 5058
UPDATE wgiimvyfwc8i 5756
INSERT 1tmgsmdsi2sbh 435
RECEIVED OBJECT MESSAGE 1tmgsmdsi2sbh 435
UPDATE 1mqtq9gwil0py 8978
INSERT ts5nfho8b4wb 3167
RECEIVED OBJECT MESSAGE ts5nfho8b4wb 3167
INSERT j878nzzqf3b3 3553
RECEIVED OBJECT MESSAGE j878nzzqf3b3 3553
INSERT 1upuhpirvr0a7 938
RECEIVED OBJECT MESSAGE 1upuhpirvr0a7 938
UPDATE mgt58gj3d61n 3538
INSERT 1q5smfr3dh6df 2715
RECEIVED OBJECT MESSAGE 1q5smfr3dh6df 2715
UPDATE 1r3jwyj2g5x5m 6113
UPDATE 1d6cpm9lavqwg 1972
INSERT vxc1h63kfy2w 1496
RECEIVED OBJECT MESSAGE vxc1h63kfy2w 1496
INSERT 55otw4aj7n7u 3826
RECEIVED OBJECT MESSAGE 55otw4aj7n7u 3826
INSERT 1dl36wovta1vf 8045
RECEIVED OBJECT MESSAGE 1dl36wovta1vf 8045
INSERT p1693zg2q0bl 4673
RECEIVED OBJECT MESSAGE p1693zg2q0bl 4673
INSERT 1e6c1ssik0esu 8536
RECEIVED OBJECT MESSAGE 1e6c1ssik0esu 8536
INSERT w7hlqytbosrt 2874
RECEIVED OBJECT MESSAGE w7hlqytbosrt 2874
UPDATE 152p5lvqwxco8 9516
INSERT byh8hu448t7w 8913
RECEIVED OBJECT MESSAGE byh8hu448t7w 8913
INSERT 1dajek4kv7xgb 8368
RECEIVED OBJECT MESSAGE 1dajek4kv7xgb 8368
INSERT ju0rs6v1k0dv 6421
RECEIVED OBJECT MESSAGE ju0rs6v1k0dv 6421
UPDATE 107qwp5m3bh6u 4167
INSERT uewom3s9awr0 4933
RECEIVED OBJECT MESSAGE uewom3s9awr0 4933
INSERT 1gejfty8uqly1 9279
RECEIVED OBJECT MESSAGE 1gejfty8uqly1 9279
INSERT ixpxzvpr08ww 5428
RECEIVED OBJECT MESSAGE ixpxzvpr08ww 5428
INSERT 17qi3yaxwtrxu 7411
RECEIVED OBJECT MESSAGE 17qi3yaxwtrxu 7411
UPDATE 7fgzuwkcz48e 5716
INSERT 1tbtnsilprbmm 8680
RECEIVED OBJECT MESSAGE 1tbtnsilprbmm 8680
UPDATE 1b2b771wi8waz 6832
INSERT 14v47vzyr5egt 564
RECEIVED OBJECT MESSAGE 14v47vzyr5egt 564
INSERT v30iv4nv8g5n 4052
RECEIVED OBJECT MESSAGE v30iv4nv8g5n 4052
UPDATE 1f2rqzn03y5qq 7750
REMOVE 1ibtlrunczhdk 4166
REMOVE 19fzxc64sngs1 5131
INSERT 981c8ahs90go 6356
RECEIVED OBJECT MESSAGE 1ibtlrunczhdk 4166
RECEIVED OBJECT MESSAGE 981c8ahs90go 6356
RECEIVED OBJECT MESSAGE 19fzxc64sngs1 5131
INSERT 1dm92l3kuloyc 3669
RECEIVED OBJECT MESSAGE 1dm92l3kuloyc 3669
UPDATE 3dtym6dsmm8j 6477
UPDATE jm5v2s8kq5cn 436
UPDATE 1aa1z7okdo09k 8246
INSERT jjnultqc90f4 9607
RECEIVED OBJECT MESSAGE jjnultqc90f4 9607
INSERT 1mmixox5tpnnn 3279
RECEIVED OBJECT MESSAGE 1mmixox5tpnnn 3279
INSERT 1227bi0v86b3t 4022
RECEIVED OBJECT MESSAGE 1227bi0v86b3t 4022
UPDATE stvbc1ya5lc2 8047
UPDATE 1xntj3pwd7p9x 258
UPDATE 1k6r3ryeo8ulm 6901
UPDATE b2e526chsv6o 2934
INSERT 1q0axu96bm6lg 7173
RECEIVED OBJECT MESSAGE 1q0axu96bm6lg 7173
INSERT sigdxvz05hsi 3065
RECEIVED OBJECT MESSAGE sigdxvz05hsi 3065
INSERT 1hgbown1i5fc9 6686
RECEIVED OBJECT MESSAGE 1hgbown1i5fc9 6686
UPDATE 9s7hmdtntgkd 4601
UPDATE 17ce0wl4ealwc 2423
UPDATE 1a96qcppjpe5f 7212
UPDATE 152pp0q3zk8zg 4671
INSERT tdlhx4yk0pmg 9733
RECEIVED OBJECT MESSAGE tdlhx4yk0pmg 9733
UPDATE 622um5vc14mq 7765
INSERT 1lsx0iiup6gg5 6041
RECEIVED OBJECT MESSAGE 1lsx0iiup6gg5 6041
UPDATE 16ekljbz9e387 4309
UPDATE 1cezkqse6hvd0 3349
UPDATE 1a6olflotven8 1087
INSERT 1au7gq4p6mv8c 3261
RECEIVED OBJECT MESSAGE 1au7gq4p6mv8c 3261
INSERT 1em7bsp3g29w 2566
RECEIVED OBJECT MESSAGE 1em7bsp3g29w 2566
INSERT 1a4mkxcpjug71 6167
RECEIVED OBJECT MESSAGE 1a4mkxcpjug71 6167
UPDATE pmw4v5xst33l 5002
UPDATE 1mpp7mfpl5kru 682
UPDATE 10z3im4emd96i 4022
UPDATE 1jxworov8e1z9 9844
INSERT 1sedtpmbhtquc 270
RECEIVED OBJECT MESSAGE 1sedtpmbhtquc 270
UPDATE lwesgdzgwpl0 8422
UPDATE gvjyxm2c7jd0 5366
INSERT moiepveyd9xt 9931
RECEIVED OBJECT MESSAGE moiepveyd9xt 9931
INSERT a8kfudlzxcxy 6156
RECEIVED OBJECT MESSAGE a8kfudlzxcxy 6156
UPDATE o0hx0zv9qgzg 3469
INSERT ugs4gnhafozq 6680
RECEIVED OBJECT MESSAGE ugs4gnhafozq 6680
INSERT sdheu51y4yfx 8489
RECEIVED OBJECT MESSAGE sdheu51y4yfx 8489
UPDATE sq6j7i0baitp 6699
INSERT v2ojrjfekv9k 4533
RECEIVED OBJECT MESSAGE v2ojrjfekv9k 4533
UPDATE 10wfve0yjl4k4 4589
INSERT uqwxetfuogwt 9849
RECEIVED OBJECT MESSAGE uqwxetfuogwt 9849
UPDATE 1ush98l72a9d3 8258
UPDATE owyg8kl65g8j 4156
INSERT pb8v2gufr6rh 6497
RECEIVED OBJECT MESSAGE pb8v2gufr6rh 6497
INSERT dmfq7qiueu7x 9955
RECEIVED OBJECT MESSAGE dmfq7qiueu7x 9955
INSERT 13ke7pnvs9ylu 7792
RECEIVED OBJECT MESSAGE 13ke7pnvs9ylu 7792
INSERT y7kbegscmisj 896
RECEIVED OBJECT MESSAGE y7kbegscmisj 896
UPDATE 2iwm9v61fvhy 749
INSERT bdmcdk1ogf9l 6628
RECEIVED OBJECT MESSAGE bdmcdk1ogf9l 6628
INSERT 18bysp3l4gb0f 6056
RECEIVED OBJECT MESSAGE 18bysp3l4gb0f 6056
UPDATE 16hr14gvrsh1k 2122
UPDATE hyp4ojojxdes 8218
UPDATE 1khy3p4q9dclk 747
UPDATE 1uzuvg1xslcky 9561
UPDATE 1kgdpavy2chez 519
INSERT 1mox60j02zh85 9798
RECEIVED OBJECT MESSAGE 1mox60j02zh85 9798
INSERT ci128g9jo27u 7739
RECEIVED OBJECT MESSAGE ci128g9jo27u 7739
INSERT 6j5syb4xpota 9066
RECEIVED OBJECT MESSAGE 6j5syb4xpota 9066
UPDATE 652v8hhke5ai 4884
UPDATE 1u1dzud98psop 2480
UPDATE q6ci13nicvri 9825
INSERT 1n81wvaep5nr6 6212
RECEIVED OBJECT MESSAGE 1n81wvaep5nr6 6212
INSERT 18ncq6m6s1acl 3285
RECEIVED OBJECT MESSAGE 18ncq6m6s1acl 3285
INSERT zjtnfrh2ajsj 3407
RECEIVED OBJECT MESSAGE zjtnfrh2ajsj 3407
UPDATE q5ug6q1daqpn 6675
INSERT uzaeixyt6g0l 610
RECEIVED OBJECT MESSAGE uzaeixyt6g0l 610
INSERT rpftez66w9rr 1695
RECEIVED OBJECT MESSAGE rpftez66w9rr 1695
UPDATE 135a8m7uhk006 4194
UPDATE 14exorxy3klbg 5424
UPDATE 14r1nopyggixk 7215
UPDATE 1izhgww1gazcn 5707
UPDATE gz1k3p3nyzcz 30
UPDATE fnf7ggv9gy8s 7314
UPDATE 2ql8f9rupz8g 1206
UPDATE 7ylaxoxi9uua 4279
INSERT x5wphn4pz0xf 7045
RECEIVED OBJECT MESSAGE x5wphn4pz0xf 7045
UPDATE b8x13s890p7v 3471
INSERT ij7897v2tqp2 8531
RECEIVED OBJECT MESSAGE ij7897v2tqp2 8531
UPDATE 1ir4dulll98sj 7765
INSERT hwiqxp6cx32v 2775
RECEIVED OBJECT MESSAGE hwiqxp6cx32v 2775
INSERT 19lblxbbkdoi0 7419
RECEIVED OBJECT MESSAGE 19lblxbbkdoi0 7419
INSERT 1dgxsd9fcwthk 6299
RECEIVED OBJECT MESSAGE 1dgxsd9fcwthk 6299
UPDATE 17owe7epcdanl 7480
UPDATE qvzncb916e1l 6856
INSERT 1c5pamjalavz0 6374
RECEIVED OBJECT MESSAGE 1c5pamjalavz0 6374
INSERT 1xndfh1zhsm8t 7993
RECEIVED OBJECT MESSAGE 1xndfh1zhsm8t 7993
INSERT 1kz8donqnnc8g 2967
RECEIVED OBJECT MESSAGE 1kz8donqnnc8g 2967
INSERT 19qq0303net89 1906
RECEIVED OBJECT MESSAGE 19qq0303net89 1906
INSERT sax47077f9ka 3016
RECEIVED OBJECT MESSAGE sax47077f9ka 3016
INSERT pufs4n4rhr1e 7255
RECEIVED OBJECT MESSAGE pufs4n4rhr1e 7255
INSERT 1e5kf8rlv0kav 5159
RECEIVED OBJECT MESSAGE 1e5kf8rlv0kav 5159
INSERT 1ulnvxkxowbly 7415
RECEIVED OBJECT MESSAGE 1ulnvxkxowbly 7415
UPDATE t9edlkta9x06 7882
UPDATE 1pkra952k87nb 2277
INSERT 1v4x4t69qoyvv 1948
RECEIVED OBJECT MESSAGE 1v4x4t69qoyvv 1948
INSERT ynlrxbwg2o86 9056
RECEIVED OBJECT MESSAGE ynlrxbwg2o86 9056
INSERT 1pjurxwpfoafo 2735
RECEIVED OBJECT MESSAGE 1pjurxwpfoafo 2735
UPDATE sxec5z2fdc8t 9386
INSERT 1po7ankt6q1e6 1401
RECEIVED OBJECT MESSAGE 1po7ankt6q1e6 1401
UPDATE 10wjk2jzd5rgb 1541
UPDATE 1rs2vq5yy7603 4562
INSERT 12l3q5i8xcyem 6099
RECEIVED OBJECT MESSAGE 12l3q5i8xcyem 6099
INSERT 105ia9hr5xg3t 4083
RECEIVED OBJECT MESSAGE 105ia9hr5xg3t 4083
INSERT naup4tcu7c2w 9001
RECEIVED OBJECT MESSAGE naup4tcu7c2w 9001
INSERT r97pmgdg7z4y 4698
RECEIVED OBJECT MESSAGE r97pmgdg7z4y 4698
INSERT 11bba4337czby 444
RECEIVED OBJECT MESSAGE 11bba4337czby 444
INSERT lcodtofso3fy 3621
RECEIVED OBJECT MESSAGE lcodtofso3fy 3621
INSERT o2l983w9jov8 8912
RECEIVED OBJECT MESSAGE o2l983w9jov8 8912
INSERT 1urnou35ulywu 697
RECEIVED OBJECT MESSAGE 1urnou35ulywu 697
INSERT 1lvbw429diy7v 9018
RECEIVED OBJECT MESSAGE 1lvbw429diy7v 9018
INSERT 1p4pa3tntgroo 2444
RECEIVED OBJECT MESSAGE 1p4pa3tntgroo 2444
INSERT qxiiwynhc3gj 9248
RECEIVED OBJECT MESSAGE qxiiwynhc3gj 9248

# LOGGING CLUSTER-SERVER2
Aug 17, 2012 11:25:34 AM org.springframework.context.support.AbstractApplicationContext prepareRefresh
INFO: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@1e701a84: startup date [Fri Aug 17 11:25:34 CEST 2012]; root of context hierarchy
Aug 17, 2012 11:25:35 AM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [spring-config.xml]
Aug 17, 2012 11:25:35 AM org.springframework.core.io.support.PropertiesLoaderSupport loadProperties
INFO: Loading properties file from class path resource [spring.properties]
Aug 17, 2012 11:25:35 AM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@1e9c800e: defining beans [company,sessionfactory,datasource,propertyConfigurer,transactionManager,org.springframework.aop.config.internalAutoProxyCreator,org.springframework.transaction.annotation.AnnotationTransactionAttributeSource#0,org.springframework.transaction.interceptor.TransactionInterceptor#0,org.springframework.transaction.config.internalTransactionAdvisor,connectionfactory,destination,jmstemplate,org.springframework.jms.listener.DefaultMessageListenerContainer#0,companymdp]; root of factory hierarchy
Aug 17, 2012 11:25:35 AM org.springframework.transaction.jta.JtaTransactionManager checkUserTransactionAndTransactionManager
INFO: Using JTA UserTransaction: ClientTM[cluster-server2+axis-into-ict.nl:9002+base_domain+t3+]
Aug 17, 2012 11:25:35 AM org.springframework.transaction.jta.JtaTransactionManager checkUserTransactionAndTransactionManager
INFO: Using JTA TransactionManager: ClientTM[cluster-server2+axis-into-ict.nl:9002+base_domain+t3+]
Aug 17, 2012 11:25:35 AM org.springframework.transaction.jta.JtaTransactionManager initTransactionSynchronizationRegistry
INFO: Using JTA TransactionSynchronizationRegistry: ClientTM[cluster-server2+axis-into-ict.nl:9002+base_domain+t3+]
Aug 17, 2012 11:25:35 AM org.hibernate.annotations.common.Version <clinit>
INFO: HCANN000001: Hibernate Commons Annotations {4.0.1.Final}
Aug 17, 2012 11:25:35 AM org.hibernate.Version logVersion
INFO: HHH000412: Hibernate Core {4.1.2.Final}
Aug 17, 2012 11:25:35 AM org.hibernate.cfg.Environment <clinit>
INFO: HHH000206: hibernate.properties not found
Aug 17, 2012 11:25:35 AM org.hibernate.cfg.Environment buildBytecodeProvider
INFO: HHH000021: Bytecode provider name : javassist
Aug 17, 2012 11:25:36 AM org.hibernate.internal.util.xml.DTDEntityResolver resolveEntity
WARN: HHH000223: Recognized obsolete hibernate namespace http://hibernate.sourceforge.net/. Use namespace http://www.hibernate.org/dtd/ instead. Refer to Hibernate 3.6 Migration Guide!
Aug 17, 2012 11:25:36 AM org.hibernate.dialect.Dialect <init>
INFO: HHH000400: Using dialect: org.hibernate.dialect.Oracle10gDialect
Aug 17, 2012 11:25:36 AM org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory <init>
INFO: HHH000397: Using ASTQueryTranslatorFactory
Aug 17, 2012 11:25:37 AM org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup start
INFO: Starting beans in phase 2147483647
INSERT 1ro7iaej9wd9k 8795
RECEIVED OBJECT MESSAGE 1ro7iaej9wd9k 8795
INSERT 1b1fcai34faga 6314
RECEIVED OBJECT MESSAGE 1b1fcai34faga 6314
INSERT 1v57ryimse5uf 9755
RECEIVED OBJECT MESSAGE 1v57ryimse5uf 9755
INSERT yx01xew3z2jh 9354
RECEIVED OBJECT MESSAGE yx01xew3z2jh 9354
UPDATE um8jioyfz2xw 3576
UPDATE fzzhd0wiga1j 5555
UPDATE g2yjl64y5r26 9223
UPDATE 1ajjz2xwnb0xy 6102
INSERT sc8lrc1ltn1c 3679
RECEIVED OBJECT MESSAGE sc8lrc1ltn1c 3679
UPDATE 19g20kvci9lgc 3349
INSERT yekowaz2yvvv 2135
RECEIVED OBJECT MESSAGE yekowaz2yvvv 2135
UPDATE hgoeapqkrf3h 9331
INSERT qoioytyju0pd 8055
RECEIVED OBJECT MESSAGE qoioytyju0pd 8055
UPDATE q9kkzncyf9vg 5423
INSERT 1e0hjp4authf2 7252
RECEIVED OBJECT MESSAGE 1e0hjp4authf2 7252
INSERT kfuv64fpg5xh 9006
RECEIVED OBJECT MESSAGE kfuv64fpg5xh 9006
UPDATE 1bzhy6v1trxil 2834
INSERT 3jeohjlsc4ai 3093
RECEIVED OBJECT MESSAGE 3jeohjlsc4ai 3093
INSERT dlbnur9etkbu 9469
RECEIVED OBJECT MESSAGE dlbnur9etkbu 9469
UPDATE 1p02mjgb66m5c 8668
UPDATE 1nkdo6q2dok30 6031
INSERT 1memxxtvl9d7j 8365
RECEIVED OBJECT MESSAGE 1memxxtvl9d7j 8365
INSERT 18qq5qqnowk38 4747
RECEIVED OBJECT MESSAGE 18qq5qqnowk38 4747
INSERT ojxr6mn4bkpf 4212
RECEIVED OBJECT MESSAGE ojxr6mn4bkpf 4212
INSERT 1s4hpfjw4uckk 8261
RECEIVED OBJECT MESSAGE 1s4hpfjw4uckk 8261
UPDATE a7267e4nojm5 7415
UPDATE 121eduf2de5o7 1629
INSERT 1jp76tnlkkk23 5448
RECEIVED OBJECT MESSAGE 1jp76tnlkkk23 5448
INSERT b3cvx730ikth 129
RECEIVED OBJECT MESSAGE b3cvx730ikth 129
UPDATE 11pwub752h6e4 4900
INSERT 1rra35pu4kx34 5472
RECEIVED OBJECT MESSAGE 1rra35pu4kx34 5472
UPDATE 1s0zsfne0divn 4096
UPDATE 1rtoy57gl7bhu 2614
UPDATE ezzxkiss0quj 3927
UPDATE 1l9zzie9sjjwx 8543
INSERT 1hufcx719vzm1 3106
RECEIVED OBJECT MESSAGE 1hufcx719vzm1 3106
UPDATE 1wifye5nhv9fh 8054
INSERT m2e4uladi9g0 5162
RECEIVED OBJECT MESSAGE m2e4uladi9g0 5162
INSERT arag8dj5j6k8 9491
RECEIVED OBJECT MESSAGE arag8dj5j6k8 9491
UPDATE vchq9193eelj 4236
INSERT r2lu5002hm8q 3371
RECEIVED OBJECT MESSAGE r2lu5002hm8q 3371
INSERT ah5ivj9eb08g 6552
RECEIVED OBJECT MESSAGE ah5ivj9eb08g 6552
INSERT 9wbnmdm841qe 5687
RECEIVED OBJECT MESSAGE 9wbnmdm841qe 5687
INSERT m7dk38en2pa6 2771
RECEIVED OBJECT MESSAGE m7dk38en2pa6 2771
INSERT 2ccmfrlth7qi 7890
RECEIVED OBJECT MESSAGE 2ccmfrlth7qi 7890
INSERT 1j4u6kowyig5h 7865
RECEIVED OBJECT MESSAGE 1j4u6kowyig5h 7865
UPDATE 71ter5vkj56o 6106
INSERT 9o7qpmcbl4vh 1349
RECEIVED OBJECT MESSAGE 9o7qpmcbl4vh 1349
INSERT lmr4oqlu0o0o 5685
RECEIVED OBJECT MESSAGE lmr4oqlu0o0o 5685
INSERT y2du188lqyjy 1659
RECEIVED OBJECT MESSAGE y2du188lqyjy 1659
INSERT ud58jd9xkmir 4338
RECEIVED OBJECT MESSAGE ud58jd9xkmir 4338
INSERT 33sskk07r3j 2585
RECEIVED OBJECT MESSAGE 33sskk07r3j 2585
INSERT 1o3gde8i54tgt 1874
RECEIVED OBJECT MESSAGE 1o3gde8i54tgt 1874
INSERT i2ixrboywlv1 4780
RECEIVED OBJECT MESSAGE i2ixrboywlv1 4780
INSERT zwtyym43rr61 9273
RECEIVED OBJECT MESSAGE zwtyym43rr61 9273
INSERT 1im8h986ltsjb 4391
RECEIVED OBJECT MESSAGE 1im8h986ltsjb 4391
INSERT 12bcoiykhyxbz 2045
RECEIVED OBJECT MESSAGE 12bcoiykhyxbz 2045
INSERT 344m66wfi4pj 2932
RECEIVED OBJECT MESSAGE 344m66wfi4pj 2932
INSERT avu4p8rk1u49 1110
RECEIVED OBJECT MESSAGE avu4p8rk1u49 1110
INSERT 1qjb6odryerif 1602
RECEIVED OBJECT MESSAGE 1qjb6odryerif 1602
UPDATE sd8x343td04w 8207
UPDATE 1nkz2pyuxq6uf 8336
INSERT 1wd44tnl8sb21 3082
RECEIVED OBJECT MESSAGE 1wd44tnl8sb21 3082
INSERT 2osxfuuxdxbm 6578
RECEIVED OBJECT MESSAGE 2osxfuuxdxbm 6578
UPDATE 1siosbc26v8pk 3497
INSERT 136pfbsg1r4pm 6778
RECEIVED OBJECT MESSAGE 136pfbsg1r4pm 6778
INSERT 1hbb0a41y2s2v 6981
RECEIVED OBJECT MESSAGE 1hbb0a41y2s2v 6981
INSERT fqxegdpt6rl4 9823
RECEIVED OBJECT MESSAGE fqxegdpt6rl4 9823
REMOVE z66prw2lb7bp 1849
RECEIVED OBJECT MESSAGE z66prw2lb7bp 1849
INSERT 11rh97d29fhky 9445
RECEIVED OBJECT MESSAGE 11rh97d29fhky 9445
UPDATE ebh10c5v33z1 8965
UPDATE 1vd660sbipcp1 8809
UPDATE 15xnralb7gz9s 9511
INSERT 1kd8n3a1hs1e1 9622
RECEIVED OBJECT MESSAGE 1kd8n3a1hs1e1 9622
INSERT 1lrqknnl2wa9b 4794
RECEIVED OBJECT MESSAGE 1lrqknnl2wa9b 4794
INSERT 1sihm871ppxgs 5251
RECEIVED OBJECT MESSAGE 1sihm871ppxgs 5251
UPDATE 1hl9r1c3jtsww 7388
INSERT 1ktajkpo6zkrt 5676
RECEIVED OBJECT MESSAGE 1ktajkpo6zkrt 5676
UPDATE 2dja2aq45ctz 3865
INSERT 161mpey6d6vr5 4319
RECEIVED OBJECT MESSAGE 161mpey6d6vr5 4319
UPDATE 1cb45pn2a4avu 4032
UPDATE s0lllla2u71m 976
INSERT 5upsxkqis7yg 9610
RECEIVED OBJECT MESSAGE 5upsxkqis7yg 9610
UPDATE 1nz3rzwkpow1r 7827
INSERT 13vklojl41kiz 2462
RECEIVED OBJECT MESSAGE 13vklojl41kiz 2462
INSERT zdtjhvacufbe 4345
RECEIVED OBJECT MESSAGE zdtjhvacufbe 4345
UPDATE s7wc62ybsumu 1280
INSERT 4pbi0tkgevwx 8057
RECEIVED OBJECT MESSAGE 4pbi0tkgevwx 8057
INSERT 6ip3sgwf3g3v 5447
RECEIVED OBJECT MESSAGE 6ip3sgwf3g3v 5447
INSERT 2rsxw9lvikhb 5606
RECEIVED OBJECT MESSAGE 2rsxw9lvikhb 5606
INSERT ubllgdw263e 5556
RECEIVED OBJECT MESSAGE ubllgdw263e 5556
INSERT 1recoqmjit876 9174
RECEIVED OBJECT MESSAGE 1recoqmjit876 9174
INSERT sor1hnjar6c2 5791
RECEIVED OBJECT MESSAGE sor1hnjar6c2 5791
INSERT 1d7r2ebnidhwx 2335
RECEIVED OBJECT MESSAGE 1d7r2ebnidhwx 2335
INSERT 1rtu3pc4ds3t0 7070
RECEIVED OBJECT MESSAGE 1rtu3pc4ds3t0 7070
UPDATE 1s79t8kafadq1 8465
UPDATE ax93q52uqnsk 1767
REMOVE e6n7hkwvo8yp 4487
RECEIVED OBJECT MESSAGE e6n7hkwvo8yp 4487
INSERT 1j6hs0edfxrkf 8433
RECEIVED OBJECT MESSAGE 1j6hs0edfxrkf 8433
INSERT 1lcd1uj5ikznc 7508
RECEIVED OBJECT MESSAGE 1lcd1uj5ikznc 7508
UPDATE sedm4wcjn1b 8343
UPDATE 1afyqhqvum90o 9944
INSERT 1v1r9p1dqm52t 210
RECEIVED OBJECT MESSAGE 1v1r9p1dqm52t 210
INSERT sg0hnmqe4dem 1375
RECEIVED OBJECT MESSAGE sg0hnmqe4dem 1375
UPDATE 75t1sh43ixpo 3927
UPDATE vg47h8u8d22f 6993
UPDATE fck5afipv4pa 5141
UPDATE 1cyykiwoedg49 8312
INSERT f10y81rg9yml 6747
RECEIVED OBJECT MESSAGE f10y81rg9yml 6747
INSERT 7l3f1g1mee1m 1420
RECEIVED OBJECT MESSAGE 7l3f1g1mee1m 1420
INSERT m28pdjg9x95g 8483
RECEIVED OBJECT MESSAGE m28pdjg9x95g 8483
INSERT 1j8xcvo5xnpjb 7360
RECEIVED OBJECT MESSAGE 1j8xcvo5xnpjb 7360
UPDATE 1sew10cgbiejp 3257
UPDATE 16hhdy988j64o 6329
UPDATE qzsxk5twh1de 5195
UPDATE 1agu0r1l7510z 431
INSERT 637vkjsrwpi3 4456
RECEIVED OBJECT MESSAGE 637vkjsrwpi3 4456
INSERT v3ugf4y8ckrz 3271
RECEIVED OBJECT MESSAGE v3ugf4y8ckrz 3271
INSERT 1fle12g0d0zx5 8707
RECEIVED OBJECT MESSAGE 1fle12g0d0zx5 8707
INSERT 1xej9zfi0t8dm 2928
RECEIVED OBJECT MESSAGE 1xej9zfi0t8dm 2928
UPDATE pwiwzjaoubbk 8390
INSERT 1f70u0ia40x6k 401
RECEIVED OBJECT MESSAGE 1f70u0ia40x6k 401
UPDATE art5ao3f00em 9900
INSERT vn759fh7ubd0 1720
RECEIVED OBJECT MESSAGE vn759fh7ubd0 1720
INSERT 1w1bllg3eosbz 7590
RECEIVED OBJECT MESSAGE 1w1bllg3eosbz 7590
UPDATE 9pxmv4vpmddc 9141
INSERT w6nh1tlqir3t 8431
RECEIVED OBJECT MESSAGE w6nh1tlqir3t 8431
INSERT 111j0mkmjrhlr 7553
RECEIVED OBJECT MESSAGE 111j0mkmjrhlr 7553
INSERT hyg2h3m7qgd8 7779
RECEIVED OBJECT MESSAGE hyg2h3m7qgd8 7779
INSERT 14nkhq5vf2fs8 4448
RECEIVED OBJECT MESSAGE 14nkhq5vf2fs8 4448
INSERT 15xckpgp1svzs 4840
RECEIVED OBJECT MESSAGE 15xckpgp1svzs 4840
INSERT 4vtaoozysimn 7844
RECEIVED OBJECT MESSAGE 4vtaoozysimn 7844
INSERT duxumkaj9u9d 837
RECEIVED OBJECT MESSAGE duxumkaj9u9d 837
INSERT njkgl8ck5yl2 5271
RECEIVED OBJECT MESSAGE njkgl8ck5yl2 5271
INSERT 1x9ykjycto7pz 29
RECEIVED OBJECT MESSAGE 1x9ykjycto7pz 29
UPDATE 7tu7toe52nw9 8115
INSERT x1y805u22iy3 6559
RECEIVED OBJECT MESSAGE x1y805u22iy3 6559
INSERT m9pbcrb569xa 367
RECEIVED OBJECT MESSAGE m9pbcrb569xa 367
UPDATE j1oi3zdc1a9h 5242
UPDATE fiqdxeb9qmqa 1877
INSERT 302l3uigwltl 6179
RECEIVED OBJECT MESSAGE 302l3uigwltl 6179
INSERT xx85e4f0zbhy 3213
RECEIVED OBJECT MESSAGE xx85e4f0zbhy 3213
INSERT veji44ccim7q 188
RECEIVED OBJECT MESSAGE veji44ccim7q 188
INSERT sx7u7fe5gq30 6652
RECEIVED OBJECT MESSAGE sx7u7fe5gq30 6652
INSERT 1w8ryh4viycl8 4882
RECEIVED OBJECT MESSAGE 1w8ryh4viycl8 4882
INSERT 100ahar3u9h9x 2920
RECEIVED OBJECT MESSAGE 100ahar3u9h9x 2920
INSERT vwgz1hmx2ty4 72
RECEIVED OBJECT MESSAGE vwgz1hmx2ty4 72
INSERT 1erhw9czpwzpo 4113
RECEIVED OBJECT MESSAGE 1erhw9czpwzpo 4113
INSERT 95vj41irgy5u 3379
RECEIVED OBJECT MESSAGE 95vj41irgy5u 3379
INSERT 1gh5lvz2o9xnw 4171
RECEIVED OBJECT MESSAGE 1gh5lvz2o9xnw 4171
INSERT 18isrf6smcuh1 2392
RECEIVED OBJECT MESSAGE 18isrf6smcuh1 2392
INSERT 1tiek05d1byq2 511
RECEIVED OBJECT MESSAGE 1tiek05d1byq2 511
INSERT 1am7ssut5lhw6 1647
RECEIVED OBJECT MESSAGE 1am7ssut5lhw6 1647
INSERT 1mepqwdhs4a14 7800
RECEIVED OBJECT MESSAGE 1mepqwdhs4a14 7800
UPDATE 1btsuhaqfxheg 7232
INSERT 1axl8f9uwwiza 9581
RECEIVED OBJECT MESSAGE 1axl8f9uwwiza 9581
INSERT 1r8ctl0p5sojt 3607
RECEIVED OBJECT MESSAGE 1r8ctl0p5sojt 3607
UPDATE r09sraod9bdu 9494
INSERT 1oxjinecd0x1t 8898
RECEIVED OBJECT MESSAGE 1oxjinecd0x1t 8898
INSERT ubbramifgk2l 7782
RECEIVED OBJECT MESSAGE ubbramifgk2l 7782
UPDATE 1s0h223s7gmcp 5523
UPDATE 1fjirpbx38a21 4219
INSERT 7vdwh61fljb7 6492
RECEIVED OBJECT MESSAGE 7vdwh61fljb7 6492
UPDATE 4uuxh5dxormc 244
UPDATE 1wtvmom1lmwm8 9559
INSERT wu47ep5bxl2o 4597
RECEIVED OBJECT MESSAGE wu47ep5bxl2o 4597
INSERT 1icnimpttk4px 4539
RECEIVED OBJECT MESSAGE 1icnimpttk4px 4539
UPDATE 112vbkll1rj32 1642
UPDATE qub5j6dhxlqm 4851
UPDATE 90pt6t3vrrwv 5812
INSERT 1j39u3t15rvm4 8527
RECEIVED OBJECT MESSAGE 1j39u3t15rvm4 8527
UPDATE pvrxzd0n4iu7 1087
INSERT 1vzia89yavuuf 1959
RECEIVED OBJECT MESSAGE 1vzia89yavuuf 1959
INSERT 1nq2d0t41i47k 8819
RECEIVED OBJECT MESSAGE 1nq2d0t41i47k 8819
UPDATE xs8ick4ea6s2 4154
INSERT 1p0zwzuk5ah6b 5207
RECEIVED OBJECT MESSAGE 1p0zwzuk5ah6b 5207
INSERT 1gfatuxk2o62j 4241
RECEIVED OBJECT MESSAGE 1gfatuxk2o62j 4241
INSERT evgpnsmtvk5j 259
INSERT 1xliyynq7e0li 8309
RECEIVED OBJECT MESSAGE evgpnsmtvk5j 259
UPDATE 5s65mfw0uxo6 4748
RECEIVED OBJECT MESSAGE 1xliyynq7e0li 8309
UPDATE 151ot0gb16gc 7844
INSERT 9hrb8vo6bxdd 4201
RECEIVED OBJECT MESSAGE 9hrb8vo6bxdd 4201
INSERT laqk54r7rfwb 7216
RECEIVED OBJECT MESSAGE laqk54r7rfwb 7216
UPDATE 1grmm4am2frd5 7082
INSERT q2c0b960x97e 5383
RECEIVED OBJECT MESSAGE q2c0b960x97e 5383
UPDATE 1itkw8bmozyd1 8031
INSERT 1f2m3mtwe5cv8 518
RECEIVED OBJECT MESSAGE 1f2m3mtwe5cv8 518
UPDATE g5kf2dr94i2q 9129
UPDATE 14tafi90oyyc5 9494

We have also showed how a Spring Hibernate application can be migrated from JBoss AS7 to WebLogic 12c (or vice versa). The things to keep in mind when migrating Spring from one application server to another, is the classloading and how the used resources are to be configured on the application server.

References

[1] The Red Hat product reference library.
[2] JBoss Web Framework Kit 2: Spring Installation Guide for use with JBoss Web Framework Kit Edition 2.0.0.
[3] HornetQ User Manual.
[3] Spring Framework Reference Documentation.
[4] JMS (Java Message Service).

We are bringing you a free course on WebLogic Server. In the course you will learn the following:

• Introduction
  • WebLogic Server architecture (domain, admin server, managed server, node manager, cluster, machine)
  • WebLogic Server internal architecture (listen threads, socket muxer, execute queue, execute threads)
• JVM Tuning
  • Code generation
  • Memory management
  • Garbage collection performance
  • JRockit Mission Control
• Deployment
  • Packaging applications recommendations
  • Deployment descriptors
  • Deployment plans
  • Work Manager configuration
• Diagnostic Framework (logging, instrumentation, harvesting, watching and notifying)
• Class Loading
  • Bundled libraries
  • Shared libraries
  • Filtering class loader
• Security
  • JAAS introduction
  • Embedded LDAP
  • WebLogic Server security providers
  • Secure communications (SSL and TLS)
• Configure Resources
  • Configure data sources
  • Monitoring resources
  • Configure JMS environment
  • Migrating singleton services (leasing, migratable targets, service migration policy)
• Clustering
  • Unicast or multicast?
  • Node manager and machine set-up
  • Denial of service configuration
  • Network channels
  • Vertical and horizontal scaling
  • Caching by using Coherence
  • Apache HTTP Server and WebLogic plug-in set-up
• Scripting (WLST)

Note that this is a course you can do at your own pace. The exercises lead you step-by-step through the process of installing, configuring, deploying etcetera. The material consists of the following:

• installation, the necessary software to set-up the environment:
  • WebLogic Server
  • JRockit JVM
  • Apache HTTP Server
  • WebLogic Server Web Server Plugins
  • The Grinder
• software, contains the deployments (.ear and .war files) that are used in the exercises.
• voorbeelden.zip (means examples.zip in Dutch), contains the code that is used to build the applications and an example configuration that you are going to set-up in the exercises.
• weblogic-opgaven.pdf, the exercises. In the beginning of the exercises, references are made to a virtual environment. This environment can be anything you like. In the exercises a VMware virtual machine with Linux is taken as the basis.
• weblogic-sheets.pdf, the sheets.

Have fun!

We are bringing you a free course on GlassFish Server. In the course you will learn the following:

• Introduction
  • GlassFish Server architecture
  • GlassFish Server internal architecture (listen threads, threads pools, containers)
  • Command-line utility
  • Admin console
• JVM Tuning
  • Code generation
  • Memory management
  • Garbage collection performance
  • JConsole and JVisualVM
• Deployment
  • Packaging applications recommendations
  • Deployment descriptors
  • Server logging
  • Front-end configuration using the Apache HTTP Server and mod_jk
• Class Loading
• Security
  • JAAS introduction
  • Role-based security
  • Secure communications (SSL and TLS)
• Configure Resources
  • Configure data sources
  • Monitoring resources
  • Configure JMS environment
• Clustering
  • Unicast or multicast?
  • Denial of service configuration
  • Vertical and horizontal scaling
  • Caching
  • Load balancing
  • Load testing

Note that this is a course you can do at your own pace. The exercises lead you step-by-step through the process of installing, configuring, deploying etcetera. The material consists of the following:

• installation, the necessary software to set-up the environment:
  • GlassFish Application Server
  • Java SE Development Kit
  • Apache HTTP Server
  • mod_jk
  • The Grinder
• software, contains the deployments (.ear and .war files) that are used in the exercises.
• voorbeelden.zip (means examples.zip in Dutch), contains the code that is used to build the applications and an example configuration that you are going to set-up in the exercises.
• glassfish-opgaven.pdf, the exercises. In the beginning of the exercises, references are made to a virtual environment. This environment can be anything you like. In the exercises a vm-ware virtual machine with Linux is taken as the basis.
• glassfish-sheets.pdf, the sheets.

Have fun!

JavaOne 2012

Presenting at JavaOne 2012: Harness the Power and Performance of GlassFish Tuning for High Availability. The presentation slides can be found here.

In this post we show a step-by-step example of how to set-up a high available tuned Java EE environment. We start with installing a Java Virtual Machine (in this case HotSpot) and JBoss Application Server. We continue with configuring a standalone server, tune the operating system and the Java Virtual Machine. Subsequently, we show how to set-up data source and JMS resources that are used by an application (consisting of a servlet, persistence, a stateless enterprise bean and a message-driven bean). Have some words about thread-pooling and enterprise beans. Show how to use the command-line interface to obtain run-time statistics. Show the steps involved in setting up a cluster by using standalone servers and configure a load balancer. Finally, we take a look at how to configure a domain in order to manage multiple server instances from a single control point.

JBoss AS7 is designed around a new kernel, which is now based on:

• JBoss Modules – handle class loading of resources in the container. Sort of a thin bootstrap wrapper for executing an application in a modular environment.
• Modular Service Container – provides a way to install, uninstall, and manage services used by a container. It further enables resources injection into services and dependency management between services.

Preparing the environment

Some operating system tweaks are worth considering when we do not want to run against system restrictions:

• Packet loss minimization – The operating system buffers must be large enough to handle incoming network traffic while the application is paused during garbage collection. Usually UDP (User Datagram Protocol) is used in order to transmit multicast messages to server instances in a cluster; to limit the need to retransmit UDP messages the size of the operating system buffers must be set appropriately to avoid excessive UDP datagram loss.
• Maximum number of open file descriptors – Most operating systems handle sockets as a form of file access and use file descriptors to keep track of which sockets are open. To contain the resources per process, the operating system restricts the number of file descriptors per process. Linux limits the number of open file descriptors per process, by default this is equal to 1024. It could be that the 1024 limit does not offer optimal performance.
• TCP/IP – On some systems the default value for the time wait interval is too high and needs to be adjusted. When the number approaches the maximum number of file descriptors per process, the application’s throughput will degrade, i.e., new connections have to wait for a free space in the application’s file descriptor table.
• Timesources – Linux has several timesources to choose from; the fastest being TSC (time stamp counter) and is used by default. However, if during start-up inconsistencies are found Linux switches to a slower timesource. This can have a negative performance impact.
• Network interface card (NIC) – Configure the network card at it’s maximum link speed and at full duplex.

In the post Tuning GlassFish for Performance all the tweaks are explained in detail and the steps involved in adjusting the operating system are explained as well. The post also contains suggestions in tuning the application server and Java Virtual Machine which are applicable to the JBoss application server as well.

Installation

• Install a JDK (of which a distribution can be downloaded here).
• Install JBoss (of which a distribution can be downloaded here).

We will use the versions: jdk-6u31-linux-x64.bin for the JDK and jboss-as-7.1.0.Final.zip for JBoss. The installation steps are straight forward, for the JDK we can just run the bin file. JBoss can be installed by unzipping the file. When we are done we have the following directory structure:

/home/jboss
	/jboss-as-7.1.0.Final (${JBOSS_HOME)
	/jdk1.6.0_31 (${JAVA_HOME})

After the set-up, we perform a startup test to validate that there are no major problems with the Java VM/operating system combination. To test the set-up navigate to ${JBOSS_HOME}/bin and run standalone.sh. Before running standalone.sh add the following lines:

#!/bin/sh

# These lines must be added
export JAVA_HOME=/home/jboss/jdk1.6.0_31
export PATH=${PATH}:${JAVA_HOME}/bin

To set JVM parameters we can edit standalone.conf, for example,

#
# Specify options to pass to the Java VM.
#
if [ "x$JAVA_OPTS" = "x" ]; then
   JAVA_OPTS="-server -Xms512m -Xmx512m -XX:NewRatio=2 -XX:PermSize=256m -XX:MaxPermSize=256m -XX:+UseParallelGC -XX:ParallelGCThreads=2 -XX:MaxGCPauseMillis=200 -XX:GCTimeRatio=19 -XX:+UseParallelOldGC -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000"
   JAVA_OPTS="$JAVA_OPTS -Djboss.modules.system.pkgs=$JBOSS_MODULES_SYSTEM_PKGS -Djava.awt.headless=true"
   JAVA_OPTS="$JAVA_OPTS -Djboss.server.default.config=standalone.xml"
else
   echo "JAVA_OPTS already set in environment; overriding default settings with values: $JAVA_OPTS"
fi

More information on tuning the JVM can be found here. Note that the default configuration is standalone.xml. This configuration does not by default include all subsystems, for example, messaging is not included. The configuration file standalone-full.xml has all the subsystems included. In the examples below we will work with standalone.xml and add specific subsystems when needed, hereby keeping the memory footprint of the application server to what is needed. When the standalone.sh is run the following is observed:

=========================================================================

  JBoss Bootstrap Environment

  JBOSS_HOME: /home/jboss/jboss-as-7.1.0.Final

  JAVA: /home/jboss/jdk1.6.0_31/bin/java

  JAVA_OPTS:  -XX:+UseCompressedOops -XX:+TieredCompilation -server -Xms512m -Xmx512m -XX:NewRatio=2 -XX:PermSize=256m -XX:MaxPermSize=256m -XX:+UseParallelGC -XX:ParallelGCThreads=2 -XX:MaxGCPauseMillis=200 -XX:GCTimeRatio=19 -XX:+UseParallelOldGC -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Djboss.server.default.config=standalone.xml

=========================================================================

12:32:01,842 INFO  [org.jboss.modules] JBoss Modules version 1.1.1.GA
12:32:02,010 INFO  [org.jboss.msc] JBoss MSC version 1.0.2.GA
12:32:02,057 INFO  [org.jboss.as] JBAS015899: JBoss AS 7.1.0.Final "Thunder" starting
12:32:03,047 INFO  [org.xnio] XNIO Version 3.0.3.GA
12:32:03,063 INFO  [org.jboss.as.server] JBAS015888: Creating http management service using socket-binding (management-http)
12:32:03,078 INFO  [org.xnio.nio] XNIO NIO Implementation Version 3.0.3.GA
12:32:03,088 INFO  [org.jboss.remoting] JBoss Remoting version 3.2.2.GA
12:32:03,145 INFO  [org.jboss.as.logging] JBAS011502: Removing bootstrap log handlers
12:32:03,151 INFO  [org.jboss.as.configadmin] (ServerService Thread Pool -- 26) JBAS016200: Activating ConfigAdmin Subsystem
12:32:03,212 INFO  [org.jboss.as.security] (ServerService Thread Pool -- 44) JBAS013101: Activating Security Subsystem
12:32:03,216 INFO  [org.jboss.as.security] (MSC service thread 1-3) JBAS013100: Current PicketBox version=4.0.6.final
12:32:03,231 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 31) JBAS010280: Activating Infinispan subsystem.
12:32:03,237 INFO  [org.jboss.as.osgi] (ServerService Thread Pool -- 39) JBAS011940: Activating OSGi Subsystem
12:32:03,282 INFO  [org.jboss.as.webservices] (ServerService Thread Pool -- 48) JBAS015537: Activating WebServices Extension
12:32:03,347 INFO  [org.jboss.as.naming] (ServerService Thread Pool -- 38) JBAS011800: Activating Naming Subsystem
12:32:03,399 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 27) JBAS010403: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
12:32:03,406 INFO  [org.jboss.as.connector] (MSC service thread 1-2) JBAS010408: Starting JCA Subsystem (JBoss IronJacamar 1.0.7.Final)
12:32:03,478 INFO  [org.jboss.as.naming] (MSC service thread 1-1) JBAS011802: Starting Naming Service
12:32:03,605 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-1) JBAS015400: Bound mail session [java:jboss/mail/Default]
12:32:03,633 INFO  [org.jboss.ws.common.management.AbstractServerConfig] (MSC service thread 1-2) JBoss Web Services - Stack CXF Server 4.0.1.GA
12:32:03,716 INFO  [org.apache.coyote.http11.Http11Protocol] (MSC service thread 1-4) Starting Coyote HTTP/1.1 on http--127.0.0.1-8080
12:32:03,976 INFO  [org.jboss.as.remoting] (MSC service thread 1-2) JBAS017100: Listening on /127.0.0.1:9999
12:32:03,984 INFO  [org.jboss.as.server.deployment.scanner] (MSC service thread 1-1) JBAS015012: Started FileSystemDeploymentService for directory /home/jboss/jboss-as-7.1.0.Final/standalone/deployments
12:32:03,986 INFO  [org.jboss.as.remoting] (MSC service thread 1-2) JBAS017100: Listening on /127.0.0.1:4447
12:32:04,119 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-3) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]
12:32:04,136 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss AS 7.1.0.Final "Thunder" started in 2726ms - Started 134 of 205 services (70 services are passive or on-demand)

To verify that the server is reachable, enter the following URL: http://localhost:8080.

To use the command line interface, we can start jboss-cli.sh (also add the JAVA_HOME and PATH variables to the script). To stop the started server, we can use

[jboss@axis-into-ict bin]$ ./jboss-cli.sh
You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.
[disconnected /] connect
[standalone@localhost:9999 /] :shutdown
{"outcome" => "success"}

We can also shuwdown the server by using:

[jboss@axis-into-ict bin]$ ./jboss-cli.sh --connect command=:shutdown
{"outcome" => "success"}

When browsing through the application server folders we see that its file system is divided into two parts: standalone servers (${JBOSS_HOME}/standalone) and domain servers (${JBOSS_HOME}/domain). An application server node which is not configured as part of a domain is qualified as a standalone server.

The ${JBOSS_HOME}/bin folder is where we start application server instances. As we already saw a standalone server can be started by using standalone.sh. When using a domain we will use domain.sh, which starts the application servers specified in the domain configuration file as we will see later on. The ${JBOSS_HOME}/modules folder contains the application server’s set of libraries, which are part of the server distribution. Each module is a pluggable unit. By using a static file system approach we can load modules, all we need to do is provide the location of the modules. When looking at the standalone.sh script (line 177 and on) we see an example of how to start the server and loading modules, for example, we could use

[jboss@axis-into-ict ~]$ cd jdk1.6.0_31/bin/
[jboss@axis-into-ict bin]$ ./java -jar /home/jboss/jboss-as-7.1.0.Final/jboss-modules.jar -mp /home/jboss/jboss-as-7.1.0.Final/modules/ org.jboss.as.standalone -Djboss.home.dir=/home/jboss/jboss-as-7.1.0.Final/

The module path (-mp) argument points to the root directory which will be searched by the module loader for module definitions. A module is defined by using an XML descriptor, for example,

<module xmlns="urn:jboss:module:1.1" name="javax.jms.api">
    <dependencies>
        <module name="javax.transaction.api" export="true"/>
    </dependencies>
    <resources>
        <resource-root path="jboss-jms-api_1.1_spec-1.0.0.Final.jar"/>
        <!-- Insert resources here -->
    </resources>
</module>

A module definition contains two main elements: the module dependencies and the resources defined in the module.

When we want to use the admin console, we first have to add a user. To this end run add-user.sh (${JBOSS_HOME}/bin) and follow the instructions given on the screen, for example,

[jboss@axis-into-ict bin]$ ./add-user.sh 

What type of user do you wish to add?
 a) Management User (mgmt-users.properties)
 b) Application User (application-users.properties)
(a): 

Enter the details of the new user to add.
Realm (ManagementRealm) :
Username : jboss
Password :
Re-enter Password :
About to add user 'jboss' for realm 'ManagementRealm'
Is this correct yes/no? yes
Added user 'jboss' to file '/home/jboss/jboss-as-7.1.0.Final/standalone/configuration/mgmt-users.properties'
Added user 'jboss' to file '/home/jboss/jboss-as-7.1.0.Final/domain/configuration/mgmt-users.properties'

Enter the following URL to access the admin console: http://hostname:9990/console and enter the log in credentials.

Configuration

The structure of the application server is maintained into a single file, which acts as a main reference for all server configurations. The default configuration files are named standalone.xml for standalone servers and domain.xml for an application server domain. An application server domain can be seen as a specialized server configuration, which also includes the domain and host controller set-up, which we will see later on. For now we will focus on the standalone configuration. The standalone.xml file is located in ${JBOSS_HOME}/standalone/configuration. An example looks as follows:

<server xmlns="urn:jboss:domain:1.1">
    <extensions>
        <extension module="org.jboss.as.clustering.infinispan"/>
        <extension module="org.jboss.as.configadmin"/>
		...
    </extensions>
    <management>
        <security-realms>
            <security-realm name="ManagementRealm">
                <authentication>
                    <properties path="mgmt-users.properties" relative-to="jboss.server.config.dir"/>
                </authentication>
            </security-realm>
            <security-realm name="ApplicationRealm">
                <authentication>
                    <properties path="application-users.properties" relative-to="jboss.server.config.dir"/>
                </authentication>
            </security-realm>
        </security-realms>
        <management-interfaces>
            <native-interface security-realm="ManagementRealm">
                <socket-binding native="management-native"/>
            </native-interface>
            <http-interface security-realm="ManagementRealm">
                <socket-binding http="management-http"/>
            </http-interface>
        </management-interfaces>
    </management>
    <profile>
        <subsystem xmlns="urn:jboss:domain:logging:1.1">...</subsystem>
        <subsystem xmlns="urn:jboss:domain:configadmin:1.0"/>
        <subsystem xmlns="urn:jboss:domain:datasources:1.0">...</subsystem>
        <subsystem xmlns="urn:jboss:domain:deployment-scanner:1.1">...</subsystem>
        <subsystem xmlns="urn:jboss:domain:ee:1.0"/>
        <subsystem xmlns="urn:jboss:domain:ejb3:1.2">...</subsystem>
        <subsystem xmlns="urn:jboss:domain:infinispan:1.1" default-cache-container="hibernate">...</subsystem>
        <subsystem xmlns="urn:jboss:domain:jaxrs:1.0"/>
        <subsystem xmlns="urn:jboss:domain:jca:1.1">...</subsystem>
        <subsystem xmlns="urn:jboss:domain:jdr:1.0"/>
        <subsystem xmlns="urn:jboss:domain:jmx:1.1">...</subsystem>
        <subsystem xmlns="urn:jboss:domain:jpa:1.0">...</subsystem>
        <subsystem xmlns="urn:jboss:domain:mail:1.0">...</subsystem>
        <subsystem xmlns="urn:jboss:domain:naming:1.1"/>
        <subsystem xmlns="urn:jboss:domain:osgi:1.2" activation="lazy">...</subsystem>
        <subsystem xmlns="urn:jboss:domain:pojo:1.0"/>
        <subsystem xmlns="urn:jboss:domain:remoting:1.1">...</subsystem>
        <subsystem xmlns="urn:jboss:domain:resource-adapters:1.0"/>
        <subsystem xmlns="urn:jboss:domain:sar:1.0"/>
        <subsystem xmlns="urn:jboss:domain:security:1.1">...</subsystem>
        <subsystem xmlns="urn:jboss:domain:threads:1.1"/>
        <subsystem xmlns="urn:jboss:domain:transactions:1.1">...</subsystem>
        <subsystem xmlns="urn:jboss:domain:web:1.1" default-virtual-server="default-host" native="false">...</subsystem>
        <subsystem xmlns="urn:jboss:domain:webservices:1.1">...</subsystem>
        <subsystem xmlns="urn:jboss:domain:weld:1.0"/>
    </profile>
    <interfaces>
        <interface name="management">
            <inet-address value="${jboss.bind.address.management:127.0.0.1}"/>
        </interface>
        <interface name="public">
            <inet-address value="${jboss.bind.address:127.0.0.1}"/>
        </interface>
    </interfaces>
    <socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
        <socket-binding name="http" port="8080"/>
        <socket-binding name="https" port="8443"/>
        <socket-binding name="management-native" interface="management" port="${jboss.management.native.port:9999}"/>
        <socket-binding name="management-http" interface="management" port="${jboss.management.http.port:9990}"/>
        <socket-binding name="management-https" interface="management" port="${jboss.management.https.port:9443}"/>
        <socket-binding name="osgi-http" interface="management" port="8090"/>
        <socket-binding name="remoting" port="4447"/>
        <socket-binding name="txn-recovery-environment" port="4712"/>
        <socket-binding name="txn-status-manager" port="4713"/>
        <outbound-socket-binding name="mail-smtp">
            <remote-destination host="localhost" port="25"/>
        </outbound-socket-binding>
    </socket-binding-group>
</server>

The application server contains a list of basic modules, called extensions, which are shared by all services. Extensions can be seen as a special type of module, which are used to extend the functionalities of the application server. They are stored in the ${JBOSS_HOME}/modules folder. Each extension is in turn picked up by the application server classloader at boot time, before any deployment. The application server has administration and management features that include a command-line interface (listens on port 9999) and an administration console (listens on port 9990). The profile can be seen as a collection of subsystems. Each subsystem contains a subset of functionalities used by the application server. For example, the web subsystem contains the definition of a set of connectors used by the container, the messaging subsystem defines the JMS configuration, and so on. The interfaces section contains the network interfaces and host names or IP addresses to which the application server is bound. Note that by default the server is bound to 127.0.0.1, which means it can only be reached locally. The IP address or host name can be changed at will, for example,

<interfaces>
	<interface name="management">
		<inet-address value="${jboss.bind.address.management:192.168.1.66}"/>
	</interface>
	<interface name="public">
		<inet-address value="${jboss.bind.address:192.168.1.66}"/>
	</interface>
</interfaces>

or even better gather the information from the network interface card, for example,

<interfaces>
	<interface name="management">
		<!--inet-address value="${jboss.bind.address.management:127.0.0.1}"/-->
		<nic name="eth0"/>
	</interface>
	<interface name="public">
		<!--inet-address value="${jboss.bind.address:127.0.0.1}"/-->
		<nic name="eth0"/>
	</interface>
</interfaces>

When we would now start the server the following is observed:

[jboss@axis-into-ict bin]$ ./standalone.sh
=========================================================================

  JBoss Bootstrap Environment

  JBOSS_HOME: /home/jboss/jboss-as-7.1.0.Final

  JAVA: /home/jboss/jdk1.6.0_31/bin/java

  JAVA_OPTS:  -XX:+UseCompressedOops -XX:+TieredCompilation -server -Xms512m -Xmx512m -XX:NewRatio=2 -XX:PermSize=256m -XX:MaxPermSize=256m -XX:+UseParallelGC -XX:ParallelGCThreads=2 -XX:MaxGCPauseMillis=200 -XX:GCTimeRatio=19 -XX:+UseParallelOldGC -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Djboss.server.default.config=standalone.xml

=========================================================================

16:29:17,748 INFO  [org.jboss.modules] JBoss Modules version 1.1.1.GA
16:29:17,935 INFO  [org.jboss.msc] JBoss MSC version 1.0.2.GA
16:29:17,983 INFO  [org.jboss.as] JBAS015899: JBoss AS 7.1.0.Final "Thunder" starting
16:29:18,991 INFO  [org.xnio] XNIO Version 3.0.3.GA
16:29:18,998 INFO  [org.jboss.as.server] JBAS015888: Creating http management service using socket-binding (management-http)
16:29:19,023 INFO  [org.xnio.nio] XNIO NIO Implementation Version 3.0.3.GA
16:29:19,043 INFO  [org.jboss.remoting] JBoss Remoting version 3.2.2.GA
16:29:19,070 INFO  [org.jboss.as.logging] JBAS011502: Removing bootstrap log handlers
16:29:19,086 INFO  [org.jboss.as.configadmin] (ServerService Thread Pool -- 26) JBAS016200: Activating ConfigAdmin Subsystem
16:29:19,167 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 31) JBAS010280: Activating Infinispan subsystem.
16:29:19,210 INFO  [org.jboss.as.webservices] (ServerService Thread Pool -- 48) JBAS015537: Activating WebServices Extension
16:29:19,214 INFO  [org.jboss.as.security] (ServerService Thread Pool -- 44) JBAS013101: Activating Security Subsystem
16:29:19,224 INFO  [org.jboss.as.security] (MSC service thread 1-1) JBAS013100: Current PicketBox version=4.0.6.final
16:29:19,226 INFO  [org.jboss.as.osgi] (ServerService Thread Pool -- 39) JBAS011940: Activating OSGi Subsystem
16:29:19,228 INFO  [org.jboss.as.naming] (ServerService Thread Pool -- 38) JBAS011800: Activating Naming Subsystem
16:29:19,290 INFO  [org.jboss.as.connector] (MSC service thread 1-2) JBAS010408: Starting JCA Subsystem (JBoss IronJacamar 1.0.7.Final)
16:29:19,351 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 27) JBAS010403: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
16:29:19,367 INFO  [org.jboss.as.naming] (MSC service thread 1-2) JBAS011802: Starting Naming Service
16:29:19,371 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-2) JBAS015400: Bound mail session [java:jboss/mail/Default]
16:29:19,586 INFO  [org.jboss.ws.common.management.AbstractServerConfig] (MSC service thread 1-4) JBoss Web Services - Stack CXF Server 4.0.1.GA
16:29:19,694 INFO  [org.apache.coyote.http11.Http11Protocol] (MSC service thread 1-3) Starting Coyote HTTP/1.1 on http--192.168.1.66-8080
16:29:19,915 INFO  [org.jboss.as.remoting] (MSC service thread 1-1) JBAS017100: Listening on /192.168.1.66:4447
16:29:19,916 INFO  [org.jboss.as.remoting] (MSC service thread 1-3) JBAS017100: Listening on /192.168.1.66:9999
16:29:19,949 INFO  [org.jboss.as.server.deployment.scanner] (MSC service thread 1-3) JBAS015012: Started FileSystemDeploymentService for directory /home/jboss/jboss-as-7.1.0.Final/standalone/deployments
16:29:20,101 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-4) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]
16:29:20,115 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss AS 7.1.0.Final "Thunder" started in 2615ms - Started 134 of 205 services (70 services are passive or on-demand)

The server is listening on another IP address than the 127.0.0.1, which is of course more useful. When using the command-line interface, we have to make sure we connect to the right host, for example,

[jboss@axis-into-ict bin]$ ./jboss-cli.sh
You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.
[disconnected /] connect
The controller is not available at localhost:9999
[disconnected /] connect 192.168.1.66:9999
[standalone@192.168.1.66:9999 /]

The command-line interface is a management tool for a domain or standalone server. It allows a user to connect to the domain controller or standalone server and execute management operations available through the de-typed management model. The native-management API is the entry point for management clients that rely on the application server’s native protocol to integrate with the management layer. It uses an open binary protocol and RPC style API based on a small number of Java types to describe and execute management operations against a domain or standalone server.

To get back at the configuration file. A socket binding makes up a named configuration of a socket. Here, we can configure the network ports, such as native management (command-line interface), which by default runs on port 9999.

Thread pools

Thread pools improve performance, and provide a means of resource management when a large number of tasks are executed asynchronously. By configuring the thread pool we can tune specific areas. The application server thread pool configuration can include:

• A thread factory creates new threads when needed. The thread factory is not included by default in the server configuration as it relies on defaults, which in general do not need to be modified.
• A bounded thread pool (or blocking bounded thread pool) is the most common kind of pool used by the application server, as it prevents resource exhaustion by defining a constraint on the thread pool size. This thread pool is also the most complex, i.e., it has to maintain a fixed-length queue and two pool sizes (core size and maximum size). When a new task is submitted, a new thread is created when the number of running threads is less than the core size. Otherwise, if there is room in the queue, the task is queued. When none of these options apply, the executor can create a new thread when the maximum size is not yet reached, otherwise the task can either be blocked (in the case of a blocking bounded thread pool) until there is room in the queue or it can be passed to a so-called hand-off executor (when one is specified, otherwise the task will be rejected).
• A unbounded thread pool has a core size and a queue with no upper bound. When a task is submitted and the number of running threads is less than the core size, a new thread is created. Otherwise, the task is placed in a queue. If too many tasks are submitted, an out of memory error can occur.
• A queueless thread pool (or blocking queueless thread pool) is a thread pool with no queue and has the same logic as the bounded thread pool (without the queue storage option).
• A scheduled thread pool is used for activities on the server-side that require running periodically or with delays.

An example configuration looks as follows

<subsystem xmlns="urn:jboss:domain:threads:1.1">
	<blocking-bounded-queue-thread-pool name="http-thread-pool">
		<queue-length count="500" per-cpu="200"/>
		<core-threads count="5" per-cpu="2"/>
		<max-threads count="10" per-cpu="3"/>
		<keepalive-time time="30" unit="seconds"/>
	</blocking-bounded-queue-thread-pool>
</subsystem>

In the configuration above we did not define a thread-factory. By not defining a thread-factory an appropriate default thread factory will be used. To view the configuration we can use the following (note that by using read-resource-description we get a detailed description of all the elements the can be configured)

[standalone@192.168.1.66:9999 /] /subsystem=threads/blocking-bounded-queue-thread-pool=http-thread-pool:read-resource-description
{
    "outcome" => "success",
    "result" => {
        "description" => "A thread pool executor with a bounded queue where threads submittings tasks may block. Such a thread pool has a core and maximum size and a specified queue length.  When a task is submitted, if the number of running threads is less than the core size, a new thread is created.  Otherwise, if there is room in the queue, the task is enqueued. Otherwise, if the number of running threads is less than the maximum size, a new thread is created. Otherwise, the caller blocks until room becomes available in the queue.",
        "attributes" => {
            "core-threads" => {
                "type" => INT,
                "description" => "The core thread pool size which is smaller than the maximum pool size. If undefined, the core thread pool size is the same as the maximum thread pool size.",
                "expressions-allowed" => true,
                "nillable" => true,
                "min" => 0L,
                "max" => 2147483647L,
                "access-type" => "read-write",
                "storage" => "configuration",
                "restart-required" => "no-services"
            },
            "current-thread-count" => {
                "type" => INT,
                "description" => "The current number of threads in the pool.",
                "expressions-allowed" => false,
                "nillable" => false,
                "access-type" => "metric",
                "storage" => "runtime"
            },
            "largest-thread-count" => {
                "type" => INT,
                "description" => "The largest number of threads that have ever simultaneously been in the pool.",
                "expressions-allowed" => false,
                "nillable" => false,
                "access-type" => "metric",
                "storage" => "runtime"
            },
            "keepalive-time" => {
                "type" => OBJECT,
                "description" => "Used to specify the amount of time that pool threads should be kept running when idle; if not specified, threads will run until the executor is shut down.",
                "expressions-allowed" => false,
                "nillable" => true,
                "value-type" => {
                    "time" => {
                        "type" => LONG,
                        "required" => true,
                        "description" => "The time"
                    },
                    "unit" => {
                        "type" => STRING,
                        "required" => true,
                        "description" => "The time unit"
                    }
                },
                "access-type" => "read-write",
                "storage" => "configuration",
                "restart-required" => "no-services"
            },
            "thread-factory" => {
                "type" => STRING,
                "description" => "Specifies the name of a specific thread factory to use to create worker threads. If not defined an appropriate default thread factory will be used.",
                "expressions-allowed" => false,
                "nillable" => true,
                "min-length" => 1L,
                "max-length" => 2147483647L,
                "access-type" => "read-write",
                "storage" => "configuration",
                "restart-required" => "all-services"
            },
            "queue-length" => {
                "type" => INT,
                "description" => "The queue length.",
                "expressions-allowed" => true,
                "nillable" => false,
                "min" => 0L,
                "max" => 2147483647L,
                "access-type" => "read-write",
                "storage" => "configuration",
                "restart-required" => "all-services"
            },
            "max-threads" => {
                "type" => INT,
                "description" => "The maximum thread pool size.",
                "expressions-allowed" => true,
                "nillable" => false,
                "min" => 0L,
                "max" => 2147483647L,
                "access-type" => "read-write",
                "storage" => "configuration",
                "restart-required" => "no-services"
            },
            "name" => {
                "type" => STRING,
                "description" => "The name of the thread pool.",
                "expressions-allowed" => false,
                "nillable" => true,
                "min-length" => 1L,
                "max-length" => 2147483647L,
                "access-type" => "read-only",
                "storage" => "configuration"
            },
            "rejected-count" => {
                "type" => INT,
                "description" => "The number of tasks that have been passed to the handoff-executor (if one is specified) or discarded.",
                "expressions-allowed" => false,
                "nillable" => false,
                "access-type" => "metric",
                "storage" => "runtime"
            },
            "allow-core-timeout" => {
                "type" => BOOLEAN,
                "description" => "Whether core threads may time out.",
                "expressions-allowed" => false,
                "nillable" => true,
                "default" => false,
                "access-type" => "read-write",
                "storage" => "configuration",
                "restart-required" => "no-services"
            }
        },
        "operations" => undefined,
        "children" => {}
    }
}
[standalone@192.168.1.66:9999 /] /subsystem=threads/blocking-bounded-queue-thread-pool=http-thread-pool:read-resource
{
    "outcome" => "success",
    "result" => {
        "allow-core-timeout" => false,
        "core-threads" => 50,
        "keepalive-time" => {
            "time" => 10L,
            "unit" => "SECONDS"
        },
        "max-threads" => 50,
        "name" => "http-thread-pool",
        "queue-length" => 50,
        "thread-factory" => undefined
    }
}

Data source

A note up front is in order. When installing and configuring an Oracle database on, for example, a Windows machine, it usually is bound to localhost. To change this, we have to edit listener.ora and tnsnames.ora (both are located in ${ORACLE_HOME}/product/11.2.0/dbhome_1/NETWORK/ADMIN). For example,

# listener.ora Network Configuration File: c:\oracle\product\11.2.0\dbhome_1\network\admin\listener.ora
# Generated by Oracle configuration tools.

SID_LIST_LISTENER =
  (SID_LIST =
    (SID_DESC =
      (SID_NAME = CLRExtProc)
      (ORACLE_HOME = c:\oracle\product\11.2.0\dbhome_1)
      (PROGRAM = extproc)
      (ENVS = "EXTPROC_DLLS=ONLY:c:\oracle\product\11.2.0\dbhome_1\bin\oraclr11.dll")
    )
  )

LISTENER =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1521))
      (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.65)(PORT = 1521))
    )
  )

ADR_BASE_LISTENER = c:\oracle

# tnsnames.ora Network Configuration File: c:\oracle\product\11.2.0\dbhome_1\network\admin\tnsnames.ora
# Generated by Oracle configuration tools.

ORACLR_CONNECTION_DATA =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1521))
    )
    (CONNECT_DATA =
      (SID = CLRExtProc)
      (PRESENTATION = RO)
    )
  )

ORCL11 =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.65)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = orcl11)
    )
  )

LISTENER_ORCL11 =
  (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.65)(PORT = 1521))

The procedure for installing a new module requires copying the .jar libraries in the appropriate modules path and adding a module.xml file, which declares the module and its dependencies. For the module path we are going to use ${JBOSS_HOME}/modules/${MODULE}/main, in which ${MODULE} will be com/oracle/database. We have the following directory structure

${JBOSS_HOME}/modules
	/com/oracle/database
		/main
			module.xml
			ojdbc6.jar

The contents of module.xml are the following

<module xmlns="urn:jboss:module:1.1" name="com.oracle.database">
    <resources>
        <resource-root path="ojdbc6.jar"/>
    </resources>
    <dependencies>
        <module name="javax.api"/>
        <module name="javax.transaction.api"/>
    </dependencies>
</module>

To configure a data source, find the entry subsystem xmlns="urn:jboss:domain:datasources:1.0" in standalone.xml, and add the following:

<subsystem xmlns="urn:jboss:domain:datasources:1.0">
	<datasources>
		<datasource jndi-name="java:/jdbc/OracleDS" pool-name="OracleDS" enabled="true" jta="true" use-java-context="true" use-ccm="true">
			<connection-url>jdbc:oracle:thin:@192.168.1.65:1521:orcl11</connection-url>
			<driver>oracle</driver>
			<pool>
				<min-pool-size>1</min-pool-size>
				<max-pool-size>15</max-pool-size>
				<prefill>true</prefill>
				<use-strict-min>true</use-strict-min>
		    </pool>
			<security>
				<user-name>example</user-name>
				<password>example</password>
			</security>
		    <statement>
				<prepared-statement-cache-size>10</prepared-statement-cache-size>
		    </statement>
			<timeout>
				<idle-timeout-minutes>0</idle-timeout-minutes>
				<query-timeout>600</query-timeout>
		    </timeout>
		</datasource>
		<drivers>
			<driver name="oracle" module="com.oracle.database">
				<driver-class>oracle.jdbc.OracleDriver</driver-class>
				<xa-datasource-class>oracle.jdbc.xa.client.OracleXADataSource</xa-datasource-class>
		    </driver>
		</drivers>
	</datasources>
</subsystem>

When the server is started, we can check the data sources present by using the command-line interface

[jboss@axis-into-ict bin]$ ./jboss-cli.sh
You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.
[disconnected /] connect 192.168.1.66:9999
[standalone@192.168.1.66:9999 /] /subsystem=datasources:installed-drivers-list
{
    "outcome" => "success",
    "result" => [
        {
            "driver-name" => "h2",
            "deployment-name" => undefined,
            "driver-module-name" => "com.h2database.h2",
            "module-slot" => "main",
            "driver-datasource-class-name" => "",
            "driver-xa-datasource-class-name" => "org.h2.jdbcx.JdbcDataSource",
            "driver-class-name" => "org.h2.Driver",
            "driver-major-version" => 1,
            "driver-minor-version" => 3,
            "jdbc-compliant" => true
        },
        {
            "driver-name" => "oracle",
            "deployment-name" => undefined,
            "driver-module-name" => "com.oracle.database",
            "module-slot" => "main",
            "driver-datasource-class-name" => "",
            "driver-xa-datasource-class-name" => "oracle.jdbc.xa.client.OracleXADataSource",
            "driver-class-name" => "oracle.jdbc.OracleDriver",
            "driver-major-version" => 11,
            "driver-minor-version" => 2,
            "jdbc-compliant" => true
        }
    ]
}

To test a connection

[standalone@192.168.1.66:9999 /] /subsystem=datasources/data-source=OracleDS:test-connection-in-pool
{
    "outcome" => "success",
    "result" => [true]
}

To obtain the configuration of the data source

[standalone@192.168.1.66:9999 /] /subsystem=datasources/data-source=OracleDS:read-resource
{
    "outcome" => "success",
    "result" => {
        "allocation-retry" => undefined,
        "allocation-retry-wait-millis" => undefined,
        "background-validation" => undefined,
        "background-validation-millis" => undefined,
        "blocking-timeout-wait-millis" => undefined,
        "check-valid-connection-sql" => undefined,
        "connection-properties" => undefined,
        "connection-url" => "jdbc:oracle:thin:@192.168.1.65:1521:orcl11",
        "datasource-class" => undefined,
        "driver-class" => undefined,
        "driver-name" => "oracle",
        "enabled" => true,
        "exception-sorter-class-name" => undefined,
        "exception-sorter-properties" => undefined,
        "flush-strategy" => undefined,
        "idle-timeout-minutes" => 0L,
        "jndi-name" => "java:/jdbc/OracleDS",
        "jta" => true,
        "max-pool-size" => 15,
        "min-pool-size" => 1,
        "new-connection-sql" => undefined,
        "password" => "example",
        "pool-prefill" => true,
        "pool-use-strict-min" => true,
        "prepared-statements-cache-size" => 10L,
        "query-timeout" => 600L,
        "reauth-plugin-class-name" => undefined,
        "reauth-plugin-properties" => undefined,
        "security-domain" => undefined,
        "set-tx-query-timeout" => "false",
        "share-prepared-statements" => "false",
        "spy" => "false",
        "stale-connection-checker-class-name" => undefined,
        "stale-connection-checker-properties" => undefined,
        "track-statements" => "\"NOWARN\"",
        "transaction-isolation" => undefined,
        "url-delimiter" => undefined,
        "url-selector-strategy-class-name" => undefined,
        "use-ccm" => true,
        "use-fast-fail" => "false",
        "use-java-context" => true,
        "use-try-lock" => undefined,
        "user-name" => "example",
        "valid-connection-checker-class-name" => undefined,
        "valid-connection-checker-properties" => undefined,
        "validate-on-match" => "false",
        "statistics" => {
            "pool" => undefined,
            "jdbc" => undefined
        }
    }
}

JMS

The JMS server configuration is done through the messaging subsystem. Which is not enabled when the default standalone.xml configuration is used. To enable the messaging subsystem we add the following to the standalone.xml configuration file (note that this can be copied from standalone-full.xml)

<server xmlns="urn:jboss:domain:1.1">
    <extensions>
		...
        <extension module="org.jboss.as.messaging"/>
		...
    </extensions>
    <management>...</management>
    <profile>
		...
        <subsystem xmlns="urn:jboss:domain:messaging:1.1">
            <hornetq-server>
                <persistence-enabled>true</persistence-enabled>
                <journal-file-size>102400</journal-file-size>
                <journal-min-files>2</journal-min-files>
                <connectors>
                    <netty-connector name="netty" socket-binding="messaging"/>
                    <netty-connector name="netty-throughput" socket-binding="messaging-throughput">
                        <param key="batch-delay" value="50"/>
                    </netty-connector>
                    <in-vm-connector name="in-vm" server-id="0"/>
                </connectors>
                <acceptors>
                    <netty-acceptor name="netty" socket-binding="messaging"/>
                    <netty-acceptor name="netty-throughput" socket-binding="messaging-throughput">
                        <param key="batch-delay" value="50"/>
                        <param key="direct-deliver" value="false"/>
                    </netty-acceptor>
                    <in-vm-acceptor name="in-vm" server-id="0"/>
                </acceptors>
                <security-settings>
                    <security-setting match="#">
                        <permission type="send" roles="guest"/>
                        <permission type="consume" roles="guest"/>
                        <permission type="createNonDurableQueue" roles="guest"/>
                        <permission type="deleteNonDurableQueue" roles="guest"/>
                    </security-setting>
                </security-settings>
                <address-settings>
                    <address-setting match="#">
                        <dead-letter-address>jms.queue.DLQ</dead-letter-address>
                        <expiry-address>jms.queue.ExpiryQueue</expiry-address>
                        <redelivery-delay>0</redelivery-delay>
                        <max-size-bytes>10485760</max-size-bytes>
                        <address-full-policy>BLOCK</address-full-policy>
                        <message-counter-history-day-limit>10</message-counter-history-day-limit>
                    </address-setting>
                </address-settings>
                <jms-connection-factories>
                    <connection-factory name="InVmConnectionFactory">
                        <connectors>
                            <connector-ref connector-name="in-vm"/>
                        </connectors>
                        <entries>
                            <entry name="java:/ConnectionFactory"/>
                        </entries>
                    </connection-factory>
                    <connection-factory name="RemoteConnectionFactory">
                        <connectors>
                            <connector-ref connector-name="netty"/>
                        </connectors>
                        <entries>
                            <entry name="RemoteConnectionFactory"/>
                            <entry name="java:jboss/exported/jms/RemoteConnectionFactory"/>
                        </entries>
                    </connection-factory>
                    <pooled-connection-factory name="hornetq-ra">
                        <transaction mode="xa"/>
                        <connectors>
                            <connector-ref connector-name="in-vm"/>
                        </connectors>
                        <entries>
                            <entry name="java:/JmsXA"/>
                        </entries>
                    </pooled-connection-factory>
                </jms-connection-factories>
                <jms-destinations>
                    <jms-queue name="testQueue">
                        <entry name="java:/queue/test"/>
                        <entry name="java:jboss/exported/jms/queue/test"/>
                    </jms-queue>
                    <jms-topic name="testTopic">
                        <entry name="java:/topic/test"/>
                        <entry name="java:jboss/exported/jms/topic/test"/>
                    </jms-topic>
                </jms-destinations>
            </hornetq-server>
        </subsystem>
    </profile>
    <interfaces>...</interfaces>
    <socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
	    ...
        <socket-binding name="messaging" port="5445"/>
        <socket-binding name="messaging-throughput" port="5455"/>
		...
    </socket-binding-group>
</server>

The transport of JMS messages is a key part of messaging system tuning. HornetQ uses Netty (a NIO client server framework) as its network library. The HornetQ transports configuration consist of acceptors and connectors. An acceptor defines which type of connection is accepted by the HornetQ server. A connector defines how to connect to a HornetQ server. HornetQ defines two the following types:

• In-VM (Intra Virtual Machine) – can be used when both HornetQ client and server run in the same virtual machine.
• netty – used when HornetQ client and server run in different virtual machines.

The JMS connection factories can be split into two kinds: In-VM connections and connections factories that can be used by remote clients. Each connection factory does reference a connector declaration, that is associated with a socket binding. The connection factory entry declaration specifies the JNDI name under which the factory will be exposed. Queues and topics are sub resources of the messaging subsystem. Each entry refers to a JNDI name of the queue or topic.

To check the configuration when the server is restarted, we can use

[standalone@192.168.1.66:9999 /] /subsystem=messaging/hornetq-server=default/connection-factory=InVmConnectionFactory:read-resource
{
    "outcome" => "success",
    "result" => {
        "auto-group" => false,
        "block-on-acknowledge" => false,
        "block-on-durable-send" => true,
        "block-on-non-durable-send" => false,
        "cache-large-message-client" => false,
        "call-timeout" => 30000L,
        "client-failure-check-period" => 30000L,
        "client-id" => undefined,
        "compress-large-messages" => false,
        "confirmation-window-size" => -1,
        "connection-load-balancing-policy-class-name" => "org.hornetq.api.core.client.loadbalance.RoundRobinConnectionLoadBalancingPolicy",
        "connection-ttl" => 60000L,
        "connector" => {"in-vm" => undefined},
        "consumer-max-rate" => -1,
        "consumer-window-size" => 1048576,
        "discovery-group-name" => undefined,
        "discovery-initial-wait-timeout" => undefined,
        "dups-ok-batch-size" => 1048576,
        "entries" => ["java:/ConnectionFactory"],
        "failover-on-initial-connection" => false,
        "failover-on-server-shutdown" => undefined,
        "group-id" => undefined,
        "ha" => false,
        "max-retry-interval" => 2000L,
        "min-large-message-size" => 102400,
        "pre-acknowledge" => false,
        "producer-max-rate" => -1,
        "producer-window-size" => 65536,
        "reconnect-attempts" => 0,
        "retry-interval" => 2000L,
        "retry-interval-multiplier" => 1.0,
        "scheduled-thread-pool-max-size" => 5,
        "thread-pool-max-size" => -1,
        "transaction-batch-size" => 1048576,
        "use-global-pools" => true
    }
}

[standalone@192.168.1.66:9999 /] /subsystem=messaging/hornetq-server=default/jms-queue=testQueue:read-resource
{
    "outcome" => "success",
    "result" => {
        "durable" => true,
        "entries" => [
            "java:/queue/test",
            "java:jboss/exported/jms/queue/test"
        ],
        "selector" => undefined
    }
}

Enterprise beans

By default, the following configuration is present

<subsystem xmlns="urn:jboss:domain:ejb3:1.2">
	<session-bean>
		<stateless>
			<bean-instance-pool-ref pool-name="slsb-strict-max-pool"/>
		</stateless>
		<stateful default-access-timeout="5000" cache-ref="simple"/>
		<singleton default-access-timeout="5000"/>
	</session-bean>
	<mdb>
		<resource-adapter-ref resource-adapter-name="hornetq-ra"/>
		<bean-instance-pool-ref pool-name="mdb-strict-max-pool"/>
	</mdb>
	<pools>
		<bean-instance-pools>
			<strict-max-pool name="slsb-strict-max-pool" max-pool-size="20" instance-acquisition-timeout="5" instance-acquisition-timeout-unit="MINUTES"/>
			<strict-max-pool name="mdb-strict-max-pool" max-pool-size="20" instance-acquisition-timeout="5" instance-acquisition-timeout-unit="MINUTES"/>
		</bean-instance-pools>
	</pools>
	<caches>
		<cache name="simple" aliases="NoPassivationCache"/>
		<cache name="passivating" passivation-store-ref="file" aliases="SimpleStatefulCache"/>
	</caches>
	<passivation-stores>
		<file-passivation-store name="file"/>
	</passivation-stores>
	<async thread-pool-name="default"/>
	<timer-service thread-pool-name="default">
		<data-store path="timer-service-data" relative-to="jboss.server.data.dir"/>
	</timer-service>
	<remote connector-ref="remoting-connector" thread-pool-name="default"/>
	<thread-pools>
		<thread-pool name="default">
			<max-threads count="10"/>
			<keepalive-time time="100" unit="milliseconds"/>
		</thread-pool>
	</thread-pools>
</subsystem>

One of the key responsibilities of the EJB container is the management of EJB component lifecycles. Bean instances are pooled and reused by the container to reduce the number of object instantiations. For example the lifecycle of a stateless session bean is as follows:

• The client obtains a reference to the stateless session bean. There are three options here. First, the reference can be injected into the client via an @EJB annotation. Second, the reference can be bound into the client’s local java:comp/env environment, using the @EJB annotation at class-level or a deployment descriptor entry, and the client can retrieve the reference with a JNDI lookup. Third, the client can look up the stateless session bean in the server’s global JNDI tree.
• The client invokes a business method on the bean reference.
• The container can reuse an existing instance of the stateless session bean from a pool, if available, or can instantiate a new bean instance. If a bean instance is newly created, the container will first perform dependency injection. If the stateless session bean implements the optional javax.ejb.SessionBean interface, the container will call setSessionContext. The session context will also be injected to any SessionContext field marked with the @Resource annotation. After completing injection of the session context and other dependencies, the container will call any methods marked with the @PostConstruct annotation.
• The container starts a transaction, if appropriate. This is controlled by the @javax.ejb.TransactionManagement and @javax.ejb.TransactionAttribute annotations.
• The container invokes the called business method on the bean instance, and the bean performs the desired operation.
• The container commits the transaction, if appropriate.
• The results of the business method call are returned to the client.
• The client may invoke additional business methods on the bean reference, each of which may end up invoking methods on a different bean instance.
• At some point, if the container decides to reduce the size of the bean instance pool, the container invokes any @PreDestroy methods on the bean instance, or ejbRemove if the bean implements the SessionBean interface. It is important to understand that this decision is not related to any client action.

There is a stronger link between the lifecycle of a stateful session bean and the client’s use of its reference:

• The client obtains a reference to the stateful session bean.
• The container instantiates a new bean instance. The container will perform dependency injection. If the stateful session bean implements the optional javax.ejb.SessionBean interface, the container will call setSessionContext. The session context will also be injected to any SessionContext field marked with the @Resource annotation. Any methods marked with the @PostConstruct will then be called.
• The client invokes a business method on the bean reference.
• The container starts a transaction, if appropriate.
• The container invokes a business method on the bean instance, and the bean performs the desired operation.
• The container commits the transaction, if appropriate.
• The results of the business method call are returned to the client.
• The client may invoke additional business methods on the bean reference and is assured that these additional calls will go to the same instance of the bean.
• The client calls a method annotated with @javax.ejb.Remove when it is done with the stateful session bean.
• The container invokes any @PreDestroy methods on the bean instance, or ejbRemove if the bean implements the SessionBean interface.

This introduction to the lifecycle of session EJB components represents a simplified view of the process. Additional complexities are introduced by limitations in the pool and cache sizes that we need to understand to configure the application properly.

EJB Pooling – JBoss maintains a pool of stateless session EJB instances for each stateless session bean deployment. This pool improves performance, because a client request can be handled immediately by any free initialized EJB instance. By default, the pool starts empty, and grows on demand. This can be controlled with the max-pool-size parameter. Within the pool configuration, there is also set a instance-acquisition-timeout of 5 minutes, which will come into play if the number of requests are larger than the maximum number of beans in the pool. If we set max-pool-size, we are choosing deliberately to throttle the application. If there are no idle bean instances, an thread making a new request will block until a bean becomes available or the transaction times out (so it must not be set too low). Message-driven beans are pooled in a manner very similar to stateless session beans. The number of message-driven bean instances can be controlled by using the max-pool-size parameter.

Stateful session bean instances are created as they are needed to service client requests. Between requests these instances reside in a bean-specific cache in the active state, ready for the next request. The size of the cache is limited by the max-size element (which is available for a passivation store). So long as the application never requires more than max-size instances of the stateful session bean at any given time to service all concurrent clients, there is no contention for the cache and performance is optimal. If we limit the number of beans in the cache, the managing of the cache is fairly active:

• If the cache is full, bean instances that are not being used at that moment for client requests are subject to passivation.
• If bean-managed transaction demarcation is used, a transaction may not be committed or rolled back at the end of a business method call. This leaves the bean instance associated with the transaction, pinned in the cache, and not eligible for passivation. Applications that keep transactions open between stateful session bean calls do not scale well and are difficult to manage.
• Passivation logic is controlled by the cache type; least recently used (LRU), passivates based the max-size and the time when the bean has not been used; not recently used (NRU), passivates beans only when the number of active beans approaches the max-size setting. The NRU strategy is lazy; the LRU strategy is eager. Although the LRU setting can be a convenient way of enforcing idle timeouts on the resources the objects encapsulate, it requires the cache to keep track of the bean’s access time and maintain an ordered list that gets updated after each bean access. Unless we have a good reason to need idle timeouts strictly enforced, most applications should probably use the NRU algorithm.

Passivation of beans means serialization of non-transient data in the bean to disk storage to release the memory used by the bean. The next request for the passivated bean will require activation, the reverse process, where bean elements are read from the disk store and the active bean instance is recreated in memory. Needless to say, passivation and activation cycles can be extremely expensive. The application should always call a @Remove method to delete the active bean instance from the cache when a client is through using the instance. Failure to call a @Remove method leaves the bean instance in the active state and consumes one slot in the cache, requiring eventual passivation during cache management to make room for additional client beans.

By using the command-line interface we retrieve some configuration information, for example,

[standalone@192.168.1.66:9999 /] /subsystem=ejb3:read-resource
{
    "outcome" => "success",
    "result" => {
        "cluster-passivation-store" => undefined,
        "default-clustered-sfsb-cache" => undefined,
        "default-entity-bean-instance-pool" => undefined,
        "default-entity-bean-optimistic-locking" => undefined,
        "default-mdb-instance-pool" => "mdb-strict-max-pool",
        "default-resource-adapter-name" => "hornetq-ra",
        "default-sfsb-cache" => "simple",
        "default-singleton-bean-access-timeout" => 5000L,
        "default-slsb-instance-pool" => "slsb-strict-max-pool",
        "default-stateful-bean-access-timeout" => 5000L,
        "in-vm-remote-interface-invocation-pass-by-value" => "true",
        "cache" => {
            "simple" => undefined,
            "passivating" => undefined
        },
        "file-passivation-store" => {"file" => undefined},
        "service" => {
            "timer-service" => undefined,
            "remote" => undefined,
            "async" => undefined
        },
        "strict-max-bean-instance-pool" => {
            "slsb-strict-max-pool" => undefined,
            "mdb-strict-max-pool" => undefined
        },
        "thread-pool" => {"default" => undefined}
    }
}

To get insight in the file passivation configuration we can use

[standalone@192.168.1.66:9999 /] /subsystem=ejb3/file-passivation-store=file:read-resource
{
    "outcome" => "success",
    "result" => {
        "groups-path" => "ejb3/groups",
        "idle-timeout" => 300L,
        "idle-timeout-unit" => "SECONDS",
        "max-size" => 100000,
        "name" => "file",
        "relative-to" => "jboss.server.data.dir",
        "sessions-path" => "ejb3/sessions",
        "subdirectory-count" => 100
    }
}

Or the stateless session bean pool configuration

[standalone@192.168.1.66:9999 /] /subsystem=ejb3/strict-max-bean-instance-pool=slsb-strict-max-pool:read-resource
{
    "outcome" => "success",
    "result" => {
        "max-pool-size" => 20,
        "name" => "slsb-strict-max-pool",
        "timeout" => 5L,
        "timeout-unit" => "MINUTES"
    }
}

Application

The application consists of a servlet that calls a stateless enterprise bean, which in turn (based on the task to be performed); inserts data in the database and sends a message to the queue; updates data in the database; or removes data from the database and also sends a message to the queue. We have to following persistence configuration,

<persistence xmlns="http://java.sun.com/xml/ns/persistence" version="2.0">
    <persistence-unit name="PersonPersistenceUnit">
        <provider>org.hibernate.ejb.HibernatePersistence</provider>
        <jta-data-source>java:/jdbc/OracleDS</jta-data-source>
        <class>model.entities.Person</class>
    </persistence-unit>
</persistence>

Note that in JBoss, Hibernate is the default persistence provider and we do not need to include the classes as these are already present on the application server. The jta-data-source is mapped to the data source we have configured above with a JNDI name of java:/jdbc/OracleDS. The stateless enterprise bean looks as follows

package model.logic;

import model.entities.Person;

import javax.ejb.Remote;

@Remote
public interface Company {
    public void insertPerson(Person person);

    public void removePerson(Integer sofinummer);

    public void updatePerson(Person person);
}

and has the following implementation

package model.logic;

import model.entities.Person;

import javax.annotation.PostConstruct;
import javax.annotation.PreDestroy;
import javax.annotation.Resource;
import javax.ejb.Stateless;
import javax.jms.*;
import javax.persistence.EntityManager;
import javax.persistence.PersistenceContext;

@Stateless(name = "Company", mappedName = "Company")
public class CompanyBean implements Company {

    @PersistenceContext(unitName = "PersonPersistenceUnit")
    private EntityManager entityManager;

    @Resource(name = "java:/JmsXA")
    private ConnectionFactory connectionFactory;
    @Resource(name = "java:/queue/test")
    private Queue destination;

    private Connection connection = null;
    private Session session = null;
    private MessageProducer messageProducer = null;

    @PostConstruct
    public void init() {
        try {
            connection = connectionFactory.createConnection();
            session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
            messageProducer = session.createProducer(destination);
        } catch (JMSException e) {
            if (connection != null) {
                try {
                    connection.close();
                } catch (JMSException f) {
                    e.printStackTrace();
                }
            }
        }
    }

    @PreDestroy
    public void release() {
        try {
            if (connection != null) {
                connection.close();
            }
        } catch (JMSException e) {
            e.printStackTrace();
        }
    }

    public void insertPerson(Person person) {
        if (findPerson(person.getSofinummer()) == null) {
            entityManager.persist(person);
            sendMessage(person);
        } else {
            updatePerson(person);
        }
    }

    public void removePerson(Integer sofinummer) {
        Person person = findPerson(sofinummer);
        if (person != null) {
            entityManager.remove(entityManager.merge(person));
            sendMessage(person);
        }
    }

    public void updatePerson(Person person) {
        entityManager.merge(person);
    }

    private Person findPerson(Integer sofinummer) {
        return entityManager.find(Person.class, sofinummer);
    }

    private void sendMessage(Person person) {
        try {
            ObjectMessage message = session.createObjectMessage();
            message.setObject(person);
            messageProducer.send(message);
        } catch (JMSException e) {
            e.printStackTrace();
        }
    }
}

Here, we inject a JMS connection factory (with JNDI java:/JmsXA) and a JMS queue (with JNDI java:/queue/test). The enterprise bean is initialized by setting-up a message producer, that will be used in the insert and remove tasks. The message-driven bean looks as follows

@MessageDriven(name = "CompanyMDB", activationConfig = {
        @ActivationConfigProperty(propertyName = "destinationType", propertyValue = "javax.jms.Queue"),
        @ActivationConfigProperty(propertyName = "destination", propertyValue = "java:/queue/test")
})
public class CompanyMDB implements MessageListener {

    public void onMessage(Message message) {
        if (message instanceof ObjectMessage) {
            ObjectMessage objectMessage = (ObjectMessage) message;
            try {
                System.out.println("RECEIVED OBJECT MESSAGE " + objectMessage.getObject());
            } catch (JMSException e) {
                e.printStackTrace();
            }
        }

        if (message instanceof TextMessage) {
            TextMessage textMessage = (TextMessage) message;
            try {
                System.out.println("RECEIVED TEXT MESSAGE " + textMessage.getText());
            } catch (JMSException e) {
                e.printStackTrace();
            }
        }
    }
}

The servlet looks as follows

package userinterface.servlets;

import model.entities.Person;
import model.logic.Company;

import javax.ejb.EJB;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.PrintWriter;
import java.util.Random;

@WebServlet(name = "TestServlet", urlPatterns = "/testservlet")
public class TestServlet extends HttpServlet {

    @EJB(name = "Company", mappedName = "java:global/LoadTest6/Model/Company")
    Company company;

    private Random generator = null;

    @Override
    public void init() throws ServletException {
        generator = new Random();
    }

    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        Person person = createPerson();

        if (generator.nextDouble() < 0.01) {
            company.removePerson(person.getSofinummer());
        } else {
            company.insertPerson(person);
        }
    }

    @Override
    public void destroy() {
        generator = null;
    }

    private Person createPerson() {
        Person person = new Person();
        person.setNaam(Long.toString(Math.abs(generator.nextLong()), 36));
        person.setSofinummer(generator.nextInt(10000));
        return person;
    }
}

When an enterprise bean is deployed is will be bound to JNDI as java:global/<deployment-name>/<module-name>/<mapped-name>.

Deployment

In the standalone mode JBoss comes with a deployment scanner. Its job is to monitor a directory for new files and to deploy those files. In the standalone.xml configuration file the following entry can be found

<subsystem xmlns="urn:jboss:domain:deployment-scanner:1.1">
	<deployment-scanner path="deployments" relative-to="jboss.server.base.dir" scan-interval="5000"/>
</subsystem>

To auto-deploy an application we can put our .ear or .war file into the ${JBOSS_HOME}/deployments directory. We can also use the preferred command-line interface

[jboss@axis-into-ict bin]$ ./jboss-cli.sh
You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.
[disconnected /] connect 192.168.1.66:9999
[standalone@192.168.1.66:9999 /] deploy /home/jboss/jboss-as-7.1.0.Final/deploy/LoadTest6.ear

The following is observed in the logging

[jboss@axis-into-ict bin]$ ./standalone.sh
=========================================================================

  JBoss Bootstrap Environment

  JBOSS_HOME: /home/jboss/jboss-as-7.1.0.Final

  JAVA: /home/jboss/jdk1.6.0_31/bin/java

  JAVA_OPTS:  -XX:+UseCompressedOops -XX:+TieredCompilation -server -Xms512m -Xmx512m -XX:NewRatio=2 -XX:PermSize=256m -XX:MaxPermSize=256m -XX:+UseParallelGC -XX:ParallelGCThreads=2 -XX:MaxGCPauseMillis=200 -XX:GCTimeRatio=19 -XX:+UseParallelOldGC -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Djboss.server.default.config=standalone.xml

=========================================================================

13:45:06,255 INFO  [org.jboss.modules] JBoss Modules version 1.1.1.GA
13:45:06,420 INFO  [org.jboss.msc] JBoss MSC version 1.0.2.GA
13:45:06,464 INFO  [org.jboss.as] JBAS015899: JBoss AS 7.1.0.Final "Thunder" starting
13:45:07,295 INFO  [org.xnio] XNIO Version 3.0.3.GA
13:45:07,302 INFO  [org.jboss.as.server] JBAS015888: Creating http management service using socket-binding (management-http)
13:45:07,312 INFO  [org.xnio.nio] XNIO NIO Implementation Version 3.0.3.GA
13:45:07,320 INFO  [org.jboss.remoting] JBoss Remoting version 3.2.2.GA
13:45:07,356 INFO  [org.jboss.as.logging] JBAS011502: Removing bootstrap log handlers
13:45:07,400 INFO  [org.jboss.as.configadmin] (ServerService Thread Pool -- 27) JBAS016200: Activating ConfigAdmin Subsystem
13:45:07,405 INFO  [org.jboss.as.webservices] (ServerService Thread Pool -- 50) JBAS015537: Activating WebServices Extension
13:45:07,420 INFO  [org.jboss.as.osgi] (ServerService Thread Pool -- 41) JBAS011940: Activating OSGi Subsystem
13:45:07,422 INFO  [org.jboss.as.security] (ServerService Thread Pool -- 46) JBAS013101: Activating Security Subsystem
13:45:07,424 INFO  [org.jboss.as.naming] (ServerService Thread Pool -- 40) JBAS011800: Activating Naming Subsystem
13:45:07,442 INFO  [org.jboss.as.security] (MSC service thread 1-2) JBAS013100: Current PicketBox version=4.0.6.final
13:45:07,471 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 32) JBAS010280: Activating Infinispan subsystem.
13:45:07,523 INFO  [org.jboss.as.connector] (MSC service thread 1-1) JBAS010408: Starting JCA Subsystem (JBoss IronJacamar 1.0.7.Final)
13:45:07,546 INFO  [org.jboss.as.naming] (MSC service thread 1-4) JBAS011802: Starting Naming Service
13:45:07,645 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 28) JBAS010403: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
13:45:07,730 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-4) JBAS015400: Bound mail session [java:jboss/mail/Default]
13:45:07,773 INFO  [org.jboss.ws.common.management.AbstractServerConfig] (MSC service thread 1-3) JBoss Web Services - Stack CXF Server 4.0.1.GA
13:45:07,822 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 28) JBAS010403: Deploying JDBC-compliant driver class oracle.jdbc.OracleDriver (version 11.2)
13:45:07,923 INFO  [org.apache.coyote.http11.Http11Protocol] (MSC service thread 1-4) Starting Coyote HTTP/1.1 on http-axis-into-ict.nl-192.168.1.66-8080
13:45:07,938 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-3) live server is starting with configuration HornetQ Configuration (clustered=false,backup=false,sharedStore=true,journalDirectory=/home/jboss/jboss-as-7.1.0.Final/standalone/data/messagingjournal,bindingsDirectory=/home/jboss/jboss-as-7.1.0.Final/standalone/data/messagingbindings,largeMessagesDirectory=/home/jboss/jboss-as-7.1.0.Final/standalone/data/messaginglargemessages,pagingDirectory=/home/jboss/jboss-as-7.1.0.Final/standalone/data/messagingpaging)
13:45:07,946 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-3) Waiting to obtain live lock
13:45:08,014 INFO  [org.hornetq.core.persistence.impl.journal.JournalStorageManager] (MSC service thread 1-3) Using AIO Journal
13:45:08,217 INFO  [org.jboss.as.remoting] (MSC service thread 1-1) JBAS017100: Listening on /192.168.1.66:9999
13:45:08,250 INFO  [org.jboss.as.server.deployment.scanner] (MSC service thread 1-2) JBAS015012: Started FileSystemDeploymentService for directory /home/jboss/jboss-as-7.1.0.Final/standalone/deployments
13:45:08,267 INFO  [org.jboss.as.remoting] (MSC service thread 1-1) JBAS017100: Listening on axis-into-ict.nl/192.168.1.66:4447
13:45:08,272 INFO  [org.hornetq.core.server.impl.AIOFileLockNodeManager] (MSC service thread 1-3) Waiting to obtain live lock
13:45:08,273 INFO  [org.hornetq.core.server.impl.AIOFileLockNodeManager] (MSC service thread 1-3) Live Server Obtained live lock
13:45:08,339 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]
13:45:08,340 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) JBAS010400: Bound data source [java:/jdbc/OracleDS]
13:45:08,649 INFO  [org.hornetq.core.remoting.impl.netty.NettyAcceptor] (MSC service thread 1-3) Started Netty Acceptor version 3.2.5.Final-a96d88c axis-into-ict.nl:5445 for CORE protocol
13:45:08,651 INFO  [org.hornetq.core.remoting.impl.netty.NettyAcceptor] (MSC service thread 1-3) Started Netty Acceptor version 3.2.5.Final-a96d88c axis-into-ict.nl:5455 for CORE protocol
13:45:08,652 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-3) Server is now live
13:45:08,653 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-3) HornetQ Server version 2.2.11.Final (HQ_2_2_11_FINAL_AS7, 122) [a0959f8b-5cbd-11e1-899c-000c2976c82d]) started
13:45:08,656 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-1) trying to deploy queue jms.topic.testTopic
13:45:08,682 INFO  [org.jboss.as.messaging] (MSC service thread 1-1) JBAS011601: Bound messaging object to jndi name java:/topic/test
13:45:08,683 INFO  [org.jboss.as.messaging] (MSC service thread 1-1) JBAS011601: Bound messaging object to jndi name java:jboss/exported/jms/topic/test
13:45:08,684 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-3) trying to deploy queue jms.queue.testQueue
13:45:08,689 INFO  [org.jboss.as.messaging] (MSC service thread 1-3) JBAS011601: Bound messaging object to jndi name java:jboss/exported/jms/queue/test
13:45:08,689 INFO  [org.jboss.as.messaging] (MSC service thread 1-3) JBAS011601: Bound messaging object to jndi name java:/queue/test
13:45:08,704 INFO  [org.jboss.as.messaging] (MSC service thread 1-4) JBAS011601: Bound messaging object to jndi name java:jboss/exported/jms/RemoteConnectionFactory
13:45:08,706 INFO  [org.jboss.as.messaging] (MSC service thread 1-4) JBAS011601: Bound messaging object to jndi name java:/RemoteConnectionFactory
13:45:08,712 INFO  [org.jboss.as.messaging] (MSC service thread 1-2) JBAS011601: Bound messaging object to jndi name java:/ConnectionFactory
13:45:08,754 INFO  [org.jboss.as.deployment.connector] (MSC service thread 1-2) JBAS010406: Registered connection factory java:/JmsXA
13:45:08,763 INFO  [org.hornetq.ra.HornetQResourceAdapter] (MSC service thread 1-2) HornetQ resource adaptor started
13:45:08,763 INFO  [org.jboss.as.connector.services.ResourceAdapterActivatorService$ResourceAdapterActivator] (MSC service thread 1-2) IJ020002: Deployed: file://RaActivatorhornetq-ra
13:45:08,767 INFO  [org.jboss.as.deployment.connector] (MSC service thread 1-4) JBAS010401: Bound JCA ConnectionFactory [java:/JmsXA]
13:45:08,844 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss AS 7.1.0.Final "Thunder" started in 2784ms - Started 157 of 231 services (72 services are passive or on-demand)
13:46:30,698 INFO  [org.jboss.as.repository] (management-handler-threads - 6) JBAS014900: Content added at location /home/jboss/jboss-as-7.1.0.Final/standalone/data/content/16/1f51dde7f085c822cc4c68b306d57f1bee902d/content
13:46:30,714 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-4) JBAS015876: Starting deployment of "LoadTest6.ear"
13:46:30,773 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-2) JBAS015876: Starting deployment of "Web.war"
13:46:30,773 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-1) JBAS015876: Starting deployment of "Model.jar"
13:46:30,838 INFO  [org.jboss.as.jpa] (MSC service thread 1-4) JBAS011401: Read persistence.xml for PersonPersistenceUnit
13:46:30,961 INFO  [org.jboss.as.ejb3.deployment.processors.EjbJndiBindingsDeploymentUnitProcessor] (MSC service thread 1-2) JNDI bindings for session bean named Company in deployment unit subdeployment "Model.jar" of deployment "LoadTest6.ear" are as follows:

        java:global/LoadTest6/Model/Company!model.logic.Company
        java:app/Model/Company!model.logic.Company
        java:module/Company!model.logic.Company
        java:jboss/exported/LoadTest6/Model/Company!model.logic.Company
        java:global/LoadTest6/Model/Company
        java:app/Model/Company
        java:module/Company

13:46:31,177 INFO  [org.jboss.as.jpa] (MSC service thread 1-2) JBAS011402: Starting Persistence Unit Service 'LoadTest6.ear/Model.jar#PersonPersistenceUnit'
13:46:31,234 INFO  [org.jboss.as.ejb3] (MSC service thread 1-4) JBAS014142: Started message driven bean 'CompanyMDB' with 'hornetq-ra' resource adapter
13:46:31,360 INFO  [org.jboss.web] (MSC service thread 1-3) JBAS018210: Registering web context: /LoadTest6
13:46:31,387 INFO  [org.hibernate.annotations.common.Version] (MSC service thread 1-2) HCANN000001: Hibernate Commons Annotations {4.0.1.Final}
13:46:31,393 INFO  [org.hibernate.Version] (MSC service thread 1-2) HHH000412: Hibernate Core {4.0.1.Final}
13:46:31,397 INFO  [org.hibernate.cfg.Environment] (MSC service thread 1-2) HHH000206: hibernate.properties not found
13:46:31,398 INFO  [org.hibernate.cfg.Environment] (MSC service thread 1-2) HHH000021: Bytecode provider name : javassist
13:46:31,414 INFO  [org.hibernate.ejb.Ejb3Configuration] (MSC service thread 1-2) HHH000204: Processing PersistenceUnitInfo [
        name: PersonPersistenceUnit
        ...]
13:46:31,532 INFO  [org.hibernate.service.jdbc.connections.internal.ConnectionProviderInitiator] (MSC service thread 1-2) HHH000130: Instantiating explicit connection provider: org.hibernate.ejb.connection.InjectedDataSourceConnectionProvider
13:46:31,639 INFO  [org.hibernate.dialect.Dialect] (MSC service thread 1-2) HHH000400: Using dialect: org.hibernate.dialect.Oracle10gDialect
13:46:31,656 INFO  [org.hibernate.engine.transaction.internal.TransactionFactoryInitiator] (MSC service thread 1-2) HHH000268: Transaction strategy: org.hibernate.engine.transaction.internal.jta.CMTTransactionFactory
13:46:31,660 INFO  [org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory] (MSC service thread 1-2) HHH000397: Using ASTQueryTranslatorFactory
13:46:31,698 INFO  [org.hibernate.validator.util.Version] (MSC service thread 1-2) Hibernate Validator 4.2.0.Final
13:46:32,054 INFO  [org.jboss.as.server] (management-handler-threads - 6) JBAS018559: Deployed "LoadTest6.ear"

When we do a few tests (access the application by using http://hostname:8080/LoadTest6/testservlet) the following is observed in the logging:

13:50:37,002 INFO  [org.jboss.ejb.client] (http-axis-into-ict.nl-192.168.1.66-8080-1) JBoss EJB Client version 1.0.2.Final
13:50:37,484 INFO  [stdout] (Thread-5 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE 1a0100uhg5tiu 7907
13:50:39,692 INFO  [stdout] (Thread-6 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE iuhpgb6pcs13 9223
13:50:40,164 INFO  [stdout] (Thread-5 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE 6m08798r5ehs 2500
13:50:40,621 INFO  [stdout] (Thread-6 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE 12wx9zs77r9cx 4262
13:50:41,044 INFO  [stdout] (Thread-5 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE 6dh6zc60tw1y 285
13:50:41,850 INFO  [stdout] (Thread-6 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE extyyn0pov1w 674
13:50:42,242 INFO  [stdout] (Thread-5 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE pv0ypyftodqy 8381
13:50:42,625 INFO  [stdout] (Thread-6 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE 1xh0e0cn7ggsa 5954
13:50:43,026 INFO  [stdout] (Thread-5 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE 1fiaxr5gsxkuo 4480
13:50:43,470 INFO  [stdout] (Thread-6 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE 1dnmh60s6fl47 7487
13:50:43,763 INFO  [stdout] (Thread-5 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE 15mxq08abiozd 6698
13:50:44,183 INFO  [stdout] (Thread-6 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE hzgxdkqw1zo7 9573
13:50:44,433 INFO  [stdout] (Thread-5 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE 4mvfve8o8gfb 1940
13:50:44,810 INFO  [stdout] (Thread-6 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE 1skgi6orrjiaw 3535
13:50:45,066 INFO  [stdout] (Thread-5 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE vb52wrkqgx81 5712
13:50:45,598 INFO  [stdout] (Thread-6 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE 1rxts2saz9fhi 7821
13:50:45,697 INFO  [stdout] (Thread-5 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE j8gonmtmsd8g 4627
13:50:45,824 INFO  [stdout] (Thread-5 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE da4b2qrqpt7b 9219
13:50:45,864 INFO  [stdout] (Thread-6 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE yqlwfar66vfk 4998
13:50:45,880 INFO  [stdout] (Thread-4 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE pzqok192zmnz 7551
13:50:45,882 INFO  [stdout] (Thread-5 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE jcyf4wttbwmy 1071
13:50:45,901 INFO  [stdout] (Thread-5 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE 1jg4yksazwtkx 5685
13:50:45,904 INFO  [stdout] (Thread-8 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE 1mv38p7onaidx 4399
13:50:45,915 INFO  [stdout] (Thread-7 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE wvsj6gq17e2i 586
13:50:45,936 INFO  [stdout] (Thread-8 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE 1c9ij29bk00ru 3746
13:50:45,973 INFO  [stdout] (Thread-7 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE ue08qs0k9unr 6877
13:50:46,008 INFO  [stdout] (Thread-8 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE hb372i5a3dai 3762
13:50:46,048 INFO  [stdout] (Thread-7 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE cg1pr42138dr 1066
13:50:46,086 INFO  [stdout] (Thread-8 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE 1k28o323a0k52 3510
13:50:46,121 INFO  [stdout] (Thread-7 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE fo0pn8x69hr9 2643
13:50:46,194 INFO  [stdout] (Thread-8 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE 1134xboronc8u 3323
13:50:46,235 INFO  [stdout] (Thread-7 (HornetQ-client-global-threads-1977385357)) RECEIVED OBJECT MESSAGE 16znhvr4pu6hy 5842

To obtain run-time information we can use the command-line interface, for example,

[jboss@axis-into-ict ~]$ cd jboss-as-7.1.0.Final/bin/
[jboss@axis-into-ict bin]$ ./jboss-cli.sh
You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.
[disconnected /] connect 192.168.1.66:9999
[standalone@192.168.1.66:9999 /] cd subsystem=messaging/hornetq-server=default/jms-queue=testQueue
[standalone@192.168.1.66:9999 jms-queue=testQueue] pwd
/subsystem=messaging/hornetq-server=default/jms-queue=testQueue
[standalone@192.168.1.66:9999 jms-queue=testQueue] read-attribute delivering-count
0
[standalone@192.168.1.66:9999 jms-queue=testQueue] read-attribute consumer-count
15
[standalone@192.168.1.66:9999 jms-queue=testQueue] read-attribute message-count
0L
[standalone@192.168.1.66:9999 jms-queue=testQueue] read-attribute messages-added
32L
[standalone@192.168.1.66:9999 jms-queue=testQueue] cd /
[standalone@192.168.1.66:9999 /] cd subsystem=datasources/data-source=OracleDS
[standalone@192.168.1.66:9999 data-source=OracleDS] ls
connection-properties                                       statistics                                                  allocation-retry=undefined
allocation-retry-wait-millis=undefined                      background-validation=undefined                             background-validation-millis=undefined
blocking-timeout-wait-millis=undefined                      check-valid-connection-sql=undefined                        connection-url=jdbc:oracle:thin:@192.168.1.65:1521:orcl11
datasource-class=undefined                                  driver-class=undefined                                      driver-name=oracle
enabled=true                                                exception-sorter-class-name=undefined                       exception-sorter-properties=undefined
flush-strategy=undefined                                    idle-timeout-minutes=0                                      jndi-name=java:/jdbc/OracleDS
jta=true                                                    max-pool-size=15                                            min-pool-size=1
new-connection-sql=undefined                                password=example                                            pool-prefill=true
pool-use-strict-min=true                                    prepared-statements-cache-size=10                           query-timeout=600
reauth-plugin-class-name=undefined                          reauth-plugin-properties=undefined                          security-domain=undefined
set-tx-query-timeout=false                                  share-prepared-statements=false                             spy=false
stale-connection-checker-class-name=undefined               stale-connection-checker-properties=undefined               track-statements="NOWARN"
transaction-isolation=undefined                             url-delimiter=undefined                                     url-selector-strategy-class-name=undefined
use-ccm=true                                                use-fast-fail=false                                         use-java-context=true
use-try-lock=undefined                                      user-name=example                                           valid-connection-checker-class-name=undefined
valid-connection-checker-properties=undefined               validate-on-match=false
[standalone@192.168.1.66:9999 data-source=OracleDS] cd statistics=pool
[standalone@192.168.1.66:9999 statistics=pool] read-attribute AverageCreationTime
93
[standalone@192.168.1.66:9999 statistics=pool] read-attribute AvailableCount
15
[standalone@192.168.1.66:9999 statistics=pool] read-attribute ActiveCount
5
[standalone@192.168.1.66:9999 statistics=pool] read-attribute MaxWaitTime
0
[standalone@192.168.1.66:9999 statistics=pool] cd ..
[standalone@192.168.1.66:9999 data-source=OracleDS] cd statistics=jdbc
[standalone@192.168.1.66:9999 statistics=jdbc] read-attribute PreparedStatementCacheAccessCount
70
[standalone@192.168.1.66:9999 statistics=jdbc] read-attribute PreparedStatementCacheHitCount
58
[standalone@192.168.1.66:9999 statistics=jdbc] read-attribute PreparedStatementCacheMissCount
0
[standalone@192.168.1.66:9999 statistics=jdbc] cd /
[standalone@192.168.1.66:9999 /] cd deployment=LoadTest6.ear/subdeployment=Model.jar/subsystem=ejb3
[standalone@192.168.1.66:9999 subsystem=ejb3] ls
entity-bean              message-driven-bean      singleton-bean           stateful-session-bean    stateless-session-bean
[standalone@192.168.1.66:9999 subsystem=ejb3] cd message-driven-bean=CompanyMDB
[standalone@192.168.1.66:9999 message-driven-bean=CompanyMDB] read-attribute pool-current-size
2
[standalone@192.168.1.66:9999 message-driven-bean=CompanyMDB] read-attribute pool-available-count
50
[standalone@192.168.1.66:9999 message-driven-bean=CompanyMDB] cd ..
[standalone@192.168.1.66:9999 subsystem=ejb3] cd stateless-session-bean=Company
[standalone@192.168.1.66:9999 stateless-session-bean=Company] read-attribute pool-current-size
5
[standalone@192.168.1.66:9999 stateless-session-bean=Company] read-attribute pool-available-count
50
[standalone@192.168.1.66:9999 stateless-session-bean=Company] cd /
[standalone@192.168.1.66:9999 /] cd deployment=LoadTest6.ear/subdeployment=Model.jar/subsystem=jpa
[standalone@192.168.1.66:9999 subsystem=jpa] ls
hibernate-persistence-unit
[standalone@192.168.1.66:9999 subsystem=jpa] cd hibernate-persistence-unit=LoadTest6.ear\/Model.jar#PersonPersistenceUnit
[standalone@192.168.1.66:9999 hibernate-persistence-unit=LoadTest6.ear/Model.jar#PersonPersistenceUnit] ls
collection                                        entity                                            entity-cache                                      query-cache
close-statement-count=0                           collection-fetch-count=0                          collection-load-count=0                           collection-recreated-count=0
collection-remove-count=0                         collection-update-count=0                         completed-transaction-count=35                    connect-count=70
enabled=false                                     entity-delete-count=0                             entity-fetch-count=0                              entity-insert-count=0
entity-load-count=0                               entity-update-count=0                             flush-count=0                                     optimistic-failure-count=0
prepared-statement-count=70                       query-cache-hit-count=0                           query-cache-miss-count=0                          query-cache-put-count=0
query-execution-count=0                           query-execution-max-time=0                        query-execution-max-time-query-string=undefined   second-level-cache-hit-count=0
second-level-cache-miss-count=0                   second-level-cache-put-count=0                    session-close-count=0                             session-open-count=0
successful-transaction-count=35
[standalone@192.168.1.66:9999 hibernate-persistence-unit=LoadTest6.ear/Model.jar#PersonPersistenceUnit] cd ../../../
[standalone@192.168.1.66:9999 deployment=LoadTest6.ear] cd subdeployment=Web.war/subsystem=web
[standalone@192.168.1.66:9999 subsystem=web] ls
servlet                     context-root=/LoadTest6     virtual-host=default-host
[standalone@192.168.1.66:9999 subsystem=web] cd servlet=TestServlet
[standalone@192.168.1.66:9999 servlet=TestServlet] read-attribute request-count
35
[standalone@192.168.1.66:9999 servlet=TestServlet] read-attribute processing-time
2002L

For the messaging we looked at the attributes; message-count that indicates how many messages are still in the queue. For a data source it important to know if requests (tasks) are waiting for a connection from the pool. This is indicated by the max-wait-time attribute, when this differs from zero it might be useful to adjust the connection pool settings, i.e., increase the maximum number of connections in the pool when possible. Other statistics we viewed was some run-time information from the EJB and servlet environment. An alternative way to view run-time statistics is by using an MBean browser, for example, jconsole or jvisualvm. Note that in the latter case we have to install a plug-in. In jvisualvm click tools, plug-ins, available plug-ins and select visualvm-mbeans. The MBean browser has an entry jboss.as that contains all the run-time information.

We can also put the commands in a script, for example,

connect 192.168.1.66:9999
cd subsystem=messaging/hornetq-server=default/jms-queue=testQueue
pwd
read-attribute delivering-count
read-attribute consumer-count
read-attribute message-count
read-attribute messages-added
cd /
cd subsystem=datasources/data-source=OracleDS
ls
cd statistics=pool
read-attribute AverageCreationTime
read-attribute AvailableCount
read-attribute ActiveCount
read-attribute MaxWaitTime
cd ..
cd statistics=jdbc
read-attribute PreparedStatementCacheAccessCount
read-attribute PreparedStatementCacheHitCount
read-attribute PreparedStatementCacheMissCount
cd /
cd deployment=LoadTest6.ear/subdeployment=Model.jar/subsystem=ejb3
ls
cd message-driven-bean=CompanyMDB
read-attribute pool-current-size
read-attribute pool-available-count
cd ..
cd stateless-session-bean=Company
read-attribute pool-current-size
read-attribute pool-available-count
cd /
cd deployment=LoadTest6.ear/subdeployment=Model.jar/subsystem=jpa
ls
cd hibernate-persistence-unit=LoadTest6.ear\/Model.jar#PersonPersistenceUnit
ls
cd ../../../
cd subdeployment=Web.war/subsystem=web
ls
cd servlet=TestServlet
read-attribute request-count
read-attribute processing-time

To execute the script, we can use (note that we called the script example-script.cli)

[jboss@axis-into-ict ~]$ cd jboss-as-7.1.0.Final/bin/
[jboss@axis-into-ict bin]$ ./jboss-cli.sh --file=/home/jboss/jboss-as-7.1.0.Final/scripts/example-script.cli

Using the Apache HTTP Server as a front-end

General information about the Apache HTTP Server can be found here. When we want to use the Apache HTTP Server as a front-end and in particular mod_jk we have to enable an AJP connector for the server in question. To create a new connector requires us to declare a new socket binding first

[standalone@192.168.1.66:9999 /] /socket-binding-group=standard-sockets/socket-binding=ajp:add(port=9080)

The created socket binding can then be used to create a new connector

[standalone@192.168.1.66:9999 /] /subsystem=web/connector=ajp:add(socket-binding=ajp, protocol="AJP/1.3", enabled=true)

With the new connector in place, we can continue with setting-up the Apache HTTP server. To install the Apache HTTP server we can follow these steps

• Unpack the httpd-2.2.21.tar.gz file:
  • gzip -d httpd-2.2.21.tar.gz
  • tar xvf httpd-2.2.21.tar
  • cd httpd-2.2.21
• The next step is to configure:
  • ./configure –prefix=/home/jboss/apache
• Next, compile the various parts for the Apache HTTP Server by using:
  • make
• To install the Apache HTTP Server we use:
  • make install
• Open the httpd.conf (/home/jboss/apache/conf) file and adjust the following directives:
  • Listen 8888
  • ServerName ip-address
• To test the set-up, start the Apache HTTP Server:
  • Navigate to /home/jboss/apache/bin.
  • Run: ./apachectl -k start (To stop the Apache HTTP Server we can use: ./apachectl -k stop).
  • Open a browser en type the following URL: http://hostname:8888.

To install the mod_jk module we can follow these steps

• Unpack the tomcat-connectors-1.2.32-src.tar.gz file:
  • gzip -d tomcat-connectors-1.2.32-src.tar.gz
  • tar xvf tomcat-connectors-1.2.32-src.tar
  • cd tomcat-connectors-1.2.32-src/native
• The next step is to configure the module:
  • ./configure –with-apxs=/home/jboss/apache/bin/apxs
• Compile:
  • make
• In the native directory, a directory apache-2.0 is created. Here a mod_jk.so file is present. Copy this file to the directory: /home/jboss/apache/modules.

To configure the mod_jk module, create a new file mod_jk.conf in the directory /home/jboss/apache/conf. An example configuration looks as follows

LoadModule jk_module /home/jboss/apache/modules/mod_jk.so

JkWorkersFile /home/jboss/apache/conf/jboss-workers.properties

# where to put the log files for the jk module
JkLogFile /home/jboss/apache/logs/mod_jk.log

# the log level [debug|info|error]
JkLogLevel info

# log format
JkLogStampFormat "[%a %b %d %H:%M:%S %Y]"

# options to indicate to send SSL KEY SIZE
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories

# set the request log format
JkRequestLogFormat "%w %V %T"

# send requests that contain the following to JBoss
JkMount /LoadTest6/* standalone-server-worker

Next, we need to define the worker file that is specified by JkWorkersFile directive. In this case, we create a file called jboss-workers.properties in the directory /home/jboss/apache/conf. An example worker configuration is the following

# define a worker that uses ajp13
worker.list=standalone-server-worker

# set properties for the worker
worker.standalone-server-worker.type=ajp13
worker.standalone-server-worker.host=192.168.1.66 (which is the IP-address where the JBoss server is running)
worker.standalone-server-worker.port=9080 (which is AJP listen port of the JBoss Server)

To let the Apache HTTP Server pick up the configuration we add the following to the httpd.conf file

# put it near the end of the file where all the other includes are present
# mod_jk configuration
Include conf/mod_jk.conf

Restart the HTTP Server (./apachectl -k stop and ./apachectl -k start). The application can now be reached at: http://hostname:8888/LoadTest6/testservlet. Note that by using the JkMount directive, we tell mod_jk to forward requests to a configured back-end server.

We can fine tune the web container, for example,

<extensions>
	...
	<extension module="org.jboss.as.web"/>
	...
</extensions>
<profile>
	...
	<subsystem xmlns="urn:jboss:domain:web:1.1" native="false" default-virtual-server="default-host">
		<connector name="http" protocol="HTTP/1.1" scheme="http" socket-binding="http"/>
		<connector name="ajp" protocol="AJP/1.3" socket-binding="ajp" enabled="true" enable-lookups="false" executor="ajp-thread-pool" max-connections="150" max-post-size="1048576"/>
		<virtual-server name="default-host" enable-welcome-root="true">
			<alias name="localhost"/>
			<alias name="example.com"/>
		</virtual-server>
	</subsystem>
	...
</profile>
<socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
	<socket-binding name="http" port="8080"/>
	<socket-binding name="https" port="8443"/>
	...
	<socket-binding name="ajp" port="9080"/>
	...
</socket-binding-group>

The connector is a Java object, which provides an interface to Web server clients. The default configuration provides a connector configuration in which most attributes have their default values. For example, to configure a denial of service we add the following attributes:

• max-connections – the maximum number of simultaneous requests that can be made. We set this equal to the MaxClients directive configured in the Apache HTTP server.
• max-post-size – limits the maximum size of HTTP POST requests it accepts (the default is 2MB). We set this to 1048576 (1024*1024) bytes. Note that the value is based on what post size to expect from clients. Usually for a web application this is in the range of a few kilobytes. When using web service this value can be in the range of a few 100 kilobytes depending on the message sizes the web service defines, and if attachments can be send as well.
• max-save-post-size – maximum size of a POST that will be saved by the container during authentication (the default is 4kB which we will leave as it is).

In general, a JBoss application server will be fronted by an Apache HTTP server. One thing to note is that, we have to make the thread pool size equal to the MaxClients directive configured on the Apache HTTP Server (usually it is something like MaxClients 150) otherwise threads will pile up in the JBoss Server. An example thread pool configuration is the following.

<subsystem xmlns="urn:jboss:domain:threads:1.1">
	<blocking-bounded-queue-thread-pool name="ajp-thread-pool">
		<queue-length count="200"/>
		<core-threads count="150"/>
		<max-threads count="150"/>
		<keepalive-time time="30" unit="seconds"/>
	</blocking-bounded-queue-thread-pool>
</subsystem>

Note that when using a small pool with a large queue, we tend to minimize CPU usage, operating system resources and context switching. This can potentially lead to a low throughput. By using small queues we need larger a larger thread pool. This keeps the CPU busier and can lead to an significant increase in scheduling overhead and thereby decreasing throughput. To use a certain thread pool configuration in a connector we can use the executor attribute.

The attribute enable-lookups when set to true, a call to request.getRemoteHost() returns the actual hostname of the remote client. When set to false the IP address is returned. In the latter case no DNS lookup is performed and thus improving performance.

To check the configuration we can use

[standalone@192.168.1.66:9999 /] /subsystem=threads/blocking-bounded-queue-thread-pool=ajp-thread-pool:read-resource(recursive=true)
{
    "outcome" => "success",
    "result" => {
        "allow-core-timeout" => false,
        "core-threads" => 150,
        "keepalive-time" => {
            "time" => 30L,
            "unit" => "SECONDS"
        },
        "max-threads" => 150,
        "name" => "ajp-thread-pool",
        "queue-length" => 200,
        "thread-factory" => undefined
    }
}

[standalone@192.168.1.66:9999 /] /subsystem=web/connector=ajp:read-resource(recursive=true)
{
    "outcome" => "success",
    "result" => {
        "enable-lookups" => false,
        "enabled" => true,
        "executor" => "ajp-thread-pool",
        "max-connections" => 150,
        "max-post-size" => 1048576,
        "max-save-post-size" => 4096,
        "protocol" => "AJP/1.3",
        "redirect-port" => 8443,
        "scheme" => "http",
        "secure" => false,
        "socket-binding" => "ajp",
        "ssl" => undefined,
        "virtual-server" => undefined
    }
}

To see web metrics by using the admin console, after making a number of requests to http://hostname:8888/LoadTest6/testservlet. Open the console (http://hostname:9990/console), click the run-time tab, web in subsystem metrics and click the ajp row. It shows the request count, error count, processing time and max time.

Clustering

To cluster standalone servers we create two configuration files, for example, server1.xml and server2.xml, which are based on the standalone.xml with which we have been working. We first undeploy the LoadTest6 application, by using the command-line interface command: undeploy LoadTest6.ear. Copy the standalone.xml file twice and name the copies to server1.xml and server2.xml. To enable clustering we set the log file name for the server, enable jgroups, and make the messaging system clusterable in the configuration files

<server xmlns="urn:jboss:domain:1.1">
    <extensions>
		...
        <extension module="org.jboss.as.clustering.jgroups"/>
		...
    </extensions>
    <profile>
		<subsystem xmlns="urn:jboss:domain:logging:1.1">
			...
            <periodic-rotating-file-handler name="FILE">
                <formatter>
                    <pattern-formatter pattern="%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%E%n"/>
                </formatter>
                <file relative-to="jboss.server.log.dir" path="server1.log"/>
                <suffix value=".yyyy-MM-dd"/>
                <append value="true"/>
            </periodic-rotating-file-handler>
			...
        </subsystem>
		...
        <subsystem xmlns="urn:jboss:domain:jgroups:1.1" default-stack="udp">
            <stack name="udp">
                <transport type="UDP" socket-binding="jgroups-udp" diagnostics-socket-binding="jgroups-diagnostics"/>
                <protocol type="PING"/>
                <protocol type="MERGE2"/>
                <protocol type="FD_SOCK" socket-binding="jgroups-udp-fd"/>
                <protocol type="FD"/>
                <protocol type="VERIFY_SUSPECT"/>
                <protocol type="BARRIER"/>
                <protocol type="pbcast.NAKACK"/>
                <protocol type="UNICAST2"/>
                <protocol type="pbcast.STABLE"/>
                <protocol type="pbcast.GMS"/>
                <protocol type="UFC"/>
                <protocol type="MFC"/>
                <protocol type="FRAG2"/>
            </stack>
            <stack name="tcp">
                <transport type="TCP" socket-binding="jgroups-tcp" diagnostics-socket-binding="jgroups-diagnostics"/>
                <protocol type="MPING" socket-binding="jgroups-mping"/>
                <protocol type="MERGE2"/>
                <protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd"/>
                <protocol type="FD"/>
                <protocol type="VERIFY_SUSPECT"/>
                <protocol type="BARRIER"/>
                <protocol type="pbcast.NAKACK"/>
                <protocol type="UNICAST2"/>
                <protocol type="pbcast.STABLE"/>
                <protocol type="pbcast.GMS"/>
                <protocol type="UFC"/>
                <protocol type="MFC"/>
                <protocol type="FRAG2"/>
            </stack>
        </subsystem>
		...
		<subsystem xmlns="urn:jboss:domain:messaging:1.1">
            <hornetq-server>
                <clustered>true</clustered>
                <cluster-user>guest</cluster-user>
                <cluster-password>guest</cluster-password>
                <persistence-enabled>true</persistence-enabled>
                <journal-type>ASYNCIO</journal-type>
                <journal-file-size>102400</journal-file-size>
                <journal-min-files>2</journal-min-files>
                <paging-directory path="messagingpaging1"/>
                <bindings-directory path="messagingbindings1"/>
                <journal-directory path="messagingjournal1"/>
                <large-messages-directory path="messaginglargemessages1"/>
                <connectors>
                    <netty-connector name="netty" socket-binding="messaging"/>
                    <netty-connector name="netty-throughput" socket-binding="messaging-throughput">
                        <param key="batch-delay" value="50"/>
                    </netty-connector>
                    <in-vm-connector name="in-vm" server-id="0"/>
                </connectors>
                <acceptors>
                    <netty-acceptor name="netty" socket-binding="messaging"/>
                    <netty-acceptor name="netty-throughput" socket-binding="messaging-throughput">
                        <param key="batch-delay" value="50"/>
                        <param key="direct-deliver" value="false"/>
                    </netty-acceptor>
                    <in-vm-acceptor name="in-vm" server-id="0"/>
                </acceptors>
                <broadcast-groups>
                    <broadcast-group name="bg-group1">
                        <group-address>231.7.7.7</group-address>
                        <group-port>9876</group-port>
                        <broadcast-period>5000</broadcast-period>
                        <connector-ref>netty</connector-ref>
                    </broadcast-group>
                </broadcast-groups>
                <discovery-groups>
                    <discovery-group name="dg-group1">
                        <group-address>231.7.7.7</group-address>
                        <group-port>9876</group-port>
                        <refresh-timeout>10000</refresh-timeout>
                    </discovery-group>
                </discovery-groups>
                <cluster-connections>
                    <cluster-connection name="my-cluster">
                        <address>jms</address>
                        <connector-ref>netty</connector-ref>
                        <discovery-group-ref discovery-group-name="dg-group1"/>
                    </cluster-connection>
                </cluster-connections>
                <security-settings>
                    <security-setting match="#">
                        <permission type="send" roles="guest"/>
                        <permission type="consume" roles="guest"/>
                        <permission type="createNonDurableQueue" roles="guest"/>
                        <permission type="deleteNonDurableQueue" roles="guest"/>
                    </security-setting>
                </security-settings>
                <address-settings>
                    <address-setting match="#">
                        <dead-letter-address>jms.queue.DLQ</dead-letter-address>
                        <expiry-address>jms.queue.ExpiryQueue</expiry-address>
                        <redelivery-delay>0</redelivery-delay>
                        <redistribution-delay>1000</redistribution-delay>
                        <max-size-bytes>10485760</max-size-bytes>
                        <address-full-policy>BLOCK</address-full-policy>
                        <message-counter-history-day-limit>10</message-counter-history-day-limit>
                    </address-setting>
                </address-settings>
                <jms-connection-factories>
                    <connection-factory name="InVmConnectionFactory">
                        <connectors>
                            <connector-ref connector-name="in-vm"/>
                        </connectors>
                        <entries>
                            <entry name="java:/ConnectionFactory"/>
                        </entries>
                    </connection-factory>
                    <connection-factory name="RemoteConnectionFactory">
                        <connectors>
                            <connector-ref connector-name="netty"/>
                        </connectors>
                        <entries>
                            <entry name="RemoteConnectionFactory"/>
                            <entry name="java:jboss/exported/jms/RemoteConnectionFactory"/>
                        </entries>
                    </connection-factory>
                    <pooled-connection-factory name="hornetq-ra">
                        <transaction mode="xa"/>
                        <connectors>
                            <connector-ref connector-name="in-vm"/>
                        </connectors>
                        <entries>
                            <entry name="java:/JmsXA"/>
                        </entries>
                    </pooled-connection-factory>
                </jms-connection-factories>
                <jms-destinations>
                    <jms-queue name="testQueue">
                        <entry name="java:/queue/test"/>
                        <entry name="java:jboss/exported/jms/queue/test"/>
                    </jms-queue>
                    <jms-topic name="testTopic">
                        <entry name="java:/topic/test"/>
                        <entry name="java:jboss/exported/jms/topic/test"/>
                    </jms-topic>
                </jms-destinations>
            </hornetq-server>
        </subsystem>
		...
    </profile>
    <socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
		...
        <socket-binding name="jgroups-diagnostics" port="0" multicast-address="224.0.75.75" multicast-port="7500"/>
        <socket-binding name="jgroups-mping" port="0" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45700"/>
        <socket-binding name="jgroups-tcp" port="7600"/>
        <socket-binding name="jgroups-tcp-fd" port="57600"/>
        <socket-binding name="jgroups-udp" port="55200" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45688"/>
        <socket-binding name="jgroups-udp-fd" port="54200"/>
		...
    </socket-binding-group>
</server>

For server2 we use the port-offset attribute (which we set to one) in the socket binding

<server xmlns="urn:jboss:domain:1.1">
    <extensions>
		...
        <extension module="org.jboss.as.clustering.jgroups"/>
		...
    </extensions>
    <profile>
		<subsystem xmlns="urn:jboss:domain:logging:1.1">
			...
            <periodic-rotating-file-handler name="FILE">
                <formatter>
                    <pattern-formatter pattern="%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%E%n"/>
                </formatter>
                <file relative-to="jboss.server.log.dir" path="server2.log"/>
                <suffix value=".yyyy-MM-dd"/>
                <append value="true"/>
            </periodic-rotating-file-handler>
			...
        </subsystem>
		...
        <subsystem xmlns="urn:jboss:domain:jgroups:1.1" default-stack="udp">...</subsystem>
		...
		<subsystem xmlns="urn:jboss:domain:messaging:1.1">
            <hornetq-server>
                <clustered>true</clustered>
                <cluster-user>guest</cluster-user>
                <cluster-password>guest</cluster-password>
                <persistence-enabled>true</persistence-enabled>
                <journal-type>ASYNCIO</journal-type>
                <journal-file-size>102400</journal-file-size>
                <journal-min-files>2</journal-min-files>
                <paging-directory path="messagingpaging2"/>
                <bindings-directory path="messagingbindings2"/>
                <journal-directory path="messagingjournal2"/>
                <large-messages-directory path="messaginglargemessages2"/>
				...
            </hornetq-server>
        </subsystem>
		...
    </profile>
    <socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:1}">
		...
    </socket-binding-group>
</server>

To start the servers, we can use standalone.sh --server-config=server1.xml and standalone.sh --server-config=server2.xml. When the servers are started the following is shown when hornetq is connected successfully between both servers

# server1
17:24:06,129 INFO  [org.hornetq.core.server.cluster.impl.BridgeImpl] (Thread-9 (HornetQ-server-HornetQServerImpl::serverUUID=d052af0e-5efa-11e1-ad54-000c2976c82d-37774788)) Bridge ClusterConnectionBridge@4552a64d [name=sf.my-cluster.c25598f7-5efa-11e1-8e93-000c2976c82d, queue=QueueImpl[name=sf.my-cluster.c25598f7-5efa-11e1-8e93-000c2976c82d, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=d052af0e-5efa-11e1-ad54-000c2976c82d]]@35242cc9 targetConnector=ServerLocatorImpl (identity=(Cluster-connection-bridge::ClusterConnectionBridge@4552a64d [name=sf.my-cluster.c25598f7-5efa-11e1-8e93-000c2976c82d, queue=QueueImpl[name=sf.my-cluster.c25598f7-5efa-11e1-8e93-000c2976c82d, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=d052af0e-5efa-11e1-ad54-000c2976c82d]]@35242cc9 targetConnector=ServerLocatorImpl [initialConnectors=[org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5446&host=axis-into-ict-nl], discoveryGroupConfiguration=null]]::ClusterConnectionImpl@762010908[nodeUUID=d052af0e-5efa-11e1-ad54-000c2976c82d, connector=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=axis-into-ict-nl, address=jms, server=HornetQServerImpl::serverUUID=d052af0e-5efa-11e1-ad54-000c2976c82d])) [initialConnectors=[org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5446&host=axis-into-ict-nl], discoveryGroupConfiguration=null]] is connected

# server2
17:24:06,123 INFO  [org.hornetq.core.server.cluster.impl.BridgeImpl] (Thread-29 (HornetQ-server-HornetQServerImpl::serverUUID=c25598f7-5efa-11e1-8e93-000c2976c82d-1291383602)) Bridge ClusterConnectionBridge@4174b989 [name=sf.my-cluster.d052af0e-5efa-11e1-ad54-000c2976c82d, queue=QueueImpl[name=sf.my-cluster.d052af0e-5efa-11e1-ad54-000c2976c82d, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=c25598f7-5efa-11e1-8e93-000c2976c82d]]@5640f2f1 targetConnector=ServerLocatorImpl (identity=(Cluster-connection-bridge::ClusterConnectionBridge@4174b989 [name=sf.my-cluster.d052af0e-5efa-11e1-ad54-000c2976c82d, queue=QueueImpl[name=sf.my-cluster.d052af0e-5efa-11e1-ad54-000c2976c82d, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=c25598f7-5efa-11e1-8e93-000c2976c82d]]@5640f2f1 targetConnector=ServerLocatorImpl [initialConnectors=[org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=axis-into-ict-nl], discoveryGroupConfiguration=null]]::ClusterConnectionImpl@211830739[nodeUUID=c25598f7-5efa-11e1-8e93-000c2976c82d, connector=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5446&host=axis-into-ict-nl, address=jms, server=HornetQServerImpl::serverUUID=c25598f7-5efa-11e1-8e93-000c2976c82d])) [initialConnectors=[org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=axis-into-ict-nl], discoveryGroupConfiguration=null]] is connected

To deploy the application we can use the following

[jboss@axis-into-ict ~]$ cd jboss-as-7.1.0.Final/bin/
[jboss@axis-into-ict bin]$ ./jboss-cli.sh
You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.
[disconnected /] connect 192.168.1.66:9999
[standalone@192.168.1.66:9999 /] deploy /home/jboss/jboss-as-7.1.0.Final/deploy/LoadTest6.ear
[standalone@192.168.1.66:9999 /] connect 192.168.1.66:10000
[standalone@192.168.1.66:10000 /] deploy /home/jboss/jboss-as-7.1.0.Final/deploy/LoadTest6.ear11

Before, we configure the load balancing, test if the application is working. Either use the URL: http://hostname:8080/LoadTest6/testservlet or the URL: http://hostname:8081/LoadTest6/testservlet.

Configure load balancing

The load balancer decides how to dispatch requests to the back end server instances. A load balancer needs to do tasks such as distributing requests, health checking and session binding. As these are simple jobs it is rare that a load balancer will become a bottleneck. To load balance requests, we are going to use mod_jk. To configure mod_jk, open the mod_jk.conf file and add the following contents:

LoadModule jk_module /home/jboss/apache/modules/mod_jk.so

JkWorkersFile /home/jboss/apache/conf/jboss-workers.properties

# where to put the log files for the jk module
JkLogFile /home/jboss/apache/logs/mod_jk.log

# the log level [debug|info|error]
JkLogLevel info

# log format
JkLogStampFormat "[%a %b %d %H:%M:%S %Y]"

# options to indicate to send SSL KEY SIZE
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories

# set the request log format
JkRequestLogFormat "%w %V %T"

# send requests that contain the following to JBoss
JkMount /LoadTest6/* loadbalancer

The workers (jboss-workers.properties) are defined as follows:

# define a worker that uses ajp13
worker.list=server1-worker,server2-worker,loadbalancer

# set properties for the server1-worker
worker.server1-worker.type=ajp13
worker.server1-worker.host=192.168.1.66
worker.server1-worker.port=9080
worker.server1-worker.lbfactor=1
worker.server1-worker.socket_keepalive=1
worker.server1-worker.socket_timeout=300

# set properties for the server2-worker
worker.server2-worker.type=ajp13
worker.server2-worker.host=192.168.1.66
worker.server2-worker.port=9081
worker.server2-worker.lbfactor=1
worker.server2-worker.socket_keepalive=1
worker.server2-worker.socket_timeout=300

# set properties for the loadbalancer
worker.loadbalancer.type=lb
worker.loadbalancer.balance_workers=server1-worker,server2-worker

With the configuration in place, start the Apache HTTP server (./apachectl -k start) and enter the following URL: http://hostname:8888/LoadTest6/testservlet a few times to see if the requests get send to the different servers. By using the following URL’s: http://hostname:9990/console/App.html#web-metrics and http://hostname:9991/console/App.html#web-metrics, we can check how the requests are being processed. We can also use the command-line interface

[disconnected /] connect 192.168.1.66:9999
[standalone@192.168.1.66:9999 /] cd /deployment=LoadTest6.ear/subdeployment=Web.war/subsystem=web/servlet=TestServlet
[standalone@192.168.1.66:9999 servlet=TestServlet] read-attribute request-count
69
[standalone@192.168.1.66:9999 servlet=TestServlet] connect 192.168.1.66:10000
[standalone@192.168.1.66:10000 servlet=TestServlet] read-attribute request-count
68
[standalone@192.168.1.66:10000 servlet=TestServlet] read-attribute request-count
118
[standalone@192.168.1.66:10000 servlet=TestServlet] connect 192.168.1.66:9999
[standalone@192.168.1.66:9999 servlet=TestServlet] read-attribute request-count
119

As we can see the load is balanced like a charm.

Load test

We can test the environment by using the The Grinder. The following script calls the application:

from net.grinder.script.Grinder import grinder
from net.grinder.script import Test
from net.grinder.plugin.http import HTTPRequest

test1 = Test(1, "Request resource")
request1 = test1.wrap(HTTPRequest())

class TestRunner:
    def __call__(self):
        result = request1.GET("http://192.168.1.66:8888/LoadTest6/testservlet")

Edit the grinder.properties (${GRINDER_HOME}/examples) file such that the script is used, for example,

# The file name of the script to run. Relative paths are evaluated from the directory containing the properties file.
grinder.script = test.py

# The number of worker processes each agent should start.
grinder.processes = 1

# The number of worker threads each worker process should start.
grinder.threads = 1

# The number of runs each worker process will perform. When using the console set this to 0, i.e., the runs are controlled by the console.
grinder.runs = 0

Use the following scripts to start a test

setGrinderEnv.sh

#!/bin/sh
GRINDERPATH=/home/jboss/grinder-3.4
export GRINDERPATH

JAVA_HOME=/home/jboss/jdk1.6.0_31
export JAVA_HOME

USER_MEM_ARGS="-server -Xms512m -Xmx512m -XX:NewRatio=2 -XX:+UseParallelGC -XX:ParallelGCThreads=2 -XX:MaxGCPauseMillis=200 -XX:GCTimeRatio=19 -XX:+UseParallelOldGC"
export USER_MEM_ARGS
GRINDERPROPERTIES=${GRINDERPATH}/examples/grinder.properties
export GRINDERPROPERTIES
CLASSPATH=${GRINDERPATH}/lib/grinder.jar:${CLASSPATH}
export CLASSPATH

PATH=${JAVA_HOME}/bin:${PATH}
export PATH

startConsole.sh

#!/bin/sh
source setGrinderEnv.sh
java ${USER_MEM_ARGS} -cp ${CLASSPATH} net.grinder.Console

startAgent.sh

#!/bin/sh
source setGrinderEnv.sh
java ${USER_MEM_ARGS} -cp ${CLASSPATH} net.grinder.Grinder ${GRINDERPROPERTIES}

First start the startConsole.sh script and then start the startAgent.sh script. The agent will now wait for a start action from the console (click the action tab and then choose start processes)

By using jvisualvm or jconsole the JVM can be monitored and MBeans can browsed

We can clearly see when the load test was running. The test introduced no particular problems for the JVM. Run-time statistics from the command-line interface

server 1
subsystem=messaging/hornetq-server=default/jms-queue=testQueue
delivering-count (The number of messages that this queue is currently delivering to its consumers): 0
consumer-count (The number of consumers consuming messages from this queue): 15
message-count (The number of messages currently in this queue): 0L
messages-added (The number of messages added to this queue since it was created): 11987L

subsystem=datasources/data-source=OracleDS/statistics=pool
AverageCreationTime (The average time spent creating a physical connection): 2594
AvailableCount (The available count): 15
ActiveCount (The active count): 2
MaxWaitTime (The maximum wait time for a connection): 3

subsystem=datasources/data-source=OracleDS/statistics=jdbc
PreparedStatementCacheAccessCount (The number of times that the statement cache was accessed): 770567
PreparedStatementCacheHitCount (The number of times that statements from the cache were used): 770560
PreparedStatementCacheMissCount (The number of times that a statement request could not be satisfied with a statement from the cache): 0

deployment=LoadTest6.ear/subdeployment=Model.jar/subsystem=ejb3/message-driven-bean=CompanyMDB
pool-current-size (The current size of the pool): 4
pool-available-count (The number of available (i.e. not in use) instances in the pool): 12003

deployment=LoadTest6.ear/subdeployment=Model.jar/subsystem=ejb3/stateless-session-bean=Company
pool-current-size (The current size of the pool): 1
pool-available-count (The number of available (i.e. not in use) instances in the pool): 385348

deployment=LoadTest6.ear/subdeployment=Model.jar/subsystem=jpa/hibernate-persistence-unit=LoadTest6.ear\/Model.jar#PersonPersistenceUnit
completed-transaction-count (Number of completed transactions): 385329L
successful-transaction-count (Number of successful transactions): 385329L

deployment=LoadTest6.ear/subdeployment=Web.war/subsystem=web/servlet=TestServlet
request-count (Number of requests processed by this servlet): 385329
processing-time (Total execution time of the servlet's service method): 926143L

server 2
subsystem=messaging/hornetq-server=default/jms-queue=testQueue
delivering-count (The number of messages that this queue is currently delivering to its consumers): 0
consumer-count (The number of consumers consuming messages from this queue): 15
message-count (The number of messages currently in this queue): 0L
messages-added (The number of messages added to this queue since it was created): 11987L

subsystem=datasources/data-source=OracleDS/statistics=pool
AverageCreationTime (The average time spent creating a physical connection): 184
AvailableCount (The available count): 15
ActiveCount (The active count): 2
MaxWaitTime (The maximum wait time for a connection): 1

subsystem=datasources/data-source=OracleDS/statistics=jdbc
PreparedStatementCacheAccessCount (The number of times that the statement cache was accessed): 770566
PreparedStatementCacheHitCount (The number of times that statements from the cache were used): 770559
PreparedStatementCacheMissCount (The number of times that a statement request could not be satisfied with a statement from the cache): 0

deployment=LoadTest6.ear/subdeployment=Model.jar/subsystem=ejb3/message-driven-bean=CompanyMDB
pool-current-size (The current size of the pool): 5
pool-available-count (The number of available (i.e. not in use) instances in the pool): 12002

deployment=LoadTest6.ear/subdeployment=Model.jar/subsystem=ejb3/stateless-session-bean=Company
pool-current-size (The current size of the pool): 1
pool-available-count (The number of available (i.e. not in use) instances in the pool): 385344

deployment=LoadTest6.ear/subdeployment=Model.jar/subsystem=jpa/hibernate-persistence-unit=LoadTest6.ear\/Model.jar#PersonPersistenceUnit
completed-transaction-count (Number of completed transactions): 385325L
successful-transaction-count (Number of successful transactions): 385325L

deployment=LoadTest6.ear/subdeployment=Web.war/subsystem=web/servlet=TestServlet
request-count (Number of requests processed by this servlet): 385325
processing-time (Total execution time of the servlet's service method): 922760L

Overall the test was easily handled by the Apache HTTP server (load balancing) and the JBoss application servers present in the cluster.

Domains

One of the new features of JBoss AS 7 is the ability to manage multiple server instances from a single control point. A domain controller process is acting as the management point. All the server instances share a common management policy, with the domain controller acting to ensure that each server is configured according to that policy. Domains can span multiple machines, with server instances on a certain host under the control of a host controller process. One host controller instance is configured as the domain controller. The host controller on each host interacts with the domain controller to control the lifecycle of the server instances running on a particular host and to assist the domain controller in managing them. One thing to note is that the configuration of subsystems (such as messaging and data sources) within a profile does not differ from the standalone configuration (apart from the fact that a domain configuration can have multiple profiles whereas a standalone configuration can only have one). What we need to get the hang of when working with domains are the domain and host controller configurations.

When the ${JBOSS_HOME}/bin/domain.sh script is run a host controller process is started. The host controller starts and stops server instances on the particular host where it was started. Each host controller is configured by using the ${JBOSS_HOME}/domain/configuration/host.xml file. This file contains

• The list of application server names that must run on the host.
• The configuration how the host controller must contact the domain controller to register itself and access the domain configuration. This can either be how to contact a remote domain controller or telling the host controller to act itself as a domain controller.

As already mentioned one host controller must act as the domain controller (central management point). The domain controller maintains a central management policy and makes sure the host controllers are aware of the contents. It also make sure that each server instance is configured with respect to the central management policy. The central management policy is by default stored in the ${JBOSS_HOME}/domain/configuration/domain.xml file on the filesystem where the domain controller is present (it is not necessary to have the domain.xml on the filesystems where other host controllers are running).

When working with a domain we also need to introduce the concept of a server group, which is a set of server instances that will be managed and configured as one. In a managed domain each application server instance is a member of a server group. A domain can have multiple server groups. Different server groups can be configured with different profiles and deployments.

An example domain.xml look as follows:

<domain xmlns="urn:jboss:domain:1.1">
    <extensions>
		<extension module="org.jboss.as.clustering.infinispan"/>
        <extension module="org.jboss.as.clustering.jgroups"/>
		...
        <extension module="org.jboss.as.messaging"/>
		...
        <extension module="org.jboss.as.weld"/>
	</extensions>
    <system-properties>
        <property name="java.net.preferIPv4Stack" value="true"/>
    </system-properties>
    <profiles>
        <profile name="default">
            <subsystem xmlns="urn:jboss:domain:logging:1.1">...</subsystem>
			...
            <subsystem xmlns="urn:jboss:domain:weld:1.0"/>
        </profile>
        <profile name="ha">
            <subsystem xmlns="urn:jboss:domain:logging:1.1">...</subsystem>
			...
            <subsystem xmlns="urn:jboss:domain:jgroups:1.1" default-stack="udp">...</subsystem>
			...
            <subsystem xmlns="urn:jboss:domain:weld:1.0"/>
        </profile>
        <profile name="full">
            <subsystem xmlns="urn:jboss:domain:logging:1.1">...</subsystem>
			...
            <subsystem xmlns="urn:jboss:domain:messaging:1.1">...</subsystem>
			...
            <subsystem xmlns="urn:jboss:domain:weld:1.0"/>
        </profile>
        <profile name="full-ha">
            <subsystem xmlns="urn:jboss:domain:logging:1.1">...</subsystem>
			...
			<subsystem xmlns="urn:jboss:domain:datasources:1.0">
                <datasources>
                    ...
                    <datasource jta="true" jndi-name="java:/jdbc/OracleDS" pool-name="OracleDS" enabled="true" use-java-context="true" use-ccm="true">
                        <connection-url>jdbc:oracle:thin:@192.168.1.65:1521:orcl11</connection-url>
                        <driver>oracle</driver>
                        <pool>
                            <min-pool-size>1</min-pool-size>
                            <max-pool-size>15</max-pool-size>
                            <prefill>true</prefill>
                            <use-strict-min>true</use-strict-min>
                        </pool>
                        <security>
                            <user-name>example</user-name>
                            <password>example</password>
                        </security>
                        <timeout>
                            <idle-timeout-minutes>0</idle-timeout-minutes>
                            <query-timeout>600</query-timeout>
                        </timeout>
                        <statement>
                            <prepared-statement-cache-size>10</prepared-statement-cache-size>
                        </statement>
                    </datasource>
                    <drivers>
					    ...
                        <driver name="oracle" module="com.oracle.database">
                            <driver-class>oracle.jdbc.OracleDriver</driver-class>
                            <xa-datasource-class>oracle.jdbc.xa.client.OracleXADataSource</xa-datasource-class>
                        </driver>
                    </drivers>
                </datasources>
            </subsystem>
			...
            <subsystem xmlns="urn:jboss:domain:jgroups:1.1" default-stack="udp">...</subsystem>
			...
            <subsystem xmlns="urn:jboss:domain:messaging:1.1">...</subsystem>
			...
			<subsystem xmlns="urn:jboss:domain:web:1.1" default-virtual-server="default-host" native="false">
				...
                <connector name="ajp" protocol="AJP/1.3" socket-binding="ajp" enable-lookups="false" max-post-size="1048576" enabled="true" max-connections="150"/>
				...
            </subsystem>
			...
			<subsystem xmlns="urn:jboss:domain:weld:1.0"/>
        </profile>
    </profiles>
    <!--Named interfaces that can be referenced elsewhere in the configuration. The configuration
        for how to associate these logical names with an actual network interface can either
        be specified here or can be declared on a per-host basis in the equivalent element in host.xml.
        These default configurations require the binding specification to be done in host.xml.-->
    <interfaces>
        <interface name="management"/>
        <interface name="public"/>
        <interface name="unsecure"/>
    </interfaces>
    <socket-binding-groups>
        <socket-binding-group name="standard-sockets" default-interface="public">...</socket-binding-group>
        <socket-binding-group name="ha-sockets" default-interface="public">
			...
			<socket-binding name="ajp" port="9080"/>
			...
		</socket-binding-group>
    </socket-binding-groups>
	<deployments>
        <deployment name="LoadTest6.ear" runtime-name="LoadTest6.ear">
            <content sha1="161f51dde7f085c822cc4c68b306d57f1bee902d"/>
        </deployment>
    </deployments>
    <server-groups>
        <server-group name="standard-server-group" profile="full">
            <jvm name="default"/>
            <socket-binding-group ref="standard-sockets"/>
        </server-group>
        <server-group name="ha-server-group" profile="full-ha">
            <jvm name="default"/>
            <socket-binding-group ref="ha-sockets"/>
			<deployments>
                <deployment name="LoadTest6.ear" runtime-name="LoadTest6.ear"/>
            </deployments>
        </server-group>
    </server-groups>
</domain>

In the example above we have set-up different profiles:

• default – a basic server set-up
• ha – a basic cluster set-up
• full – a server set-up with messaging enabled
• full-ha – a cluster set-up with messaging enabled

Different socket-binding groups one for the default and full profiles and one for the ha and full-ha profiles. Further, we defined two server groups, each mapped to a different profile. Note that server groups can have their own settings, such as JVM, socket-binding-group and deployments. To map server instances to a server group, we use a host configuration, for example,

<host name="axis-into-ict.nl" xmlns="urn:jboss:domain:1.1">
    <management>
        <security-realms>
            <security-realm name="ManagementRealm">
                <authentication>
                    <properties path="mgmt-users.properties" relative-to="jboss.domain.config.dir"/>
                </authentication>
            </security-realm>
            <security-realm name="ApplicationRealm">
                <authentication>
                    <properties path="application-users.properties" relative-to="jboss.domain.config.dir" />
                </authentication>
            </security-realm>
        </security-realms>
        <management-interfaces>
            <native-interface security-realm="ManagementRealm">
                <socket interface="management" port="${jboss.management.native.port:9999}"/>
            </native-interface>
            <http-interface security-realm="ManagementRealm">
                <socket interface="management" port="${jboss.management.http.port:9990}"/>
            </http-interface>
        </management-interfaces>
    </management>
    <domain-controller>
       <local/>
       <!-- remote domain controller configuration: remote host="${jboss.domain.master.address}" port="${jboss.domain.master.port:9999}" -->
    </domain-controller>
    <interfaces>
        <interface name="management">
            <nic name="eth0"/>
        </interface>
        <interface name="public">
           <nic name="eth0"/>
        </interface>
        <interface name="unsecure">
            <inet-address value="127.0.0.1"/>
        </interface>
    </interfaces>
    <jvms>
    	<jvm name="default">
            <heap size="512m" max-size="512m"/>
            <permgen size="256m" max-size="256m"/>
            <jvm-options>
                <option value="-server"/>
                <option value="-XX:NewRatio=2"/>
                <option value="-XX:+UseParallelGC"/>
                <option value="-XX:ParallelGCThreads=2"/>
                <option value="-XX:MaxGCPauseMillis=200"/>
                <option value="-XX:GCTimeRatio=19"/>
                <option value="-XX:+UseParallelOldGC"/>
            </jvm-options>
        </jvm>
    </jvms>
    <servers>
        <server name="server1" group="ha-server-group" auto-start="true">
        </server>
        <server name="server2" group="ha-server-group" auto-start="true">
            <socket-bindings port-offset="1"/>
        </server>
    </servers>
</host>

Configuring a host to act as the domain controller is done through the domain-controller declaration in host.xml. If it includes the local element, then this host will become the domain controller. A host acting as the domain controller must expose a native (i.e. non-HTTP) management interface on an address accessible to the other hosts in the domain. Exposing an HTTP management interface is not required, but is recommended as it allows the administration console to work. The configuration above also contains the network interfaces which we mapped to the network interface card eth0.

We can start the domain by using domain.sh located in the ${JBOSS_HOME}/bin directory (do not forget to define JAVA_HOME and PATH before running domain.sh)

[jboss@axis-into-ict ~]$ cd jboss-as-7.1.0.Final/bin/
[jboss@axis-into-ict bin]$ ./domain.sh
=========================================================================

  JBoss Bootstrap Environment

  JBOSS_HOME: /home/jboss/jboss-as-7.1.0.Final

  JAVA: /home/jboss/jdk1.6.0_31/bin/java

  JAVA_OPTS: -server -Xms512m -Xmx512m -XX:NewRatio=2 -XX:PermSize=256m -XX:MaxPermSize=256m -XX:+UseParallelGC -XX:ParallelGCThreads=2 -XX:MaxGCPauseMillis=200 -XX:GCTimeRatio=19 -XX:+UseParallelOldGC -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Djboss.domain.default.config=domain.xml -Djboss.host.default.config=host.xml

=========================================================================

10:13:12,861 INFO  [org.jboss.modules] (main) JBoss Modules version 1.1.1.GA
10:13:13,459 INFO  [org.jboss.as.process.Host Controller.status] (main) JBAS012017: Starting process 'Host Controller'
[Host Controller] 10:13:14,908 INFO  [org.jboss.modules] (main) JBoss Modules version 1.1.1.GA
[Host Controller] 10:13:15,288 INFO  [org.jboss.msc] (main) JBoss MSC version 1.0.2.GA
[Host Controller] 10:13:15,603 INFO  [org.jboss.as] (MSC service thread 1-3) JBAS015899: JBoss AS 7.1.0.Final "Thunder" starting
[Host Controller] 10:13:17,216 INFO  [org.xnio] (MSC service thread 1-1) XNIO Version 3.0.3.GA
[Host Controller] 10:13:17,237 INFO  [org.xnio.nio] (MSC service thread 1-1) XNIO NIO Implementation Version 3.0.3.GA
[Host Controller] 10:13:17,764 INFO  [org.jboss.remoting] (MSC service thread 1-1) JBoss Remoting version 3.2.2.GA
[Host Controller] 10:13:21,800 INFO  [org.jboss.as] (Controller Boot Thread) JBAS010902: Creating http management service using network interface (management) port (9990) securePort (-1)
[Host Controller] 10:13:21,829 INFO  [org.jboss.as.remoting] (MSC service thread 1-1) JBAS017100: Listening on /192.168.1.66:9999
[Host Controller] 10:13:22,129 INFO  [org.jboss.as.host.controller] (Controller Boot Thread) JBAS010922: Starting server server1
[Host Controller] 10:13:22,197 INFO  [org.jboss.as.host.controller] (Controller Boot Thread) JBAS010922: Starting server server2
10:13:22,199 INFO  [org.jboss.as.process.Server:server1.status] (ProcessController-threads - 3) JBAS012017: Starting process 'Server:server1'
[Host Controller] 10:13:22,216 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss AS 7.1.0.Final "Thunder" (Host Controller) started in 8647ms - Started 11 of 11 services (0 services are passive or on-demand)
10:13:22,292 INFO  [org.jboss.as.process.Server:server2.status] (ProcessController-threads - 3) JBAS012017: Starting process 'Server:server2'
[Server:server1] 10:13:22,976 INFO  [org.jboss.modules] (main) JBoss Modules version 1.1.1.GA
[Server:server2] 10:13:23,121 INFO  [org.jboss.modules] (main) JBoss Modules version 1.1.1.GA
[Server:server1] 10:13:23,671 INFO  [org.jboss.msc] (main) JBoss MSC version 1.0.2.GA
[Server:server2] 10:13:23,672 INFO  [org.jboss.msc] (main) JBoss MSC version 1.0.2.GA
[Server:server1] 10:13:23,833 INFO  [org.jboss.as] (MSC service thread 1-4) JBAS015899: JBoss AS 7.1.0.Final "Thunder" starting
[Server:server2] 10:13:23,850 INFO  [org.jboss.as] (MSC service thread 1-4) JBAS015899: JBoss AS 7.1.0.Final "Thunder" starting
[Server:server1] 10:13:24,097 INFO  [org.xnio] (MSC service thread 1-3) XNIO Version 3.0.3.GA
[Server:server1] 10:13:24,127 INFO  [org.xnio.nio] (MSC service thread 1-3) XNIO NIO Implementation Version 3.0.3.GA
[Server:server2] 10:13:24,135 INFO  [org.xnio] (MSC service thread 1-2) XNIO Version 3.0.3.GA
[Server:server1] 10:13:24,161 INFO  [org.jboss.remoting] (MSC service thread 1-3) JBoss Remoting version 3.2.2.GA
[Server:server2] 10:13:24,204 INFO  [org.xnio.nio] (MSC service thread 1-2) XNIO NIO Implementation Version 3.0.3.GA
[Server:server2] 10:13:24,236 INFO  [org.jboss.remoting] (MSC service thread 1-2) JBoss Remoting version 3.2.2.GA
[Server:server1] 10:13:26,666 INFO  [org.jboss.as.logging] (MSC service thread 1-3) JBAS011502: Removing bootstrap log handlers
[Server:server2] 10:13:26,867 INFO  [org.jboss.as.logging] (MSC service thread 1-1) JBAS011502: Removing bootstrap log handlers
[Server:server2] 10:13:26,890 INFO  [org.jboss.as.webservices] (ServerService Thread Pool -- 32) JBAS015537: Activating WebServices Extension
[Server:server2] 10:13:26,902 INFO  [org.jboss.as.security] (ServerService Thread Pool -- 36) JBAS013101: Activating Security Subsystem
[Server:server2] 10:13:26,993 INFO  [org.jboss.as.configadmin] (ServerService Thread Pool -- 59) JBAS016200: Activating ConfigAdmin Subsystem
[Server:server2] 10:13:26,975 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 55) JBAS010280: Activating Infinispan subsystem.
[Server:server2] 10:13:26,974 INFO  [org.jboss.as.jacorb] (ServerService Thread Pool -- 54) JBAS016300: Activating JacORB Subsystem
[Server:server2] 10:13:26,964 INFO  [org.jboss.as.clustering.jgroups] (ServerService Thread Pool -- 49) JBAS010260: Activating JGroups subsystem.
[Server:server2] 10:13:26,950 INFO  [org.jboss.as.osgi] (ServerService Thread Pool -- 41) JBAS011940: Activating OSGi Subsystem
[Server:server2] 10:13:26,914 INFO  [org.jboss.as.naming] (ServerService Thread Pool -- 42) JBAS011800: Activating Naming Subsystem
[Server:server2] 10:13:27,225 INFO  [org.jboss.as.security] (MSC service thread 1-1) JBAS013100: Current PicketBox version=4.0.6.final
[Server:server2] 10:13:27,568 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 58) JBAS010403: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
[Server:server2] 10:13:27,719 INFO  [org.jboss.as.connector] (MSC service thread 1-4) JBAS010408: Starting JCA Subsystem (JBoss IronJacamar 1.0.7.Final)
[Server:server1] 10:13:27,837 INFO  [org.jboss.as.naming] (MSC service thread 1-4) JBAS011802: Starting Naming Service
[Server:server2] 10:13:27,973 INFO  [org.jboss.as.naming] (MSC service thread 1-1) JBAS011802: Starting Naming Service
[Server:server2] 10:13:27,998 INFO  [org.jboss.as.jaxr] (MSC service thread 1-1) Binding JAXR ConnectionFactory: java:jboss/jaxr/ConnectionFactory
[Server:server1] 10:13:27,858 INFO  [org.jboss.as.connector] (MSC service thread 1-2) JBAS010408: Starting JCA Subsystem (JBoss IronJacamar 1.0.7.Final)
[Server:server2] 10:13:28,163 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-1) JBAS015400: Bound mail session [java:jboss/mail/Default]
[Server:server1] 10:13:28,187 INFO  [org.jboss.as.jaxr] (MSC service thread 1-4) Binding JAXR ConnectionFactory: java:jboss/jaxr/ConnectionFactory
[Server:server2] 10:13:28,300 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 58) JBAS010403: Deploying JDBC-compliant driver class oracle.jdbc.OracleDriver (version 11.2)
[Server:server1] 10:13:28,297 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 58) JBAS010403: Deploying JDBC-compliant driver class oracle.jdbc.OracleDriver (version 11.2)
[Host Controller] 10:13:28,359 INFO  [org.jboss.as.domain.controller.mgmt] (proxy-threads - 1) JBAS010920: Server [Server:server1] registered using connection [Channel ID 6cdb4665 (inbound) of Remoting connection 71ee88dd to /192.168.1.66:59624]
[Host Controller] 10:13:28,443 INFO  [org.jboss.as.domain.controller.mgmt] (proxy-threads - 1) JBAS010920: Server [Server:server2] registered using connection [Channel ID 025391e5 (inbound) of Remoting connection 633a494d to /192.168.1.66:55081]
[Host Controller] 10:13:28,472 INFO  [org.jboss.as.host.controller] (proxy-threads - 1) JBAS010919: Registering server server1
[Server:server1] 10:13:28,862 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-4) JBAS015400: Bound mail session [java:jboss/mail/Default]
[Host Controller] 10:13:28,874 INFO  [org.jboss.as.host.controller] (proxy-threads - 2) JBAS010919: Registering server server2
[Server:server2] 10:13:29,228 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 55) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be pasivated.
[Server:server1] 10:13:29,225 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 55) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be pasivated.
[Server:server1] 10:13:29,387 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 55) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be pasivated.
[Server:server2] 10:13:29,449 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 55) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be pasivated.
[Server:server2] 10:13:29,672 INFO  [org.apache.coyote.ajp.AjpAprProtocol] (MSC service thread 1-2) Starting Coyote AJP/1.3 on ajp-axis-into-ict.nl-192.168.1.66-9081
[Server:server1] 10:13:29,687 INFO  [org.apache.coyote.ajp.AjpAprProtocol] (MSC service thread 1-1) Starting Coyote AJP/1.3 on ajp-axis-into-ict.nl-192.168.1.66-9080
[Server:server2] 10:13:29,815 INFO  [org.jboss.ws.common.management.AbstractServerConfig] (MSC service thread 1-4) JBoss Web Services - Stack CXF Server 4.0.1.GA
[Server:server2] 10:13:29,817 INFO  [org.apache.coyote.http11.Http11AprProtocol] (MSC service thread 1-1) Starting Coyote HTTP/1.1 on http-axis-into-ict.nl-192.168.1.66-8081
[Server:server1] 10:13:29,839 INFO  [org.jboss.ws.common.management.AbstractServerConfig] (MSC service thread 1-3) JBoss Web Services - Stack CXF Server 4.0.1.GA
[Server:server2] 10:13:29,846 INFO  [org.jboss.as.modcluster] (MSC service thread 1-3) JBAS011704: Mod_cluster uses default load balancer provider
[Server:server1] 10:13:29,853 INFO  [org.apache.coyote.http11.Http11AprProtocol] (MSC service thread 1-4) Starting Coyote HTTP/1.1 on http-axis-into-ict.nl-192.168.1.66-8080
[Server:server1] 10:13:29,879 INFO  [org.jboss.as.modcluster] (MSC service thread 1-2) JBAS011704: Mod_cluster uses default load balancer provider
[Server:server1] 10:13:30,055 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-3) live server is starting with configuration HornetQ Configuration (clustered=false,backup=false,sharedStore=true,journalDirectory=/home/jboss/jboss-as-7.1.0.Final/domain/servers/server1/data/messagingjournal,bindingsDirectory=/home/jboss/jboss-as-7.1.0.Final/domain/servers/server1/data/messagingbindings,largeMessagesDirectory=/home/jboss/jboss-as-7.1.0.Final/domain/servers/server1/data/messaginglargemessages,pagingDirectory=/home/jboss/jboss-as-7.1.0.Final/domain/servers/server1/data/messagingpaging)
[Server:server2] 10:13:30,068 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-2) live server is starting with configuration HornetQ Configuration (clustered=false,backup=false,sharedStore=true,journalDirectory=/home/jboss/jboss-as-7.1.0.Final/domain/servers/server2/data/messagingjournal,bindingsDirectory=/home/jboss/jboss-as-7.1.0.Final/domain/servers/server2/data/messagingbindings,largeMessagesDirectory=/home/jboss/jboss-as-7.1.0.Final/domain/servers/server2/data/messaginglargemessages,pagingDirectory=/home/jboss/jboss-as-7.1.0.Final/domain/servers/server2/data/messagingpaging)
[Server:server1] 10:13:30,090 INFO  [org.jboss.modcluster.ModClusterService] (MSC service thread 1-2) Initializing mod_cluster 1.2.0.Final
[Server:server1] 10:13:30,091 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-3) Waiting to obtain live lock
[Server:server2] 10:13:30,102 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-2) Waiting to obtain live lock
[Server:server2] 10:13:30,131 INFO  [org.jboss.modcluster.ModClusterService] (MSC service thread 1-3) Initializing mod_cluster 1.2.0.Final
[Server:server1] 10:13:30,208 INFO  [org.jboss.modcluster.advertise.impl.AdvertiseListenerImpl] (MSC service thread 1-2) Listening to proxy advertisements on 224.0.1.105:23364
[Server:server2] 10:13:30,264 INFO  [org.jboss.modcluster.advertise.impl.AdvertiseListenerImpl] (MSC service thread 1-3) Listening to proxy advertisements on 224.0.1.105:23364
[Server:server1] 10:13:30,338 INFO  [org.hornetq.core.persistence.impl.journal.JournalStorageManager] (MSC service thread 1-3) Using AIO Journal
[Server:server2] 10:13:30,386 INFO  [org.hornetq.core.persistence.impl.journal.JournalStorageManager] (MSC service thread 1-2) Using AIO Journal
[Server:server1] 10:13:30,816 INFO  [org.hornetq.core.server.impl.AIOFileLockNodeManager] (MSC service thread 1-3) Waiting to obtain live lock
[Server:server1] 10:13:30,817 INFO  [org.hornetq.core.server.impl.AIOFileLockNodeManager] (MSC service thread 1-3) Live Server Obtained live lock
[Server:server2] 10:13:30,954 INFO  [org.hornetq.core.server.impl.AIOFileLockNodeManager] (MSC service thread 1-2) Waiting to obtain live lock
[Server:server2] 10:13:30,954 INFO  [org.hornetq.core.server.impl.AIOFileLockNodeManager] (MSC service thread 1-2) Live Server Obtained live lock
[Server:server2] 10:13:31,115 INFO  [org.jboss.as.jacorb] (MSC service thread 1-4) JBAS016330: CORBA ORB Service started
[Server:server1] 10:13:31,108 INFO  [org.jboss.as.jacorb] (MSC service thread 1-1) JBAS016330: CORBA ORB Service started
[Server:server1] 10:13:31,341 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-4) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]
[Server:server2] 10:13:31,438 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]
[Server:server2] 10:13:31,439 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) JBAS010400: Bound data source [java:/jdbc/OracleDS]
[Server:server1] 10:13:31,448 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-2) JBAS010400: Bound data source [java:/jdbc/OracleDS]
[Server:server1] 10:13:31,449 INFO  [org.jboss.as.remoting] (MSC service thread 1-4) JBAS017100: Listening on axis-into-ict.nl/192.168.1.66:4447
[Server:server2] 10:13:31,462 INFO  [org.jboss.as.remoting] (MSC service thread 1-3) JBAS017100: Listening on axis-into-ict.nl/192.168.1.66:4448
[Server:server2] 10:13:31,554 INFO  [org.jboss.as.jacorb] (MSC service thread 1-4) JBAS016328: CORBA Naming Service started
[Server:server1] 10:13:31,587 INFO  [org.jboss.as.jacorb] (MSC service thread 1-1) JBAS016328: CORBA Naming Service started
[Server:server2] 10:13:32,254 INFO  [org.hornetq.core.remoting.impl.netty.NettyAcceptor] (MSC service thread 1-2) Started Netty Acceptor version 3.2.5.Final-a96d88c axis-into-ict.nl:5446 for CORE protocol
[Server:server2] 10:13:32,257 INFO  [org.hornetq.core.remoting.impl.netty.NettyAcceptor] (MSC service thread 1-2) Started Netty Acceptor version 3.2.5.Final-a96d88c axis-into-ict.nl:5456 for CORE protocol
[Server:server1] 10:13:32,257 INFO  [org.hornetq.core.remoting.impl.netty.NettyAcceptor] (MSC service thread 1-3) Started Netty Acceptor version 3.2.5.Final-a96d88c axis-into-ict.nl:5445 for CORE protocol
[Server:server2] 10:13:32,258 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-2) Server is now live
[Server:server2] 10:13:32,259 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-2) HornetQ Server version 2.2.11.Final (HQ_2_2_11_FINAL_AS7, 122) [2b25644c-61f8-11e1-a2e3-000c2976c82d]) started
[Server:server1] 10:13:32,260 INFO  [org.hornetq.core.remoting.impl.netty.NettyAcceptor] (MSC service thread 1-3) Started Netty Acceptor version 3.2.5.Final-a96d88c axis-into-ict.nl:5455 for CORE protocol
[Server:server1] 10:13:32,262 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-3) Server is now live
[Server:server1] 10:13:32,263 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-3) HornetQ Server version 2.2.11.Final (HQ_2_2_11_FINAL_AS7, 122) [2b200d12-61f8-11e1-9621-000c2976c82d]) started
[Server:server1] 10:13:32,338 INFO  [org.jboss.as.messaging] (MSC service thread 1-3) JBAS011601: Bound messaging object to jndi name java:/ConnectionFactory
[Server:server2] 10:13:32,342 INFO  [org.jboss.as.messaging] (MSC service thread 1-2) JBAS011601: Bound messaging object to jndi name java:jboss/exported/jms/RemoteConnectionFactory
[Server:server2] 10:13:32,344 INFO  [org.jboss.as.messaging] (MSC service thread 1-2) JBAS011601: Bound messaging object to jndi name java:/RemoteConnectionFactory
[Server:server1] 10:13:32,343 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-1) trying to deploy queue jms.queue.testQueue
[Server:server2] 10:13:32,347 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-3) trying to deploy queue jms.queue.testQueue
[Server:server2] 10:13:32,356 INFO  [org.jboss.as.messaging] (MSC service thread 1-3) JBAS011601: Bound messaging object to jndi name java:/queue/test
[Server:server2] 10:13:32,362 INFO  [org.jboss.as.messaging] (MSC service thread 1-3) JBAS011601: Bound messaging object to jndi name java:jboss/exported/jms/queue/test
[Server:server2] 10:13:32,368 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-4) trying to deploy queue jms.topic.testTopic
[Server:server1] 10:13:32,368 INFO  [org.jboss.as.messaging] (MSC service thread 1-1) JBAS011601: Bound messaging object to jndi name java:/queue/test
[Server:server1] 10:13:32,370 INFO  [org.jboss.as.messaging] (MSC service thread 1-1) JBAS011601: Bound messaging object to jndi name java:jboss/exported/jms/queue/test
[Server:server1] 10:13:32,371 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-2) trying to deploy queue jms.topic.testTopic
[Server:server1] 10:13:32,442 INFO  [org.jboss.as.deployment.connector] (MSC service thread 1-1) JBAS010406: Registered connection factory java:/JmsXA
[Server:server2] 10:13:32,445 INFO  [org.jboss.as.deployment.connector] (MSC service thread 1-2) JBAS010406: Registered connection factory java:/JmsXA
[Server:server2] 10:13:32,468 INFO  [org.hornetq.ra.HornetQResourceAdapter] (MSC service thread 1-2) HornetQ resource adaptor started
[Server:server1] 10:13:32,470 INFO  [org.jboss.as.messaging] (MSC service thread 1-2) JBAS011601: Bound messaging object to jndi name java:/topic/test
[Server:server1] 10:13:32,472 INFO  [org.jboss.as.messaging] (MSC service thread 1-2) JBAS011601: Bound messaging object to jndi name java:jboss/exported/jms/topic/test
[Server:server2] 10:13:32,469 INFO  [org.jboss.as.messaging] (MSC service thread 1-4) JBAS011601: Bound messaging object to jndi name java:/topic/test
[Server:server1] 10:13:32,473 INFO  [org.jboss.as.messaging] (MSC service thread 1-4) JBAS011601: Bound messaging object to jndi name java:jboss/exported/jms/RemoteConnectionFactory
[Server:server1] 10:13:32,474 INFO  [org.jboss.as.messaging] (MSC service thread 1-4) JBAS011601: Bound messaging object to jndi name java:/RemoteConnectionFactory
[Server:server2] 10:13:32,475 INFO  [org.jboss.as.connector.services.ResourceAdapterActivatorService$ResourceAdapterActivator] (MSC service thread 1-2) IJ020002: Deployed: file://RaActivatorhornetq-ra
[Server:server2] 10:13:32,478 INFO  [org.jboss.as.messaging] (MSC service thread 1-4) JBAS011601: Bound messaging object to jndi name java:jboss/exported/jms/topic/test
[Server:server2] 10:13:32,480 INFO  [org.jboss.as.messaging] (MSC service thread 1-1) JBAS011601: Bound messaging object to jndi name java:/ConnectionFactory
[Server:server1] 10:13:32,482 INFO  [org.hornetq.ra.HornetQResourceAdapter] (MSC service thread 1-1) HornetQ resource adaptor started
[Server:server2] 10:13:32,483 INFO  [org.jboss.as.deployment.connector] (MSC service thread 1-2) JBAS010401: Bound JCA ConnectionFactory [java:/JmsXA]
[Server:server1] 10:13:32,490 INFO  [org.jboss.as.connector.services.ResourceAdapterActivatorService$ResourceAdapterActivator] (MSC service thread 1-1) IJ020002: Deployed: file://RaActivatorhornetq-ra
[Server:server1] 10:13:32,497 INFO  [org.jboss.as.deployment.connector] (MSC service thread 1-2) JBAS010401: Bound JCA ConnectionFactory [java:/JmsXA]
[Server:server2] 10:13:32,504 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss AS 7.1.0.Final "Thunder" started in 10041ms - Started 169 of 292 services (121 services are passive or on-demand)
[Server:server1] 10:13:32,513 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss AS 7.1.0.Final "Thunder" started in 10204ms - Started 169 of 292 services (121 services are passive or on-demand)

When starting the domain a process controller is active is as well to start the host controller and the servers. The following shows the process controller log (that is located in the ${JBOSS_HOME}/domain/log directory; in this directory the log file of the host controller is present as well)

10:13:12,861 INFO  [org.jboss.modules] (main) JBoss Modules version 1.1.1.GA
10:13:13,459 INFO  [org.jboss.as.process.Host Controller.status] (main) JBAS012017: Starting process 'Host Controller'
10:13:22,199 INFO  [org.jboss.as.process.Server:server1.status] (ProcessController-threads - 3) JBAS012017: Starting process 'Server:server1'
10:13:22,292 INFO  [org.jboss.as.process.Server:server2.status] (ProcessController-threads - 3) JBAS012017: Starting process 'Server:server2'

In a domain set-up each server automatically gets its own directory, i.e.,

${JBOSS_HOME}/domain
	/configuration
		domain.xml
		host.xml
	/servers
		/server1
			/data
			/log
		/server2
			/data
			/log

Now the question arises: which use cases are appropriate for a domain and which are appropriate for standalone servers? A domain is about coordinated multi-server management, i.e., it offers a central point through which we can manage multiple servers, with the capability to keep the configuration consistent among the servers in a particular server group. The choice between using a domain or standalone server is about how server are managed. As we saw above we can set-up a group of standalone servers to form a cluster. One thing to note here, as we look at the directory structure provided out of the box by using a domain, is that the servers get there own directory structure, i.e., we get data directories that contain among others the HornetQ message bindings and journals and log directories. Recall that in the standalone case, we had to configure this ourselves. So from a management point of view a domain would be the obvious choice when multiple servers are involved.

Once the domain controller is configured we can configure the host controllers that should join the domain. A host controller configuration requires the following steps

• Define a unique name for the host controller within the domain.
• Configure the management interface and interface binding on an address that is accessible to the domain controller.
• Tell the host controller how to find the domain controller so it can register itself with the domain.
• Define the JVM and the servers to run on the host.

A basic example of a host controller configuration looks as follows:

<host name="hostname" xmlns="urn:jboss:domain:1.0">
	<management>
        <security-realms>
            <security-realm name="ManagementRealm">
                <authentication>
                    <properties path="mgmt-users.properties" relative-to="jboss.domain.config.dir"/>
                </authentication>
            </security-realm>
        </security-realms>
        <management-interfaces>
            <native-interface security-realm="ManagementRealm">
                <socket interface="management" port="9999"/>
            </native-interface>
        </management-interfaces>
    </management>
	<domain-controller>
       <remote host="hostname or IP-address of the domain controller host" port="9999"/>
    </domain-controller>
    <interfaces>
        <interface name="management">
            <nic name="eth0"/>
        </interface>
    </interfaces>
	<jvms>
    	<jvm name="default">
            <heap size="512m" max-size="512m"/>
            <permgen size="256m" max-size="256m"/>
            <jvm-options>
                <option value="-server"/>
                <option value="-XX:NewRatio=2"/>
                <option value="-XX:+UseParallelGC"/>
                <option value="-XX:ParallelGCThreads=2"/>
                <option value="-XX:MaxGCPauseMillis=200"/>
                <option value="-XX:GCTimeRatio=19"/>
                <option value="-XX:+UseParallelOldGC"/>
            </jvm-options>
        </jvm>
    </jvms>
    <servers>
        <server name="server3" group="ha-server-group" auto-start="true">
			<socket-bindings port-offset="2"/>
        </server>
        <server name="server4" group="ha-server-group" auto-start="true">
            <socket-bindings port-offset="3"/>
        </server>
		<server name="server5" group="ha-server-group" auto-start="true">
            <socket-bindings port-offset="4"/>
        </server>
    </servers>
</host>

Let us deploy the same application to our server-group (ha-server-group). To this end we need to configure a data source for the database. We also want to use the Apache HTTP server for the load balancing, which means we need to configure an AJP connector and socket binding. The domain.xml example presented above already contains the these configurations. From the start-up log we can see that AJP is bound to 192.168.1.66 and listening on ports 9080 for server1 and 9081 for server2. To deploy the application we can use the command-line interface. When doing this we first need to connect to domain controller which is listening on 192.168.1.66:9999 (see log output above)

[jboss@axis-into-ict ~]$ cd jboss-as-7.1.0.Final/bin/
[jboss@axis-into-ict bin]$ ./jboss-cli.sh
You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.
[disconnected /] connect 192.168.1.66:9999
[domain@192.168.1.66:9999 /] deploy /home/jboss/jboss-as-7.1.0.Final/deploy/LoadTest6.ear --server-groups=ha-server-group

When the application gets deployed the following is observed in the logging

[Host Controller] 10:31:14,061 INFO  [org.jboss.as.repository] (management-handler-threads -  JBAS014900: Content added at location /home/jboss/jboss-as-7.1.0.Final/domain/data/content/16/1f51dde7f085c822cc4c68b306d57f1bee902d/content
[Server:server1] 10:31:14,259 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-3) JBAS015876: Starting deployment of "LoadTest6.ear"
[Server:server2] 10:31:14,262 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-1) JBAS015876: Starting deployment of "LoadTest6.ear"
[Server:server1] 10:31:14,315 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-4) JBAS015876: Starting deployment of "Model.jar"
[Server:server2] 10:31:14,315 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-4) JBAS015876: Starting deployment of "Web.war"
[Server:server2] 10:31:14,316 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-4) JBAS015876: Starting deployment of "Model.jar"
[Server:server1] 10:31:14,316 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-4) JBAS015876: Starting deployment of "Web.war"
[Server:server2] 10:31:14,421 INFO  [org.jboss.as.jpa] (MSC service thread 1-4) JBAS011401: Read persistence.xml for PersonPersistenceUnit
[Server:server1] 10:31:14,423 INFO  [org.jboss.as.jpa] (MSC service thread 1-4) JBAS011401: Read persistence.xml for PersonPersistenceUnit
[Server:server1] 10:31:14,528 INFO  [org.jboss.as.ejb3.deployment.processors.EjbJndiBindingsDeploymentUnitProcessor] (MSC service thread 1-3) JNDI bindings for session bean named Company in deployment unit subdeployment "Model.jar" of deployment "LoadTest6.ear" are as follows:
[Server:server1]
[Server:server1]        java:global/LoadTest6/Model/Company!model.logic.Company
[Server:server1]        java:app/Model/Company!model.logic.Company
[Server:server1]        java:module/Company!model.logic.Company
[Server:server1]        java:jboss/exported/LoadTest6/Model/Company!model.logic.Company
[Server:server1]        java:global/LoadTest6/Model/Company
[Server:server1]        java:app/Model/Company
[Server:server1]        java:module/Company
[Server:server1]
[Server:server2] 10:31:14,576 INFO  [org.jboss.as.ejb3.deployment.processors.EjbJndiBindingsDeploymentUnitProcessor] (MSC service thread 1-2) JNDI bindings for session bean named Company in deployment unit subdeployment "Model.jar" of deployment "LoadTest6.ear" are as follows:
[Server:server2]
[Server:server2]        java:global/LoadTest6/Model/Company!model.logic.Company
[Server:server2]        java:app/Model/Company!model.logic.Company
[Server:server2]        java:module/Company!model.logic.Company
[Server:server2]        java:jboss/exported/LoadTest6/Model/Company!model.logic.Company
[Server:server2]        java:global/LoadTest6/Model/Company
[Server:server2]        java:app/Model/Company
[Server:server2]        java:module/Company
[Server:server2]
[Server:server2] 10:31:15,167 INFO  [org.jboss.as.jpa] (MSC service thread 1-4) JBAS011402: Starting Persistence Unit Service 'LoadTest6.ear/Model.jar#PersonPersistenceUnit'
[Server:server1] 10:31:15,182 INFO  [org.jboss.as.jpa] (MSC service thread 1-3) JBAS011402: Starting Persistence Unit Service 'LoadTest6.ear/Model.jar#PersonPersistenceUnit'
[Server:server2] 10:31:15,377 INFO  [org.jboss.as.ejb3] (MSC service thread 1-1) JBAS014142: Started message driven bean 'CompanyMDB' with 'hornetq-ra' resource adapter
[Server:server1] 10:31:15,381 INFO  [org.jboss.as.ejb3] (MSC service thread 1-4) JBAS014142: Started message driven bean 'CompanyMDB' with 'hornetq-ra' resource adapter
[Server:server2] 10:31:15,558 INFO  [org.jboss.web] (MSC service thread 1-3) JBAS018210: Registering web context: /LoadTest6
[Server:server1] 10:31:15,563 INFO  [org.jboss.web] (MSC service thread 1-2) JBAS018210: Registering web context: /LoadTest6
[Server:server2] 10:31:15,613 INFO  [org.hibernate.annotations.common.Version] (MSC service thread 1-4) HCANN000001: Hibernate Commons Annotations {4.0.1.Final}
[Server:server1] 10:31:15,613 INFO  [org.hibernate.annotations.common.Version] (MSC service thread 1-3) HCANN000001: Hibernate Commons Annotations {4.0.1.Final}
[Server:server1] 10:31:15,622 INFO  [org.hibernate.Version] (MSC service thread 1-3) HHH000412: Hibernate Core {4.0.1.Final}
[Server:server2] 10:31:15,622 INFO  [org.hibernate.Version] (MSC service thread 1-4) HHH000412: Hibernate Core {4.0.1.Final}
[Server:server1] 10:31:15,624 INFO  [org.hibernate.cfg.Environment] (MSC service thread 1-3) HHH000206: hibernate.properties not found
[Server:server2] 10:31:15,626 INFO  [org.hibernate.cfg.Environment] (MSC service thread 1-4) HHH000206: hibernate.properties not found
[Server:server1] 10:31:15,626 INFO  [org.hibernate.cfg.Environment] (MSC service thread 1-3) HHH000021: Bytecode provider name : javassist
[Server:server2] 10:31:15,628 INFO  [org.hibernate.cfg.Environment] (MSC service thread 1-4) HHH000021: Bytecode provider name : javassist
[Server:server1] 10:31:15,680 INFO  [org.hibernate.ejb.Ejb3Configuration] (MSC service thread 1-3) HHH000204: Processing PersistenceUnitInfo [
[Server:server1]        name: PersonPersistenceUnit
[Server:server1]        ...]
[Server:server2] 10:31:15,682 INFO  [org.hibernate.ejb.Ejb3Configuration] (MSC service thread 1-4) HHH000204: Processing PersistenceUnitInfo [
[Server:server2]        name: PersonPersistenceUnit
[Server:server2]        ...]
[Server:server2] 10:31:15,879 INFO  [org.hibernate.service.jdbc.connections.internal.ConnectionProviderInitiator] (MSC service thread 1-4) HHH000130: Instantiating explicit connection provider: org.hibernate.ejb.connection.InjectedDataSourceConnectionProvider
[Server:server1] 10:31:15,882 INFO  [org.hibernate.service.jdbc.connections.internal.ConnectionProviderInitiator] (MSC service thread 1-3) HHH000130: Instantiating explicit connection provider: org.hibernate.ejb.connection.InjectedDataSourceConnectionProvider
[Server:server1] 10:31:16,034 INFO  [org.hibernate.dialect.Dialect] (MSC service thread 1-3) HHH000400: Using dialect: org.hibernate.dialect.Oracle10gDialect
[Server:server2] 10:31:16,034 INFO  [org.hibernate.dialect.Dialect] (MSC service thread 1-4) HHH000400: Using dialect: org.hibernate.dialect.Oracle10gDialect
[Server:server2] 10:31:16,052 INFO  [org.hibernate.engine.transaction.internal.TransactionFactoryInitiator] (MSC service thread 1-4) HHH000268: Transaction strategy: org.hibernate.engine.transaction.internal.jta.CMTTransactionFactory
[Server:server1] 10:31:16,052 INFO  [org.hibernate.engine.transaction.internal.TransactionFactoryInitiator] (MSC service thread 1-3) HHH000268: Transaction strategy: org.hibernate.engine.transaction.internal.jta.CMTTransactionFactory
[Server:server1] 10:31:16,061 INFO  [org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory] (MSC service thread 1-3) HHH000397: Using ASTQueryTranslatorFactory
[Server:server2] 10:31:16,062 INFO  [org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory] (MSC service thread 1-4) HHH000397: Using ASTQueryTranslatorFactory
[Server:server2] 10:31:16,144 INFO  [org.hibernate.validator.util.Version] (MSC service thread 1-4) Hibernate Validator 4.2.0.Final
[Server:server1] 10:31:16,145 INFO  [org.hibernate.validator.util.Version] (MSC service thread 1-3) Hibernate Validator 4.2.0.Final
[Server:server1] 10:31:17,007 INFO  [org.jboss.as.server] (host-controller-connection-threads - 3) JBAS018559: Deployed "LoadTest6.ear"
[Server:server2] 10:31:17,007 INFO  [org.jboss.as.server] (host-controller-connection-threads - 3) JBAS018559: Deployed "LoadTest6.ear"

As the servers are listening on the same IP addresses and AJP ports as in the standalone case, we do not need to edit anything in the Apache HTTP server configuration. Start the Apache HTTP server and enter the URL: http://hostname:8888/LoadTest6/testservlet and make a few requests to see if everything is working. Open the admin console (http://192.168.1.66:9990/console) and enter the necessary credentials to log in (these we have created in the beginning by using add-user.sh). In the admin console, click server instances and select a particular server. Subsequently, click web to see how many requests were processed by the AJP connector. Let us perform the load test again to see how the environment is performing. When using the jvisualvm we need to know the process IDs of the servers within the domain. To this end we can use the ps command to get information on the running Java process

[jboss@axis-into-ict ~]$ ps -ef|grep java
jboss    7277  7263  0 10:13 pts/2    00:00:06 /home/jboss/jdk1.6.0_31/bin/java -D[Process Controller] -server -Xms512m -Xmx512m -XX:NewRatio=2 -XX:PermSize=256m -XX:MaxPermSize=256m -XX:+UseParallelGC -XX:ParallelGCThreads=2 -XX:MaxGCPauseMillis=200 -XX:GCTimeRatio=19 -XX:+UseParallelOldGC -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Djboss.domain.default.config=domain.xml -Djboss.host.default.config=host.xml -Dorg.jboss.boot.log.file=/home/jboss/jboss-as-7.1.0.Final/domain/log/process-controller.log -Dlogging.configuration=file:/home/jboss/jboss-as-7.1.0.Final/domain/configuration/logging.properties -jar /home/jboss/jboss-as-7.1.0.Final/jboss-modules.jar -mp /home/jboss/jboss-as-7.1.0.Final/modules org.jboss.as.process-controller -jboss-home /home/jboss/jboss-as-7.1.0.Final -jvm /home/jboss/jdk1.6.0_31/bin/java -- -Dorg.jboss.boot.log.file=/home/jboss/jboss-as-7.1.0.Final/domain/log/host-controller.log -Dlogging.configuration=file:/home/jboss/jboss-as-7.1.0.Final/domain/configuration/logging.properties -server -Xms512m -Xmx512m -XX:NewRatio=2 -XX:PermSize=256m -XX:MaxPermSize=256m -XX:+UseParallelGC -XX:ParallelGCThreads=2 -XX:MaxGCPauseMillis=200 -XX:GCTimeRatio=19 -XX:+UseParallelOldGC -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Djboss.domain.default.config=domain.xml -Djboss.host.default.config=host.xml -- -default-jvm /home/jboss/jdk1.6.0_31/bin/java
jboss    7294  7277  0 10:13 pts/2    00:00:11 /home/jboss/jdk1.6.0_31/bin/java -D[Host Controller] -Dorg.jboss.boot.log.file=/home/jboss/jboss-as-7.1.0.Final/domain/log/host-controller.log -Dlogging.configuration=file:/home/jboss/jboss-as-7.1.0.Final/domain/configuration/logging.properties -server -Xms512m -Xmx512m -XX:NewRatio=2 -XX:PermSize=256m -XX:MaxPermSize=256m -XX:+UseParallelGC -XX:ParallelGCThreads=2 -XX:MaxGCPauseMillis=200 -XX:GCTimeRatio=19 -XX:+UseParallelOldGC -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Djboss.domain.default.config=domain.xml -Djboss.host.default.config=host.xml -jar /home/jboss/jboss-as-7.1.0.Final/jboss-modules.jar -mp /home/jboss/jboss-as-7.1.0.Final/modules -jaxpmodule javax.xml.jaxp-provider org.jboss.as.host-controller --pc-address localhost.localdomain --pc-port 60207 -default-jvm /home/jboss/jdk1.6.0_31/bin/java -Djboss.home.dir=/home/jboss/jboss-as-7.1.0.Final
jboss    7347  7277  9 10:13 pts/2    00:06:43 /home/jboss/jdk1.6.0_31/bin/java -D[Server:server1] -XX:PermSize=256m -XX:MaxPermSize=256m -Xms512m -Xmx512m -server -XX:NewRatio=2 -XX:+UseParallelGC -XX:ParallelGCThreads=2 -XX:MaxGCPauseMillis=200 -XX:GCTimeRatio=19 -XX:+UseParallelOldGC -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Dsun.rmi.dgc.client.gcInterval=3600000 -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.server.gcInterval=3600000 -Djava.awt.headless=true -Djboss.host.default.config=host.xml -D[Host=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djboss.home.dir=/home/jboss/jboss-as-7.1.0.Final -Djboss.domain.default.config=domain.xml -Djava.net.preferIPv4Stack=true -Dorg.jboss.boot.log.file=/home/jboss/jboss-as-7.1.0.Final/domain/servers/server1/log/boot.log -Dlogging.configuration=file:/home/jboss/jboss-as-7.1.0.Final/domain/configuration/logging.properties -jar jboss-modules.jar -mp modules -jaxpmodule javax.xml.jaxp-provider org.jboss.as.server
jboss    7364  7277  9 10:13 pts/2    00:06:44 /home/jboss/jdk1.6.0_31/bin/java -D[Server:server2] -XX:PermSize=256m -XX:MaxPermSize=256m -Xms512m -Xmx512m -server -XX:NewRatio=2 -XX:+UseParallelGC -XX:ParallelGCThreads=2 -XX:MaxGCPauseMillis=200 -XX:GCTimeRatio=19 -XX:+UseParallelOldGC -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Dsun.rmi.dgc.client.gcInterval=3600000 -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.server.gcInterval=3600000 -Djava.awt.headless=true -Djboss.host.default.config=host.xml -D[Host=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djboss.home.dir=/home/jboss/jboss-as-7.1.0.Final -Djboss.domain.default.config=domain.xml -Djava.net.preferIPv4Stack=true -Dorg.jboss.boot.log.file=/home/jboss/jboss-as-7.1.0.Final/domain/servers/server2/log/boot.log -Dlogging.configuration=file:/home/jboss/jboss-as-7.1.0.Final/domain/configuration/logging.properties -jar jboss-modules.jar -mp modules -jaxpmodule javax.xml.jaxp-provider org.jboss.as.server

As we can see the first one (7277) is the process controller; the second (7294) is the host controller; the third (7347) and fourth (7364) are respectively server1 and server2. The load test statistics are as follows

To get request count and processing time information we use the command-line interface

[jboss@axis-into-ict ~]$ cd jboss-as-7.1.0.Final/bin
[jboss@axis-into-ict bin]$ ./jboss-cli.sh
You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.
[disconnected /] connect 192.168.1.66:9999
[domain@192.168.1.66:9999 /] cd host=axis-into-ict.nl/server=server1
[domain@192.168.1.66:9999 server=server1] cd deployment=LoadTest6.ear/subdeployment=Web.war/subsystem=web
[domain@192.168.1.66:9999 subsystem=web] cd servlet=TestServlet
[domain@192.168.1.66:9999 servlet=TestServlet] read-attribute request-count
394063
[domain@192.168.1.66:9999 servlet=TestServlet] read-attribute processing-time
930637L
[domain@192.168.1.66:9999 servlet=TestServlet] cd /
[domain@192.168.1.66:9999 /] cd host=axis-into-ict.nl/server=server2
[domain@192.168.1.66:9999 server=server2] cd deployment=LoadTest6.ear/subdeployment=Web.war/subsystem=web
[domain@192.168.1.66:9999 subsystem=web] cd servlet=TestServlet
[domain@192.168.1.66:9999 servlet=TestServlet] read-attribute request-count
394056
[domain@192.168.1.66:9999 servlet=TestServlet] read-attribute processing-time
930302L

To see how the JVMs are doing during the load test we use jvisualvm

The test introduced no particular problems for the JVM.

References

[1] Marchioni, “JBoss AS7 Configuration, Deployment and Administration”, PACKT Publishing, Birmingham, UK, 2011. Essential reading material.
[2] JBoss Documentation.

In this post, we set-up a WebLogic cluster that spans multiple machines. We start with installing the WebLogic software on the machines involved. Next, we create the WebLogic domain by using the WebLogic Scripting Tool (WLST). Use pack and unpack to create managed server directories on remote machines. Create start and stop scripts for the environment. Deploy an application and tune it by using deployment override descriptors. Set-up load balancing by using the Apache HTTP Server and show the steps involved in scaling the environment. Finally, we perform a load test by using The Grinder and use JRockit Mission Control to obtain insight in the performance of the cluster.

Install WebLogic

First, we choose an installation directory, for example /home/oracle/weblogic12.1.1. This will be our middleware home in the installation of WebLogic. To keep things simple use the same directory structure on both machines.

To install JRockit we follow these steps:

• Run the file jrrt-4.0.1-1.6.0-linux-x64.bin (JRockit can be downloaded here).
• Click next on the welcome screen.
• Define the install directory /home/oracle/jrrt-4.0.1-1.6.0 and click next.
• Optionally select extra components and click next.
• When the installation is finished click done.

To WebLogic we follow these steps:

• Navigate to the JDK’s bin directory (/home/oracle/jrrt-4.0.1-1.6.0/bin).
• Enter the following command: ./java -d64 -Xms1024m -Xmx1024m -jar wls1211_generic.jar (WebLogic can be downloaded here).
• Click next on the welcome screen.
• Define the Middleware-Home directory and click next.
• Fill in support credentials if applicable and click next.
• Select custom as installation type and click next.
• De-select the evaluation database and click next.
• Select the installed JDK (if the JDK is not shown, click browse and browse to the root of the JDK) and click next.
• Accept the defaults in the directories screen and click next.
• Click next to start the installation.
• De-select run quickstart and click done.

Create domain

To create our domain, we are going to use WLST. The following script, creates the domain in production mode, disables hostname verification, creates two managed server that are clustered and are present on different machines and creates the artifacts needed to run the application, such as JMS and a data source.

beahome = '/home/oracle/weblogic12.1.1';
pathseparator = '/';
adminusername = 'weblogic';
adminpassword = 'magic12c';
adminservername='AdminServer';
adminserverurl='t3://172.31.0.113:7001';
domainname = 'base_domain';
domaindirectory = beahome + pathseparator + 'user_projects' + pathseparator + 'domains' + pathseparator + domainname;
domaintemplate = beahome + pathseparator + 'wlserver_12.1' + pathseparator + 'common' + pathseparator + 'templates' + pathseparator + 'domains' + pathseparator + 'wls.jar';
jvmdirectory = '/home/oracle/jrrt-4.0.1-1.6.0';

print 'CREATE DOMAIN';
readTemplate(domaintemplate);
setOption('DomainName', domainname);
setOption('OverwriteDomain', 'true');
setOption('ServerStartMode', 'prod');
cd('/Security/base_domain/User/weblogic');
cmo.setName(adminusername);
cmo.setUserPassword(adminpassword);
cd('/');
writeDomain(domaindirectory);

# When using startServer command on WebLogic12c, we have to perform the following steps (1 and 2):
# Starting weblogic server ...
# WLST-WLS-1326447719560: Exception in thread "Main Thread" java.lang.AssertionError: JAX-WS 2.2 API is required, but an older version was found in the JDK.
# WLST-WLS-1326447719560: Use the endorsed standards override mechanism (http://java.sun.com/javase/6/docs/technotes/guides/standards/).
# WLST-WLS-1326447719560: 1) locate the bundled Java EE 6 endorsed directory in $WL_HOME/endorsed.
# WLST-WLS-1326447719560: 2) copy those JAR files to $JAVA_HOME/jre/lib/endorsed OR add the endorsed directory to the value specified by system property java.endorsed.dirs.
print 'START ADMIN SERVER';
startServer(adminservername, domainname, adminserverurl, adminusername, adminpassword, domaindirectory);

print 'CONNECT TO ADMIN SERVER';
connect(adminusername, adminpassword, adminserverurl);

print 'START EDIT MODE';
edit();
startEdit();

#print 'CHANGE NODEMANAGER USERNAME AND PASSWORD';
#cd('/SecurityConfiguration/' + domainname);
#cmo.setNodeManagerUsername(adminusername);
#set('NodeManagerPasswordEncrypted', encrypt(adminpassword, domaindirectory));
#cd('/');

print 'DISABLE HOSTNAME VERIFICATION';
cd('/Servers/' + adminservername + '/SSL/' + adminservername);
cmo.setHostnameVerificationIgnored(true);
cmo.setHostnameVerifier(None);
cmo.setTwoWaySSLEnabled(false);
cmo.setClientCertificateEnforced(false);
cd('/');

print 'SAVE AND ACTIVATE CHANGES';
save();
activate(block='true');

print 'SHUTDOWN THE ADMIN SERVER';
shutdown(block='true');

print 'START ADMIN SERVER';
startServer(adminservername, domainname, adminserverurl, adminusername, adminpassword, domaindirectory);

print 'CONNECT TO ADMIN SERVER';
connect(adminusername, adminpassword, adminserverurl);

print 'START EDIT MODE';
edit();
startEdit();

print 'CREATE MACHINE: machine1';
machine1 = cmo.createUnixMachine('machine1');
machine1.setPostBindUIDEnabled(true);
machine1.setPostBindUID('oracle');
machine1.getNodeManager().setListenAddress('172.31.0.175');
machine1.getNodeManager().setNMType('ssl');

print 'CREATE MACHINE: machine2';
machine2 = cmo.createUnixMachine('machine2');
machine2.setPostBindUIDEnabled(true);
machine2.setPostBindUID('oracle');
machine2.getNodeManager().setListenAddress('172.31.0.113');
machine2.getNodeManager().setNMType('ssl');

print 'CREATE CLUSTER: CLUSTER';
cluster = cmo.createCluster('cluster');
cluster.setClusterMessagingMode('unicast');

print 'CREATE MANAGED SERVER: server1';
server1 = cmo.createServer('server1');
server1.setListenPort(7002);
server1.setListenAddress('172.31.0.175');
server1.setAutoRestart(true);
server1.setAutoKillIfFailed(true);
server1.setRestartMax(2);
server1.setRestartDelaySeconds(10);
server1.getServerStart().setJavaHome(jvmdirectory);
server1.getServerStart().setJavaVendor('Oracle');
server1.getServerStart().setArguments('-jrockit -Xms1024m -Xmx1024m -Xns256m -Xgc:throughput');

print 'CREATE MANAGED SERVER: server2';
server2 = cmo.createServer('server2');
server2.setListenPort(7003);
server2.setListenAddress('172.31.0.113');
server2.setAutoRestart(true);
server2.setAutoKillIfFailed(true);
server2.setRestartMax(2);
server2.setRestartDelaySeconds(10);
server2.getServerStart().setJavaHome(jvmdirectory);
server2.getServerStart().setJavaVendor('Oracle');
server2.getServerStart().setArguments('-jrockit -Xms1024m -Xmx1024m -Xns256m -Xgc:throughput');

print 'ADD MANAGED SERVERS TO CLUSTER';
server1.setCluster(cluster);
server2.setCluster(cluster);

print 'ADD MANAGED SERVERS TO MACHINE';
server1.setMachine(machine1);
server2.setMachine(machine2);

print 'SAVE AND ACTIVATE CHANGES';
save();
activate(block='true');

print 'START EDIT MODE';
startEdit();

print 'CREATE FILESTORE FOR SERVER1';
filestore1 = cmo.createFileStore('FileStore1');
filestore1.setDirectory(beahome + pathseparator + 'deploy');
targets = filestore1.getTargets();
targets.append(server1);
filestore1.setTargets(targets);

print 'CREATE JMS SERVER FOR SERVER1';
jmsserver1 = cmo.createJMSServer('JMSServer1');
jmsserver1.setPersistentStore(filestore1);
jmsserver1.setTargets(targets);

targets.remove(server1);
targets.append(server2);

print 'CREATE FILESTORE FOR SERVER2';
filestore2 = cmo.createFileStore('FileStore2');
filestore2.setDirectory(beahome + pathseparator + 'deploy');
filestore2.setTargets(targets);

print 'CREATE JMS SERVER FOR SERVER2';
jmsserver2 = cmo.createJMSServer('JMSServer2');
jmsserver2.setPersistentStore(filestore2);
jmsserver2.setTargets(targets);

targets.remove(server2);
targets.append(cluster);

print 'CREATE JMS SYSTEM MODULE';
module = cmo.createJMSSystemResource('SystemModule');
module.setTargets(targets);

print 'CREATE SUBDEPLOYMENT';
module.createSubDeployment('SubDeployment');
cd('/JMSSystemResources/SystemModule/SubDeployments/SubDeployment');
set('Targets',jarray.array([ObjectName('com.bea:Name=JMSServer1,Type=JMSServer'), ObjectName('com.bea:Name=JMSServer2,Type=JMSServer')], ObjectName));
cd('/');

resource = module.getJMSResource();

print 'CREATE CONNECTION FACTORY';
resource.createConnectionFactory('ConnectionFactory');
connectionfactory = resource.lookupConnectionFactory('ConnectionFactory');
connectionfactory.setJNDIName('jms/ConnectionFactory');
connectionfactory.setDefaultTargetingEnabled(true);
connectionfactory.getTransactionParams().setTransactionTimeout(3600);
connectionfactory.getTransactionParams().setXAConnectionFactoryEnabled(true);

print 'CREATE UNIFORM DISTRIBUTED QUEUE';
resource.createUniformDistributedQueue('DistributedQueue');
distributedqueue = resource.lookupUniformDistributedQueue('DistributedQueue');
distributedqueue.setJNDIName('jms/CompanyQueue');
distributedqueue.setLoadBalancingPolicy('Round-Robin');
distributedqueue.setSubDeploymentName('SubDeployment');

print 'CREATE DATA SOURCE';
datasource = cmo.createJDBCSystemResource('DataSource');
datasource.setTargets(targets);
jdbcResource = datasource.getJDBCResource();
jdbcResource.setName('DataSource');
names = ['jdbc/exampleDS'];
dataSourceParams = jdbcResource.getJDBCDataSourceParams();
dataSourceParams.setJNDINames(names);
dataSourceParams.setGlobalTransactionsProtocol('LoggingLastResource');
driverParams = jdbcResource.getJDBCDriverParams();
driverParams.setUrl('jdbc:oracle:thin:@hostname:1521:sid');
driverParams.setDriverName('oracle.jdbc.OracleDriver');
driverParams.setPassword('password');
driverProperties = driverParams.getProperties();
driverProperties.createProperty('user');
userProperty = driverProperties.lookupProperty('user');
userProperty.setValue('username');
connectionPoolParams = jdbcResource.getJDBCConnectionPoolParams();
connectionPoolParams.setTestTableName('SQL SELECT 1 FROM DUAL');
connectionPoolParams.setConnectionCreationRetryFrequencySeconds(100);

print 'SAVE AND ACTIVATE CHANGES';
save();
activate(block='true');

print 'SHUTDOWN THE ADMIN SERVER';
shutdown(block='true');

To run the script we can use the following (which also shows the output logging):

[oracle@edu-wls-rh ~]$ cd weblogic12.1.1/wlserver_12.1/common/bin/
[oracle@edu-wls-rh bin]$ ./wlst.sh /home/oracle/weblogic12.1.1/deploy/scripts/create_environment.py 

CLASSPATH=/home/oracle/weblogic12.1.1/patch_wls1211/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/home/oracle/weblogic12.1.1/patch_ocp371/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/home/oracle/jrrt-4.0.1-1.6.0/lib/tools.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/server/lib/weblogic_sp.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/server/lib/weblogic.jar:/home/oracle/weblogic12.1.1/modules/features/weblogic.server.modules_12.1.1.0.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/server/lib/webservices.jar:/home/oracle/weblogic12.1.1/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/home/oracle/weblogic12.1.1/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar::/home/oracle/weblogic12.1.1/utils/config/10.3/config-launch.jar::/home/oracle/weblogic12.1.1/wlserver_12.1/common/derby/lib/derbynet.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/common/derby/lib/derbyclient.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/common/derby/lib/derbytools.jar::

Initializing WebLogic Scripting Tool (WLST) ...

Welcome to WebLogic Server Administration Scripting Shell

Type help() for help on available commands

CREATE DOMAIN
START ADMIN SERVER
Starting weblogic server ...
WLST-WLS-1326701511109: <Jan 16, 2012 9:11:51 AM CET> <Info> <Security> <BEA-090905> <Disabling CryptoJ JCE Provider self-integrity check for better startup performance. To enable this check, specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true>
WLST-WLS-1326701511109: <Jan 16, 2012 9:11:51 AM CET> <Info> <Security> <BEA-090906> <Changing the default Random Number Generator in RSA CryptoJ from ECDRBG to FIPS186PRNG. To disable this change, specify -Dweblogic.security.allowCryptoJDefaultPRNG=true>
WLST-WLS-1326701511109: <Jan 16, 2012 9:11:52 AM CET> <Info> <WebLogicServer> <BEA-000377> <Starting WebLogic Server with Oracle JRockit(R) Version R28.0.1-21-133393-1.6.0_20-20100512-2126-linux-x86_64 from Oracle Corporation.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:11:52 AM CET> <Info> <Management> <BEA-141107> <Version: WebLogic Server 12.1.1.0  Wed Dec 7 08:40:57 PST 2011 1445491 >
WLST-WLS-1326701511109: <Jan 16, 2012 9:11:54 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:11:54 AM CET> <Info> <WorkManager> <BEA-002900> <Initializing self-tuning thread pool.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:11:54 AM CET> <Notice> <Log Management> <BEA-170019> <The server log file /home/oracle/weblogic12.1.1/user_projects/domains/base_domain/servers/AdminServer/logs/AdminServer.log is opened. All server side log events will be written to this file.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:11:57 AM CET> <Notice> <Security> <BEA-090082> <Security initializing using security realm myrealm.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:00 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STANDBY.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:00 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:05 AM CET> <Notice> <Log Management> <BEA-170027> <The server has successfully established a connection with the Domain level Diagnostic Service.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:05 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to ADMIN.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:05 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to RESUMING.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:05 AM CET> <Notice> <Server> <BEA-002613> <Channel "Default[1]" is now listening on fe80:0:0:0:20c:29ff:feb9:e2cc:7001 for protocols iiop, t3, ldap, snmp, http.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:05 AM CET> <Notice> <Server> <BEA-002613> <Channel "Default" is now listening on 172.31.0.113:7001 for protocols iiop, t3, ldap, snmp, http.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:05 AM CET> <Notice> <Server> <BEA-002613> <Channel "Default[2]" is now listening on 127.0.0.1:7001 for protocols iiop, t3, ldap, snmp, http.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:05 AM CET> <Notice> <Server> <BEA-002613> <Channel "Default[3]" is now listening on 0:0:0:0:0:0:0:1:7001 for protocols iiop, t3, ldap, snmp, http.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:05 AM CET> <Notice> <WebLogicServer> <BEA-000329> <Started the WebLogic Server Administration Server "AdminServer" for domain "base_domain" running in production mode.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:05 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to RUNNING.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:05 AM CET> <Notice> <WebLogicServer> <BEA-000360> <The server started in RUNNING mode.>
Server started successfully.
CONNECT TO ADMIN SERVER
Connecting to t3://172.31.0.113:7001 with userid weblogic ...
Successfully connected to Admin Server 'AdminServer' that belongs to domain 'base_domain'.

Warning: An insecure protocol was used to connect to the
server. To ensure on-the-wire security, the SSL port or
Admin port should be used instead.

START EDIT MODE
Location changed to edit tree. This is a writable tree with
DomainMBean as the root. To make changes you will need to start
an edit session via startEdit(). 

For more help, use help(edit)

Starting an edit session ...
Started edit session, please be sure to save and activate your
changes once you are done.
DISABLE HOSTNAME VERIFICATION
SAVE AND ACTIVATE CHANGES
Saving all your changes ...
Saved all your changes successfully.
Activating all your changes, this may take a while ...
The edit lock associated with this edit session is released
once the activation is completed.
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:11 AM CET> <Warning> <Management> <BEA-141239> <The non-dynamic attribute HostnameVerificationIgnored on weblogic.management.configuration.SSLMBeanImpl@4c9421a2([base_domain]/Servers[AdminServer]/SSL[AdminServer]) has been changed. This may require redeploying or rebooting configured entities.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:11 AM CET> <Warning> <Management> <BEA-141238> <A non-dynamic change has been made which affects the server AdminServer. This server must be rebooted in order to consume this change.> 

The following non-dynamic attribute(s) have been changed on MBeans
that require server re-start:
MBean Changed : com.bea:Name=AdminServer,Type=SSL,Server=AdminServer
Attributes changed : HostnameVerificationIgnored

Activation completed
SHUTDOWN THE ADMIN SERVER
Shutting down the server AdminServer with force=false while connected to AdminServer ...
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:11 AM CET> <Alert> <WebLogicServer> <BEA-000396> <Server shutdown has been requested by weblogic.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:11 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to SUSPENDING.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:11 AM CET> <Notice> <HTTP> <BEA-101278> <There are no active sessions. The Web service is ready to suspend.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:11 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to ADMIN.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:11 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to SHUTTING_DOWN.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:11 AM CET> <Notice> <Server> <BEA-002607> <Channel "Default[3]", listening on 0:0:0:0:0:0:0:1:7001, was shut down.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:11 AM CET> <Notice> <Server> <BEA-002607> <Channel "Default", listening on 172.31.0.113:7001, was shut down.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:11 AM CET> <Notice> <Server> <BEA-002607> <Channel "Default[2]", listening on 127.0.0.1:7001, was shut down.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:11 AM CET> <Notice> <Server> <BEA-002607> <Channel "Default[1]", listening on fe80:0:0:0:20c:29ff:feb9:e2cc:7001, was shut down.>
<Jan 16, 2012 9:12:11 AM CET> <Warning> <JNDI> <BEA-050001> <WLContext.close() was called in a different thread than the one in which it was created.>
WLST-WLS-1326701511109: <Jan 16, 2012 9:12:11 AM CET> <Warning> <Management> <BEA-141269> <The temporary bean tree updateDeploymentContext(Mon Jan 16 09:12:11 CET 2012) was allocated for an undo, get, or activate operation, but has not been garbage collected.>
WLST-WLS-1326701511109: Stopped draining WLST-WLS-1326701511109
WLST-WLS-1326701511109: Stopped draining WLST-WLS-1326701511109
WLST lost connection to the WebLogic Server that you were
connected to, this may happen if the server was shutdown or
partitioned. You will have to re-connect to the server once the
server is available.
Disconnected from weblogic server: AdminServer
Disconnected from weblogic server:
START ADMIN SERVER
Starting weblogic server ...
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:15 AM CET> <Info> <Security> <BEA-090905> <Disabling CryptoJ JCE Provider self-integrity check for better startup performance. To enable this check, specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:15 AM CET> <Info> <Security> <BEA-090906> <Changing the default Random Number Generator in RSA CryptoJ from ECDRBG to FIPS186PRNG. To disable this change, specify -Dweblogic.security.allowCryptoJDefaultPRNG=true>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:15 AM CET> <Info> <WebLogicServer> <BEA-000377> <Starting WebLogic Server with Oracle JRockit(R) Version R28.0.1-21-133393-1.6.0_20-20100512-2126-linux-x86_64 from Oracle Corporation.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:16 AM CET> <Info> <Management> <BEA-141107> <Version: WebLogic Server 12.1.1.0  Wed Dec 7 08:40:57 PST 2011 1445491 >
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:18 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:18 AM CET> <Info> <WorkManager> <BEA-002900> <Initializing self-tuning thread pool.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:18 AM CET> <Notice> <Log Management> <BEA-170019> <The server log file /home/oracle/weblogic12.1.1/user_projects/domains/base_domain/servers/AdminServer/logs/AdminServer.log is opened. All server side log events will be written to this file.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:21 AM CET> <Notice> <Security> <BEA-090082> <Security initializing using security realm myrealm.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:24 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STANDBY.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:24 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:28 AM CET> <Notice> <Log Management> <BEA-170027> <The server has successfully established a connection with the Domain level Diagnostic Service.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:28 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to ADMIN.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:28 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to RESUMING.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:28 AM CET> <Notice> <Server> <BEA-002613> <Channel "Default[1]" is now listening on fe80:0:0:0:20c:29ff:feb9:e2cc:7001 for protocols iiop, t3, ldap, snmp, http.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:28 AM CET> <Notice> <Server> <BEA-002613> <Channel "Default" is now listening on 172.31.0.113:7001 for protocols iiop, t3, ldap, snmp, http.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:28 AM CET> <Notice> <Server> <BEA-002613> <Channel "Default[2]" is now listening on 127.0.0.1:7001 for protocols iiop, t3, ldap, snmp, http.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:28 AM CET> <Notice> <Server> <BEA-002613> <Channel "Default[3]" is now listening on 0:0:0:0:0:0:0:1:7001 for protocols iiop, t3, ldap, snmp, http.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:28 AM CET> <Notice> <WebLogicServer> <BEA-000329> <Started the WebLogic Server Administration Server "AdminServer" for domain "base_domain" running in production mode.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:28 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to RUNNING.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:28 AM CET> <Notice> <WebLogicServer> <BEA-000360> <The server started in RUNNING mode.>
Server started successfully.
CONNECT TO ADMIN SERVER
Connecting to t3://172.31.0.113:7001 with userid weblogic ...
Successfully connected to Admin Server 'AdminServer' that belongs to domain 'base_domain'.

Warning: An insecure protocol was used to connect to the
server. To ensure on-the-wire security, the SSL port or
Admin port should be used instead.

START EDIT MODE

Starting an edit session ...
Started edit session, please be sure to save and activate your
changes once you are done.
CREATE MACHINE: machine1
CREATE MACHINE: machine2
CREATE CLUSTER: CLUSTER
CREATE MANAGED SERVER: server1
CREATE MANAGED SERVER: server2
ADD MANAGED SERVERS TO CLUSTER
ADD MANAGED SERVERS TO MACHINE
SAVE AND ACTIVATE CHANGES
Saving all your changes ...
Saved all your changes successfully.
Activating all your changes, this may take a while ...
The edit lock associated with this edit session is released
once the activation is completed.
Activation completed
START EDIT MODE
Starting an edit session ...
Started edit session, please be sure to save and activate your
changes once you are done.
CREATE FILESTORE FOR SERVER1
CREATE JMS SERVER FOR SERVER1
CREATE FILESTORE FOR SERVER2
CREATE JMS SERVER FOR SERVER2
CREATE JMS SYSTEM MODULE
CREATE SUBDEPLOYMENT
CREATE CONNECTION FACTORY
CREATE UNIFORM DISTRIBUTED QUEUE
CREATE DATA SOURCE
SAVE AND ACTIVATE CHANGES
Saving all your changes ...
Saved all your changes successfully.
Activating all your changes, this may take a while ...
The edit lock associated with this edit session is released
once the activation is completed.
Activation completed
SHUTDOWN THE ADMIN SERVER
Shutting down the server AdminServer with force=false while connected to AdminServer ...
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:34 AM CET> <Alert> <WebLogicServer> <BEA-000396> <Server shutdown has been requested by weblogic.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:34 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to SUSPENDING.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:34 AM CET> <Notice> <HTTP> <BEA-101278> <There are no active sessions. The Web service is ready to suspend.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:34 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to ADMIN.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:34 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to SHUTTING_DOWN.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:34 AM CET> <Notice> <Server> <BEA-002607> <Channel "Default", listening on 172.31.0.113:7001, was shut down.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:34 AM CET> <Notice> <Server> <BEA-002607> <Channel "Default[2]", listening on 127.0.0.1:7001, was shut down.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:34 AM CET> <Notice> <Server> <BEA-002607> <Channel "Default[3]", listening on 0:0:0:0:0:0:0:1:7001, was shut down.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:34 AM CET> <Notice> <Server> <BEA-002607> <Channel "Default[1]", listening on fe80:0:0:0:20c:29ff:feb9:e2cc:7001, was shut down.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:34 AM CET> <Warning> <Management> <BEA-141269> <The temporary bean tree updateDeploymentContext(Mon Jan 16 09:12:34 CET 2012) was allocated for an undo, get, or activate operation, but has not been garbage collected.>
WLST-WLS-1326701534609: <Jan 16, 2012 9:12:34 AM CET> <Warning> <Management> <BEA-141269> <The temporary bean tree updateDeploymentContext(Mon Jan 16 09:12:32 CET 2012) was allocated for an undo, get, or activate operation, but has not been garbage collected.>
WLST-WLS-1326701534609: Stopped draining WLST-WLS-1326701534609
WLST-WLS-1326701534609: Stopped draining WLST-WLS-1326701534609
WLST lost connection to the WebLogic Server that you were
connected to, this may happen if the server was shutdown or
partitioned. You will have to re-connect to the server once the
server is available.
Disconnected from weblogic server: AdminServer
Disconnected from weblogic server:

Next, we add a boot.properties file in the ${DOMAIN_HOME}/servers/AdminServer/security directory:

[oracle@edu-wls-rh ~]$ cd weblogic12.1.1/user_projects/domains/base_domain/servers/AdminServer/
[oracle@edu-wls-rh AdminServer]$ mkdir security
[oracle@edu-wls-rh AdminServer]$ vi boot.properties

Add the following name-value pairs to the boot.properties file:

username=weblogic
password=magic12c

Note that when the admin server is started these values will be encrypted. The node manager requires authentication to start and stop managed servers. The first time the node manager is started it communicates with the admin server to obtain a username and password that will be used by the admin server to authenticate the node manager. As we created the domain in production mode, random node manager credits are created. When we want to use, for example, nmConnect we have to set the node manager username and password to known values. To this end start the admin server open the admin console:

• Click on base_domain, security, general and click on the advanced link.
• Edit the NodeManager Username and Password. In our case, we set these to respectively weblogic and magic12c.
• Start the node manager by using the startNodeManager.sh script.

WebLogic provides two command-line utilities, pack and unpack, that helps to create new managed server directories on remote machines, for example on the 172.31.0.113 machine we do the following (first navigate to the ${WL_HOME}/common/bin directory):

[oracle@edu-wls-rh ~]$ cd weblogic12.1.1/wlserver_12.1/common/bin/
[oracle@edu-wls-rh bin]$ ./pack.sh -managed=true -domain=/home/oracle/weblogic12.1.1/user_projects/domains/base_domain -template=/home/oracle/temp/base_domain.jar -template_name=base_domain
<< read domain from "/home/oracle/weblogic12.1.1/user_projects/domains/base_domain"
>>  succeed: read domain from "/home/oracle/weblogic12.1.1/user_projects/domains/base_domain"
<< set config option Managed to "true"
>>  succeed: set config option Managed to "true"
<< write template to "/home/oracle/temp/base_domain.jar"
....................................................................................................
>>  succeed: write template to "/home/oracle/temp/base_domain.jar"
<< close template
>>  succeed: close template

Next, we copy the created base_domain.jar to the 172.31.0.175 machine, for example, by using:

[oracle@edu-wls-rh bin]$ scp oracle@172.31.0.113:/home/oracle/temp/base_domain.jar oracle@172.31.0.175:/home/oracle/temp/
oracle@172.31.0.113's password:
oracle@172.31.0.175's password:
base_domain.jar                                                                                  100%   45KB  44.9KB/s   00:00
Connection to 172.31.0.113 closed.

To use unpack on the 172.31.0.175 machine we can do the following (first navigate to the ${WL_HOME}/common/bin directory):

[oracle@middleware-magic ~]$ cd weblogic12.1.1/wlserver_12.1/common/bin/
[oracle@middleware-magic bin]$ ./unpack.sh -domain=/home/oracle/weblogic12.1.1/user_projects/domains/base_domain -template=/home/oracle/temp/base_domain.jar
<< read template from "/home/oracle/temp/base_domain.jar"
>>  succeed: read template from "/home/oracle/temp/base_domain.jar"
<< set config option DomainName to "base_domain"
>>  succeed: set config option DomainName to "base_domain"
<< write Domain to "/home/oracle/weblogic12.1.1/user_projects/domains/base_domain"
...............................................................................................
>>  succeed: write Domain to "/home/oracle/weblogic12.1.1/user_projects/domains/base_domain"
<< close template
>>  succeed: close template

To test the set-up, start the admin server (by using ./startWebLogic.sh located in the ${DOMAIN_HOME} directory). Subsequently, start the node managers on both machines (by using ./startNodeManager.sh located in the ${WL_HOME}/server/bin directory). When the node manager is started for the first time a nodemanager.properties file is created in the node manager home directory (which is ${WL_HOME}/common/nodemanager by default). We edit the nodemanager.properties and set the property StartScriptEnabled=false (the default is true). When doing this, we need to copy the ${WL_HOME}/endorsed directory to the jrrt-4.0.1-1.6.0/jre/lib/endorsed directory. When the changes are made the node manager needs to be restarted. Subsequently, use the admin console to the start the managed servers.

Start and stop scripts

In general, it is recommended to start the node manager when the machine boots. In this case, we need to know where to put our custom commands that will be called when the system boots. Note that RedHat Enterprise Linux has a /etc/rc.d/rc.local file, that can be used to put custom commands in. Let us do it in the recommended script-based way. Note that Unix-based systems specify so-called run levels, and that for each run level, scripts can be defined that start a certain service. These scripts are located in the /etc/rc.d/init.d directory. This allows for services to be started when the system boots or to be stopped on system shutdown. The different run levels are specified by a specific directory in the /etc/rc.d directory, i.e.,

• rc0.d – contains scripts that are executed on system shutdown.
• rc1.d – contains scripts for single-user mode.
• rc2.d – contains scripts for multi-user mode.
• rc3.d – contains scripts for multi-user mode and networking.
• rc4.d – not used.
• rc5.d – same as rc3.d plus some graphical stuff.
• rc6.d – contains scripts that are executed on system reboot.

The boot sequence is as follows: in the /etc/inittab file the starting runlevel is defined, the script /etc/rc.d/rc.sysinit is called and /etc/rc.d/rc is run. The rc script looks in the /etc/rc.d/rc<start-runlevel>.d to execute the K**<script-name> scripts with the stop option. After this the S**<script-name> scripts are executed with the start option. Note that scripts are started in numerical order, i.e., the S10network script is executed before the S80sendmail script.

To create a node manager ‘service’, we first create basic start and stop scripts for the node manager. To this end, we will use WLST. To start the node manager by using WLST we can use the following:

bea_home = '/home/oracle/weblogic12.1.1';
pathseparator = '/';
listen_address = '172.31.0.175';
nodemanager_listen_port = '5556';
domain_name='base_domain';

node_manager_home = bea_home + pathseparator + 'wlserver_12.1' + pathseparator + 'common' + pathseparator + 'nodemanager';

startNodeManager(verbose='true', NodeManagerHome=node_manager_home, ListenPort=nodemanager_listen_port, ListenAddress=listen_address);

To stop the node manager by using WLST we can use the following:

bea_home = '/home/oracle/weblogic12.1.1';
pathseparator = '/';
admin_username = 'weblogic';
admin_password = 'magic12c';
listen_address = '172.31.0.175';
nodemanager_listen_port = '5556';
admin_server_listen_port = '7001';
domain_name = 'base_domain';

domain_home = bea_home + pathseparator + 'user_projects' + pathseparator + 'domains' + pathseparator + domain_name;
admin_server_url='t3://' + listen_address + ':' + admin_server_listen_port;

nmConnect(admin_username, admin_password, listen_address, nodemanager_listen_port, domain_name, domain_home, 'ssl');
stopNodeManager();

Create two shell scripts that respectively run the start and stop script, i.e.,

#!/bin/sh

BEA_HOME="/home/oracle/weblogic12.1.1"
export BEA_HOME

WL_HOME="${BEA_HOME}/wlserver_12.1"
export WL_HOME

. ${WL_HOME}/common/bin/wlst.sh ${BEA_HOME}/deploy/scripts/startNodeManager.py

#!/bin/sh

BEA_HOME="/home/oracle/weblogic12.1.1"
export BEA_HOME

WL_HOME="${BEA_HOME}/wlserver_12.1"
export WL_HOME

. ${WL_HOME}/common/bin/wlst.sh ${BEA_HOME}/deploy/scripts/stopNodeManager.py

Create a boot script (nodemanager) under the root user and place this script in the /etc/rc.d/init.d directory. An example boot script for the node manager looks as follows:

#!/bin/sh
#
# chkconfig: 235 91 35
# description: starts and stops the node manager
#
#
. /etc/rc.d/init.d/functions

RETVAL=0
SERVICE="nodemanager"

start() {
	echo "Starting Node Manager"
	su - oracle -c "/home/oracle/weblogic12.1.1/deploy/scripts/NodeManagerStartService.sh" >/dev/null 2>&1
	RETVAL=$?
	[ $RETVAL -eq 0 ] && success || failure
	echo
	[ $RETVAL -eq 0 ] && touch /var/lock/subsys/${SERVICE}
	return $RETVAL
} 

stop() {
	echo "Stopping Node Manager"
	su - oracle -c "/home/oracle/weblogic12.1.1/deploy/scripts/NodeManagerStopService.sh" >/dev/null 2>&1
	RETVAL=$?
	[ $RETVAL -eq 0 ] && success || failure
	echo
	[ $RETVAL -eq 0 ] && rm -r /var/lock/subsys/${SERVICE}
	return $RETVAL
} 

restart() {
	stop
	start
} 

case "$1" in
	start)
		start
		;;
	stop)
		stop
		;;
	restart)
		restart
		;;
	*)
		echo $"Usage: $0 {start|stop|restart}"
		exit 1
esac 

exit $?

By using the chkconfig command we can update the runlevel information for system services, for example, chkconfig --add nodemanager. To test the set-up shut the system down and start it again. To check if the nodemanager is running we can use either ps -ef|grep java or netstat -anp|grep :5556 which assumes the node manager is listening on port 5556.

By using WLST we can also create start and stop scripts for the admin server and the managed servers, for example,

bea_home = '/home/oracle/weblogic12.1.1';
pathseparator = '/';
admin_username = 'weblogic';
admin_password = 'magic12c';
listen_address_machine1 = '172.31.0.175';
listen_address_machine2 = '172.31.0.113';
nodemanager_listen_port = '5556';
admin_server_listen_port = '7001';
domain_name = 'base_domain';

domain_home = bea_home + pathseparator + 'user_projects' + pathseparator + 'domains' + pathseparator + domain_name;
admin_server_url='t3://' + listen_address_machine2 + ':' + admin_server_listen_port;

print 'CONNECT TO NODE MANAGER ON MACHINE2';
nmConnect(admin_username, admin_password, listen_address_machine2, nodemanager_listen_port, domain_name, domain_home, 'ssl');

print 'START ADMIN SERVER ON MACHINE2';
nmStart('AdminServer');

print 'CONNECT TO ADMIN SERVER';
connect(admin_username, admin_password, admin_server_url);

print 'START MANAGED SERVERS ON MACHINE2';
start('server2','Server');

print 'DISCONNECT FROM NODE MANAGER ON MACHINE2';
nmDisconnect();

print 'CONNECT TO NODE MANAGER ON MACHINE1';
nmConnect(admin_username, admin_password, listen_address_machine1, nodemanager_listen_port, domain_name, domain_home, 'ssl');

print 'START MANAGED SERVERS ON MACHINE1';
start('server1','Server');
# server3 will be added in a later stadium when we scale the cluster
#start('server3','Server');

print 'DISCONNECT FROM THE ADMIN SERVER';
disconnect();

print 'DISCONNECT FROM NODE MANAGER ON MACHINE1';
nmDisconnect();

bea_home = '/home/oracle/weblogic12.1.1';
pathseparator = '/';
admin_username = 'weblogic';
admin_password = 'magic12c';
listen_address_machine1 = '172.31.0.175';
listen_address_machine2 = '172.31.0.113';
nodemanager_listen_port = '5556';
admin_server_listen_port = '7001';
domain_name = 'base_domain';

domain_home = bea_home + pathseparator + 'user_projects' + pathseparator + 'domains' + pathseparator + domain_name;
admin_server_url='t3://' + listen_address_machine2 + ':' + admin_server_listen_port;

print 'CONNECT TO NODE MANAGER ON MACHINE1';
nmConnect(admin_username, admin_password, listen_address_machine1, nodemanager_listen_port, domain_name, domain_home, 'ssl');

print 'CONNECT TO ADMIN SERVER';
connect(admin_username, admin_password, admin_server_url);

print 'STOPPING MANAGED SERVERS ON MACHINE1';
shutdown('server1','Server','true',1000,'true');
# server3 will be added in a later stadium when we scale the cluster
#shutdown('server3','Server','true',1000,'true');

print 'DISCONNECT FROM NODE MANAGER ON MACHINE1';
nmDisconnect();

print 'CONNECT TO NODE MANAGER ON MACHINE2';
nmConnect(admin_username, admin_password, listen_address_machine1, nodemanager_listen_port, domain_name, domain_home, 'ssl');

print 'STOPPING MANAGED SERVERS ON MACHINE2';
shutdown('server2','Server','true',1000,'true');

print 'STOPPING ADMIN SERVER ON MACHINE2';
shutdown('AdminServer','Server','true',1000,'true');

print 'DISCONNECT FROM NODE MANAGER ON MACHINE2';
nmDisconnect();

For these WLST scripts, we create two shell scripts that respectively run the start and stop script, i.e.,

#!/bin/sh

BEA_HOME="/home/oracle/weblogic12.1.1"
export BEA_HOME

WL_HOME="${BEA_HOME}/wlserver_12.1"
export WL_HOME

. ${WL_HOME}/common/bin/wlst.sh ${BEA_HOME}/deploy/scripts/startDomain.py

#!/bin/sh

BEA_HOME="/home/oracle/weblogic12.1.1"
export BEA_HOME

WL_HOME="${BEA_HOME}/wlserver_12.1"
export WL_HOME

. ${WL_HOME}/common/bin/wlst.sh ${BEA_HOME}/deploy/scripts/stopDomain.py

To start the domain we can use:

[oracle@edu-wls-rh scripts]$ ./DomainStartService.sh 

CLASSPATH=/home/oracle/weblogic12.1.1/patch_wls1211/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/home/oracle/weblogic12.1.1/patch_ocp371/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/home/oracle/jrrt-4.0.1-1.6.0/lib/tools.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/server/lib/weblogic_sp.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/server/lib/weblogic.jar:/home/oracle/weblogic12.1.1/modules/features/weblogic.server.modules_12.1.1.0.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/server/lib/webservices.jar:/home/oracle/weblogic12.1.1/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/home/oracle/weblogic12.1.1/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar::/home/oracle/weblogic12.1.1/utils/config/10.3/config-launch.jar::/home/oracle/weblogic12.1.1/wlserver_12.1/common/derby/lib/derbynet.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/common/derby/lib/derbyclient.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/common/derby/lib/derbytools.jar::

Initializing WebLogic Scripting Tool (WLST) ...

Welcome to WebLogic Server Administration Scripting Shell

Type help() for help on available commands

CONNECT TO NODE MANAGER ON MACHINE2
Connecting to Node Manager ...
<Jan 16, 2012 1:59:54 PM CET> <Info> <Security> <BEA-090905> <Disabling CryptoJ JCE Provider self-integrity check for better startup performance. To enable this check, specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true>
<Jan 16, 2012 1:59:54 PM CET> <Info> <Security> <BEA-090906> <Changing the default Random Number Generator in RSA CryptoJ from ECDRBG to FIPS186PRNG. To disable this change, specify -Dweblogic.security.allowCryptoJDefaultPRNG=true>
Successfully Connected to Node Manager.
START ADMIN SERVER ON MACHINE2
Starting server AdminServer ...
Successfully started server AdminServer ...
CONNECT TO ADMIN SERVER
Connecting to t3://172.31.0.113:7001 with userid weblogic ...
Successfully connected to Admin Server 'AdminServer' that belongs to domain 'base_domain'.

Warning: An insecure protocol was used to connect to the
server. To ensure on-the-wire security, the SSL port or
Admin port should be used instead.

START MANAGED SERVERS ON MACHINE2

Starting server server2 ...............................................................
Server with name server2 started successfully
DISCONNECT FROM NODE MANAGER ON MACHINE2
Successfully disconnected from Node Manager.
CONNECT TO NODE MANAGER ON MACHINE1
Connecting to Node Manager ...
Successfully Connected to Node Manager.
START MANAGED SERVERS ON MACHINE1

Starting server server1 ..................................................................................
Server with name server1 started successfully
DISCONNECT FROM THE ADMIN SERVER
Disconnected from weblogic server: AdminServer
DISCONNECT FROM NODE MANAGER ON MACHINE1
Successfully disconnected from Node Manager.

To stop the domain we can use:

[oracle@edu-wls-rh scripts]$ ./DomainStopService.sh 

CLASSPATH=/home/oracle/weblogic12.1.1/patch_wls1211/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/home/oracle/weblogic12.1.1/patch_ocp371/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/home/oracle/jrrt-4.0.1-1.6.0/lib/tools.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/server/lib/weblogic_sp.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/server/lib/weblogic.jar:/home/oracle/weblogic12.1.1/modules/features/weblogic.server.modules_12.1.1.0.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/server/lib/webservices.jar:/home/oracle/weblogic12.1.1/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/home/oracle/weblogic12.1.1/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar::/home/oracle/weblogic12.1.1/utils/config/10.3/config-launch.jar::/home/oracle/weblogic12.1.1/wlserver_12.1/common/derby/lib/derbynet.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/common/derby/lib/derbyclient.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/common/derby/lib/derbytools.jar::

Initializing WebLogic Scripting Tool (WLST) ...

Welcome to WebLogic Server Administration Scripting Shell

Type help() for help on available commands

CONNECT TO NODE MANAGER ON MACHINE1
Connecting to Node Manager ...
<Jan 16, 2012 2:05:40 PM CET> <Info> <Security> <BEA-090905> <Disabling CryptoJ JCE Provider self-integrity check for better startup performance. To enable this check, specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true>
<Jan 16, 2012 2:05:40 PM CET> <Info> <Security> <BEA-090906> <Changing the default Random Number Generator in RSA CryptoJ from ECDRBG to FIPS186PRNG. To disable this change, specify -Dweblogic.security.allowCryptoJDefaultPRNG=true>
Successfully Connected to Node Manager.
CONNECT TO ADMIN SERVER
Connecting to t3://172.31.0.113:7001 with userid weblogic ...
Successfully connected to Admin Server 'AdminServer' that belongs to domain 'base_domain'.

Warning: An insecure protocol was used to connect to the
server. To ensure on-the-wire security, the SSL port or
Admin port should be used instead.

STOPPING MANAGED SERVERS ON MACHINE1
Shutting down the server server1 with force=true while connected to AdminServer ...
DISCONNECT FROM NODE MANAGER ON MACHINE1
Successfully disconnected from Node Manager.
CONNECT TO NODE MANAGER ON MACHINE2
Connecting to Node Manager ...
Successfully Connected to Node Manager.
STOPPING MANAGED SERVERS ON MACHINE2
Shutting down the server server2 with force=true while connected to AdminServer ...
STOPPING ADMIN SERVER ON MACHINE2
Shutting down the server AdminServer with force=true while connected to AdminServer ...
WLST lost connection to the WebLogic Server that you were
connected to, this may happen if the server was shutdown or
partitioned. You will have to re-connect to the server once the
server is available.
Disconnected from weblogic server: AdminServer
Disconnected from weblogic server:
DISCONNECT FROM NODE MANAGER ON MACHINE2
Successfully disconnected from Node Manager.

Deployment

The application can be found in the post WebLogic12c in Action (in the ‘My first JavaEE6 application’ section). First, we create the following directory structure in /home/oracle/weblogic12.1.1/deploy:

/loadtest
	/app
		LoadTest6.ear
	/plan

By using weblogic.PlanGenerator, we generate a deployment plan:

[oracle@edu-wls-rh ~]$ cd weblogic12.1.1/wlserver_12.1/server/bin/
[oracle@edu-wls-rh bin]$ . ./setWLSEnv.sh
CLASSPATH=/home/oracle/weblogic12.1.1/patch_wls1211/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/home/oracle/weblogic12.1.1/patch_ocp371/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/home/oracle/jrrt-4.0.1-1.6.0/lib/tools.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/server/lib/weblogic_sp.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/server/lib/weblogic.jar:/home/oracle/weblogic12.1.1/modules/features/weblogic.server.modules_12.1.1.0.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/server/lib/webservices.jar:/home/oracle/weblogic12.1.1/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/home/oracle/weblogic12.1.1/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar:

PATH=/home/oracle/weblogic12.1.1/wlserver_12.1/server/bin:/home/oracle/weblogic12.1.1/modules/org.apache.ant_1.7.1/bin:/home/oracle/jrrt-4.0.1-1.6.0/jre/bin:/home/oracle/jrrt-4.0.1-1.6.0/bin:/usr/kerberos/bin:/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/home/oracle/bin

Your environment has been set.
[oracle@edu-wls-rh bin]$ java weblogic.PlanGenerator -plan /home/oracle/weblogic12.1.1/deploy/loadtest/plan/Plan.xml -root /home/oracle/weblogic12.1.1/deploy/loadtest/ /home/oracle/weblogic12.1.1/deploy/loadtest/app/LoadTest6.ear
Generating plan for application /home/oracle/weblogic12.1.1/deploy/loadtest/app/LoadTest6.ear
Export option is: dependencies
Exporting properties...
Saving plan to /home/oracle/weblogic12.1.1/deploy/loadtest/plan/Plan.xml...
<Jan 17, 2012 1:56:43 PM CET> <Info> <J2EE Deployment SPI> <BEA-260086> <The descriptor information at /home/oracle/weblogic12.1.1/deploy/loadtest/plan/Web.war/WEB-INF/weblogic.xml was saved to the configuration area.>
<Jan 17, 2012 1:56:43 PM CET> <Info> <J2EE Deployment SPI> <BEA-260072> <Saved configuration for application, LoadTest6.ear>

When the deployment plan has been generated, the files Plan.xml and weblogic.xml are created automatically. The other files, weblogic-application.xml and weblogic-ejb-jar.xml, we have to create ourselves according to the below directory structure:

/loadtest
	/app
		LoadTest6.ear
	/plan
		/LoadTest6.ear
			/META-INF
				weblogic-application.xml
		/Model.jar
			/META-INF
				weblogic-ejb-jar.xml
		/Web.war
			/WEB-INF
				weblogic.xml
		Plan.xml

in which the deployment overrides (weblogic-application.xml, weblogic-ejb-jar.xml and weblogic.xml) have the following contents:

<weblogic-application xmlns="http://xmlns.oracle.com/weblogic/weblogic-application" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/weblogic/weblogic-application http://xmlns.oracle.com/weblogic/weblogic-application/1.4/weblogic-application.xsd">
</weblogic-application>

<weblogic-ejb-jar xmlns="http://xmlns.oracle.com/weblogic/weblogic-ejb-jar" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/weblogic/weblogic-ejb-jar http://xmlns.oracle.com/weblogic/weblogic-ejb-jar/1.1/weblogic-ejb-jar.xsd">
	<weblogic-enterprise-bean>
		<ejb-name>Company</ejb-name>
		<enable-call-by-reference>True</enable-call-by-reference>
	</weblogic-enterprise-bean>
</weblogic-ejb-jar>

<weblogic-web-app xmlns="http://xmlns.oracle.com/weblogic/weblogic-web-app" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/weblogic/weblogic-web-app http://xmlns.oracle.com/weblogic/weblogic-web-app/1.4/weblogic-web-app.xsd">
</weblogic-web-app>

The Java EE specification requires that EJB components invoked through their remote interfaces must use pass-by-value semantics, meaning that method parameters are copied during the invocation. Changes made to a parameter in the bean method are not reflected in the caller’s version of the object. Copying method parameters is required in the case of a true remote invocation, of course, because the parameters are serialized by the underlying RMI infrastructure before being provided to the bean method. Pass-by-value semantics are also required between components located in different enterprise applications in the same Java virtual machine due to classloader constraints. EJB components located in the same enterprise application archive (.ear) file are loaded by the same classloader and have the option of using pass-by-reference semantics for all invocations, eliminating the unnecessary copying of parameters passed during the invocation and improving performance. By setting the enable-call-by-reference parameter to true in weblogic-ejb-jar.xml, we enable this feature a specific bean in the application. Local references always use pass-by-reference semantics and are unaffected by the enable-call-by-reference setting. When we deploy an EJB with a remote interface and do not enable call by reference, WebLogic will issue a warning of the performance cost, i.e.,

<Jan 16, 2012 5:01:56 PM CET> <Warning> <EJB> <BEA-010202> <Call-by-reference is not enabled for EJB Company. The server will have better performance if it is enabled. To enable call-by-reference, set the enable-call-by-reference element to True in the weblogic-ejb-jar.xml deployment descriptor or corresponding annotation for this EJB.>

Load balancing

To install the Apache HTTP Server we can follow these steps:

• Unpack the httpd-2.2.21.tar.gz file:
  • gzip -d httpd-2.2.21.tar.gz
  • tar xvf httpd-2.2.21.tar
  • cd httpd-2.2.21
• The next step is to configure:
  • ./configure --prefix=/home/oracle/weblogic12.1.1/apache
• Next, compile the various parts for the Apache HTTP Server by using:
  • make
• To install the Apache HTTP Server we use:
  • make install
• Open the httpd.conf (/home/oracle/weblogic12.1.1/apache/conf) file and adjust the following directives:
  • Listen 7777
  • ServerName 172.31.0.113
• To test the set-up, start the Apache HTTP Server:
  • Navigate to /home/oracle/weblogic12.1.1/apache/bin.
  • Run: ./apachectl -k start (To stop the Apache HTTP Server we can use: ./apachectl -k stop).
  • Open a browser en type the following URL: http://172.31.0.113:7777.

To install the the WebLogic plugin we can follow these steps:

• The WebLogic plugins can be downloaded here.
• Unzip the distribution that is appropriate for the operating system, for example, WLSPlugin11g-64bitApache2.2-linux64-x86_64.zip.
• Copy the libclntsh.so.11.1, libnnz11.so and libwlssl.so files to the ${APACHE_HOME}/lib directory. Note in our case ${APACHE_HOME} is /home/oracle/weblogic12.1.1/apache.
• Copy the mod_wl.so file to the ${APACHE_HOME}/modules directory.
• Create a configuration file (for example, mod_wl.conf) for the WebLogic plugin and place this file in the ${APACHE_HOME}/conf directory.

The contents of the mod_wl.conf look as follows:

LoadModule weblogic_module   "/home/oracle/weblogic12.1.1/apache/modules/mod_wl.so"

<IfModule weblogic_module>
	ConnectTimeoutSecs 10
	ConnectRetrySecs 2
	DebugConfigInfo ON
	WLSocketTimeoutSecs 2
	WLIOTimeoutSecs 300
	Idempotent ON
	FileCaching ON
	KeepAliveSecs 20
	KeepAliveEnabled ON
	DynamicServerList ON
	WLProxySSL OFF
</IfModule>

<Location /LoadTest6>
	SetHandler weblogic-handler
	WebLogicCluster 172.31.0.175:7002,172.31.0.113:7003
</Location>

To let the Apache HTTP Server pick up the configuration we add the following to the httpd.conf file:

# put it near the end of the file where all the other includes are present
# mod_wl configuration
Include conf/mod_wl.conf

Restart the HTTP Server (./apachectl -k stop and ./apachectl -k start).

We are not happy with the load (do not know why, just a hypotheses) and need an extra server. We have two scaling options: vertical and horizontal scaling. Vertical scaling relates to adding more CPUs to a machine. To better utilize the server hardware we can add more WebLogic instances to the machine that could lead to increased application throughput. To determine if this is indeed the case we need to benchmark. Benchmarking for scalability is about measuring resource utilization. Good scalability means that service levels can be maintained while the workload is increased. If an application does not scale well, it is not fully utilizing the hardware. Consequently, throughput will degrade. Ideally, a linear load increase should lead to a linear degradation in service levels and performance. Linear scalability can be approached when so-called share nothing clusters are used. The nodes provide the same functionality and know nothing about other nodes in the cluster (no HTTP session replication). In this case, the computing ability of the cluster increases almost linearly as more nodes are added to the cluster, if the back-end information systems, such as a database, are powerful enough.

Applications that ‘share nothing’ are usually sharing state through the database. The application-tier can scale as far as when the database becomes a bottleneck. In general, relying on a single shared resource will eventually cause contention for that resource and thus limit the scalability. Caching is a good resolution. When we cache data at the application-tier we avoid calls to the database (and also avoid relational data to object data conversions). Caching solutions, such as Coherence, provide different kind of caching, i.e., replicated and partitioned. Replicated does not scale well when cache writes are involved as the data needs to be replicated across all the nodes in the grid. A partioned cache, on the other hand, scales very well when cache writes are involved as data ownership is spread throughout the cluster (the system automatically rebalances the data when the number of nodes in the grid changes – we do not need to decide on how to partition the data, it comes out of the box). Another plus is that access to the cache means at most one network trip, this in order to maintain linear scalability. An optimization on read-access can be made when data can be obtained locally (sticky access) in this case a hybrid solution such as the near cache can be applied.

Horizontal scaling relates to adding more machines to the environment, which gives a failover capability that we cannot get with vertical scaling. A good approach is to combine both scaling techniques to obtain better CPU utilization and failover capability.

In order to create a new server based on server1 open the admin console:

• Click environment, servers, select server1 and click clone.
• Change the server name to server3 and listen port to 7004.
• Click OK.

Create a JMS server and a file store and target these to server3:

• Click services, messaging, JMS servers.
• Click new and enter the name as JMSServer3.
• Click create a new store:
  • Choose file store as type and click next.
  • Enter the following parameters:
    • name: FileStore3
    • target: server3
    • directory: /home/oracle/weblogic12.1.1/deploy
  • Click OK.
• Click next and choose as target, server3.
• Click finish.

Add the created JMS server to the subdeployment of the JMS module:

• Click services, messaging, JMS modules and choose systemmodule.
• Click the subdeployments tab and choose subdeployment.
• Add JMSServer3 and click save.

Start server3 by using the admin console:

• Click environment, servers.
• Click the control tab.
• Select server3 and click start.

To test the configuration we can use the URL: http://172.31.0.113:7777/LoadTest6/?__WebLogicBridgeConfig. When the URL is accessed the General Server List should show the servers in the cluster, i.e.,

WebLogic Server Plugin version 1.1, <WLSPLUGINS_11.1.1.4.0_LINUX.X64_101209.1115>
Query String: '?__WebLogicBridgeConfig'
WebLogic Cluster List:
General Server List:
Host: '172.31.0.175' Port: 7004 SecurePort: 0 Status: OK
Host: '172.31.0.113' Port: 7003 SecurePort: 0 Status: OK
Host: '172.31.0.175' Port: 7002 SecurePort: 0 Status: OK
ConnectRetrySecs: '2'
ConnectTimeoutSecs: '10'
WLCookieName: JSESSIONID
Debug: 'no debugging'
DebugConfigInfo: 'ON'
DefaultFileName: ''
DisableCookie2Server: OFF
DynamicServerList: 'ON'
ErrorPage: ''
FileCaching: ON
Idempotent: ON
KeepAliveEnabled: ON
KeepAliveSecs: 20
MaxPostSize: '0'
MaxSkipTime: '10'
PathPrepend: ''
PathTrim: ''
QueryFromRequest: OFF
WLForwardUriUnparsed: OFF
WLAllowDoubleEscapedURI: OFF
SecureProxy: 'OFF'
StatPath: 'false'
WLDNSRefreshInterval: '0'
WLIOTimeoutSecs(old name is HungServerRecoverSecs): '300'
WLLogFile: ''
WLSocketTimeoutSecs: '2'
WLProxySSL: OFF
WLProxyPassThrough: OFF
WLProxySSLPassThrough: OFF

Runtime statistics:
requests: 0
successful requests: 0
Exception objects created: 0
Exception Objects deleted: 0
URL Objects created: 1
URL Objects deleted: 0
connections recycled: 0
UNKNOWN_ERROR_CODE exceptions: 0
CONNECTION_REFUSED exceptions: 0
CONNECTION_TIMEOUT exceptions: 0
READ_ERROR_FROM_CLIENT exceptions: 0
READ_ERROR_FROM_SERVER exceptions: 0
READ_ERROR_FROM_FILE exceptions: 0
WRITE_ERROR_TO_CLIENT exceptions: 0
WRITE_ERROR_TO_SERVER exceptions: 0
WRITE_ERROR_TO_FILE exceptions: 0
READ_TIMEOUT exceptions: 0
WRITE_TIMEOUT exceptions: 0
UNKNOWN_HOST exceptions: 0
NO_RESOURCES exceptions: 0
PROTOCOL_ERROR exceptions: 0
CONFIG_ERROR exceptions: 0
FAILOVER_REQUIRED exceptions: 0
POST_TIMEOUT exceptions: 0
REQUEST_ENTITY_TOO_LARGE exceptions: 0
HALF_OPEN_SOCKET_RETRY exceptions: 0
BAD_REQUEST_FROM_CLIENT exceptions: 0
SSL_INIT_ERROR exceptions: 0
Build date/time: Dec 20 2010 22:11:08
Change Number: 1013

The runtime statistics depict results when a number of requests (http://172.31.0.113:7777/LoadTest6/testservlet) are made. The new server is added to the general server list (which is due to the used DynamicServerList configuration). As the application is stateless and no sessions are created (no users need to login), we have no session binding. To get an idea how the load is balanced, click in the admin console on deployments, LoadTest6 and subsequently click on the workload, monitoring tab.

Testing

We can test the environment by using the The Grinder. To set-up The Grinder, we create the following scripts:

A script to set the environment (setGrinderEnv.sh):

#!/bin/sh
GRINDERPATH=/home/oracle/grinder-3.4
export GRINDERPATH

JAVA_HOME=/home/oracle/jdk1.6.0_24
export JAVA_HOME

USER_MEM_ARGS="-server -Xms512m -Xmx512m -XX:NewRatio=2 -XX:+UseParallelGC -XX:ParallelGCThreads=2 -XX:MaxGCPauseMillis=200 -XX:GCTimeRatio=19 -XX:+UseParallelOldGC"
export USER_MEM_ARGS
GRINDERPROPERTIES=${GRINDERPATH}/examples/grinder.properties
export GRINDERPROPERTIES
CLASSPATH=${GRINDERPATH}/lib/grinder.jar:${CLASSPATH}
export CLASSPATH

PATH=${JAVA_HOME}/bin:${PATH}
export PATH

A script to start the console (startConsole.sh):

#!/bin/sh
source setGrinderEnv.sh
java ${USER_MEM_ARGS} -cp ${CLASSPATH} net.grinder.Console

A script to start an agent that starts the test processes (startAgent.sh):

#!/bin/sh
source setGrinderEnv.sh
java ${USER_MEM_ARGS} -cp ${CLASSPATH} net.grinder.Grinder ${GRINDERPROPERTIES}

The grinder.properties file has, among others, the following properties:

# The file name of the script to run.
#
# Relative paths are evaluated from the directory containing the
# properties file. The default is "grinder.py".
grinder.script = test.py

# The number of worker processes each agent should start. The default
# is 1.
grinder.processes = 1

# The number of worker threads each worker process should start. The
# default is 1.
grinder.threads = 1

# The number of runs each worker process will perform. When using the
# console this is usually set to 0, meaning "run until the console
# sneds a stop or reset signal". The default is 1.
grinder.runs = 0

The test.py script has the following contents:

from net.grinder.script.Grinder import grinder
from net.grinder.script import Test
from net.grinder.plugin.http import HTTPRequest

test1 = Test(1, "Request resource")
request1 = test1.wrap(HTTPRequest())

class TestRunner:
    def __call__(self):
        result = request1.GET("http://172.31.0.113:7777/LoadTest6/testservlet")

Start the test by running startConsole.sh and subsequently startAgent.sh:

During the loadtest we are interested in monitoring the paging. The Linux memory handler manages the allocation of physical memory by freeing portions of physical memory when possible. All processes use memory, but each process does not need all its allocated memory all the time. Taking advantage of this fact, the kernel frees up physical memory by writing some or all of a process’ memory to disk until it is needed again. The kernel uses paging and swapping to perform this memory management. Paging refers to writing portions (pages) of a process’ memory to disk. Swapping refers to writing the entire process to disk. When pages are written to disk, the event is called a page-out, and when pages are returned to physical memory, the event is called a page-in. A page fault occurs when the kernel needs a page, finds it does not exist in physical memory because it has been paged-out, and re-reads it in from disk. When the kernel detects that memory is running low, it attempts to free up memory by paging out. Though this may happen briefly from time to time, if page-outs are plentiful and constant, the kernel can reach a point where it is actually spending more time managing paging activity than running the applications, and system performance suffers. To monitor paging we can use, for example, vmstat 60 10 (which runs vmstat with ten updates, 60 seconds apart):

# Output machine1 (where server1 and server3 are running)
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 2  0      0 1185576  40744 521408    0    0    63    10  384  347  6  3 90  1  0
 2  0      0 1178368  40852 527824    0    0     0   132 2921 2509 21 10 68  0  0
 1  0      0 1174384  40924 530588    0    0     0    63 2442 2130 17  9 74  0  0
 0  0      0 1169796  41020 534336    0    0     0    77 2798 2408 19 10 71  0  0
 0  0      0 1162728  41124 540188    0    0     1   130 2941 2614 18 11 70  0  0
 1  0      0 1155028  41216 544020    0    0     0    76 2899 2549 21 10 69  0  0
 0  0      0 1150812  41316 548804    0    0     0   105 3130 2790 21 11 68  0  0
 0  0      0 1140256  41420 554224    0    0     0   120 2947 2595 19 11 69  0  0
 1  0      0 1121904  41528 561132    0    0     0   140 2896 2557 19 11 70  0  0
 3  0      0 1110760  41628 567212    0    0     0   123 2963 2657 18 13 68  0  0

Output machine2 (where the admin server and server2 are running)
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 1622200  34556 574508    0    0    86    15  642  305  2  0 97  1  0
 0  0      0 1617552  34656 578236    0    0     0    87 2685 1967  5  2 93  0  0
 0  0      0 1615476  34748 580480    0    0     0    55 2309 1562  3  1 95  0  0
 0  0      0 1612872  34844 582992    0    0     0    64 2616 1898  4  2 95  0  0
 1  0      0 1609964  34948 585576    0    0     0    73 2735 2017  4  2 94  0  0
 1  0      0 1605508  35052 589116    0    0     0    81 2712 1985  4  2 94  0  0
 1  0      0 1602076  35152 592352    0    0     0    77 2913 2313  5  2 93  0  0
 0  0      0 1596448  35260 595104    0    0     0    69 2773 2095  4  2 94  0  0
 0  0      0 1590000  35360 599212    0    0     0    92 2693 2038  4  2 94  0  0
 0  0      0 1587164  35456 601748    0    0     0   106 2633 1861  5  2 93  0  0

 FIELD DESCRIPTION FOR VM MODE
   Procs
       r: The number of processes waiting for run time.
       b: The number of processes in uninterruptible sleep.

   Memory
       swpd: the amount of virtual memory used.
       free: the amount of idle memory.
       buff: the amount of memory used as buffers.
       cache: the amount of memory used as cache.
       inact: the amount of inactive memory. (-a option)
       active: the amount of active memory. (-a option)

   Swap
       si: Amount of memory swapped in from disk (/s).
       so: Amount of memory swapped to disk (/s).

   IO
       bi: Blocks received from a block device (blocks/s).
       bo: Blocks sent to a block device (blocks/s).

   System
       in: The number of interrupts per second, including the clock.
       cs: The number of context switches per second.

   CPU
       These are percentages of total CPU time.
       us: Time spent running non-kernel code. (user time, including nice time)
       sy: Time spent running kernel code. (system time)
	   id: Time spent idle. Prior to Linux 2.5.41, this includes IO-wait time.
       wa: Time spent waiting for IO. Prior to Linux 2.5.41, included in idle.
       st: Time stolen from a virtual machine. Prior to Linux 2.6.11, unknown.

The values for si and so are both zero, indicating there are no page-ins and page-outs.

Let us look at some of the diagnostics collected by WebLogic. To this end, open the admin console, click deployments and click the monitoring tab. This page displays monitoring information for all applications deployed to the domain. The JMS tab displays monitoring information for all JMS destinations (note that the application only sends a message when a new person is added):

JMSServer1!JMSServer1.TemporaryQueue0 - Bytes Received Count: 1230, Messages Received Count: 40
JMSServer2!JMSServer2.TemporaryQueue0 - Bytes Received Count: 1230, Messages Received Count: 40
JMSServer3!JMSServer3.TemporaryQueue0 - Bytes Received Count: 1230, Messages Received Count: 40
SystemModule!JMSServer1@DistributedQueue - Bytes Received Count: 719274, Messages Received Count: 2347
SystemModule!JMSServer2@DistributedQueue - Bytes Received Count: 703990, Messages Received Count: 2297
SystemModule!JMSServer3@DistributedQueue - Bytes Received Count: 715648, Messages Received Count: 2335

The EJB (stateless and message-driven) tab displays monitoring information for all the Enterprise JavaBeans (EJBs):

Company	example - Pooled Beans Current Count: 3, Access Total Count: 348936, Transactions Committed Total Count: 348899, Transactions Rolled Back Total Count: 37
CompanyMDB example - Access Total Count: 6979, Processed Message Count: 2347, Transactions Committed Total Count: 6979

The JDBC tab displays monitoring information for all JDBC data sources:

DataSource server1 - Active Connections High Count: 2, Connection Delay Time: 155, PrepStmt Cache Access Count: 105396, Reserve Request Count (cummulative running count of requests for a connection): 116715, Waiting For Connection Total (cumulative running count of requests for a connection that had to wait before getting a connection) 0
DataSource server2 - Active Connections High Count: 2, Connection Delay Time: 127, PrepStmt Cache Access Count: 10173, Reserve Request Count (cummulative running count of requests for a connection): 116716, Waiting For Connection Total (cumulative running count of requests for a connection that had to wait before getting a connection) 0
DataSource server3 - Active Connections High Count: 2, Connection Delay Time: 143, PrepStmt Cache Access Count: 99600, Reserve Request Count (cummulative running count of requests for a connection): 116698, Waiting For Connection Total (cumulative running count of requests for a connection that had to wait before getting a connection) 0

The workload tab shows statistics for the Work Managers, constraints, and policies that are configured for application deployments:

default server1 - Pending Requests: 0, Completed Requests: 118664
default server2 - Pending Requests: 0, Completed Requests: 118616
default server3 - Pending Requests: 0, Completed Requests: 118635

We use the JRockit Mission Control Flight Recording, to see if there are any hick-ups due to garbage collections:

One thing to note is that the garbage collections did not run at same time:

• server1 (172.31.0.175): 03:46:18 154ms – 03:46:18 295ms
• server2 (172.31.0.113): 03:43:29 508ms – 03:43:29 751ms
• server3 (172.31.0.175): 03:43:52 399ms – 03:43:52 650ms

In general, JVM instances running on the same machine will typically not run the garbage collection at the same time. This means that we will have a JVM available to process application requests on other available CPUs. This is an advantage of vertical scaling that leads to a higher application throughput.

To get some insight what effect the garbage collection had on the application we use the events environment of the flight recording. We enable the following events:

• Java application – Java blocked
• Java application – Java wait
• Java virtual machine – GC – garbage collection
• WebLogic Server – EJB – EJB business method invocation
• WebLogic Server – JDBC – JDBC statement execute
• WebLogic Server – Servlet – Servlet invocation

The thread group ‘thread group for queue: weblogic.socket.Muxer’ contain the muxer threads. As a native muxer is used and we have 2 CPUs available, the number of threads is equal to #CPUs + 1 = 3. These threads are always showing this behavior, i.e., one thread at a time is active; picking requests of the sockets and put it in the execute queue. Here it will be picked up by an execute thread from the thread group ‘pooled threads’, which will process it, i.e., the servlet invocation (light green), the EJB business method invocation (blue) and the JDBC statement execution (purple). Note that during the garbage collection the execute threads have to wait (yellow) and a pause in the processing of the work defined for the execute thread is introduced.

References

[1] WebLogic 12c Documentation Library.
[2] Monitoring Virtual Memory with vmstat.

In this post we are going to put WebLogic 12c to work. We start with setting-up our playground by installing VMWare WorkStation and creating a RedHat Enterprise Linux virtual machine. We continue by setting-up a basic WebLogic domain. Next, we set-up a cluster. Here, we show by using a WLST script how to set-up service migration for JTA and JMS. Once the clustered environment is set-up, we build an application that uses JavaEE6 features in order to test the environment.

Set-up our playground

To create a virtual machine we are going to use VMWare WorkStation (VMware-workstation-5.5.6-80404.exe).

• Open VMWare WorkStation and click new, virtual machine.
• Click next on the welcome screen.
• Choose custom and click next.
• Choose a certain virtual machine format and click next.
• Choose a guest operating system. In this example we are going to use RedHat Enterprise Linux. Click next.
• Enter the following parameters:
  • virtual machine name: RedHat Enterprise Linux 5
  • location: c:\temp\vmredhat\
• Click next.
• Choose the number of processors and click next.
• Configure the amount of memory for the virtual machine and click next.
• Select the network type and click next.
• Select the I/O adapter type and click next.
• Select create a new virtual disk and click next.
• Enter the disk capacity (select allocate disk space now and split disk into 2GB files) and click next. One important thing to note is that the disk capacity cannot be changed, this in contrast to the number of processors and the amount of memory that can be adjusted at will once the virtual machine is created.
• Enter the name for the disk file.
• Click finish

Next RedHat Enterprise Linux must be installed in the virtual machine. When an iso file is used that is located on disk, we have to edit the virtual machine:

• In VMWare WorkStation click edit virtual machine settings.
• Click on CD-ROM and select use iso image and browse to the appropriate location.

Start the virtual machine and on first start-up the RedHat Enterprise Linux installer will run:

• Press enter to start the installation (note that various install options are available, for example linux mem=2048m noprobe, see the function buttons for more details).
• The CD media test screen is shown, run the test to be sure the iso image (rhel-server-5.7-x86_64-dvd.iso) is correct. Note that, usually when a 64-bit environment is to be virtualized the VT option of the (in this case Intel) processor must be enabled.
• After a few moments the welcome screen is shown, click next.
• Choose the installation language and click next.
• Choose the keyboard type to be used and click next.
• Enter the installation number (can also be skipped) when applicable and click next.
• The hard drive partitioning screen is shown. Usually the default is suitable for most cases:
  • select – remove linux partitions on selected drives and create default layout.
  • to review the layout, select review and modify partition layout.
• Click next (a warning is shown, click yes).
• The partition layout can be modified at will, which we are not going to do, click next
• Accept the defaults on the boot loader screen and click next.
• Edit the network devices when necessary and click next.
• Choose the location (for time settings) and click next
• Enter the password for the superuser (root) and click next.
• The option to customize the packages to be installed is offered, click next.
• When the customize option was chosen we can choose which packages to install (the defaults are in general fine), click next.
• Click next to start the installation and be patient for a while.
• Click reboot (the installation is finished).

On first start up, RedHat Enterprise Linux offers the system to be configured:

• Click forward on the welcome screen.
• The license agreement is shown, click yes and click forward.
• The option to configure the firewall is offered (disable), click forward.
• Security Enhanced Linux (enforcing), click forward.
• Disable kdump, click forward.
• Set the date and time and click forward.
• Configure software updates and click forward.
• Create a user:
  • username: oracle
  • fullname: oracle
  • password: magic11g
  • confirm password: magic11g
• Click forward.
• Configure a sound card, click forward.
• Install additional Cd’s when needed, click finish.

To get some performance enhancement install VMWare tools:

• When the virtual machine is running, in VMWare WorkStation, click VM and choose install VMWare tools.
• To install VMWare tools by hand, obtain a distribution, for example, VMwareTools-4.0.0-208167.tar.gz and follow these steps:
  • su root
  • tar xzf VMwareTools-4.0.0-208167.tar.gz
  • cd vmware-tools-distrib
  • ./vmware-install.pl
  • follow the instructions presented on the screen (which means pressing enter a lot).
• Log out and log in again – now we are able to adjust, for example, by clicking system, preferences, the screen resolution.

Now that we have our playground ready we continue with installing WebLogic 12c.

Set-up a basic WebLogic environment

First, we choose an installation directory, for example /home/oracle/weblogic12.1.1. This will be our middleware home in the installation of WebLogic.

To install JRockit we follow these steps:

• Run the file jrrt-4.0.1-1.6.0-linux-x64.bin (JRockit can be downloaded here).
• Click next on the welcome screen.
• Define the install directory /home/oracle/jrrt-4.0.1-1.6.0 and click next.
• Optionally select extra components and click next.
• When the installation is finished click done.

To WebLogic we follow these steps:

• Navigate to the JDK’s bin directory (/home/oracle/jrrt-4.0.1-1.6.0/bin).
• Enter the following command: ./java -d64 -Xms1024m -Xmx1024m -jar wls1211_generic.jar (WebLogic can be downloaded here).
• Click next on the welcome screen.
• Define the Middleware-Home directory and click next.
• Fill in support credentials if applicable and click next.
• Select custom as installation type and click next.
• De-select the evaluation database and click next.
• Select the installed JDK (if the JDK is not shown, click browse and browse to the root of the JDK) and click next.
• Accept the defaults in the directories screen and click next.
• Click next to start the installation.
• De-select run quickstart and click done.

Next, we configure a domain. A WebLogic domain is a logical grouping of server instances that are controlled through an admin server. When creating a domain, our first step is to set-up the admin server, i.e., create the files that define the admin server. To this end we run the configuration wizard. Navigate to the ${WL_HOME}/common/bin directory and run config.sh:

• Select Create a New WebLogic Domain.
• Select Generate a Domain Configured Automatically to Support the Following Products: Basic WebLogic Server Domain 12.1.1.0 [wlserver_12.1].
• Enter a domain name, for example base_domain.
• Enter a username and password, for example weblogic and magic11g.
• Choose a mode (for example, production mode) and select a JDK.
• Note that the configuration wizard can create managed servers, cluster and machines.
• Check the summary and click create.

When using WebLogic scripting tool, we can create a domain in production mode as follows:

beahome = '/home/oracle/weblogic12.1.1';
pathseparator = '/';
adminusername = 'weblogic';
adminpassword = 'magic11g';
domainname = 'base_domain';
domaintemplate = beahome + pathseparator + 'wlserver_10.3' + pathseparator + 'common' + pathseparator + 'templates' + pathseparator + 'domains' + pathseparator + 'wls.jar';
domainlocation = beahome + pathseparator + 'user_projects' + pathseparator + 'domains' + pathseparator + domainname;

readTemplate(domaintemplate);
setOption('DomainName', domainname);
setOption('OverwriteDomain', 'true');
setOption('ServerStartMode', 'prod');
cd('/Security/base_domain/User/weblogic');
cmo.setName(adminusername);
cmo.setUserPassword(adminpassword);
cd('/');
writeDomain(domainlocation);

In production mode the command-line will prompt for a username and password on start-up. To overcome this we add a boot.properties file. First create the directory ${DOMAIN_HOME}/servers/AdminServer/security and add a new file: boot.properties. Open the file in a text editor and add the following name-value pairs:

username=weblogic
password=magic11g

To start the admin server, open a command shell, navigate to the ${DOMAIN_HOME} directory and run startWebLogic.sh. The admin console can be reached at http://hostname:7001/console.

Set-up a clustered environment

To set-up the clustered environment we are going to use the following script:

userhome = '/home/oracle';
pathseparator = '/';

adminusername='weblogic';
adminpassword='magic11g';

jvmlocation = userhome + pathseparator + 'jrrt-4.0.1-1.6.0';

print 'CONNECT TO ADMIN SERVER';
connect(adminusername, adminpassword);

print 'START EDIT MODE';
edit();
startEdit();

print 'CREATE MACHINE';
machine = cmo.createMachine('TestMachine');
machine.getNodeManager().setNMType('ssl');

print 'CREATE CLUSTER';
cluster = cmo.createCluster('TestCluster');
cluster.setClusterMessagingMode('unicast');

print 'CREATE MANAGED SERVER: testSERVER1';
testServer1 = cmo.createServer('TestServer1');
testServer1.setListenPort(7002);
testServer1.setAutoRestart(true);
testServer1.setAutoKillIfFailed(true);
testServer1.setRestartMax(2);
testServer1.setRestartDelaySeconds(10);
testServer1.getServerStart().setJavaHome(jvmlocation);
testServer1.getServerStart().setJavaVendor('Oracle');
testServer1.getServerStart().setArguments('-jrockit -Xms1024m -Xmx1024m -Xns256m -Xgc:throughput');

print 'CREATE MANAGED SERVER: testSERVER2';
testServer2 = cmo.createServer('TestServer2');
testServer2.setListenPort(7003);
testServer2.setAutoRestart(true);
testServer2.setAutoKillIfFailed(true);
testServer2.setRestartMax(2);
testServer2.setRestartDelaySeconds(10);
testServer2.getServerStart().setJavaHome(jvmlocation);
testServer2.getServerStart().setJavaVendor('Oracle');
testServer1.getServerStart().setArguments('-jrockit -Xms1024m -Xmx1024m -Xns256m -Xgc:throughput');

print 'ADD MANAGED SERVERS TO CLUSTER';
testServer1.setCluster(cluster);
testServer2.setCluster(cluster);

print 'ADD MANAGED SERVERS TO MACHINE';
testServer1.setMachine(machine);
testServer2.setMachine(machine);

print 'CONFIGURE MIGRATION SERVICE';
cluster.setMigrationBasis('consensus');
cluster.setAdditionalAutoMigrationAttempts(3);
cluster.setMillisToSleepBetweenAutoMigrationAttempts(180000);
cluster.getDatabaseLessLeasingBasis().setMemberDiscoveryTimeout(30);
cluster.getDatabaseLessLeasingBasis().setLeaderHeartbeatPeriod(10);
candidatemachines = cluster.getCandidateMachinesForMigratableServers();
candidatemachines.append(machine);
cd('/Clusters/TestCluster');
set('CandidateMachinesForMigratableServers',candidatemachines);
cd('/');

print 'CONFIGURE MIGRATABLE TARGETS';
migratabletargets = cmo.getMigratableTargets();
for migratabletarget in migratabletargets:
	migratabletarget.setMigrationPolicy('exactly-once');
	cd('/MigratableTargets/' + migratabletarget.getName());
	set('ConstrainedCandidateServers',jarray.array([ObjectName('com.bea:Name=TestServer1,Type=Server'), ObjectName('com.bea:Name=TestServer2,Type=Server')], ObjectName))
	cd('/');

migratabletargetserver = migratabletargets[0];

print 'CREATE FILESTORE';
filestore = cmo.createFileStore('FileStore');
filestore.setDirectory('/home/oracle/weblogic12.1.1/deploy');
targets = filestore.getTargets();
targets.append(migratabletargetserver);
filestore.setTargets(targets);

print 'CREATE JMS SERVER';
jmsserver = cmo.createJMSServer('JMSServer');
jmsserver.setPersistentStore(filestore);
jmsserver.setTargets(targets);

print 'CREATE PATH SERVICE';
pathservice = cmo.createPathService('PathService');
pathservice.setPersistentStore(filestore);
pathservice.setTargets(targets);

targets.remove(migratabletargetserver);
targets.append(cluster);

print 'CREATE JMS SYSTEM MODULE';
module = cmo.createJMSSystemResource('SystemModule');
module.setTargets(targets);

targets.remove(cluster);
targets.append(jmsserver);

print 'CREATE SUBDEPLOYMENT';
module.createSubDeployment('SubDeployment');
subdeployment = module.lookupSubDeployment('SubDeployment');
subdeployment.setTargets(targets);

resource = module.getJMSResource();

print 'CREATE CONNECTION FACTORY';
resource.createConnectionFactory('ConnectionFactory');
connectionfactory = resource.lookupConnectionFactory('ConnectionFactory');
connectionfactory.setJNDIName('jms/ConnectionFactory');
connectionfactory.setDefaultTargetingEnabled(true);
connectionfactory.getDefaultDeliveryParams().setDefaultUnitOfOrder('.System');
connectionfactory.getTransactionParams().setTransactionTimeout(3600);
connectionfactory.getTransactionParams().setXAConnectionFactoryEnabled(true);

print 'CREATE UNIFORM DISTRIBUTED QUEUE';
resource.createUniformDistributedQueue('DistributedQueue');
distributedqueue = resource.lookupUniformDistributedQueue('DistributedQueue');
distributedqueue.setJNDIName('jms/CompanyQueue');
distributedqueue.setLoadBalancingPolicy('Round-Robin');
distributedqueue.setSubDeploymentName('SubDeployment');
distributedqueue.setUnitOfOrderRouting('PathService');

targets.remove(jmsserver);
targets.append(cluster);

print 'CREATE DATA SOURCE';
datasource = cmo.createJDBCSystemResource('DataSource');
datasource.setTargets(targets);
jdbcResource = datasource.getJDBCResource();
jdbcResource.setName('DataSource');
names = ['jdbc/exampleDS'];
dataSourceParams = jdbcResource.getJDBCDataSourceParams();
dataSourceParams.setJNDINames(names);
dataSourceParams.setGlobalTransactionsProtocol('LoggingLastResource');
driverParams = jdbcResource.getJDBCDriverParams();
driverParams.setUrl('jdbc:oracle:thin:@hostname:1521:SID');
driverParams.setDriverName('oracle.jdbc.OracleDriver');
driverParams.setPassword('password');
driverProperties = driverParams.getProperties();
driverProperties.createProperty('user');
userProperty = driverProperties.lookupProperty('user');
userProperty.setValue('username');
connectionPoolParams = jdbcResource.getJDBCConnectionPoolParams();
connectionPoolParams.setTestTableName('SQL SELECT 1 FROM DUAL');
connectionPoolParams.setConnectionCreationRetryFrequencySeconds(100);

print 'SAVE AND ACTIVATE CHANGES';
save();
activate(block='true');

print 'START EDIT MODE';
edit();
startEdit();

print 'CONFIGURE AUTOMATIC JTA MIGRATION';
cd('/Servers/TestServer1/JTAMigratableTarget/TestServer1');
set('ConstrainedCandidateServers',jarray.array([ObjectName('com.bea:Name=TestServer1,Type=Server'), ObjectName('com.bea:Name=TestServer2,Type=Server')], ObjectName));
cmo.setMigrationPolicy('failure-recovery');
cd('/');
cd('/Servers/TestServer2/JTAMigratableTarget/TestServer2');
set('ConstrainedCandidateServers',jarray.array([ObjectName('com.bea:Name=TestServer1,Type=Server'), ObjectName('com.bea:Name=TestServer2,Type=Server')], ObjectName));
cmo.setMigrationPolicy('failure-recovery');
cd('/');

print 'SAVE AND ACTIVATE CHANGES';
save();
activate(block='true');

The script:

• Creates a machine called TestMachine.
• Creates a cluster called TestCluster.
• Creates two managed servers, TestServer1 and TestServer2, that are coupled to the TestMachine and TestCluster. The JVM is tuned by using -jrockit -Xms1024m -Xmx1024m -Xns256m -Xgc:throughput. By choosing throughput as optimization strategy the following defaults are present:
  • The compaction is configured as -XXcompaction:abortable=false,percentage=6.25,heapParts=4096,maxReferences=299900.
  • The thread local area size is configured as -XXtlasize:min=2k,preferred=16k,wastelimit=2k. Note that the preferred size depends on the heap size and lies between 16k and 64k.

  Additional tuning may be necessary when compaction causes long garbage collection pauses. To find out the impact compaction has on the garbage collection pause time, we can run a flight recording and examine the compaction pause parts of old garbage collections. In general, compaction pausetime depends on the compaction ratio (percentage or externalPercentage and internalPercentage) and the maximum number of references. In multi-threaded applications where threads allocate lots of objects, it might be beneficial to increase the TLA size. Caution must be taken, however, to not make the TLA size too large as this increases the fragmentation and as a result more garbage collections need to be run in order to allocate new objects. More information on tuning the JVM can be found in the post Fast, Faster, JRockit.

• Configures a migration service that uses consensus based leasing. The migration policy is set to Auto-Migrate Exactly-Once Services which means that the service will run if at least one candidate server is available in the cluster. Note that this can lead to the case that all migratable targets are running on a single server. The migratable target performs health monitoring on the deployed migratable services and has a direct communication channel to the leasing system. When bad health is detected the migratable target requests the lease to be released in order to trigger a migration:
  • In the case of JTA, the server defaults to shutting down if the JTA system reports itself unhealthy, for example, if an I/O error occurs when accessing the default store. When a server fails, JTA is migrated to a candidate server.
  • In the case of JMS, the JMS server communicates its health to the monitoring system. When a dependent service such as a persistent store fails, for example due to errors in the I/O layer, it is detected by the migration framework. In this case the JMS server along with the persistent store (and path service when configured) is migrated to a candidate server.
• Creates a JMS environment that consists of the following elements:
  • File Store – to persist messages that can be retrieved in the case of a server failure.
  • JMS Server – JMS servers act as management containers for the queues and topics in JMS modules that are targeted to them. A JMS server’s primary responsibility for its destinations is to maintain information on what persistent store is used for any persistent messages that arrive on the destinations, and to maintain the states of durable subscribers created on the destinations.
  • Path Service – A path service is persistent map that can be used to store the mapping of a group of messages in a Message Unit-of-Order to a messaging resource in a cluster. It provides a way to enforce ordering by pinning messages to a member of a cluster hosting servlets, distributed queue members, or store-and-forward agents. Note that the FileStore, JMSServer and PathService are all targeted to a migratable target.
  • JMS Module – JMS system resources are configured and stored as modules similar to standard Java EE modules. Such resources include queues, topics, connection factories, templates, destination keys, quota, distributed queues, distributed topics, foreign servers, and JMS store-and-forward (SAF) parameters. The JMS Module is targeted to the TestCluster.
  • JMS Resources:
    • Connection Factory is XA enabled and has the UnitOfOrder set to System. The connection factory is targeted directly to the JMS module.
    • Uniform Distributed Queue with a round-robin load balancing policy and has a unit of order routing that uses the PathService. Note that the uniform distributed queue is targeted to a SubDeployment that is targeted to the JMSServer.
• Creates a data source that has global transactions enabled by using logging last resource.

The script can be executed by using:

[oracle@edu-wls-rh ~]$ cd weblogic12.1.1/wlserver_12.1/common/bin/
[oracle@edu-wls-rh bin]$ ./wlst.sh /home/oracle/weblogic12.1.1/deploy/scripts/create-environment.py 

CLASSPATH=/home/oracle/weblogic12.1.1/patch_wls1211/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/home/oracle/weblogic12.1.1/patch_ocp371/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/home/oracle/jrrt-4.0.1-1.6.0/lib/tools.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/server/lib/weblogic_sp.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/server/lib/weblogic.jar:/home/oracle/weblogic12.1.1/modules/features/weblogic.server.modules_12.1.1.0.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/server/lib/webservices.jar:/home/oracle/weblogic12.1.1/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/home/oracle/weblogic12.1.1/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar::/home/oracle/weblogic12.1.1/utils/config/10.3/config-launch.jar::/home/oracle/weblogic12.1.1/wlserver_12.1/common/derby/lib/derbynet.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/common/derby/lib/derbyclient.jar:/home/oracle/weblogic12.1.1/wlserver_12.1/common/derby/lib/derbytools.jar::

Initializing WebLogic Scripting Tool (WLST) ...

Jython scans all the jar files it can find at first startup. Depending on the system, this process may take a few minutes to complete, and WLST may not return a prompt right away.

Welcome to WebLogic Server Administration Scripting Shell

Type help() for help on available commands

CONNECT TO ADMIN SERVER
Connecting to t3://localhost:7001 with userid weblogic ...
Successfully connected to Admin Server 'AdminServer' that belongs to domain 'base_domain'.

Warning: An insecure protocol was used to connect to the server. To ensure on-the-wire security, the SSL port or Admin port should be used instead.

START EDIT MODE
Location changed to edit tree. This is a writable tree with DomainMBean as the root. To make changes you will need to start an edit session via startEdit(). 

For more help, use help(edit)

Starting an edit session ...
Started edit session, please be sure to save and activate your changes once you are done.
CREATE MACHINE
CREATE CLUSTER
CREATE MANAGED SERVER: testSERVER1
CREATE MANAGED SERVER: testSERVER2
ADD MANAGED SERVERS TO CLUSTER
ADD MANAGED SERVERS TO MACHINE
CONFIGURE MIGRATION SERVICE
CONFIGURE MIGRATABLE TARGETS
CREATE FILESTORE
CREATE JMS SERVER
CREATE PATH SERVICE
CREATE JMS SYSTEM MODULE
CREATE SUBDEPLOYMENT
CREATE CONNECTION FACTORY
CREATE UNIFORM DISTRIBUTED QUEUE
CREATE DATA SOURCE
SAVE AND ACTIVATE CHANGES
Saving all your changes ...
Saved all your changes successfully.
Activating all your changes, this may take a while ...
The edit lock associated with this edit session is released once the activation is completed.
Activation completed
START EDIT MODE
Already in Edit Tree

Starting an edit session ...
Started edit session, please be sure to save and activate your changes once you are done.
CONFIGURE AUTOMATIC JTA MIGRATION
SAVE AND ACTIVATE CHANGES
Saving all your changes ...
Saved all your changes successfully.
Activating all your changes, this may take a while ...
The edit lock associated with this edit session is released once the activation is completed.

The following non-dynamic attribute(s) have been changed on MBeans that require server re-start:
MBean Changed : com.bea:Name=TestServer1,Type=Server
Attributes changed : Cluster, ListenPort, RestartDelaySeconds, AutoKillIfFailed, Machine, Cluster, ListenPort, RestartDelaySeconds, AutoKillIfFailed, Machine

MBean Changed : com.bea:Name=TestServer1,Type=JTAMigratableTarget,Server=TestServer1
Attributes changed : ConstrainedCandidateServers, ConstrainedCandidateServers, MigrationPolicy

MBean Changed : com.bea:Name=TestServer2,Type=JTAMigratableTarget,Server=TestServer2
Attributes changed : ConstrainedCandidateServers, ConstrainedCandidateServers, MigrationPolicy

MBean Changed : com.bea:Name=TestServer2,Type=Server
Attributes changed : Cluster, ListenPort, RestartDelaySeconds, AutoKillIfFailed, Machine, Cluster, ListenPort, RestartDelaySeconds, AutoKillIfFailed, Machine

Activation completed

Next, we will test if the set-up is working. To this end we need to configure the node manager. The node manager provides a mechanism to start and stop server instances, monitors the health of the servers and restarts servers in the case of failure. The node manager requires authentication to start and stop managed servers. The first time the node manager is started it communicates with the admin server to obtain a username and password that will be used by the admin server to authenticate the node manager. When the domain is created in production mode, random node manager credits are created. When we want to use, for example, nmConnect we have to set the node manager username and password to known values. To this end open the admin console:

• Click on base_domain, security, general and click on the advanced link.
• Edit the NodeManager Username and Password. In our case, we set these to respectively weblogic and magic11g.
• Start the node manager by using the startNodeManager.sh script.

When the node manager is started use the admin console to start the managed servers. When everything is running the node managers and managed servers are properly configured, i.e., the node managers are initialized with the password files they need to accept commands from the admin server and when the managed servers are started on all candidate machines the node manager and domain directory are properly initialized.

To start the environment by using WLST we can use the following scripts. To start the node manager:

bea_home = '/home/oracle/weblogic12.1.1';
pathseparator = '/';
listen_port = '5556';
listen_address = 'localhost';

node_manager_home = bea_home + pathseparator + 'wlserver_12.1' + pathseparator + 'common' + pathseparator + 'nodemanager';

startNodeManager(verbose='true', NodeManagerHome=node_manager_home, ListenPort=listen_port, ListenAddress=listen_address);

To start the servers:

bea_home = '/home/oracle/weblogic12.1.1';
pathseparator = '/';
admin_username = 'weblogic';
admin_password = 'magic11g';
listen_address = 'localhost';
listen_port = '5556';
admin_server_url='t3://localhost:7001'
domain_name = 'base_domain';
domain_home = bea_home + pathseparator + 'user_projects' + pathseparator + 'domains' + pathseparator + domain_name;

print 'CONNECT TO NODE MANAGER';
nmConnect(admin_username, admin_password, listen_address, listen_port, domain_name, domain_home, 'ssl');

print 'START ADMIN SERVER ONLY ON THE MACHINE WHERE THE ADMIN SERVER IS PRESENT';
nmStart('AdminServer');

print 'CONNECT TO ADMIN SERVER';
connect(admin_username, admin_password, admin_server_url);

print 'START MANAGED SERVERS ON THE MACHINE';
start('TestServer1','Server');
start('TestServer2','Server');

To stop the node manager:

bea_home = '/home/oracle/weblogic12.1.1';
pathseparator = '/';
admin_username = 'weblogic';

admin_password = 'magic11g';
listen_address = 'localhost';
listen_port = '5556';

domain_name = 'base_domain';
domain_home = bea_home + pathseparator + 'user_projects' + pathseparator + 'domains' + pathseparator + domain_name;

nmConnect(admin_username, admin_password, listen_address, listen_port, domain_name, domain_home, 'ssl');
stopNodeManager();

To stop the servers:

bea_home = '/home/oracle/weblogic12.1.1';
pathseparator = '/';
admin_username = 'weblogic';
admin_password = 'magic11g';
listen_address = 'localhost';
listen_port = '5556';
admin_server_url='t3://localhost:7001'
domain_name = 'base_domain';
domain_home = bea_home + pathseparator + 'user_projects' + pathseparator + 'domains' + pathseparator + domain_name;

print 'CONNECT TO NODE MANAGER';
nmConnect(admin_username, admin_password, listen_address, listen_port, domain_name, domain_home, 'ssl');

print 'CONNECT TO ADMIN SERVER';
connect(admin_username, admin_password, admin_server_url);

print 'STOPPING SERVERS ON THE MACHINE';
shutdown('TestServer1','Server','true',1000,'true');
shutdown('TestServer2','Server','true',1000,'true');

print 'STOPPING ADMIN SERVER ONLY ON THE MACHINE WHERE THE ADMIN SERVER IS PRESENT';
shutdown('AdminServer','Server','true',1000,'true');

Create shell scripts that run the scripts, for example,

#!/bin/sh

BEA_HOME="/home/oracle/weblogic12.1.1"
export BEA_HOME

WL_HOME="/home/oracle/weblogic12.1.1/wlserver_12.1"
export WL_HOME

. ${WL_HOME}/common/bin/wlst.sh ${BEA_HOME}/deploy/scripts/startNodeManager.py

My first JavaEE6 application

The application uses Java Persistence and Java Messaging, i.e., we have the following persistence.xml file

<persistence xmlns="http://java.sun.com/xml/ns/persistence" version="2.0">
    <persistence-unit name="PersonPersistenceUnit">
        <provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>
        <jta-data-source>jdbc/exampleDS</jta-data-source>
        <class>model.entities.Person</class>
    </persistence-unit>
</persistence>

We use EclipseLink as the persistence provider and use a JTA-enabled data source. We have the following stateless enterprise bean:

package model.logic;

import model.entities.Person;

import javax.annotation.PostConstruct;
import javax.annotation.PreDestroy;
import javax.annotation.Resource;
import javax.ejb.Stateless;
import javax.jms.*;
import javax.persistence.EntityManager;
import javax.persistence.PersistenceContext;

@Stateless(name = "Company", mappedName = "ejb/Company")
public class CompanyBean implements Company {

    @PersistenceContext(unitName = "PersonPersistenceUnit")
    private EntityManager entityManager;

    @Resource(name = "jms/ConnectionFactory")
    private ConnectionFactory connectionFactory;
    @Resource(name = "jms/CompanyQueue")
    private Queue destination;

    private Connection connection = null;
    private Session session = null;
    private MessageProducer messageProducer = null;

    @PostConstruct
    public void init() {
        try {
            connection = connectionFactory.createConnection();
            session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
            messageProducer = session.createProducer(destination);
        } catch (JMSException e) {
            if (connection != null) {
                try {
                    connection.close();
                } catch (JMSException f) {
                    e.printStackTrace();
                }
            }
        }
    }

    @PreDestroy
    public void release() {
        try {
            if (connection != null) {
                connection.close();
            }
        } catch (JMSException e) {
            e.printStackTrace();
        }
    }

    public void insertPerson(Person person) {
        if (findPerson(person.getSofinummer()) == null) {
            entityManager.persist(person);
            sendMessage(person);
        } else {
            updatePerson(person);
        }
    }

    public void removePerson(Integer sofinummer) {
        Person person = findPerson(sofinummer);
        if (person != null) {
            entityManager.remove(entityManager.merge(person));
            sendMessage(person);
        }
    }

    public void updatePerson(Person person) {
        entityManager.merge(person);
    }

    private Person findPerson(Integer sofinummer) {
        return entityManager.find(Person.class, sofinummer);
    }

    private void sendMessage(Person person) {
        try {
            ObjectMessage message = session.createObjectMessage();
            message.setObject(person);
            messageProducer.send(message);
        } catch (JMSException e) {
            e.printStackTrace();
        }
    }
}

We inject an entity manager, JMS connection factory and JMS queue into the bean. By using the EJB lifecycle (PostConstruct) we set-up a JMS message producer (note that by using PreDestroy the created resources are cleaned-up). We accept the defaults for the transaction manager, i.e., required. The method insertPerson, inserts a person into the database and sends a message to the queue, if the person does not exist. If on the other hand, the person does exist the person information is updated in the database. Note that we are using two resources, which means that we have to resort to a two-phase commit. In the server set-up this was configured by using the logging last resources (emulates a two phase commit) and by using XA for the JMS connection factory. For the stateless enterprise bean we have defined the following remote interface:

package model.logic;

import model.entities.Person;

import javax.ejb.Remote;

@Remote
public interface Company {
    public void insertPerson(Person person);

    public void removePerson(Integer sofinummer);

    public void updatePerson(Person person);
}

We have the following message-driven bean:

package model.logic;

import javax.ejb.ActivationConfigProperty;
import javax.ejb.MessageDriven;
import javax.jms.*;

@MessageDriven(mappedName = "jms/CompanyQueue", activationConfig = {
        @ActivationConfigProperty(propertyName = "destinationName", propertyValue = "jms/CompanyQueue"),
        @ActivationConfigProperty(propertyName = "destinationType", propertyValue = "javax.jms.Queue")
})
public class CompanyMDB implements MessageListener {

    public void onMessage(Message message) {
        if (message instanceof ObjectMessage) {
            ObjectMessage objectMessage = (ObjectMessage) message;
            try {
                System.out.println("RECEIVED OBJECT MESSAGE " + objectMessage.getObject());
            } catch (JMSException e) {
                e.printStackTrace();
            }
        }

        if (message instanceof TextMessage) {
            TextMessage textMessage = (TextMessage) message;
            try {
                System.out.println("RECEIVED TEXT MESSAGE " + textMessage.getText());
            } catch (JMSException e) {
                e.printStackTrace();
            }
        }
    }
}

The message-driven bean is configured to listen to the jms/CompanyQueue (which is a distributed queue). The following servlet is used for testing:

package userinterface.servlets;

import model.entities.Person;
import model.logic.Company;

import javax.ejb.EJB;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.PrintWriter;
import java.util.Random;

@WebServlet(name = "TestServlet", urlPatterns = "/testservlet")
public class TestServlet extends HttpServlet {

    @EJB(name = "Company", mappedName = "ejb/Company")
    Company company;

    private Random generator = null;

    @Override
    public void init() throws ServletException {
        generator = new Random();
    }

    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        Person person = createPerson();

        if (generator.nextDouble() < 0.01) {
            company.removePerson(person.getSofinummer());
        } else {
            company.insertPerson(person);
        }
    }

    @Override
    public void destroy() {
        generator = null;
    }

    private Person createPerson() {
        Person person = new Person();
        person.setNaam(Long.toString(Math.abs(generator.nextLong()), 36));
        person.setSofinummer(generator.nextInt(10000));
        return person;
    }
}

Note that the servlet is configured by using the WebServlet annotation. We inject the stateless enterprise bean by using the EJB annotation. Package the application as follows:

LoadTest6.ear
	META-INF
		application.xml
	Model.jar
		META-INF
			ejb-jar.xml
			persistence.xml
		packages
	Web.war
		WEB-INF
			classes
				packages
			web.xml

in which the deployment descriptors are defined as follows:

application.xml

<application xmlns="http://java.sun.com/xml/ns/javaee"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/application_6.xsd"
             version="6">
    <module id="Model">
        <ejb>Model.jar</ejb>
    </module>
    <module id="Web">
        <web>
            <web-uri>Web.war</web-uri>
            <context-root>LoadTest6</context-root>
        </web>
    </module>
</application>

ejb-jar.xml

<ejb-jar xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/ejb-jar_3_1.xsd"
         version="3.1">
</ejb-jar>

web.xml

<web-app xmlns="http://java.sun.com/xml/ns/javaee"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
           version="3.0">
</web-app>

One thing to note is that the deployment descriptors ejb-jar.xml and web.xml are empty, most configuration can in JavaEE6 be accomplished by using annotations. In order to build the application we need javaee-api-6.0.jar on the class path, which can be downloaded here.

Testing

We test the environment by using the The Grinder. The following script calls the application:

from net.grinder.script.Grinder import grinder
from net.grinder.script import Test
from net.grinder.plugin.http import HTTPRequest

test1 = Test(1, "Request resource")
request1 = test1.wrap(HTTPRequest())

class TestRunner:
    def __call__(self):
        result = request1.GET("http://172.31.0.113:7777/LoadTest6/testservlet")

Before, we start the test we configure a load balancer. The load balancer decides how to dispatch requests to the back end server instances. A load balancer needs to do tasks such as distributing requests, health checking and session binding. To load balance requests, we are going to use the web server plug-in. To configure the web server plug-in (mod_wl_ohs), open the mod_wl_ohs.conf file and add the following contents:

LoadModule weblogic_module   "${ORACLE_HOME}/ohs/modules/mod_wl_ohs.so"

<IfModule weblogic_module>
	ConnectTimeoutSecs 10
	ConnectRetrySecs 2
	DebugConfigInfo ON
	WLSocketTimeoutSecs 2
	WLIOTimeoutSecs 300
	Idempotent ON
	FileCaching ON
	KeepAliveSecs 20
	KeepAliveEnabled ON
	DynamicServerList ON
	WLProxySSL OFF
</IfModule>

<Location /LoadTest6>
	SetHandler weblogic-handler
	WebLogicCluster 172.31.0.113:7002,172.31.0.113:7003
</Location>

WebLogic offers an extensive diagnostics framework. Each created resources has its own monitoring environment that gives information how the runtime of a certain sub-system is doing. This information can also be displayed graphically by using the WLDF console extension, which can be enabled in the admin console (preferences, extensions tab and enable the diagnostics console extension – to make the changes active the server has to be restarted). The monitoring dashboard can be reached at: http://hostname:7001/console/dashboard. For example, to view how the memory of the individual JVMs are doing we can start the JVM Runtime Heap view:

It is also possible to create our own views, for example, say we are interested in the current number of connections in the connection pool and want to know if requests are waiting for connections. To this end, click on my views and click on the new view icon. To add graphs to our view click on the metric browser tab, select the TestServer1, select the JDBCConnectionPool, DataSource, and drag and drop the metrics ActiveConnectionsCurrentCount and WaitingForConnectionCurrentCount on the view:

References

[1] WebLogic Server 12c Release 1 (12.1.1) Documentation Library
[2] The Java EE 6 Tutorial

In this post we show a step-by-step example of how to set-up a high available tuned Java EE environment. We start with installing a Java Virtual Machine (in this case HotSpot) and GlassFish Server. We continue with configuring a domain and tune the Java Virtual Machine for the admin server (and the default environment). Subsequently, we show how to set-up a standalone instance on which we deploy an application that uses JBoss RichFaces and Coherence, show how to set-up security for the application and create a deployment plan that is used to configure the application. Next, we show the steps involved in setting up a cluster and configure a load balancer. Subsequently, we show how to create resources needed for Java messaging and database communication. Once the resources are configured, we deploy a Java EE application that uses these resources. Finally, when everything is in-place we create a benchmark to test the application.

Install GlassFish

We first create a user (on the operating system in this case RedHat Linux), for example, oracle. To create a user run the following commands under the root user:

• useradd oracle
• passwd magic11g

This creates a /home/oracle directory. To delete the user we can use: userdel -r oracle the option -r removes the /home/oracle directory.

Next, we adjust the /etc/hosts file. Use /sbin/ifconfig to retrieve the IP address, for example, eth0 – inet addr:172.31.0.149:

127.0.0.1       localhost.localdomain localhost
::1             localhost6.localdomain6 localhost6
172.31.0.149    edu-glf-rh

The name edu-glf-rh is our DNS-name that was mapped to 127.0.0.1, which gets an IP from DHCP (usually in server environments we have to make sure we have a fixed IP address).

Adjust the number of open file descriptors. As root add the following to /etc/security/limits.conf:

oracle soft nofile 8192
oracle hard nofile 8192

Execute the ulimit -u 8192 command. As we are using the Apache HTTP Server we need to install the following libraries:

yum install gcc
yum install gcc-c++

Log out and log in again, but now as the user oracle.

The following software versions will be used in the installation:

• HotSpot JVM – jdk-6u24-linux-x64.bin
• GlassFish Server – ogs-3.1.1-unix-ml.sh

To install HotSpot we follow these steps:

• Run in the /home/oracle directory the file jdk-6u24-linux-x64.bin (when necessary adjust the execute permissions, chmod ug=rwx <filename>).

To install GlassFish we follow these steps:

• Open a command shell and define JAVA_HOME, i.e., export JAVA_HOME=/home/oracle/jdk1.6.0_24.
• Start the installer (./ogs-3.1.1-unix-ml.sh).
• Click next in the welcome screen.
• Choose custom installation and click next.
• Choose the install only option and click next.
• Specify the install directory: /home/oracle/glassfish3 (which is the default) and click next.
• Choose the installed JDK (which is selected by default, if not then specify the directory /home/oracle/jdk1.6.0_24).
• Click next, deselect the update tool and click next.
• Click install and when the installation is finished click exit.

Configure a domain

A GlassFish domain is a logical grouping of server instances that are controlled through a domain administration server. When creating a domain our first step is to set-up the domain administration server. To this end we run the installer (./ogs-3.1.1-unix-ml.sh) again:

• Click next in the welcome screen.
• Choose custom installation and click next.
• Choose configure an existing installation and click next.
• Specify the install directory: /home/oracle/glassfish3 and click next.
• Choose create a server domain and click next.
• Enter the following parameters:
  • domain name: base_domain
  • admin port: 4848
  • http port: 8080
  • username: glassfish
  • password: magic11g
  • reenter password: magic11g
  • deselect start domain after creation
• Click next (domain is being configured).
• Click exit.

Note that the domain administration server is used to configure, manage and monitor servers in a domain. The domain administration server is a GlassFish server instance with extra applications deployed on it that provide administrative capabilities. Other GlassFish server instances (standalone instances and clustered instances) also contain extra applications that the domain administration server uses to send information to them. It further maintains an XML repository (domain.xml) in the ${DOMAIN_HOME}/config directory.

One thing to note is that the domain administration server is not clusterable and when it goes down, we cannot administer our domain. In general, we can just restart the admin server. But what if the machine on which the admin server runs fails? In this case we have to make sure that the files the domain administration server relies on (the admin server directory and the domain configuration files) are available on another machine, such that domain administration server can be restarted on this machine. In GlassFish terms a machine is represented by a node, i.e., a node is a host on which GlassFish Server software has been installed. For each machine that contains server instances, a host must exist. By creating a server domain a node by the name of localhost-<domain-name&tgt; is created automatically.

Apply a patch

When using the Apache HTTP Server and in particular mod_jk as a front-end for the GlassFish Server a NullPointerException is thrown. To this end we have to replace the grizzly-utils.jar in the directory /home/oracle/glassfish3/glassfish/modules (the old one can be removed) by a patched one that can be obtained here.

Getting acquainted

The asadmin utility (present in the /home/oracle/glassfish3/bin directory) can be used for administrative tasks as well as the lifecycle of the GlassFish Server. Note that GlassFish also provides an admin console that can be used. To start the domain we can use:

To stop the domain we can use:

[sourcecode language="text" wraplines="false"
./asadmin stop-domain base_domain

Waiting for the domain to stop ......
Command stop-domain executed successfully.

When the domain administration server is started the admin console can be reached at: http://hostname:4848. As mentioned before the admin console offers an alternative to the asadmin utility in order to configure a GlassFish domain.

The following gives some examples on how to retrieve various sorts of information by using the asadmin utility:

• ./asadmin list-commands - shows all the available commands.
• ./asadmin help <command> - shows the help for a specific command.
• ./asadmin version - shows the GlassFish Server version.
• ./asadmin list-applications [--type <type>] - shows the deployed applications.
• ./asadmin list-containers - shows the application containers.
• ./asadmin list-modules - shows all the available modules.
• ./asadmin show-component-status <application-name> - shows the status of a deployed application.
• ./asadmin uptime - show the time the domain administration server is running.
• ./asadmin list-jvm-options - shows the used JVM options.
• ./asadmin generate-jvm-report [--type summary] - generate a JVM report that contains information about threads, classes, memory and loggers for the domain administration server.
• ./asadmin list-threadpools <server-name> - shows the existing thread pools.

GlassFish maintains one or more thread pools. When GlassFish receives a request, the request is mapped to a request thread from a specific thread pool. The request subsequently processes the request and returns the result. This means we have to make sure that the resources are plenty. For example, when request threads use connections from a connection pool and there are more requests threads asking for connections than there are connections in the connection pool, we get some contention, i.e., a request thread has to wait when a connection becomes available. Note that this adds time to the response a user is waiting for, thus we have to make sure enough resources are available.

The domain configuration is maintained in the domain.xml file, located in the /home/oracle/glassfish3/glassfish/domains/base_domain/config directory The used Java version is maintained in the asenv.conf file (AS_JAVA=/home/oracle/jdk1.6.0_24), located in the /home/oracle/glassfish3/glassfish/config directory.

Tune the JVM

JVM parameters can be added by using the admin console:

• Open the admin console (http://hostname:4848).
• Click on server (Admin Server) and click on configuration: server-config.
• Click on the link JVM Settings and subsequently on the JVM options tab to add or adjust parameters.
• To make the changes active, the server needs to be restarted.

One thing to note is that GlassFish uses a so-called configuration on which a standalone server or cluster of servers is based. When creating a server the option is offered to choose a configuration (which is usually the default-config). In the same manner as described above we can adjust the JVM parameters of the default configuration, which can than be used as a basis for all the servers in the domain.

When using GlassFish (or any other application server for that matter) it is beneficial to tune for application throughput. GlassFish is a multi-threaded environment and to let this run as smoothly as possible we need to give the threads as much resources as possible, hence the choice for the throughput optimization configuration. In the post Tune the JVM that runs Coherence Revisited a detailed discussion is given on how to set up the JVM parameters if we want to tune the HotSpot to optimize for throughput, for example, we can use:

-server -Xms1024m -Xmx1024m -XX:NewRatio=2 -XX:+UseParallelGC -XX:ParallelGCThreads=2 -XX:MaxGCPauseMillis=200 -XX:GCTimeRatio=19 -XX:+UseParallelOldGC -XX:+UseTLAB -XX:LargePageSizeInBytes=2048k -XX:+UseLargePages

in which,

• -server: selects the JIT compiler.
• -Xms: sets the initial heap size.
• -Xmx: sets the maximum heap size.
• -XX:NewRatio=N: sets the young generation to heap size / (1 + N).
• -XX:+UseParallelGC: selects the parallel collector for minor collections.
• -XX:ParallelGCThreads: sets the number of garbage collector threads, i.e., the number of CPUs to be used when collecting garbage.
• -XX:MaxGCPauseMillis: sets the maximum pause time goal.
• -XX:GCTimeRatio=N: sets the throughput goal that is measured in terms of the time spent doing garbage collection versus the time spent outside of garbage collection.
• -XX:+UseParallelOldGC: selects the parallel collector for major collections.
• -XX:+UseTLAB: enables thread-local object allocation.
• -XX:LargePageSizeInBytes: sets the large page size used for the Java heap.
• -XX:+UseLargePages: use large page memory. Note that when the large pages option is enabled, additional operating system configuration is necessary. The steps involved are discussed in detail in the post Tune the JVM that runs Coherence in particular the section Call profiling and large pages.

We only have talked about tuning the JVM, note that there can be additional resources that need to be tuned.

Standalone instance that runs JBoss RichFaces and uses Coherence

In the post JBoss RichFaces, Facelets and Coherence on WebLogic we already showed how to build an application that uses JBoss RichFaces and integrate Coherence. When using GlassFish the JBoss configuration has to be adjusted a little, i.e., in the faces-config.xml we have to configure a org.ajax4jsf.application.AjaxViewHandler:

<faces-config xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-facesconfig_1_2.xsd" version="1.2">
    <application>
        <message-bundle>userinterface.resources.messagebundle</message-bundle>
        <locale-config>
            <supported-locale>en</supported-locale>
            <supported-locale>nl</supported-locale>
        </locale-config>
        <view-handler>org.ajax4jsf.application.AjaxViewHandler</view-handler>
    </application>
	...
</faces-config>

in the web.xml we have to configure the org.ajax4jsf.VIEW_HANDLERS property in we want to use facelets:

<web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" version="2.5">
    <context-param>
        <param-name>org.ajax4jsf.VIEW_HANDLERS</param-name>
        <param-value>com.sun.facelets.FaceletViewHandler</param-value>
    </context-param>
    <context-param>
        <param-name>javax.faces.DEFAULT_SUFFIX</param-name>
        <param-value>.xhtml</param-value>
    </context-param>
    <context-param>
        <param-name>javax.faces.STATE_SAVING_METHOD</param-name>
        <param-value>server</param-value>
    </context-param>
    <context-param>
        <param-name>org.richfaces.SKIN</param-name>
        <param-value>blueSky</param-value>
    </context-param>
    <context-param>
        <param-name>org.richfaces.CONTROL_SKINNING</param-name>
        <param-value>enable</param-value>
    </context-param>
    <filter>
        <filter-name>RichFaces Filter</filter-name>
        <filter-class>org.ajax4jsf.Filter</filter-class>
    </filter>
    <filter-mapping>
        <filter-name>RichFaces Filter</filter-name>
        <servlet-name>Faces Servlet</servlet-name>
        <dispatcher>REQUEST</dispatcher>
        <dispatcher>FORWARD</dispatcher>
        <dispatcher>INCLUDE</dispatcher>
    </filter-mapping>
    <servlet>
        <servlet-name>Faces Servlet</servlet-name>
        <servlet-class>javax.faces.webapp.FacesServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
    </servlet>
    <servlet-mapping>
        <servlet-name>Faces Servlet</servlet-name>
        <url-pattern>/faces/*</url-pattern>
    </servlet-mapping>
    <listener>
        <listener-class>com.sun.faces.config.ConfigureListener</listener-class>
    </listener>
    <listener>
        <listener-class>userinterface.listeners.ContextListener</listener-class>
    </listener>
    <security-constraint>
        <web-resource-collection>
            <web-resource-name>All</web-resource-name>
            <url-pattern>/faces/*</url-pattern>
        </web-resource-collection>
        <auth-constraint>
            <role-name>MANAGER</role-name>
            <role-name>EMPLOYEE</role-name>
        </auth-constraint>
        <user-data-constraint>
            <transport-guarantee>NONE</transport-guarantee>
        </user-data-constraint>
    </security-constraint>
    <login-config>
        <auth-method>FORM</auth-method>
        <form-login-config>
            <form-login-page>/login.jspx</form-login-page>
            <form-error-page>/login.jspx</form-error-page>
        </form-login-config>
    </login-config>
    <security-role>
        <role-name>MANAGER</role-name>
    </security-role>
    <security-role>
        <role-name>EMPLOYEE</role-name>
    </security-role>
</web-app>

To create a standalone instance we can use the following steps:

• In the admin console, click on standalone instances and click new.
• Enter the following parameters:
  • instance name: app-server1
  • node: localhost-base_domain
  • configuration: default config (make a copy of the selected configuration)
• Click OK.

When using asadmin we can use: ./asadmin create-instance --node localhost-base_domain app-server1. By leaving out --config a copy of the default-config configuration will be created by the name of app-server1-config. By using ./asadmin delete-instance app-server1 the instance can be deleted when necessary.

To start the server:

• In the admin console, click on standalone instances, select app-server1 and click start

Note that the listen port information can be found in the app-server1, properties environment. The HTTP listen port is set by using the HTTP_LISTENER_PORT property, for example, 28080.

To start and stop standalone instances by using asadmin, we can respectively use: ./asadmin start-instance app-server1 and ./asadmin stop-instance app-server1.

To deploy the application, we first create a new directory ${GLASSFISH_HOME}/deploy/application. In this directory we copy our WAR file. To deploy applications by using the admin console we can follow these steps:

• Click on standalone instances and on app-server1.
• Click on the applications tab and click deploy.
• Enter the following parameters:
  • location: choose local packaged file..., click browse files and navigate to the war file.
  • type: web application
  • context root: application
  • application name: application
  • status: enabled
  • targets: app-server1
• Click OK.

To deploy an application by using asadmin we can use: ./asadmin deploy --name application --contextroot application --enabled=true --target app-server1 /home/oracle/glassfish3/deploy/application/application.war. When the application is successfully deployed it can be reached at: http://hostname:28080/application/faces/overview.xhtml.

Logging

The admin server log files are located in the ${DOMAIN_HOME}/logs directory. The server instance log files are located in the ${NODE_HOME}/<server-instance-name>/logs directory (in our case ${NODE_HOME} is equal to /home/oracle/glassfish3/glassfish/nodes/localhost-base_domain and server-instance-name is app-server1).

By using the admin console log files can be viewed as follows:

• Click on standalone instances and choose a particular server listed in the table and click view log files.

Using the Apache HTTP Server as a front-end

General information about the Apache HTTP Server can be found here. When we want to use the Apache HTTP Server as a front-end and in particular mod_jk we have to enable the JK listener for the server in question. In order to accomplish this we follow these steps:

• In the admin console, click on standalone instances and choose app-server1.
• Click on the configuration link app-server1-config.
• Click on the network listeners link and choose http-listener-1.
• Enable the JK Listener option.

We also need to increase the max thread pool size of the servers for which we have configured the JK listener. The default is rather low, namely 5. We have to make the max thread pool size equal to the MaxClients directive configured on the Apache HTTP Server (usually it is something like MaxClients 150) otherwise threads will pile up in the GlassFish Server. To adjust the max thread pool size, we can follow these steps:

• In the admin console, open the tree configurations, app-server1-config, thread pools, http-thread-pool.
• Adjust the max thread pool size attribute and set this to 150.
• Click save.
• Restart the server.

Note that it is handy when we plan to add more servers to do this for the default configuration as well. Information about which thread pool is being used by a particular listener can be found by clicking: configurations, app-server1-config, network config, network listeners, http-listener-1. The thread pool attribute contains the thread pool that is used by the listener.

To install the Apache HTTP Server we can follow these steps:

• Unpack the httpd-2.2.21.tar.gz file:
  • gzip -d httpd-2.2.21.tar.gz
  • tar xvf httpd-2.2.21.tar
  • cd httpd-2.2.21
• The next step is to configure:
  • ./configure --prefix=/home/oracle/glassfish3/apache
• Next, compile the various parts for the Apache HTTP Server by using:
  • make
• To install the Apache HTTP Server we use:
  • make install
• Open the httpd.conf (/home/oracle/glassfish3/apache/conf) file and adjust the following directives:
  • Listen 7777
  • ServerName ip-address
• To test the set-up, start the Apache HTTP Server:
  • Navigate to /home/oracle/glassfish3/apache/bin.
  • Run: ./apachectl -k start (To stop the Apache HTTP Server we can use: ./apachectl -k stop).
  • Open a browser en type the following URL: http://<hostname>:7777.

To install the mod_jk module we can follow these steps:

• Unpack the tomcat-connectors-1.2.32-src.tar.gz file:
  • gzip -d tomcat-connectors-1.2.32-src.tar.gz
  • tar xvf tomcat-connectors-1.2.32-src.tar
  • cd tomcat-connectors-1.2.32-src/native
• The next step is to configure the module:
  • ./configure --with-apxs=/home/oracle/glassfish3/apache/bin/apxs
• Compile:
  • make
• In the native directory, a directory apache-2.0 is created. Here a mod_jk.so file is present. Copy this file to the directory: /home/oracle/glassfish3/apache/modules.

To configure the mod_jk module, create a new file mod_jk.conf in the directory /home/oracle/glassfish3/apache/conf. An example configuration looks as follows:

LoadModule jk_module /home/oracle/glassfish3/apache/modules/mod_jk.so

JkWorkersFile /home/oracle/glassfish3/apache/conf/glassfish-workers.properties

# where to put the log files for the jk module
JkLogFile /home/oracle/glassfish3/apache/logs/mod_jk.log

# the log level [debug|info|error]
JkLogLevel info

# log format
JkLogStampFormat "[%a %b %d %H:%M:%S %Y]"

# options to indicate to send SSL KEY SIZE
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories

# set the request log format
JkRequestLogFormat "%w %V %T"

# send requests that contain the following to glassfish
JkMount /application/* app-server1-worker

Next, we need to define the worker file that is specified by JkWorkersFile directive. In this case, we create a file called glassfish-workers.properties in the directory /home/oracle/glassfish3/apache/conf. An example worker configuration is the following:

# define a worker that uses ajp13
worker.list=app-server1-worker

# set properties for the worker
worker.app-server1-worker.type=ajp13
worker.app-server1-worker.host=172.31.0.149 (which is the IP-address where the GlassFish server is running)
worker.app-server1-worker.port=28080 (which is HTTP listen port of the GlassFish Server)

To let the Apache HTTP Server pick up the configuration we add the following to the httpd.conf file:

# put it near the end of the file where all the other includes are present
# mod_jk configuration
Include conf/mod_jk.conf

Restart the HTTP Server (./apachectl -k stop and ./apachectl -k start). The application can now be reached at: http://hostname:7777/application/faces/overview.xhtml. Note that by using the JkMount directive, we tell mod_jk to forward requests to a configured back-end server in our case: 172.31.0.149:28080.

Class loading

Next to the bootstrap classloader we have the following classloaders in a GlassFish environment:

• Extension (domain-dir/lib/ext) - contains the so-called optional packages.
• Public API (glassfish/modules) - jars needed for the JavaEE runtime, that can be used by deployed applications.
• Common (domain-dir/lib) - usually contains jars that contain database drivers.
• Applib (domain-dir/lib/applibs) - application specific class loading, loads classes that are specified during a deployment (classes that are loaded by this class loader overwrite the same classes loaded by the preceding class loaders).
• Archive - loads the classes contained by a WAR or an EAR file.

To experiment a little, say we exclude Coherence from our WAR deployment file. In this case we need to specify the coherence.jar in one of the classloaders. A good choice here is to let it be loaded by the Applib class loader, i.e., not all applications deployed will use Coherence. The class loaders (Extension, Public API and Common) load classes for the general environment. For example, if we want to configure connection pools for an Oracle database a good place for the Oracle database drivers is the Common class loader. Thus to let Coherence be loaded per application and the Oracle database drivers be generally loaded, we put the jars in the following directories:

${GLASSFISH_HOME}
	/glassfish
		/domains/base_domain
			/lib
				/applibs
					coherence.jar
				ojdbc6.jar
		/nodes/localhost-base_domain
			/lib
				/applibs
					coherence.jar
				ojdbc6.jar

To deploy the WAR file excluding Coherence, we can use the following steps:

• In the admin console, click on applications and click deploy.
• Enter the following parameters:
  • Choose Local Packaged File... and browse to the war file
  • type: web application
  • context root: application
  • application name: application
  • status: enabled
  • libraries: coherence.jar (here jar files can be referenced that are present in the ${NODE_HOME}/app-server1/lib/applibs directory)
  • targets: app-server1
• Click OK.

Security and deployment plans

To create user and groups in GlassFish, we follow these steps:

• In the admin console, open the tree configurations, app-server1-config, security.
• Click on realm and click file.
• Click manage users and subsequently new.
• Enter the following parameters:
  • user id: manager
  • group list: managers
  • new password: welcome1
  • confirm new password: welcome1
• Click OK.
• Create another user (employee) that belongs to the group employees.

By using asadmin we can check our configuration, for example,

./asadmin list-auth-realms app-server1
admin-realm
file
certificate

./asadmin list-file-users --authrealmname file app-server1
employee
manager

./asadmin list-file-groups --authrealmname file app-server1
file-group : managers
file-group : employees

In order to map the roles defined in the application to users or groups defined in the security realm, we need to use a deployment plan. When we have an enterprise application that has the following structure:

application.ear
	/META-INF
		application.xml
	EJB.jar
		/META-INF
			ejb-jar.xml
			persistence.xml
	WEB.war
		/WEB-INF
			web.xml

we can use the following deployment plan structure (note that deployment plans are packaged as jars):

plan.jar
	glassfish-application.xml (provides the override for application.xml)
	EJB.jar.glassfish-ejb-jar.xml (provides the override for ejb-jar.xml)
	WEB.war.glassfish-web.xml (provides the override for web.xml)

When deploying just a WAR file (as in our case), we only need the glassfish-web.xml file, i.e.,

application-deployment-plan.jar
	glassfish-web.xml

To map the application roles to groups in the security realm we can use the following as the contents for the glassfish-web.xml deployment override:

<!DOCTYPE glassfish-web-app PUBLIC "-//GlassFish.org//DTD GlassFish Application Server 3.1 Servlet 3.0//EN" "http://glassfish.org/dtds/glassfish-web-app_3_0-1.dtd">
<glassfish-web-app>
	<security-role-mapping>
        <role-name>MANAGER</role-name>
		<!--principal-name>manager</principal-name-->
        <group-name>managers</group-name>
    </security-role-mapping>
    <security-role-mapping>
        <role-name>EMPLOYEE</role-name>
        <group-name>employees</group-name>
    </security-role-mapping>
</glassfish-web-app>

The jar file for the deployment plan can be created by using, for example, jar cvf application-deployment-plan.jar glassfish-web.xml.

To deploy an application that uses a deployment plan, we need to use the asadmin command-line utility. First, we undeploy the application:

./asadmin list-applications --long=true app-server1

./asadmin disable --target app-server1 application

./asadmin undeploy --target app-server1 application

The command-line to deploy the application and the deployment plan is as follows:

./asadmin deploy --contextroot application --name application
    --deploymentplan /home/oracle/glassfish3/deploy/application/application-deployment-plan.jar
    --enabled=true --libraries coherence.jar
    --target app-server1 /home/oracle/glassfish3/deploy/application/application.war

To check if the configuration works open a browser and enter the URL for the application (http://hostname:7777/application/faces/overview.xhtml) and login by using the created credentials.

Clustering

In the example below we will create a cluster that is present on one host. In the case, we are dealing with a cluster that spans multiple hosts, the administrative operations executed by the domain administration server need a means to communicate with a remote host. When the instance on the remote host is already running the domain administration server can communicate directly. On the other hand when the instance needs to be started, communication must be with SSH (secure shell). To set-up SSH we can follow these steps:

• Edit the file /etc/ssh/sshd_config and set the parameters StrictModes and PubkeyAuthentication to yes.
• To determine if the SSH daemon (sshd) is running we can use: /sbin/service sshd status.
• /sbin/service sshd start can be used to start the SSH daemon.
• To test the set-up we can use: ssh -l user-name host-name, for example, ssh -l oracle 172.31.0.149.

When the cluster spans multiple hosts, we can use asadmin install-node host-list to install the GlassFish software on the other hosts. By using asadmin list-nodes all the nodes in the domain are displayed (including the type and hostname). The host type can be either ssh or config. A config node does not allow remote communication an ssh typed node does. To test if a particular node is reachable we can use: asadmin ping-node-ssh node-name.

GlassFish clusters use multicast to retrieve server health information (heartbeat messages). To check if multicast is available in our environment we can use:

./asadmin validate-multicast
Will use port 2048
Will use address 228.9.3.1
Will use bind interface null
Will use wait period 2,000 (in milliseconds)
Listening for data...
Sending message with content "edu-glf-rh" every 2,000 milliseconds
Received data from edu-glf-rh (loopback)
Exiting after 20 seconds. To change this timeout, use the --timeout command line option.

To create a cluster we can follow these steps:

• In the admin console, click cluster and subsequently new.
• Enter the following parameters:
  • name: app-cluster
  • configuration: default-config (make a copy)
  • add two servers: app-cluster-server1 en app-cluster-server2 (load balance weight - 100, node - localhost-base_domain)
• Click OK.

Adjust the network listener:

• In the admin console open the tree configurations, app-cluster-config, network config, network listeners and click http-listener-1.
• Enable the JK Listener option.
• Click save.

To start the cluster we can use either the admin console (clusters, select app-cluster and click start cluster) or the asadmin command-line utility:

./asadmin start-cluster --verbose=true app-cluster
start-instance app-cluster-server2
start-instance app-cluster-server1
The command start-instance executed successfully for: app-cluster-server2 app-cluster-server1
Command start-cluster executed successfully.

To retrieve the listen ports:

• In the admin console click clusters, app-cluster and the instances tab.
• Select a server instance and click on the properties tab (HTTP_LISTENER_PORT contains the listen port).

To retrieve properties by means of asadmin, we can use:

./asadmin get servers.server.app-cluster-server1.system-property.HTTP_LISTENER_PORT
servers.server.app-cluster-server1.system-property.HTTP_LISTENER_PORT.name=HTTP_LISTENER_PORT
servers.server.app-cluster-server1.system-property.HTTP_LISTENER_PORT.value=28082

./asadmin get servers.server.app-cluster-server2.system-property.HTTP_LISTENER_PORT
servers.server.app-cluster-server2.system-property.HTTP_LISTENER_PORT.name=HTTP_LISTENER_PORT
servers.server.app-cluster-server2.system-property.HTTP_LISTENER_PORT.value=28081

By using the following commands we can retrieve information regarding health, multicast and node information:

./asadmin list-clusters
app-cluster running

./asadmin get-health app-cluster
app-cluster-server1 started since Thu Dec 01 09:51:23 CET 2011
app-cluster-server2 started since Thu Dec 01 09:51:23 CET 2011

./asadmin get clusters.cluster.app-cluster
clusters.cluster.app-cluster.config-ref=app-cluster-config
clusters.cluster.app-cluster.gms-bind-interface-address=${GMS-BIND-INTERFACE-ADDRESS-app-cluster}
clusters.cluster.app-cluster.gms-enabled=true
clusters.cluster.app-cluster.gms-multicast-address=228.9.132.110
clusters.cluster.app-cluster.gms-multicast-port=29361
clusters.cluster.app-cluster.name=app-cluster

./asadmin list-nodes --long=true
NODE NAME                TYPE      NODE HOST    INSTALL DIRECTORY          REFERENCED BY
localhost-base_domain    CONFIG    localhost    /home/oracle/glassfish3    app-server1, app-cluster-server2, app-cluster-server1

Scaling

We have two scaling options: vertical and horizontal scaling. Vertical scaling relates to adding more CPUs to a machine. To better utilize the server hardware we can add more GlassFish instances to the machine that could lead to increased application throughput. To determine if this is indeed the case we need to benchmark. Benchmarking for scalability is about measuring resource utilization. Good scalability means that service levels can be maintained while the workload is increased. If an application does not scale well, it is not fully utilizing the hardware. Consequently, throughput will degrade. Ideally, a linear load increase should lead to a linear degradation in service levels and performance. Linear scalabilty can be approached when so-called share nothing clusters are used. The nodes provide the same functionality and know nothing about other nodes in the cluster (no HTTP session replication). In this case, the computing ability of the cluster increases almost linearly as more nodes are added to the cluster, if the back-end information systems, such as a database, are powerful enough.

Applications that 'share nothing' are usually sharing state through the database. The application-tier can scale as far as when the database becomes a bottleneck. In general, relying on a single shared resource will eventually cause contention for that resource and thus limit the scalability. Caching is a good resolution. When we cache data at the application-tier we avoid calls to the database (and also avoid relational data to object data conversions). When using a cluster of server instances we need to maintain multiple caches; this is not a problem for read-only data but for read/write data it is. Caching solutions, such as Coherence, provide different kind of caching, i.e., replicated and partitioned. Replicated does not scale well when cache writes are involved as the data needs to be replicated across all the nodes in the grid. A partioned cache, on the other hand, scales very well when cache writes are involved as data ownership is spread throughout the cluster (the system automatically rebalances the data when the number of nodes in the grid changes - we do not need to decide on how to partition the data, it comes out of the box). Another plus is that access to the cache means at most one network trip, this in order to maintain linear scalability. An optimization on read-access can be made when data can be obtained locally (sticky access) in this case a hybrid solution such as the near cache can be applied.

Horizontal scaling relates to adding more machines to the environment, which gives a failover capability that we cannot get with vertical scaling. A good approach is to combine both scaling techniques to obtain better CPU utilization and failover capability.

To add an extra machine to our cluster we can use the following steps:

• Configure ssh
  • yum install openssh-server
  • yum install openssh-clients
  • chkconfig sshd on
  • service sshd start
  • Test is ssh is working from the other host

    [glassfish@glassfish1 bin]$ ssh glassfish@glassfish2.com
    The authenticity of host 'glassfish2.com (192.168.1.160)' can't be established.
    RSA key fingerprint is a6:cd:73:87:2f:f3:96:1e:16:34:23:a8:51:5e:69:97.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added 'glassfish2.com,192.168.1.160' (RSA) to the list of known hosts.
    glassfish@glassfish2.com's password:
    [glassfish@glassfish2 ~]$ exit
    logout
    Connection to glassfish2.com closed.
    [glassfish@glassfish1 bin]$ ssh -l glassfish glassfish2.com
    glassfish@glassfish2.com's password:
    Last login: Fri Mar 29 13:42:46 2013 from glassfish1.com
    [glassfish@glassfish2 ~]$
    

• Install the JVM and GlassFish on both machines.
• Create a domain on host GlassFish1.
• Start the domain (asadmin start-domain base_domain).
• Enable remote administration (asadmin enable-secure-admin).
• Restart the domain (asadmin stop-domain base_domain and asadmin start-domain base_domain).
• Check if the remote administration is working, for example by using

  [glassfish@glassfish2 bin]$ ./asadmin --host glassfish1.com --port 4848 list-instances
  [
  [
    Version: V3
    Subject: CN=glassfish1.com, OU=GlassFish, O=Oracle Corporation, L=Santa Clara, ST=California, C=US
    Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11
  
    Key:  Sun RSA public key, 2048 bits
    modulus: 23240487021714962896424948633607969649221492541901259278251354022217316690568190613146558081413942287422551011017205757573869498943009098808161810308175849714018499282054244250620744466891501116668156575322374884073512287758514196369461357936242418926377510601714431282318816590915749319224521312843683532542085392111709672784400132041769359346688114343332995853786148444245754014425308575464844915206464756572993994428879805996793446709630188793718353028158966848655646660605205712353399783441346022509600162844812962400421138671723296014926498127250905535360169590799349586873306206634176842832235319451743899186797
    public exponent: 65537
    Validity: [From: Fri Mar 29 13:07:29 CET 2013,
                 To: Mon Mar 27 14:07:29 CEST 2023]
    Issuer: CN=glassfish1.com, OU=GlassFish, O=Oracle Corporation, L=Santa Clara, ST=California, C=US
    SerialNumber: [    41f25b98]
  
  Certificate Extensions: 1
  [1]: ObjectId: 2.5.29.14 Criticality=false
  SubjectKeyIdentifier [
  KeyIdentifier [
  0000: C0 69 98 7C 3A FB 96 61   D2 42 35 74 12 8C 6F A0  .i..:..a.B5t..o.
  0010: DB D1 0B 09                                        ....
  ]
  ]
  
  ]
    Algorithm: [SHA256withRSA]
    Signature:
  0000: AC 75 86 D6 43 24 0B AB   52 0F C5 BC 4A 38 6E 76  .u..C$..R...J8nv
  0010: A8 A5 E3 20 34 AE 13 3A   45 AA 27 AB 93 30 89 1A  ... 4..:E.'..0..
  0020: A8 96 33 B1 BB 6A E7 16   44 20 A0 2D 82 0A 0E 42  ..3..j..D .-...B
  0030: CF 95 DB 86 66 75 17 7F   F0 0B D5 F7 50 19 42 C1  ....fu......P.B.
  0040: F3 03 BD 76 18 B2 42 BD   CA B6 CF 32 83 3C CF C3  ...v..B....2.<..
  0050: EE BD 5D B9 A3 9E 4E E3   B9 4D E7 C4 6F 0F C0 AA  ..]...N..M..o...
  0060: 0F 84 54 0F 5C 66 9A CC   96 69 77 34 31 4A 27 6B  ..T.\f...iw41J'k
  0070: 6D 8B 7B A7 19 95 62 B0   56 7A 0F 7E 8D EA ED E4  m.....b.Vz......
  0080: D8 3D 76 CD BB 59 D8 5C   EC 5E 70 9D 67 D9 48 94  .=v..Y.\.^p.g.H.
  0090: 39 66 74 4A A5 62 A5 33   19 38 96 22 46 8A CC FD  9ftJ.b.3.8."F...
  00A0: C3 D6 79 46 29 30 6D 0B   3B E9 A9 9E 39 26 7E F1  ..yF)0m.;...9&..
  00B0: A3 11 8D 86 4B 74 10 FD   EA 3C 22 39 F7 16 93 5F  ....Kt...<"9..._
  00C0: 8C 37 62 E7 DB 02 8E 02   ED 49 48 DC 53 92 D3 59  .7b......IH.S..Y
  00D0: 22 21 EF AE DE 12 8C BD   31 0A C1 FB 5F BD 32 4A  "!......1..._.2J
  00E0: A9 E3 7C 32 9C FB B8 E4   60 F9 DF D7 7E 19 AC D3  ...2....`.......
  00F0: 3D D3 31 BC 06 7A C9 01   B1 AC 6A 38 F8 E8 5E EA  =.1..z....j8..^.
  
  ]
  Do you trust the above certificate [y|N] -->y
  Enter admin user name>  glassfish
  Enter admin password for user "glassfish">
  Nothing to list.
  Command list-instances executed successfully.
  

• Create a cluster, and create two instances, one on the local node and one on the remote host
  • In the admin console, click nodes, new. Enter a name and select SSH as the type. Next enter the hostname in the node host field, and set the SSH settings, i.e., select password as the SSH user authentication and set the SSH user name (the user under which the GlassFish Server is installed) and SSH password belonging to the user.
  • Click cluster, new. Enter a name and select a configuration.
  • Next click the created cluster, click the instances tab and click new. Enter a name and select a host. To create an instance we can also use asadmin --host glassfish1.com --port 4848 create-local-instance --cluster test-cluster test-server-glassfish2.

    [glassfish@glassfish2 bin]$ ./asadmin --host glassfish1.com --port 4848 create-local-instance --cluster test-cluster test-server-glassfish2
    Enter admin user name>  glassfish
    Enter admin password for user "glassfish">
    Rendezvoused with DAS on glassfish1.com:4848.
    Port Assignments for server instance test-server-glassfish2:
    JMX_SYSTEM_CONNECTOR_PORT=28686
    JMS_PROVIDER_PORT=27676
    HTTP_LISTENER_PORT=28080
    ASADMIN_LISTENER_PORT=24848
    JAVA_DEBUGGER_PORT=29009
    IIOP_SSL_LISTENER_PORT=23820
    IIOP_LISTENER_PORT=23700
    OSGI_SHELL_TELNET_PORT=26666
    HTTP_SSL_LISTENER_PORT=28181
    IIOP_SSL_MUTUALAUTH_PORT=23920
    Command create-local-instance executed successfully.
    

Start the servers to test if it is working

Configure load balancer

The load balancer decides how to dispatch requests to the back end server instances. A load balancer needs to do tasks such as distributing requests, health checking and session binding. As these are simple jobs it is rare that a load balancer will become a bottleneck. To load balance requests, we are going to use mod_jk. To configure mod_jk, open the mod_jk.conf file and add the following contents:

LoadModule jk_module /home/oracle/glassfish3/apache/modules/mod_jk.so

JkWorkersFile /home/oracle/glassfish3/apache/conf/glassfish-workers.properties

# where to put the log files for the jk module
JkLogFile /home/oracle/glassfish3/apache/logs/mod_jk.log

# the log level [debug|info|error]
JkLogLevel info

# log format
JkLogStampFormat "[%a %b %d %H:%M:%S %Y]"

# options to indicate to send SSL KEY SIZE
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories

# set the request log format
JkRequestLogFormat "%w %V %T"

# send requests that contain the following to glassfish
JkMount /application/* app-server1-worker

JkMount /LoadTest/* loadbalancer

The workers (glassfish-workers.properties) are defined as follows:

# define a worker that uses ajp13
worker.list=app-server1-worker,app-cluster-server1-worker,app-cluster-server2-worker,loadbalancer

# set properties for the worker
worker.app-server1-worker.type=ajp13
worker.app-server1-worker.host=172.31.0.149
worker.app-server1-worker.port=28080

# set properties for the app-cluster-server1-worker
worker.app-cluster-server1-worker.type=ajp13
worker.app-cluster-server1-worker.host=172.31.0.149
worker.app-cluster-server1-worker.port=28082
worker.app-cluster-server1-worker.lbfactor=1
worker.app-cluster-server1-worker.socket_keepalive=1
worker.app-cluster-server1-worker.socket_timeout=300

# set properties for the app-cluster-server2-worker
worker.app-cluster-server2-worker.type=ajp13
worker.app-cluster-server2-worker.host=172.31.0.149
worker.app-cluster-server2-worker.port=28081
worker.app-cluster-server2-worker.lbfactor=1
worker.app-cluster-server2-worker.socket_keepalive=1
worker.app-cluster-server2-worker.socket_timeout=300

# set properties for the loadbalancer
worker.loadbalancer.type=lb
worker.loadbalancer.balance_workers=app-cluster-server1-worker,app-cluster-server2-worker

We can also choose to use the GlassFish Load Balancer. We can use the same steps to configure Apache as presented above (./configure --with-mpm=worker --with-included-apr --prefix=apache_install_path which also enables multithreading support). Once Apache is installed we can use the load balancer configurer (that can be downloaded here). Run java -jar glassfish-lbconfigurator-3_1_1.jar to start the configuration and follow the instructions on the screen. When the configuration is finished we need to edit the loadbalancer.xml.example (located in the ${APACHE_HOME}/conf directory and rename it to loadbalancer.xml). An example configuration looks as follows

<!DOCTYPE loadbalancer PUBLIC "-//GlassFish.org//DTD GlassFish Application Server 3.1//EN" "glassfish-loadbalancer_1_3.dtd">
<loadbalancer>
    <cluster name="test-cluster"  policy="round-robin">
        <instance  name="test-server-glassfish1" enabled="true" disable-timeout-in-minutes="60" listeners="http://192.168.1.155:28080"  weight="100"/>
        <instance  name="test-server-glassfish2" enabled="true" disable-timeout-in-minutes="60" listeners="http://192.168.1.160:28080"  weight="100"/>
        <web-module context-root="SessionTest" enabled="true" disable-timeout-in-minutes="60" error-url="sun-http-lberror.html" />
        <health-checker url="/" interval-in-seconds="10" timeout-in-seconds="30" />
    </cluster>
    <property name="reload-poll-interval-in-seconds" value="60"/>
    <property name="response-timeout-in-seconds" value="30"/>
    <property name="https-routing" value="false"/>
    <property name="require-monitor-data" value="false"/>
    <property name="active-healthcheck-enabled" value="false"/>
    <property name="number-healthcheck-retries" value="3"/>
    <property name="rewrite-location" value="true"/>
</loadbalancer>

Note that when using the GlassFish Load Balancer we do not need to enable the JK Listener on the Network Listener.

Configure resources

The application we will deploy to the cluster uses Java Persistence and Java Messaging, i.e., we have the following persistence.xml file

<persistence xmlns="http://java.sun.com/xml/ns/persistence" version="1.0">
    <persistence-unit name="PersonPersistenceUnit">
        <provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>
        <jta-data-source>jdbc/exampleDS</jta-data-source>
        <class>model.entities.Person</class>
    </persistence-unit>
</persistence>

Note that EclipseLink is the default persistent provider on GlassFish, so no further jar files need to be added to the classpath. We have the following stateless enterprise bean:

package model.logic;

import model.entities.Person;

import javax.annotation.PostConstruct;
import javax.annotation.PreDestroy;
import javax.annotation.Resource;
import javax.ejb.Stateless;
import javax.jms.*;
import javax.persistence.EntityManager;
import javax.persistence.PersistenceContext;

@Stateless(name = "Company", mappedName = "ejb/Company")
public class CompanyBean implements Company {

    @PersistenceContext(unitName = "PersonPersistenceUnit")
    private EntityManager entityManager;

    @Resource(name = "jms/ConnectionFactory", mappedName = "jms/ConnectionFactory", type = ConnectionFactory.class)
    private ConnectionFactory connectionFactory;

    @Resource(name = "jms/CompanyQueue", mappedName = "jms/CompanyQueue", type = Queue.class)
    private Queue destination;

    private Connection connection = null;
    private Session session = null;
    private MessageProducer messageProducer = null;

    @PostConstruct
    public void init() {
        try {
            connection = connectionFactory.createConnection();
            session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
            messageProducer = session.createProducer(destination);
        } catch (JMSException e) {
            if (connection != null) {
                try {
                    connection.close();
                } catch (JMSException f) {
                    e.printStackTrace();
                }
            }
        }
    }

    @PreDestroy
    public void release() {
        try {
            if (connection != null) {
                connection.close();
            }
        } catch (JMSException e) {
            e.printStackTrace();
        }
    }

    public void insertPerson(Person person) {
        if (findPerson(person.getSofinummer()) == null) {
            entityManager.persist(person);
            sendMessage(person);
        } else {
            updatePerson(person);
        }
    }

    public void removePerson(Integer sofinummer) {
        Person person = findPerson(sofinummer);
        if (person != null) {
            entityManager.remove(entityManager.merge(person));
            sendMessage(person);
        }
    }

    public void updatePerson(Person person) {
        entityManager.merge(person);
    }

    private Person findPerson(Integer sofinummer) {
        return entityManager.find(Person.class, sofinummer);
    }

    private void sendMessage(Person person) {
        try {
            ObjectMessage message = session.createObjectMessage();
            message.setObject(person);
            messageProducer.send(message);
        } catch (JMSException e) {
            e.printStackTrace();
        }
    }
}

and the following message-driven bean:

package model.logic;

import model.entities.Person;

import javax.ejb.ActivationConfigProperty;
import javax.ejb.MessageDriven;
import javax.jms.*;

@MessageDriven(mappedName = "jms/CompanyQueue", activationConfig = {
        @ActivationConfigProperty(propertyName = "acknowledgeMode", propertyValue = "Auto-acknowledge"),
        @ActivationConfigProperty(propertyName = "destination", propertyValue = "CompanyQueue"),
        @ActivationConfigProperty(propertyName = "destinationType", propertyValue = "javax.jms.Queue")
})
public class CompanyMDB implements MessageListener {

    public void onMessage(Message message) {
        if (message instanceof ObjectMessage) {
            ObjectMessage objectMessage = (ObjectMessage) message;
            try {
                System.out.println("RECEIVED OBJECT MESSAGE " + objectMessage.getObject());
            } catch (JMSException e) {
                e.printStackTrace();
            }
        }

        if (message instanceof TextMessage) {
            TextMessage textMessage = (TextMessage) message;
            try {
                System.out.println("RECEIVED TEXT MESSAGE " + textMessage.getText());
            } catch (JMSException e) {
                e.printStackTrace();
            }
        }
    }
}

Three resources need thus to be configured:

• JDBC data source with JNDI-name jdbc/exampleDS.
• JMS connection factory with JNDI-name jms/ConnectionFactory.
• JMS queue with JNDI-name jms/CompanyQueue.

To create a data source we can follow these steps:

• In de admin console, open the tree resources, JDBC and click JDBC connection pools.
• Click new and enter the following parameters:
  • pool name: LoadTestConnectionPool
  • resource type: javax.sql.DataSource
  • database vendor: Oracle
• Click next and enter the following parameters:
  • url: jdbc:oracle:thin:@hostname:1521:SID
  • password: schema password
  • user: schema user
• Click finish.
• Click in the connection pool table on LoadTestConnectionPool and click ping.
• Open the tree resources, JDBC and click JDBC resources.
• Click new and enter the following parameters:
  • JNDI name: jdbc/exampleDS
  • pool name: LoadTestConnectionPool
  • status: enabled
  • targets: app-cluster

To create the JMS resources we can follow these steps:

• Open the tree resources, JMS resources and click on connection factories.
• Click new and enter the following parameters:
  • pool name: jms/ConnectionFactory
  • resource type: javax.jms.QueueConnectionFactory
  • targets: app-cluster
• Click OK.
• Click destination resources and new.
• Enter the following parameters:
  • JNDI name: jms/CompanyQueue
  • physical destination name: CompanyQueue
  • resource type: javax.jms.Queue
  • targets: app-cluster
• Click OK.

Now, that the resources are in place we can deploy the application. Create a new directory ${GLASSFISH_HOME}/deploy/loadtest and copy the EAR file to this directory. To deploy the application we can follow these steps:

• In the admin console click on the applications link and click deploy.
• Enter the following parameters:
  • Choose Local Packaged File... and browse to the EAR file
  • type: enterprise application
  • status: enabled
  • java web start: disabled
  • targets: app-cluster
• Click OK.

To test the deployment enter the following URL: http://hostname:7777/LoadTest/testservlet. Check in the database if a person has been added. In the server log file (${NODE_HOME}/app-server1/logs) we can check if the JMS system has processed the message: 2011-11-30T14:13:55.601+0100 | INFO | oracle-glassfish3.1.1 | javax.enterprise.system.std.com.sun.enterprise.server.logging | _ThreadID=19; _ThreadName=Thread-2; | RECEIVED OBJECT MESSAGE ifdfcjecgh9d 1911.

Monitoring

To get an idea how the application is performing, we can enable some monitoring:

• In the admin console, click configuration, app-cluster-config, monitoring.
• Enable monitoring for the web container, thread pool, JMS service, JDBC connection pool and EJB container components, for example, by setting the monitoring level to high.
• Click clusters, app-cluster and select one of the server instances.

Request statistics can be found in the applications and server environments. Statistics regarding the JDBC connection pool can be found in the resources environment. Statistics for the JMS resources can be found the JMS physical destinations environment of the cluster (statistics column in the destinations table).

Start and stop scripts

An example of a script that starts the environment is the following:

#!/bin/bash

GLASSFISH_HOME="/home/oracle/glassfish3"
export GLASSFISH_HOME

APACHE_HOME="${GLASSFISH_HOME}/apache/"
export APACHE_HOME

start_apache() {
	echo "Starting Apache HTTP Server"
	${APACHE_HOME}/bin/apachectl -k start
}

start_glassfish_das() {
	if [ -z "$1" ]
	then
		echo "no parameters, starting glassfish DAS for domain: base_domain"
		${GLASSFISH_HOME}/bin/asadmin start-domain base_domain
	else
		echo "starting glassfish DAS for domain: $1"
		${GLASSFISH_HOME}/bin/asadmin start-domain $1
	fi
}

start_glassfish_instance() {
	if [ -z "$1" ]
	then
		echo "no parameters, starting glassfish instance: app-server1"
		${GLASSFISH_HOME}/bin/asadmin start-instance app-server1
	else
		echo "starting glassfish instance: $1"
		${GLASSFISH_HOME}/bin/asadmin start-instance $1
	fi
}

start_glassfish_cluster() {
	if [ -z "$1" ]
	then
		echo "no parameters, starting glassfish cluster: app-cluster"
		${GLASSFISH_HOME}/bin/asadmin start-cluster --verbose=true app-cluster
	else
		echo "starting glassfish cluster: $1"
		${GLASSFISH_HOME}/bin/asadmin start-cluster --verbose=true $1
	fi
}

start_glassfish_das base_domain

start_glassfish_instance app-server1

start_glassfish_cluster app-cluster

start_apache

An example to stop the environment is the following:

#!/bin/bash

GLASSFISH_HOME="/home/oracle/glassfish3"
export GLASSFISH_HOME

APACHE_HOME="${GLASSFISH_HOME}/apache/"
export APACHE_HOME

stop_apache() {
	echo "Stopping Apache HTTP Server"
	${APACHE_HOME}/bin/apachectl -k stop
}

stop_glassfish_das() {
	if [ -z "$1" ]
	then
		echo "no parameters, stopping glassfish DAS for domain: base_domain"
		${GLASSFISH_HOME}/bin/asadmin stop-domain base_domain
	else
		echo "stopping glassfish das DAS domain: $1"
		${GLASSFISH_HOME}/bin/asadmin stop-domain $1
	fi
}

stop_glassfish_instance() {
	if [ -z "$1" ]
	then
		echo "no parameters, stopping glassfish instance: app-server1"
		${GLASSFISH_HOME}/bin/asadmin stop-instance app-server1
	else
		echo "stopping glassfish instance: $1"
		${GLASSFISH_HOME}/bin/asadmin stop-instance $1
	fi
}

stop_glassfish_cluster() {
	if [ -z "$1" ]
	then
		echo "no parameters, stopping glassfish cluster: app-cluster"
		${GLASSFISH_HOME}/bin/asadmin stop-cluster --verbose=true app-cluster
	else
		echo "stopping glassfish cluster: $1"
		${GLASSFISH_HOME}/bin/asadmin stop-cluster --verbose=true $1
	fi
}

stop_apache

stop_glassfish_cluster app-cluster

stop_glassfish_instance app-server1

stop_glassfish_das base_domain

Load test

We can test the environment by using the The Grinder. The following script calls the application:

from net.grinder.script.Grinder import grinder
from net.grinder.script import Test
from net.grinder.plugin.http import HTTPRequest

test1 = Test(1, "Request resource")
request1 = test1.wrap(HTTPRequest())

class TestRunner:
    def __call__(self):
        result = request1.GET("http://172.31.0.149:7777/LoadTest/testservlet")

Edit the grinder.properties (${GRINDER_HOME}/examples) file such that the script is used, for example,

# The file name of the script to run. Relative paths are evaluated from the directory containing the properties file.
grinder.script = test.py

# The number of worker processes each agent should start.
grinder.processes = 1

# The number of worker threads each worker process should start.
grinder.threads = 1

# The number of runs each worker process will perform. When using the console set this to 0, i.e., the runs are controlled by the console.
grinder.runs = 0

Use the following scripts to start a test:

setGrinderEnv.sh

#!/bin/sh
GRINDERPATH=/home/oracle/grinder-3.4
export GRINDERPATH

JAVA_HOME=/home/oracle/jdk1.6.0_24
export JAVA_HOME

USER_MEM_ARGS="-server -Xms512m -Xmx512m -XX:NewRatio=2 -XX:+UseParallelGC -XX:ParallelGCThreads=2 -XX:MaxGCPauseMillis=200 -XX:GCTimeRatio=19 -XX:+UseParallelOldGC"
export USER_MEM_ARGS
GRINDERPROPERTIES=${GRINDERPATH}/examples/grinder.properties
export GRINDERPROPERTIES
CLASSPATH=${GRINDERPATH}/lib/grinder.jar:${CLASSPATH}
export CLASSPATH

PATH=${JAVA_HOME}/bin:${PATH}
export PATH

startConsole.sh

#!/bin/sh
source setGrinderEnv.sh
java ${USER_MEM_ARGS} -cp ${CLASSPATH} net.grinder.Console

startAgent.sh

#!/bin/sh
source setGrinderEnv.sh
java ${USER_MEM_ARGS} -cp ${CLASSPATH} net.grinder.Grinder ${GRINDERPROPERTIES}

First start the startConsole.sh script and then start the startAgent.sh script. The agent will now wait for a start action from the console (click the action tab and then choose start processes).

Use the GlassFish admin console to see how the requests are handled. By using jvisualvm or jconsole the JVM can be monitored.

References

[1] GlassFish Server Documentation

WebLogic provides a set of MBeans that can be used to configure, monitor, and manage WebLogic Server resources. The MBeans can be organized into the following types. Runtime MBeans contain information about the runtime state of a server and its resources. The life cycle of a runtime MBean follows that of the resource for which it exposes runtime data, i.e., when we start a server instance, the server instantiates a ServerRuntimeMBean and populates it with the current runtime data. Each resource updates the data in its runtime MBean as its state changes. The resource destroys its runtime MBean when it is stopped. Configuration MBeans contain information about the configuration of servers and resources, i.e., the information that is stored in the configuration files (such as config.xml). Configuration MBeans are discussed in the post Building a Domain using the WebLogic Scripting Tool. In this post, we will look at using runtime MBeans to build monitoring scripts.

Our starting point is the DomainRuntimeServiceMBean that provides an access point for navigating to all runtime and configuration MBeans in the domain. As we are interested in the server runtime information, we can use the following:

# we first connect to the admin server
connect('weblogic','magic11g', 't3://172.31.0.106:7001');
servers = domainRuntimeService.getServerRuntimes();

To know which MBeans to use the WebLogic Server MBean Reference is mandatory. Here, we see the MBeanServer Services tree, which contains the DomainRuntimeServiceMBean. A related MBean (child MBeans) is, for example, ServerRuntimes that provides methods for retrieving runtime information about a server instance and for transitioning a server from one state to another. Thus by using domainRuntimeService.getServerRuntimes() we obtain an array containing all the runtime MBeans for the servers in the domain.

To obtain the application runtime information, we can use the ApplicationRuntimeMBean. This MBean encapsulates runtime information about a deployed Enterprise application. We obtain a ApplicationRuntimeMBean instance by using:

for server in servers:
	apps = server.getApplicationRuntimes();

An ApplicationRuntimeMBean has the following related MBeans:

• ClassRedefinitionRuntime – If class fast-swap feature is enabled for the application, return the runtime MBean to monitor and control the class fast-swap within the application.
• ComponentRuntimes – Returns the list of component runtime instances for each J2EE component (such as an EJB or a Web application) that is contained in this Enterprise application.
• KodoPersistenceUnitRuntimes – Provides an array of KodoPersistenceUnitRuntimeMBean objects for this EJB module.
• LibraryRuntimes – Returns the list of library runtime instances for each J2EE library that is contained in this Enterprise application.
• MaxThreadsConstraintRuntimes – Get the runtime MBeans for all MaxThreadsConstraints defined at the app-level.
• MinThreadsConstraintRuntimes – Get the runtime MBeans for all MinThreadsConstraints defined at the app-level.
• OptionalPackageRuntimes – Returns the list of optional package runtime instances for each J2EE optional package that is contained in this Enterprise application.
• QueryCacheRuntimes – Returns a list of QueryCacheRuntimeMBeans configured for this application.
• RequestClassRuntimes – Get the runtime MBeans for all request classes defined at the app-level.
• WorkManagerRuntimes – Returns the list of work manager runtime instances for each application-scoped work manager that is associated with this Enterprise application.
• WseeRuntimes – Returns the list of Web Service runtime instances that are contained in this Enterprise application.

As we are interested in runtime information of application components, we need to obtain an instance of the ComponentRuntimeMBean, which is a base class for all runtime MBeans that provide status of running modules. The following MBeans extend or implement this MBean type:

• AppClientComponentRuntimeMBean – This is the ComponentRuntimeMBean for J2EE Application Client Containers.
• ConnectorComponentRuntimeMBean – Generates notifications about the deployment state of resource adapters. Each resource adapter is represented by an instance of ConnectorComponentMBean.
• EJBComponentRuntimeMBean- This is the top level interface for all runtime information collected for an EJB module.
• InterceptionComponentRuntimeMBean – This is the top level interface for all runtime information collected for an Interception module.
• JDBCConnectionPoolRuntimeMBean – This class is used for monitoring a WebLogic JDBC component.
• JDBCDataSourceRuntimeMBean – This class is used for monitoring a WebLogic JDBC Data Source and its associated connection pool.
• JMSComponentRuntimeMBean – This is the top level interface for all runtime information collected for an JMS module.
• WebAppComponentRuntimeMBean – Describes a servlet component (servlet context).

To obtain ComponentRuntimeMBean instances we can use:

for app in apps:
	crs = app.getComponentRuntimes();

The ComponentRuntimeMBean exposes a Type attribute, that can be used to check which type we are dealing with, for example,

for cr in crs:
	if (cr.getType() == 'EJBComponentRuntime'):
		# do EJB runtime related stuff
	if (cr.getType() == 'WebAppComponentRuntime'):
		# do servlet runtime related stuff

Say, we want to retrieve information that concerns our message-driven beans and stateless beans. In particular, we want to check the status and the health state of our message-driven beans and get some transaction related information for our stateless beans. The EJBComponentRuntimeMBean as a related MBean called EJBRuntimeMBean that provides runtime information for a particular EJB. EJBRuntimeMBean instances can be cast to their appropriate subclass (EntityEJBRuntimeMBean, StatelessEJBRuntimeMBean, StatefulEJBRuntimeMBean or MessageDrivenEJBRuntimeMBean) to access additional runtime information for the particular EJB, for example,

ejbRTs = cr.getEJBRuntimes();
	for ejbRT in ejbRTs:
		print ' -EJBRunTime: ' + ejbRT.getName() + ' Type ' + ejbRT.getType();
		if (ejbRT.getType() == 'MessageDrivenEJBRuntime'):
			print '  -MDB Status: ' + ejbRT.getMDBStatus() + ', MDB Health State: ' + repr(ejbRT.getHealthState());
		if (ejbRT.getType() == 'StatelessEJBRuntime'):
			print '  -EJB Name: ' + ejbRT.getEJBName();
			print '  -Resources: ' + repr(ejbRT.getTransactionRuntime());

For the Web component, we can proceed in the same manner, for example, to retrieve information about the number of open sessions, the number of times a certain servlet has been invoked and the average invocation time, we can use something like:

print ' -Name: ' + cr.getName() + ', Session Current Count: ' + repr(cr.getOpenSessionsCurrentCount());
servlets = cr.getServlets();
for servlet in servlets:
	print '  -Servlet: ' + servlet.getServletName() + ', total: ' + repr(servlet.getInvocationTotalCount()) + ', average time: ' + repr(servlet.getExecutionTimeAverage());

To get runtime information about the JMS environment, we have to obtain a JMSRuntimeMBean instance. This can be obtained from a ServerRuntimeMBean, for example,

jmsRuntime = server.getJMSRuntime();

The JMSRuntimeMBean has two related MBeans, i.e.,

• JMSConnectionRuntimeMBean – This class is used for monitoring a WebLogic JMS connection and provides access to the JMSSessionRuntimeMBean, which contains runtime information about the consumers and producers.
• JMSServerRuntimeMBean – This class is used for monitoring a WebLogic JMS server and provides access to the JMSDestinationRuntimeMBean, which is used for monitoring a destination (topic or queue).

To see the number of bytes consumed by each consumer and the bytes send by each producer we can use:

connections = jmsRuntime.getConnections();
for connection in connections:
	print('-Connection Name: ' + connection.getName());
	sessions = connection.getSessions();
	for session in sessions:
		print(' -Session Name: ' + session.getName());
		consumers = session.getConsumers();
		for consumer in consumers:
			print('  -Consumer Name: ' + consumer.getName() + ', Bytes Received: ' + repr(consumer.getBytesReceivedCount()));
		producers = session.getProducers();
		for producer in producers:
			print('  -Producer Name: ' + producer.getName() + ', Bytes Send: ' + repr(producer.getBytesSentCount()));

The obtain some runtime statistics for the destinations we can use:

jmsServers = jmsRuntime.getJMSServers();
for jmsServer in jmsServers:
	print('-JMSServer: ' + jmsServer.getName());
	destinations = jmsServer.getDestinations();
	for destination in destinations:
		print(' -Destination: ' + destination.getName() + ', ' + repr(destination.getMessagesHighCount()) + ', ' + repr(destination.getConsumersHighCount()));

Runtime information concerning JDBC can be obtained from the JDBCServiceRuntimeMBean. In particular, we are interrested in some connection pools statistics, such as, if some requests are waiting for a connection. This runtime information can be obtained from the JDBCDataSourceRuntimeMBean, for example,

jdbcRuntime = server.getJDBCServiceRuntime();
datasources = jdbcRuntime.getJDBCDataSourceRuntimeMBeans();
for datasource in datasources:
	print('-Data Source: ' + datasource.getName() + ', Active Connections: ' + repr(datasource.getActiveConnectionsCurrentCount()) + ', Waiting for Connections: ' + repr(datasource.getWaitingForConnectionCurrentCount()));

This rounds-up our exploration of the WebLogic runtime MBean environment. Of course, there are lots more MBeans available that provide different kinds of information. The example above gives a starting point for the reader to explore further.

The complete script looks as follows:

connect('weblogic','magic11g', 't3://172.31.0.106:7001');

servers = domainRuntimeService.getServerRuntimes();
for server in servers:
	print 'SERVER: ' + server.getName();

	print('APPLICATION RUNTIME INFORMATION');
	apps = server.getApplicationRuntimes();
	for app in apps:
		print 'Application: ' + app.getName();
		crs = app.getComponentRuntimes();
		for cr in crs:
			print '-Component Type: ' + cr.getType();
			if (cr.getType() == 'EJBComponentRuntime'):
				ejbRTs = cr.getEJBRuntimes();
				for ejbRT in ejbRTs:
					print ' -EJBRunTime: ' + ejbRT.getName() + ' Type ' + ejbRT.getType();
					if (ejbRT.getType() == 'MessageDrivenEJBRuntime'):
						print '  -MDB Status: ' + ejbRT.getMDBStatus() + ', MDB Health State: ' + repr(ejbRT.getHealthState());
					if (ejbRT.getType() == 'StatelessEJBRuntime'):
						print '  -EJB Name: ' + ejbRT.getEJBName();
						print '  -Resources: ' + repr(ejbRT.getTransactionRuntime());
			if (cr.getType() == 'WebAppComponentRuntime'):
				print ' -Name: ' + cr.getName() + ', Session Current Count: ' + repr(cr.getOpenSessionsCurrentCount());
				servlets = cr.getServlets();
				for servlet in servlets:
					print '  -Servlet: ' + servlet.getServletName() + ', total: ' + repr(servlet.getInvocationTotalCount()) + ', average time: ' + repr(servlet.getExecutionTimeAverage());

	print('JMS RUNTIME INFORMATION');
	jmsRuntime = server.getJMSRuntime();
	connections = jmsRuntime.getConnections();
	for connection in connections:
		print('-Connection Name: ' + connection.getName());
		sessions = connection.getSessions();
		for session in sessions:
			print(' -Session Name: ' + session.getName());
			consumers = session.getConsumers();
			for consumer in consumers:
				print('  -Consumer Name: ' + consumer.getName() + ', Bytes Received: ' + repr(consumer.getBytesReceivedCount()));
			producers = session.getProducers();
			for producer in producers:
				print('  -Producer Name: ' + producer.getName() + ', Bytes Send: ' + repr(producer.getBytesSentCount()));
	jmsServers = jmsRuntime.getJMSServers();
	for jmsServer in jmsServers:
		print('-JMSServer: ' + jmsServer.getName());
		destinations = jmsServer.getDestinations();
		for destination in destinations:
			print(' -Destination: ' + destination.getName() + ', ' + repr(destination.getMessagesHighCount()) + ', ' + repr(destination.getConsumersHighCount()));

	print('JDBC RUNTIME INFORMATION');
	jdbcRuntime = server.getJDBCServiceRuntime();
	datasources = jdbcRuntime.getJDBCDataSourceRuntimeMBeans();
	for datasource in datasources:
		print('-Data Source: ' + datasource.getName() + ', Active Connections: ' + repr(datasource.getActiveConnectionsCurrentCount()) + ', Waiting for Connections: ' + repr(datasource.getWaitingForConnectionCurrentCount()));

The following shows the output when the script is run in an environment presented in the post Setting-up a High Available Tuned Java EE environment using WebLogic:

wls:/offline> execfile('c:\\temp\\probeer.py')
Connecting to t3://172.31.0.106:7001 with userid weblogic ...
Successfully connected to Admin Server 'AdminServer' that belongs to domain 'base_domain'.

Warning: An insecure protocol was used to connect to the
server. To ensure on-the-wire security, the SSL port or
Admin port should be used instead.

SERVER: AdminServer
APPLICATION RUNTIME INFORMATION
Application: bea_wls_diagnostics
-Component Type: WebAppComponentRuntime
 -Name: AdminServer_/bea_wls_diagnostics, Session Current Count: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: Accessor, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
Application: bea_wls9_async_response
-Component Type: WebAppComponentRuntime
 -Name: AdminServer_/_async, Session Current Count: 0
  -Servlet: AsyncResponseServiceSoap12Servletjms, total: 0, average time: 0
  -Servlet: AsyncResponseServiceServlethttps, total: 0, average time: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: AsyncResponseServiceSoap12Servlethttps, total: 0, average time: 0
  -Servlet: AsyncResponseServiceSoap12Servlethttp, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: AsyncResponseServiceServlethttp, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: AsyncResponseServiceServletjms, total: 0, average time: 0
Application: uddiexplorer
-Component Type: WebAppComponentRuntime
 -Name: AdminServer_/uddiexplorer, Session Current Count: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
Application: uddi
-Component Type: WebAppComponentRuntime
 -Name: AdminServer_/uddi, Session Current Count: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: uddilistener, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
Application: wls-wsat
-Component Type: WebAppComponentRuntime
 -Name: AdminServer_/wls-wsat, Session Current Count: 0
  -Servlet: ParticipantPortTypeServlethttp, total: 0, average time: 0
  -Servlet: RegistrationRequesterPortTypeServlethttp, total: 0, average time: 0
  -Servlet: ParticipantPortTypeServlethttp11, total: 0, average time: 0
  -Servlet: RegistrationRequesterPortType11Servlethttp, total: 0, average time: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: RegistrationPortTypeRPCServlethttp11, total: 0, average time: 0
  -Servlet: RegistrationPortTypeRPCServlethttp, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: CoordinatorPortTypeServlethttp, total: 0, average time: 0
  -Servlet: CoordinatorPortTypeServlethttp11, total: 0, average time: 0
Application: consoleapp
-Component Type: WebAppComponentRuntime
 -Name: AdminServer_/console, Session Current Count: 0
  -Servlet: /jsp/bridge/MessagingBridgeMonitoring.jsp, total: 4, average time: 10
  -Servlet: /layouts/tableBaseLayout_netui.jsp, total: 60, average time: 0
  -Servlet: /layouts/configBaseLayout_netui.jsp, total: 29, average time: 16
  -Servlet: /jsp/bridge/createmessagingbridge/CreateMessagingBridge.jsp, total: 3, average time: 64
  -Servlet: /jsp/navtree/tree.jsp, total: 125, average time: 2
  -Servlet: /layouts/summarypage.jsp, total: 2, average time: 0
  -Servlet: /framework/skeletons/wlsconsole/header.jsp, total: 250, average time: 0
  -Servlet: /jsp/jms/bridgedestination/JMSBridgeDestinationTable.jsp, total: 24, average time: 8
  -Servlet: /jsp/changemgmt/ChangeManager.jsp, total: 125, average time: 5
  -Servlet: /jsp/common/tableCustomizer.jsp, total: 3, average time: 175
  -Servlet: /jsp/bridge/MessagingBridgeConfigTable.jsp, total: 25, average time: 25
  -Servlet: /jsp/bridge/MessagingBridgeConfigGeneralForm.jsp, total: 13, average time: 49
  -Servlet: /framework/skeletons/wlsconsole/titlebar.jsp, total: 1250, average time: 0
  -Servlet: /jsp/systemstatus/ServerHealthStatus.jsp, total: 125, average time: 0
  -Servlet: /framework/skeletons/wlsconsole/placeholder.jsp, total: 2628, average time: 0
  -Servlet: /jsp/bridge/createmessagingbridge/SourceDestination.jsp, total: 5, average time: 12
  -Servlet: /index.jsp, total: 1, average time: 2
  -Servlet: /framework/skeletons/wlsconsole/shell.jsp, total: 250, average time: 0
  -Servlet: /framework/skeletons/wlsconsole/window.jsp, total: 2000, average time: 0
  -Servlet: /jsp/preferences/wlstadhocrecording.jsp, total: 6, average time: 129
  -Servlet: /jsp/app/DeploymentsControlTable.jsp, total: 1, average time: 135
  -Servlet: /jsp/bridge/MessagingBridgeTargetdeployForm.jsp, total: 9, average time: 17
  -Servlet: /jsp/bridge/MessagingBridgeConfigTransactionsForm.jsp, total: 2, average time: 90
  -Servlet: /jsp/contentheader/ContentMenu.jsp, total: 125, average time: 1
  -Servlet: /framework/skeletons/wlsconsole/nolayout.jsp, total: 2128, average time: 4
  -Servlet: /login/LoginForm.jsp, total: 3, average time: 38
  -Servlet: /jsp/bridge/createmessagingbridge/MessagingSourceDestinationType.jsp, total: 2, average time: 16
  -Servlet: /framework/skeletons/wlsconsole/body.jsp, total: 250, average time: 0
  -Servlet: /jsp/bridge/createmessagingbridge/MessagingBridgeDeployForm.jsp, total: 2, average time: 110
  -Servlet: /jsp/changemgmt/needlockTable.jsp, total: 50, average time: 7
  -Servlet: /framework/skeletons/wlsconsole/abstractbutton.jsp, total: 250, average time: 0
  -Servlet: AppManagerServlet, total: 162, average time: 214
  -Servlet: ResourceServlet, total: 0, average time: 0
  -Servlet: action, total: 0, average time: 0
  -Servlet: /framework/skeletons/wlsconsole/page.jsp, total: 2378, average time: 0
  -Servlet: /jsp/quicklinks/QuickLinks.jsp, total: 125, average time: 0
  -Servlet: /jsp/bridge/createmessagingbridge/MessagingBridgeFinal.jsp, total: 2, average time: 12
  -Servlet: /jsp/summary/ServicesJmsSummary.jsp, total: 2, average time: 19
  -Servlet: /framework/skeletons/wlsconsole/togglebutton.jsp, total: 500, average time: 0
  -Servlet: JSPCServlet, total: 58, average time: 64
  -Servlet: /jsp/bridge/createmessagingbridge/TargetDestination.jsp, total: 4, average time: 12
  -Servlet: /jsp/common/ActionMessages.jsp, total: 89, average time: 0
  -Servlet: /jsp/jms/bridgedestination/createjmsbridgedestination/CreateJMSBridgeDestination.jsp, total: 6, average time: 19
  -Servlet: I18nServlet, total: 0, average time: 0
  -Servlet: /jsp/navtree/treestate.jsp, total: 12, average time: 0
  -Servlet: /framework/skeletons/wlsconsole/abstractmenu.jsp, total: 82, average time: 0
  -Servlet: /jsp/contentheader/Messages.jsp, total: 125, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: /jsp/bridge/MessagingBridgeMonitoringTable.jsp, total: 5, average time: 108
  -Servlet: /jsp/common/genericdelete/confirm.jsp, total: 6, average time: 22
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: /jsp/bridge/createmessagingbridge/MessagingTargetDestinationType.jsp, total: 2, average time: 17
  -Servlet: /jsp/breadcrumb/index.jsp, total: 125, average time: 0
  -Servlet: /jsp/jms/bridgedestination/JMSBridgeDestinationConfigForm.jsp, total: 2, average time: 41
  -Servlet: /jsp/home/home1.jsp, total: 2, average time: 36
  -Servlet: /framework/skeletons/wlsconsole/book.jsp, total: 2910, average time: 0
  -Servlet: /layouts/assistantBaseLayout_netui.jsp, total: 32, average time: 0
  -Servlet: /jsp/jms/server/JMSServerTable.jsp, total: 1, average time: 77
  -Servlet: FileServlet, total: 139, average time: 1
  -Servlet: /framework/skeletons/wlsconsole/desktop.jsp, total: 250, average time: 0
  -Servlet: /jsp/bridge/MessagingBridgeConfigConnectionretryForm.jsp, total: 3, average time: 18
  -Servlet: /framework/skeletons/wlsconsole/singlelevelmenu.jsp, total: 414, average time: 0
  -Servlet: /framework/skeletons/wlsconsole/head.jsp, total: 250, average time: 0
  -Servlet: FileDefault, total: 6, average time: 0
  -Servlet: /framework/skeletons/wlsconsole/twocollayout.jsp, total: 250, average time: 29
  -Servlet: /jsp/changemgmt/needlockConfig.jsp, total: 36, average time: 3
  -Servlet: /framework/skeletons/wlsconsole/footer.jsp, total: 250, average time: 0
-Component Type: WebAppComponentRuntime
 -Name: AdminServer_/consolehelp, Session Current Count: 0
  -Servlet: /framework/skeletons/console/body.jsp, total: 6, average time: 1
  -Servlet: /framework/skeletons/console/head.jsp, total: 6, average time: 0
  -Servlet: AppManagerServlet, total: 3, average time: 649
  -Servlet: /framework/skeletons/console/twocollayout.jsp, total: 6, average time: 50
  -Servlet: /framework/skeletons/console/footer.jsp, total: 6, average time: 0
  -Servlet: /jsp/TOCView.jsp, total: 2, average time: 29
  -Servlet: portalDependencyServlet, total: 0, average time: 0
  -Servlet: /framework/skeletons/console/placeholder.jsp, total: 18, average time: 0
  -Servlet: JSPCServlet, total: 18, average time: 35
  -Servlet: /framework/skeletons/console/page.jsp, total: 12, average time: 0
  -Servlet: /framework/skeletons/console/shell.jsp, total: 6, average time: 0
  -Servlet: /framework/skeletons/console/header.jsp, total: 6, average time: 5
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: /jsp/ContentView.jsp, total: 3, average time: 9
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: /jsp/SearchView.jsp, total: 1, average time: 101
  -Servlet: /framework/skeletons/console/abstractmenu.jsp, total: 3, average time: 9
  -Servlet: /jsp/TOCException.jsp, total: 1, average time: 0
  -Servlet: /framework/skeletons/console/desktop.jsp, total: 6, average time: 1
  -Servlet: /framework/skeletons/console/singlelevelmenu.jsp, total: 6, average time: 6
  -Servlet: /framework/skeletons/console/window.jsp, total: 12, average time: 0
  -Servlet: FileServlet, total: 27, average time: 4
  -Servlet: FileDefault, total: 3, average time: 14
  -Servlet: /framework/skeletons/console/nolayout.jsp, total: 6, average time: 30
  -Servlet: /framework/skeletons/console/book.jsp, total: 12, average time: 6
Application: jms-notran-adp
-Component Type: ConnectorComponentRuntime
Application: bea_wls_deployment_internal
-Component Type: WebAppComponentRuntime
 -Name: AdminServer_/bea_wls_deployment_internal, Session Current Count: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: DeploymentService, total: 54, average time: 1
Application: bea_wls_management_internal2
-Component Type: WebAppComponentRuntime
 -Name: AdminServer_/bea_wls_management_internal2, Session Current Count: 0
  -Servlet: Bootstrap, total: 2, average time: 5
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: FileDistribution, total: 2, average time: 3
Application: mejb
-Component Type: EJBComponentRuntime
 -EJBRunTime: Mejb Type StatelessEJBRuntime
  -EJB Name: Mejb
  -Resources: [MBeanServerInvocationHandler]com.bea:Type=EJBTransactionRuntime,Location=AdminServer,ApplicationRuntime=
ejb,Name=Mejb,StatelessEJBRuntime=Mejb,EJBComponentRuntime=mejb,ServerRuntime=AdminServer
Application: bea_wls_internal
-Component Type: WebAppComponentRuntime
 -Name: AdminServer_/bea_wls_internal, Session Current Count: 0
  -Servlet: HTTPClntSend, total: 0, average time: 0
  -Servlet: ClientLogin, total: 0, average time: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: ClientClose, total: 0, average time: 0
  -Servlet: HTTPClntRecv, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: classes, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: ClientSend, total: 0, average time: 0
  -Servlet: HTTPClntLogin, total: 0, average time: 0
  -Servlet: ClientRecv, total: 0, average time: 0
  -Servlet: HTTPClntClose, total: 0, average time: 0
JMS RUNTIME INFORMATION
JDBC RUNTIME INFORMATION
SERVER: Server1
APPLICATION RUNTIME INFORMATION
Application: example
-Component Type: EJBComponentRuntime
 -EJBRunTime: CompanyMDB_SystemModule!JMSServer1@DistributedQueue Type MessageDrivenEJBRuntime
  -MDB Status: running, MDB Health State: Component:null,State:HEALTH_OK,MBean:null,ReasonCode:[MDB application example is connected to messaging system.]
 -EJBRunTime: Company Type StatelessEJBRuntime
  -EJB Name: Company
  -Resources: [MBeanServerInvocationHandler]com.bea:Type=EJBTransactionRuntime,Location=Server1,ApplicationRuntime=example,Name=Company,StatelessEJBRuntime=Company,EJBComponentRuntime=PersonModel.jar,ServerRuntime=Server1
-Component Type: WebAppComponentRuntime
 -Name: Server1_/LoadTest, Session Current Count: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: TestServlet, total: 0, average time: 0
Application: bea_wls_deployment_internal
-Component Type: WebAppComponentRuntime
 -Name: Server1_/bea_wls_deployment_internal, Session Current Count: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: DeploymentService, total: 1721, average time: 0
Application: bea_wls_internal
-Component Type: WebAppComponentRuntime
 -Name: Server1_/bea_wls_internal, Session Current Count: 0
  -Servlet: HTTPClntSend, total: 0, average time: 0
  -Servlet: ClientLogin, total: 0, average time: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: ClientClose, total: 0, average time: 0
  -Servlet: HTTPClntRecv, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: classes, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: ClientSend, total: 0, average time: 0
  -Servlet: HTTPClntLogin, total: 0, average time: 0
  -Servlet: ClientRecv, total: 0, average time: 0
  -Servlet: HTTPClntClose, total: 0, average time: 0
Application: wls-wsat
-Component Type: WebAppComponentRuntime
 -Name: Server1_/wls-wsat, Session Current Count: 0
  -Servlet: ParticipantPortTypeServlethttp, total: 0, average time: 0
  -Servlet: RegistrationRequesterPortTypeServlethttp, total: 0, average time: 0
  -Servlet: ParticipantPortTypeServlethttp11, total: 0, average time: 0
  -Servlet: RegistrationRequesterPortType11Servlethttp, total: 0, average time: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: RegistrationPortTypeRPCServlethttp11, total: 0, average time: 0
  -Servlet: RegistrationPortTypeRPCServlethttp, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: CoordinatorPortTypeServlethttp, total: 0, average time: 0
  -Servlet: CoordinatorPortTypeServlethttp11, total: 0, average time: 0
Application: uddi
-Component Type: WebAppComponentRuntime
 -Name: Server1_/uddi, Session Current Count: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: uddilistener, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
Application: bea_wls9_async_response
-Component Type: WebAppComponentRuntime
 -Name: Server1_/_async, Session Current Count: 0
  -Servlet: AsyncResponseServiceSoap12Servletjms, total: 0, average time: 0
  -Servlet: AsyncResponseServiceServlethttps, total: 0, average time: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: AsyncResponseServiceSoap12Servlethttps, total: 0, average time: 0
  -Servlet: AsyncResponseServiceSoap12Servlethttp, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: AsyncResponseServiceServlethttp, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: AsyncResponseServiceServletjms, total: 0, average time: 0
Application: uddiexplorer
-Component Type: WebAppComponentRuntime
 -Name: Server1_/uddiexplorer, Session Current Count: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
Application: bea_wls_diagnostics
-Component Type: WebAppComponentRuntime
 -Name: Server1_/bea_wls_diagnostics, Session Current Count: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: Accessor, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
Application: SystemModule
-Component Type: JMSComponentRuntime
Application: bea_wls_cluster_internal
-Component Type: WebAppComponentRuntime
 -Name: Server1_/bea_wls_cluster_internal, Session Current Count: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: MulticastSessionDataRecoveryServlet, total: 1, average time: 7
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: GroupMessageHandlerServlet, total: 0, average time: 0
  -Servlet: StateDumpServlet, total: 1, average time: 7
Application: DataSource
-Component Type: JDBCDataSourceRuntime
JMS RUNTIME INFORMATION
-Connection Name: connection13
 -Session Name: session14
  -Consumer Name: consumer17, Bytes Received: 27090L
 -Session Name: session69
 -Session Name: session68
 -Session Name: session71
  -Producer Name: producer72, Bytes Send: 27090L
 -Session Name: session70
-Connection Name: connection18
 -Session Name: session62
  -Consumer Name: consumer64, Bytes Received: 0L
 -Session Name: session29
  -Consumer Name: consumer31, Bytes Received: 0L
 -Session Name: session59
  -Consumer Name: consumer61, Bytes Received: 0L
 -Session Name: session26
  -Consumer Name: consumer28, Bytes Received: 0L
 -Session Name: session56
  -Consumer Name: consumer58, Bytes Received: 0L
 -Session Name: session20
  -Consumer Name: consumer22, Bytes Received: 0L
 -Session Name: session23
  -Consumer Name: consumer25, Bytes Received: 0L
 -Session Name: session53
  -Consumer Name: consumer55, Bytes Received: 0L
 -Session Name: session50
  -Consumer Name: consumer52, Bytes Received: 0L
 -Session Name: session47
  -Consumer Name: consumer49, Bytes Received: 0L
 -Session Name: session44
  -Consumer Name: consumer46, Bytes Received: 0L
 -Session Name: session41
  -Consumer Name: consumer43, Bytes Received: 0L
 -Session Name: session65
  -Consumer Name: consumer67, Bytes Received: 0L
 -Session Name: session38
  -Consumer Name: consumer40, Bytes Received: 0L
 -Session Name: session35
  -Consumer Name: consumer37, Bytes Received: 0L
 -Session Name: session32
  -Consumer Name: consumer34, Bytes Received: 0L
-JMSServer: JMSServer1
 -Destination: SystemModule!JMSServer1@DistributedQueue, 0L, 16L
 -Destination: JMSServer1!JMSServer1.TemporaryQueue0, 1L, 1L
JDBC RUNTIME INFORMATION
-Data Source: DataSource, Active Connections: 0, Waiting for Connections: 0
SERVER: Server2
APPLICATION RUNTIME INFORMATION
Application: SystemModule
-Component Type: JMSComponentRuntime
Application: DataSource
-Component Type: JDBCDataSourceRuntime
Application: bea_wls_cluster_internal
-Component Type: WebAppComponentRuntime
 -Name: Server2_/bea_wls_cluster_internal, Session Current Count: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: MulticastSessionDataRecoveryServlet, total: 1, average time: 7
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: GroupMessageHandlerServlet, total: 0, average time: 0
  -Servlet: StateDumpServlet, total: 0, average time: 0
Application: bea_wls_deployment_internal
-Component Type: WebAppComponentRuntime
 -Name: Server2_/bea_wls_deployment_internal, Session Current Count: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: DeploymentService, total: 1720, average time: 0
Application: bea_wls_diagnostics
-Component Type: WebAppComponentRuntime
 -Name: Server2_/bea_wls_diagnostics, Session Current Count: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: Accessor, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
Application: wls-wsat
-Component Type: WebAppComponentRuntime
 -Name: Server2_/wls-wsat, Session Current Count: 0
  -Servlet: ParticipantPortTypeServlethttp, total: 0, average time: 0
  -Servlet: RegistrationRequesterPortTypeServlethttp, total: 0, average time: 0
  -Servlet: ParticipantPortTypeServlethttp11, total: 0, average time: 0
  -Servlet: RegistrationRequesterPortType11Servlethttp, total: 0, average time: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: RegistrationPortTypeRPCServlethttp11, total: 0, average time: 0
  -Servlet: RegistrationPortTypeRPCServlethttp, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: CoordinatorPortTypeServlethttp, total: 0, average time: 0
  -Servlet: CoordinatorPortTypeServlethttp11, total: 0, average time: 0
Application: bea_wls_internal
-Component Type: WebAppComponentRuntime
 -Name: Server2_/bea_wls_internal, Session Current Count: 0
  -Servlet: HTTPClntSend, total: 0, average time: 0
  -Servlet: ClientLogin, total: 0, average time: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: ClientClose, total: 0, average time: 0
  -Servlet: HTTPClntRecv, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: classes, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: ClientSend, total: 0, average time: 0
  -Servlet: HTTPClntLogin, total: 0, average time: 0
  -Servlet: ClientRecv, total: 0, average time: 0
  -Servlet: HTTPClntClose, total: 0, average time: 0
Application: bea_wls9_async_response
-Component Type: WebAppComponentRuntime
 -Name: Server2_/_async, Session Current Count: 0
  -Servlet: AsyncResponseServiceSoap12Servletjms, total: 0, average time: 0
  -Servlet: AsyncResponseServiceServlethttps, total: 0, average time: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: AsyncResponseServiceSoap12Servlethttps, total: 0, average time: 0
  -Servlet: AsyncResponseServiceSoap12Servlethttp, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: AsyncResponseServiceServlethttp, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: AsyncResponseServiceServletjms, total: 0, average time: 0
Application: example
-Component Type: EJBComponentRuntime
 -EJBRunTime: CompanyMDB_SystemModule!JMSServer2@DistributedQueue Type MessageDrivenEJBRuntime
  -MDB Status: running, MDB Health State: Component:null,State:HEALTH_OK,MBean:null,ReasonCode:[MDB application example is connected to messaging system.]
 -EJBRunTime: Company Type StatelessEJBRuntime
  -EJB Name: Company
  -Resources: [MBeanServerInvocationHandler]com.bea:Type=EJBTransactionRuntime,Location=Server2,ApplicationRuntime=example,Name=Company,StatelessEJBRuntime=Company,EJBComponentRuntime=PersonModel.jar,ServerRuntime=Server2
-Component Type: WebAppComponentRuntime
 -Name: Server2_/LoadTest, Session Current Count: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
  -Servlet: TestServlet, total: 0, average time: 0
Application: uddi
-Component Type: WebAppComponentRuntime
 -Name: Server2_/uddi, Session Current Count: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: uddilistener, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
Application: uddiexplorer
-Component Type: WebAppComponentRuntime
 -Name: Server2_/uddiexplorer, Session Current Count: 0
  -Servlet: JspServlet, total: 0, average time: 0
  -Servlet: FileServlet, total: 0, average time: 0
  -Servlet: WebServiceServlet, total: 0, average time: 0
JMS RUNTIME INFORMATION
-Connection Name: connection18
 -Session Name: session65
  -Consumer Name: consumer67, Bytes Received: 0L
 -Session Name: session38
  -Consumer Name: consumer40, Bytes Received: 0L
 -Session Name: session35
  -Consumer Name: consumer37, Bytes Received: 0L
 -Session Name: session32
  -Consumer Name: consumer34, Bytes Received: 0L
 -Session Name: session47
  -Consumer Name: consumer49, Bytes Received: 0L
 -Session Name: session44
  -Consumer Name: consumer46, Bytes Received: 0L
 -Session Name: session41
  -Consumer Name: consumer43, Bytes Received: 0L
 -Session Name: session20
  -Consumer Name: consumer22, Bytes Received: 0L
 -Session Name: session23
  -Consumer Name: consumer25, Bytes Received: 0L
 -Session Name: session53
  -Consumer Name: consumer55, Bytes Received: 0L
 -Session Name: session50
  -Consumer Name: consumer52, Bytes Received: 0L
 -Session Name: session62
  -Consumer Name: consumer64, Bytes Received: 0L
 -Session Name: session29
  -Consumer Name: consumer31, Bytes Received: 0L
 -Session Name: session59
  -Consumer Name: consumer61, Bytes Received: 0L
 -Session Name: session26
  -Consumer Name: consumer28, Bytes Received: 0L
 -Session Name: session56
  -Consumer Name: consumer58, Bytes Received: 0L
-Connection Name: connection13
 -Session Name: session69
 -Session Name: session68
 -Session Name: session71
  -Producer Name: producer72, Bytes Send: 27058L
 -Session Name: session70
 -Session Name: session14
  -Consumer Name: consumer17, Bytes Received: 27058L
-JMSServer: JMSServer2
 -Destination: JMSServer2!JMSServer2.TemporaryQueue0, 1L, 1L
 -Destination: SystemModule!JMSServer2@DistributedQueue, 0L, 16L
JDBC RUNTIME INFORMATION
-Data Source: DataSource, Active Connections: 0, Waiting for Connections: 0

References

[1] WebLogic Server MBean Reference.

In this post we show a step-by-step example of how to set-up a high available tuned Java EE environment. We start with installing a Java Virtual Machine (in this case JRockit) and WebLogic. We continue with configuring a domain and tune the Java Virtual Machine for the admin server. Next, we show the steps involved in setting up a cluster, show how to create start and stop scripts and configure a load balancer. Subsequently, we show how to create resources needed for Java messaging and database communication. Once the resources are configured, we deploy a Java EE application that uses these resources, show how to set-up security for the application and create a deployment plan that is used to configure the application. Finally, when everything is in-place we create a benchmark to test the application and use a profiling tool such as JRockit Mission Control to obtain some performance insight. All the topics refer to previous posts that provide more detailed information.

Install WebLogic

We first create a user, for example, oracle. To create a user run the following commands under the root user:

• useradd oracle
• passwd oracle

This creates a /home/oracle directory. To delete the user we can use: userdel -r oracle the option -r removes the /home/oracle directory.

The following software versions will be used:

• JRockit JVM – jrrt-4.0.1-1.6.0-linux-x64
• WebLogic Server – wls1035_generic

First, we choose an installation directory, for example /home/oracle/weblogic. This will be our middleware home in the installation of WebLogic.

To install JRockit we follow these steps:

• Run the file jrrt-4.0.1-1.6.0-linux-x64.bin.
• Click Next on the welcome screen.
• Define the install directory /home/oracle/weblogic/jrrt-4.0.1-1.6.0 and click Next.
• Optionally select extra components and click Next.
• When the installation is finished click Done.

To install WebLogic we follow these steps:

• Navigate to the JDK’s bin directory (/home/oracle/weblogic/jrrt-4.0.1-1.6.0/bin).
• Enter the following command: ./java -d64 -Xms1024m -Xmx1024m -jar wls1031_generic.jar.
• Click Next on the welcome screen.
• Define the Middleware-Home directory which was used to install the JDK and click Next. Click Yes in the pop-up.
• Fill in support credentials if applicable and click Next.
• Select Custom as installation type and click Next.
• De-select the evaluation database and click Next.
• Select the installed JDK (if the JDK is not shown, click Browse and browse to the root of the JDK) and click Next.
• Accept the defaults in the directories screen and click Next.
• Click Next to start the installation.
• De-select Run Quickstart and click Done.

Configure a domain

A WebLogic domain is a logical grouping of server instances that are controlled through an admin server. When creating a domain our first step is to set-up the admin server, i.e., create the files that define the admin server. To this end we run the configuration wizard. Navigate to the ${WL_HOME}/common/bin directory and run config.sh:

• Select Create a New WebLogic Domain.
• Select Generate a Domain Configured Automatically to Support the Following Products: Select WebLogic Server (Required).
• Enter a domain name, for example base_domain.
• Enter a username and password, for example weblogic and magic11g.
• Configure server start-up.
• Choose a mode and select a JDK.
• Note that the configuration wizard can create managed servers, cluster and machines. These we are going to configure by using the admin console.
• Check the summary and click Create.

Note that the admin server is used to configure, manage and monitor servers in a domain. The admin server is a WebLogic server instance with extra applications deployed on it that provide administrative capabilities. Other WebLogic server instances (managed servers) also contain extra applications that the admin server uses to send information to them. It further maintains an XML repository in the ${DOMAIN_HOME}/config directory.

One thing to note is that the admin server is not clusterable and when it goes down, we cannot administer our domain. In general, we can just restart the admin server and if the node manager was used to start it, the node manager will restart it for us. But what if the machine on which the admin server runs fails? In this case we have to make sure that the files the admin server relies on (the admin server directory and the domain configuration files) are available on another machine, such that admin server can be restarted on this machine. Note that when the listen address was set explicitly this has to be corrected in the config.xml to represent the right IP-address. Only thing to watch out for, is that no more than one admin server is running at a time.

Getting acquainted

If we choose production mode as the configuration mode, the command-line will prompt for a username and password on start-up. To overcome this we add a boot.properties file. First create the directory ${DOMAIN_HOME}/servers/AdminServer/security and add a new file: boot.properties. Open the file in a text editor and add the following name-value pairs:

username=weblogic
password=magic11g

Note that when the server is started these values will be encrypted. WebLogic uses the SerializedSystemIni.dat to encrypt the password. This file is located in the ${DOMAIN_HOME}/security directory. Should it happen that the SerializedSystemIni.dat file changes somehow, we will need to encrypt the password again by using the encrypt utility:

• First make a back-up copy of the config.xml file (located in the ${DOMAIN_HOME}/config directory).
• Navigate to the ${DOMAIN_HOME} directory (An encrypted string must be encrypted by the encryption service in the WebLogic domain where it will be used. If not, the server will not be able to decrypt the string).
• Set the weblogic environment by running setWLSEnv.sh (located in the ${WL_HOME}/server/bin directory).
• Run the encrypt utility, for example, java weblogic.security.Encrypt magic11g.
• Edit the config.xml file and replace the encrypted entries (the credential-encrypted and node-manager-password-encrypted elements) with the newly generated one.

It is also possible to decrypt the password, which comes in handy when we forget our admin password. To this end we can use the following:

package middleware.magic;

import weblogic.security.internal.SerializedSystemIni;
import weblogic.security.internal.encryption.ClearOrEncryptedService;
import weblogic.security.internal.encryption.EncryptionService;

public class Decrypt {
    public static void main(String[] args) {
        String string = args[0];
        EncryptionService encryptionService = SerializedSystemIni.getEncryptionService();
        ClearOrEncryptedService clearOrEncryptedService = new ClearOrEncryptedService(encryptionService);
        System.out.println("Clear text password: " + clearOrEncryptedService.decrypt(string));
    }
}

For this to compile we need weblogic.jar (located in the ${WL_HOME}/server/lib directory) in the classpath. The following shows an example:

# navigate to the ${WL_HOME}/server/bin directory
[oracle@edu-wls-rh ~]$ cd /home/oracle/weblogic/wlserver_10.3/server/bin

# set the WebLogic environment by sourcing setWLSEnv.sh
[oracle@edu-wls-rh bin]$ . ./setWLSEnv.sh

# encrypt a password, note we are not in the ${DOMAIN_HOME} directory
[oracle@edu-wls-rh bin]$ java weblogic.security.Encrypt magic11g
Unable to initialize encryption service, verify you are in the domain directory or have specified the correct value for -Dweblogic.RootDirectory

# navigate to the ${DOMAIN_HOME} directory
[oracle@edu-wls-rh bin]$ cd /home/oracle/weblogic/user_projects/domains/base_domain

# encrypt a password
[oracle@edu-wls-rh base_domain]$ java weblogic.security.Encrypt magic11g
{AES}58WNW+BWmxd6xz1QKIE44fKy4F7FagZVs9JV8cmGks4=

# edit the classpath to include our decrypt utility, for example
[oracle@edu-wls-rh base_domain]$ export CLASSPATH=${CLASSPATH}:/home/oracle/weblogic/user_projects/domains/base_domain/lib/decrypt.jar

# decrypt the encrypted password
[oracle@edu-wls-rh base_domain]$ java middleware.magic.Decrypt {AES}58WNW+BWmxd6xz1QKIE44fKy4F7FagZVs9JV8cmGks4=
Clear text password: magic11g

To start the admin server, open a command shell, navigate to the ${DOMAIN_HOME} directory and run startWebLogic.sh. The admin console can be reached at http://hostname:7001/console.

When the server is configured in development mode we enable Lock & Edit by following these steps:

• Click on the top of the screen in the Preferences link.
• Deselect the Automatically acquire lock and activate changes
• Click Save.
• Click Release Configuration.

In this case when we need to perform any changes, we first have to click Lock & Edit in the Change Center (located in the top left). In order to make the changes active we have to click Active Changes (or Undo All Changes if we want to roll the changes back).

Tune the admin server’s JVM

JVM parameters can be added in the setDomainEnv.sh file (located in the ${DOMAIN_HOME}/bin directory). Find the entry BEA_JAVA_HOME and add the following:

BEA_JAVA_HOME="/home/oracle/weblogic/jrrt-4.0.1-1.6.0"
export BEA_JAVA_HOME

USER_MEM_ARGS="-Xms512m -Xmx512m -Xgc:throughput"
export USER_MEM_ARGS

JAVA_VENDOR="Oracle"
export JAVA_VENDOR

...

PRODUCTION_MODE="true"
export PRODUCTION_MODE

By setting the PRODUCTION_MODE to true we disable the auto-deployment ‘feature’, which not useful when using a cluster as it only deploys applications to the admin server.

When using WebLogic (or any other application server for that matter) it is beneficial to tune for application throughput. WebLogic is a highly multi-threaded environment and to let this run as smoothly as possible we need to give the threads as much resources as possible, hence the choice for the throughput optimization configuration. By choosing throughput as optimization strategy the following defaults are present:

• The nursery size (-Xns) is automatically sized to 50% of free heap.
• The compaction is configured as -XXcompaction:abortable=false,percentage=6.25,heapParts=4096,maxReferences=299900.
• The thread local area size is configured as -XXtlasize:min=2k,preferred=16k,wastelimit=2k. Note that the preferred size depends on the heap size and lies between 16k and 64k.

Additional tuning may be necessary when compaction causes long garbage collection pauses. To find out the impact compaction has on the garbage collection pausetime, we can run a flight recording and examine the compaction pause parts of old garbage collections. In general, compaction pausetime depends on the compaction ratio (percentage or externalPercentage and internalPercentage) and the maximum number of references. In multi-threaded applications where threads allocate lots of objects, it might be beneficial to increase the TLA size. Caution must be taken, however, to not make the TLA size too large as this increases the fragmentation and as a result more garbage collections need to be run in order to allocate new objects.

More information on tuning the JVM can be found in the posts:

• Optimizing WebLogic Server Performance: JVM tuning
• Fast, Faster, JRockit
• Tune the JVM that runs Coherence

and the presentation (to be) given at the UKOUG on december 7th:

We only have talked about tuning the JVM, note that there can be additional resources that need to be tuned. This is explained in the post Tuning the WebLogic Server Performance.

Clustering

First, we create a cluster (a cluster defines groups of WebLogic servers that work together to increase scalability and reliability):

• Open the admin console.
• Click Environment, Clusters.
• Click New and enter the name of the cluster and choose as messaging mode unicast.

Unicast clustering uses TCP/IP sockets to pass messages between servers. WebLogic uses a group leader as a relay point for cluster messages. As the cluster becomes larger (10+ servers) WebLogic uses multiple group leaders where each group is limited to 10 servers. If the size of the cluster gets very large, the number of sockets grows large and the burden on the group leader becomes large as more and more group leaders are communicating with one another. This can lead to that servers are dropping out of the cluster and rejoin the cluster frequently. In this case, it is recommended to use multicast for the cluster messaging. Multicast is very efficient as it uses low-level network technology to broadcast messages.

Next, we create a machine (A machine is the logical representation of the computer that hosts one or more WebLogic servers. WebLogic uses configured machine names to determine the optimum server in a cluster to which certain tasks, such as HTTP session replication, are delegated. The admin server uses the machine definition in conjunction with node manager to start managed servers):

• Click Environment, Machines.
• Click New and enter a name and choose the machine type.
• Click Next and fill in the node manager properties, such as the listen address (hostname or IP address of the machine to which the node manager is associated) and port (default 5556).
• In the case of a Unix related operating system the machine must be mapped to a user ID and/or a group ID:
  • Click the tab Configuration, General.
  • Map the machine to a user ID – Enable Post-Bind UID: check this option, Post-Bind UID: enter user ID, for example oracle.
  • Map the machine to a group ID – Enable Post-Bind GID: check this option, Post-Bind GID: enter group ID, for example oracle.

By telling WebLogic which servers run on which machine, it can be smart about the location of the replicated objects by keeping it on different machines. It also tells the admin server which node manager to use when starting and stopping managed servers.

To create managed servers we can follow these steps:

• Click Environment, Servers.
• Click New and enter the server name, listen address and port, and add the server to the configured cluster.
• Change the start up parameters, health monitoring and overload:
  • Click the tab Configuration, Server Start.
  • Enter the following parameters:
    • Java Home: /home/oracle/weblogic/jrrt-4.0.1-1.6.0
    • Java Vendor: Oracle
    • Arguments: -jrockit -Xms1024m -Xmx1024m -Xgc:throughput -XX:+UseCallProfiling -XXcompaction:abortable=false,percentage=6.25,heapParts=4096,maxReferences=299900 -XXtlaSize:min=2k,preferred=16k,wasteLimit=2k
  • Click the tab Health Monitoring and select the Auto Kill If Failed option.
  • Click the tab Overload and configure the parameters as follows:
    • Panic Action: select the ‘Exit the server process’ option (The server shuts down in the case of an out-of-memory-exception.)
    • Max Stuck Thread Time: 600
    • Stuck Thread Count: 50
    • Failure Action: Force immediate shutdown for this server

We can additionally configure a denial of service for the managed servers, for example to limit the maximum time to receive request data (post timeout) and the maximum size of a request (max post size and HTTP max message size). This can be accomplished on the HTTP, Protocols tab of the server in question. Another important parameter is the accept backlog (Tuning, Configuration tab) that controls the length of the TCP/IP listen queue (and thus limits the number of concurrent connections to the server). In cases when the server is processing a lot of concurrent HTTP requests, the accept backlog parameter must be increased to prevent ‘connection refused’ errors. Logging can be configured on the General, Logging tab. By default, WebLogic uses log rotation. To troubleshoot problems we need to have enough information available for analysis. In this case the rotation file size and the number of retained files must be set to appropriate values, values of respectively 5000 and 10 will in most cases be sufficient. If the architecture routes all the HTTP requests through a web server, that already keeps a record of HTTP access, it is beneficial to disable HTTP access logging for the managed server (this can be disabled on the HTTP, Logging tab).

WebLogic allows us to define separate network channels for cluster traffic, such as heartbeat messages and replication. To isolate heartbeat messages we create a network channel for each server in the cluster:

• Click Environment, Servers, and choose the server for which to configure a network channel.
• Click the Channels, Protocols tab, click New and enter the following parameters:
  • Name: UnicastBroadcastChannel
  • Protocol: cluster-broadcast
  • Click Next and enter the listen address (server listen address), listen port (same for all servers) – leave the external listen address and port empty.
  • Click Next and disable HTTP Enabled for This Protocol.
• Click Environment, Clusters and choose the cluster for which to set the unicast broadcast channel.
• Click on the Messaging, Configuration tab and set the Unicast Broadcast Channel to the name of the configured network channel in this case: UnicastBroadcastChannel

We can follow basically the same steps in order to configure a network channel that will be used for cluster replication:

• Click Environment, Servers, and choose the server for which to configure a network channel.
• Click the Channels, Protocols tab, click New and enter the following parameters:
  • Name: ReplicationChannel
  • Protocol: t3
  • Click Next and enter the listen address (server listen address), listen port (same for all servers) – leave the external listen address and port empty.
  • Click Next and disable HTTP Enabled for This Protocol.
• Click Environment, Clusters and choose the cluster for which to set the replication channel.
• Click on the Replication, Configuration tab and set the Replication Channel to the name of the configured network channel in this case: ReplicationChannel

As the last step add the managed servers to the appropriate machine.

Next, we will test if the set-up is working. To this end we need to configure the node manager. The node manager provides a mechanism to start and stop server instances, monitors the health of the servers and restarts servers in the case of failure. First, we need to decide where the node manager home will reside. Note that the startNodeManager.sh script (located in the ${WL_HOME}/server/bin directory) contains the following entry NODEMGR_HOME="${WL_HOME}/common/nodemanager". When starting the node manager for the first time, the script will check if the specified directory contains a nodemanager.properties file and will create one if not present. The node manager home location can be changed at will, note however that the configuration wizard creates a nodemanager.domains file in the default directory (${WL_HOME}/common/nodemanager). If we change the node manager home, we must also make sure the nodemanager.domains file is present in this directory.

The node manager requires authentication to start and stop managed servers. The first time the node manager is started it communicates with the admin server to obtain a username and password that will be used by the admin server to authenticate the node manager (this can also be done by using nmEnroll). When the domain is created in production mode, random node manager credits are created (in development mode these are the same as the admin user). When we want to use, for example, nmConnect we have to set the node manager username and password to known values. To this end open the admin console:

• Click on base_domain, security, general and click on the advanced link.
• Edit the NodeManager Username and Password. In our case, we set these to respectively weblogic and magic11g.
• Start the node manager by using the startNodeManager.sh script.

When the node manager is started use the admin console to start the managed servers. When everything is running the node managers and managed servers are properly configured, i.e., the node managers are initialized with the password files they need to accept commands from the admin server and when the managed servers are started on all candidate machines the node manager and domain directory are properly initialized.

Scaling

We have two scaling options: vertical and horizontal scaling. Vertical scaling relates to adding more CPUs to a machine. To better utilize the server hardware we can add more WebLogic instances to the machine that could lead to increased application throughput. To determine if this is indeed the case we need to benchmark. Benchmarking for scalability is about measuring resource utilization. Good scalability means that service levels can be maintained while the workload is increased. If an application does not scale well, it is not fully utilizing the hardware. Consequently, throughput will degrade. Ideally, a linear load increase should lead to a linear degradation in service levels and performance. Linear scalabilty can be approached when so-called share nothing clusters are used. The nodes provide the same functionality and know nothing about other nodes in the cluster (no HTTP session replication). In this case, the computing ability of the cluster increases almost linearly as more nodes are added to the cluster, if the back-end information systems, such as a database, are powerful enough.

Applications that ‘share nothing’ are usually sharing state through the database. The application-tier can scale as far as when the database becomes a bottleneck. In general, relying on a single shared resource will eventually cause contention for that resource and thus limit the scalability. Caching is a good resolution. When we cache data at the application-tier we avoid calls to the database (and also avoid relational data to object data conversions). When using a cluster of server instances we need to maintain multiple caches; this is not a problem for read-only data but for read/write data it is. Caching solutions, such as Coherence, provide different kind of caching, i.e., replicated and partitioned. Replicated does not scale well when cache writes are involved as the data needs to be replicated across all the nodes in the grid. A partioned cache, on the other hand, scales very well when cache writes are involved as data ownership is spread throughout the cluster (the system automatically rebalances the data when the number of nodes in the grid changes – we do not need to decide on how to partition the data, it comes out of the box). Another plus is that access to the cache means at most one network trip, this in order to maintain linear scalability. An optimization on read-access can be made when data can be obtained locally (sticky access) in this case a hybrid solution such as the near cache can be applied. Note that when Coherence*Web is used to cache HTTP session objects a near cache data structure is used. More information on Coherence*Web can be found in the post Setting-up a WebLogic Cluster that uses Coherence.

Database caching patterns are cache-aside, read/write-through or write-behind. The caching pattern cache-aside is discussed in the post Hibernate and Coherence. The caching patterns read/write-through or write-behind are discussed in the post Coherence and Hibernate: Decoupling the Database.

To scale the processing, data grids allow for a targeted execution of processing in the nodes, i.e., the map/reduce pattern (used, for example, in Google’s bigtable or Apache’s hadoop). The core of the map/reduce pattern are the following steps:

• Map step – a master node takes some input (problem), partitions it into smaller sub-problems and distributes those to worker nodes. Note that a worker node in turn can do the same.
• Reduce step – the master takes all the answers and combines them to get the output.

Sort of a divide and conquer algorithm, with the key difference that the map/reduce algorithm handles the data as key-value pairs.

Horizontal scalling relates to adding more machines to the environment, which gives a failover capability that we cannot get with vertical scaling. A good approach is to combine both scaling techniques to obtain better CPU utilization and failover capability. The post Setting-up a WebLogic Cluster that spans Multiple Machines shows an example, as the title suggests, of how to set-up a cluster that spans multiple machines.

Start and stop scripts

In general, it is recommended to start the node manager when the machine boots. In this case, we need to know where to put our custom commands that will be called when the system boots. Note that RedHat Enterprise Linux has a /etc/rc.d/rc.local file, that can be used to put custom commands in. Let us do it in the recommended script-based way. Note that Unix-based systems specify so-called run levels, and that for each run level, scripts can be defined that start a certain service. These scripts are located in the /etc/rc.d/init.d directory. This allows for services to be started when the system boots or to be stopped on system shutdown. The different run levels are specified by a specific directory in the /etc/rc.d directory, i.e.,

• rc0.d – contains scripts that are executed on system shutdown.
• rc1.d – contains scripts for single-user mode.
• rc2.d – contains scripts for multi-user mode.
• rc3.d – contains scripts for multi-user mode and networking.
• rc4.d – not used.
• rc5.d – same as rc3.d plus some graphical stuff.
• rc6.d – contains scripts that are executed on system reboot.

The boot sequence is as follows: in the /etc/inittab file the starting runlevel is defined, the script /etc/rc.d/rc.sysinit is called and /etc/rc.d/rc is run. The rc script looks in the /etc/rc.d/rc<start-runlevel>.d to execute the K**<script-name> scripts with the stop option. After this the S**<script-name> scripts are executed with the start option. Note that scripts are started in numerical order, i.e., the S10network script is executed before the S80sendmail script.

To create a node manager ‘service’, we first create basic start and stop scripts for the node manager. To this end, we will use WLST. To start the node manager by using WLST we can use the following:

bea_home = '/home/oracle/weblogic';
pathseparator = '/';
listen_port = '5556';
listen_address = 'localhost';

node_manager_home = bea_home + pathseparator + 'wlserver_10.3' + pathseparator + 'common' + pathseparator + 'nodemanager';

startNodeManager(verbose='true', NodeManagerHome=node_manager_home, ListenPort=listen_port, ListenAddress=listen_address);

To stop the node manager by using WLST we can use the following:

bea_home = '/home/oracle/weblogic';
pathseparator = '/';
admin_username = 'weblogic';

admin_password = 'magic11g';
listen_address = 'localhost';
listen_port = '5556';

domain_name = 'base_domain';
domain_home = bea_home + pathseparator + 'user_projects' + pathseparator + 'domains' + pathseparator + domain_name;

nmConnect(admin_username, admin_password, listen_address, listen_port, domain_name, domain_home, 'ssl');
stopNodeManager();

Create two shell scripts that respectively run the start and stop script, i.e.,

#!/bin/sh

BEA_HOME="/home/oracle/weblogic"
export BEA_HOME

WL_HOME="/home/oracle/weblogic/wlserver_10.3"
export WL_HOME

. ${WL_HOME}/common/bin/wlst.sh ${BEA_HOME}/deploy/scripts/startNodeManager.py

#!/bin/sh

BEA_HOME="/home/oracle/weblogic"
export BEA_HOME

WL_HOME="/home/oracle/weblogic/wlserver_10.3"
export WL_HOME

. ${WL_HOME}/common/bin/wlst.sh ${BEA_HOME}/deploy/scripts/stopNodeManager.py

Create a boot script (nodemanager) under the root user and place this script in the /etc/rc.d/init.d directory, for example,

#!/bin/sh
#
# chkconfig: 235 91 35
# description: starts and stops the node manager
#
#
. /etc/rc.d/init.d/functions

RETVAL=0
SERVICE="nodemanager"

start() {
	echo "Starting Node Manager"
	su - oracle -c "/home/oracle/weblogic/deploy/scripts/startNodeManagerService.sh" >/dev/null 2>&1
	RETVAL=$?
	[ $RETVAL -eq 0 ] && success || failure
	echo
	[ $RETVAL -eq 0 ] && touch /var/lock/subsys/${SERVICE}
	return $RETVAL
}

stop() {
	echo "Stopping Node Manager"
	su - oracle -c "/home/oracle/weblogic/deploy/scripts/stopNodeManagerService.sh" >/dev/null 2>&1
	RETVAL=$?
	[ $RETVAL -eq 0 ] && success || failure
	echo
	[ $RETVAL -eq 0 ] && rm -r /var/lock/subsys/${SERVICE}
	return $RETVAL
}

restart() {
	stop
	start
}

case "$1" in
	start)
		start
		;;
	stop)
		stop
		;;
	restart)
		restart
		;;
	*)
		echo $"Usage: $0 {start|stop|restart}"
		exit 1
esac

exit $?

By using the chkconfig command we can update the runlevel information for system services, for example, chkconfig --add nodemanager. To test the set-up shut the system down and start it again. To check if the nodemanager is running we can use either ps -ef|grep java or netstat -anp|grep :5556 which assumes the node manager is listening on port 5556.

By using WLST we can also create start and stop scripts for the admin server and the managed servers, for example,

bea_home = '/home/oracle/weblogic';
pathseparator = '/';
admin_username = 'weblogic';
admin_password = 'magic11g';
listen_address = 'localhost';
listen_port = '5556';
admin_server_url='t3://localhost:7001'
domain_name = 'base_domain';
domain_home = bea_home + pathseparator + 'user_projects' + pathseparator + 'domains' + pathseparator + domain_name;

print 'CONNECT TO NODE MANAGER';
nmConnect(admin_username, admin_password, listen_address, listen_port, domain_name, domain_home, 'ssl');

print 'START ADMIN SERVER ONLY ON THE MACHINE WHERE THE ADMIN SERVER IS PRESENT';
nmStart('AdminServer');

print 'CONNECT TO ADMIN SERVER';
connect(admin_username, admin_password, admin_server_url);

print 'START MANAGED SERVERS ON THE MACHINE';
start('Server1','Server');
start('Server2','Server');

bea_home = '/home/oracle/weblogic';
pathseparator = '/';
admin_username = 'weblogic';
admin_password = 'magic11g';
listen_address = 'localhost';
listen_port = '5556';
admin_server_url='t3://localhost:7001'
domain_name = 'base_domain';
domain_home = bea_home + pathseparator + 'user_projects' + pathseparator + 'domains' + pathseparator + domain_name;

print 'CONNECT TO NODE MANAGER';
nmConnect(admin_username, admin_password, listen_address, listen_port, domain_name, domain_home, 'ssl');

print 'CONNECT TO ADMIN SERVER';
connect(admin_username, admin_password, admin_server_url);

print 'STOPPING SERVERS ON THE MACHINE';
shutdown('Server1','Server','true',1000,'true');
shutdown('Server2','Server','true',1000,'true');

print 'STOPPING ADMIN SERVER ONLY ON THE MACHINE WHERE THE ADMIN SERVER IS PRESENT';
shutdown('AdminServer','Server','true',1000,'true');

Sometimes it can happen that the action ‘connect to node manager’ is slow. This is related to the machine’s entropy and JVM reading random bytes by using a particilar securerandom source, usually this is /dev/urandom. To overcome this problem edit the java.security file (located in the <java_home>/jre/lib/security directory) and change securerandom.source=file:/dev/urandom to securerandom.source=file:/dev/./urandom.

Configure load balancer

The load balancer decides how to dispatch requests to the back end server instances. A load balancer needs to do tasks such as distributing requests, health checking and session binding. As these are simple jobs it is rare that a load balancer will become a bottleneck. To load balance requests, we are going to use the web server plug-in. The post Setting-up Web-Tier Components in a SOA Environment describes in detail how to install and set-up the web server plug-in. To configure the web server plug-in (mod_wl_ohs), open the mod_wl_ohs.conf file and add the following contents:

LoadModule weblogic_module   "${ORACLE_HOME}/ohs/modules/mod_wl_ohs.so"

<IfModule weblogic_module>
	WebLogicCluster 172.31.0.106:8001,172.31.0.106:8011
	ConnectTimeoutSecs 10
	ConnectRetrySecs 2
	DebugConfigInfo ON
	WLSocketTimeoutSecs 2
	WLIOTimeoutSecs 300
	Idempotent ON
	FileCaching ON
	KeepAliveSecs 20
	KeepAliveEnabled ON
	DynamicServerList ON
	WLProxySSL OFF
</IfModule>

<Location /LoadTest>
	SetHandler weblogic-handler
</Location>

A handy script to show the status, start and stop the web server is the following:

#!/bin/sh

PATH="${PATH}:/home/oracle/weblogic/Oracle_WT1/opmn/bin/"
ORACLE_INSTANCE="/home/oracle/weblogic/Oracle_WT1/instances/instance1/"
export ORACLE_INSTANCE

if [ "$1" == "status" ] ; then
	opmnctl status
fi

if [ "$1" == "stopohs" ] ; then
	opmnctl stopproc process-type=OHS
fi

if [ "$1" == "startohs" ] ; then
	opmnctl startproc process-type=OHS
fi

This can be used as, for example, ./script-name.sh status. To test the configuration we can use the URL: http://web-server-hostname:7777/LoadTest/?__WebLogicBridgeConfig. When the URL is accessed the General Server List should show the servers in the cluster.

Configuring resources

The application uses Java Persistence and Java Messaging, i.e., we have the following persistence.xml file

<persistence xmlns="http://java.sun.com/xml/ns/persistence" version="1.0">
    <persistence-unit name="PersonPersistenceUnit">
        <provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>
        <jta-data-source>jdbc/exampleDS</jta-data-source>
        <class>model.entities.Person</class>
    </persistence-unit>
</persistence>

The following stateless enterprise bean

package model.logic;

import model.entities.Person;
import javax.annotation.PostConstruct;
import javax.annotation.PreDestroy;
import javax.annotation.Resource;
import javax.ejb.Stateless;
import javax.jms.*;
import javax.persistence.EntityManager;
import javax.persistence.PersistenceContext;

@Stateless(name = "Company", mappedName = "ejb/Company")
public class CompanyBean implements Company {

    @PersistenceContext(unitName = "PersonPersistenceUnit")
    private EntityManager entityManager;

    @Resource(name = "jms/ConnectionFactory", type = ConnectionFactory.class)
    private ConnectionFactory connectionFactory;
    @Resource(name = "jms/CompanyQueue", type = Queue.class)
    private Queue destination;

    private Connection connection = null;
    private Session session = null;
    private MessageProducer messageProducer = null;

    @PostConstruct
    public void init() {
        try {
            connection = connectionFactory.createConnection();
            session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
            messageProducer = session.createProducer(destination);
        } catch (JMSException e) {
            if (connection != null) {
                try {
                    connection.close();
                } catch (JMSException f) {
                    e.printStackTrace();
                }
            }
        }
    }

    @PreDestroy
    public void release() {
        try {
            if (connection != null) {
                connection.close();
            }
        } catch (JMSException e) {
            e.printStackTrace();
        }
    }

    public void insertPerson(Person person) {
        Person persontemp = findPerson(person.getSofinummer());
        if (persontemp == null) {
            entityManager.persist(person);
            sendMessage(person);
        } else {
            updatePerson(person);
        }
    }

    public void removePerson(Integer sofinummer) {
        Person person = findPerson(sofinummer);
        if (person != null) {
            entityManager.remove(entityManager.merge(person));
            sendMessage(person);
        }
    }

    public void updatePerson(Person person) {
        entityManager.merge(person);
    }

    private Person findPerson(Integer sofinummer) {
        return entityManager.find(Person.class, sofinummer);
    }

    private void sendMessage(Person person) {
        try {
            ObjectMessage message = session.createObjectMessage();
            message.setObject(person);
            messageProducer.send(message);
        } catch (JMSException e) {
            e.printStackTrace();
        }
    }
}

and the following message-driven bean:

package model.logic;

import model.entities.Person;

import javax.ejb.ActivationConfigProperty;
import javax.ejb.MessageDriven;
import javax.jms.*;

@MessageDriven(mappedName = "jms/CompanyQueue", activationConfig = {
        @ActivationConfigProperty(propertyName = "destinationName", propertyValue = "jms/CompanyQueue"),
        @ActivationConfigProperty(propertyName = "destinationType", propertyValue = "javax.jms.Queue")
})
public class CompanyMDB implements MessageListener {

    public void onMessage(Message message) {
        if (message instanceof ObjectMessage) {
            ObjectMessage objectMessage = (ObjectMessage) message;
            try {
                System.out.println("RECEIVED OBJECT MESSAGE " + objectMessage.getObject());
            } catch (JMSException e) {
                e.printStackTrace();
            }
        }

        if (message instanceof TextMessage) {
            TextMessage textMessage = (TextMessage) message;
            try {
                System.out.println("RECEIVED TEXT MESSAGE " + textMessage.getText());
            } catch (JMSException e) {
                e.printStackTrace();
            }
        }
    }
}

Three resources need thus to be configured:

• JDBC data source with JNDI-name jdbc/exampleDS.
• JMS connection factory with JNDI-name jms/ConnectionFactory.
• JMS queue with JNDI-name jms/CompanyQueue.

To create a data source we can follow these steps:

• Click Services, JDBC and select Data Sources.
• Click New (choose generic datasource) and enter the following parameters:
  • Name: DataSource
  • JNDI Name: jdbc/exampleDS
  • Database Type: select Oracle
  • Database Driver: select Oracle’s Driver (thin) for service connections; Versions: 9.0.1, 9.2.0, 10, 11
  • On the transaction options screen, select the logging last resource option. Note that we need a two-phase commit as the method CompanyBean.insertPerson is using two resources, i.e., it inserts a person into a database and sends a message to a queue. The logging last resource option ensures that once a non-XA resource commits, the XA recovery log is also committed. In this case WebLogic will have a permanent record of the transaction so that it can attempt XA recovery on the transaction.
  • On the connection properties screen fill in the connection parameters
  • Click Next, and click Test Configuration.
• Click Next, and target the data source to the appropriate servers, in our case we will target it to the cluster. Note that this will make it easier if we ever plan to scale our environment by adding more servers to the cluster. By targeting resources to the cluster the resource will automatically be available to all the servers in the cluster.

On the connection pool, configuration tab we can fine tune the connection pool settings. Here, we can set the initial size and the maximum size of the pool. When PreparedStatement objects are used, the statement cache can be configured. The advanced area contains options to make the connection pool more resilient to failures, such as the test connections on reserve attribute, enables a feature that validates connections before they are given to an application. Note that the validation is done synchronously and will thus add some overhead. By using the test frequency option, the feature to test unused connections on a frequent basis is enabled. Another important feature to enable is the connection retry frequency. When this is set to another value than zero, WebLogic will try to create the data source every so often when the database is temporarily unavailable.

Configuring a JMS environment consists of the following steps:

• Configure a persistent store when we do not want to lose any messages.
• Configure a JMS Server. JMS servers act as management containers for the queues and topics in JMS modules that are targeted to them. A JMS server’s primary responsibility for its destinations is to maintain information on what persistent store is used for any persistent messages that arrive on the destinations, and to maintain the states of durable subscribers created on the destinations.
• Configure a JMS Module. JMS system resources are configured and stored as modules similar to standard Java EE modules. Such resources include queues, topics, connection factories, templates, destination keys, quota, distributed queues, distributed topics, foreign servers, and JMS store-and-forward (SAF) parameters.
• Configure JMS system resources.

Create a persistence store:

• Click Services, Persistent Stores.
• Click New (choose Create FileStore) and enter the following parameters:
  • Name: FileStore
  • Target: a server in the cluster
  • Directory: An existing directory

Create a JMS server:

• Click Services, Messaging and select JMS Servers.
• Click New and enter the following parameters:
  • Name: JMSServer
  • Persistent Store: Select FileStore
  • Select as target the same server as the filestore was targeted to.

Note that when using a cluster we can create a persistent store/JMS server pair for every server in the cluster.

Create a JMS Module:

• Click Services, Messaging and select JMS Modules.
• Click New and enter the following parameters:
  • Name: JMSModule
  • Target the JMS module to the cluster.

Create a subdeployment (a mechanism to group resources of a JMS module and target them to a server, for example a JMS server):

• Select in the JMSModule the tab Subdeployments, click New and enter the following parameters:
  • Subdeployment Name: Subdeployment
  • Target the subdeployment to the JMS servers

Create a connection factory:

• In the JMS module click the configuration tab.
• Click New and select Connection Factory.
• Click Next and enter the following parameters:
  • Name: ConnectionFactory
  • JNDI Name: jms/ConnectionFactory
  • Make sure the XA connection factory enabled option is selected
• Click Next and check if the connection factory is targeted to the cluster.

Create a uniform distributed queue:

• In the JMS module click the configuration tab.
• Click New and select Distributed Queue.
• Click Next and enter the following parameters:
  • Name: Queue
  • JNDI Name: jms/CompanyQueue
  • Type: Uniform
• Click Next, click Advanced Targeting and select the subdeployment

The post Messaging in WebLogic Server: Best Practices provides detailed information on how to fine tune the WebLogic messaging environment. In the example above we have used a distributed queue to make the messaging resources highly available. We did, however, not configure any failover (migration) mechanism for the persistent store and the JMS server. The set-up concerning migration is explained in detail in the post Managing Failure Conditions in a WebLogic Environment.

Now, that the resources are in place we can deploy the application.

Deployment

The application to deploy consists of an enterprise bean part (already presented above) and a servlet:

package userinterface.servlets;

import model.entities.Person;
import model.logic.Company;

import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.Random;

public class TestServlet extends HttpServlet {

    private Random generator = null;

    @Override
    public void init() throws ServletException {
        generator = new Random();
    }

    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        Company company = getCompany();
        Person person = createPerson();

        if (generator.nextDouble() < 0.01) {
            company.removePerson(person.getSofinummer());
        } else {
            company.insertPerson(person);
        }
    }

    @Override
    public void destroy() {
        generator = null;
    }

    private Person createPerson() {
        Person person = new Person();
        person.setNaam(Long.toString(Math.abs(generator.nextLong()), 36));
        person.setSofinummer(generator.nextInt(10000));
        return person;
    }

    private static Company getCompany() {
        Context context = null;
        Company company = null;
        try {
            context = new InitialContext();
            company = (Company) context.lookup("ejb/Company#model.logic.Company");
        } catch (NamingException e) {
            e.printStackTrace();
        }
        return company;
    }
}

in the web.xml we have the following servlet definition:

<servlet>
	<servlet-name>TestServlet</servlet-name>
	<servlet-class>userinterface.servlets.TestServlet</servlet-class>
</servlet>
<servlet-mapping>
	<servlet-name>TestServlet</servlet-name>
	<url-pattern>/testservlet</url-pattern>
</servlet-mapping>

To deploy the application we can follow the following steps:

First create the following directory structure in /home/oracle/weblogic/deploy:

/example
	/app
		loadtest.ear
	/plan

By using this structure WebLogic will automatically create a deployment plan in the plan directory. Open the admin console and follow these steps:

• Click Lock & Edit.
• Click Deployments and click Install.
• Navigate to the example directory and select example (open directory).
• Choose install this deployment as an application, target the application to the cluster and further accept the defaults.
• Click Finish.
• Do not click on Active Changes yet as we are going to edit the deployment plan.

Security

We have enabled security in the application by using the following entry in the web.xml file

<security-constraint>
	<web-resource-collection>
		<web-resource-name>All</web-resource-name>
		<url-pattern>/testservlet/*</url-pattern>
	</web-resource-collection>
	<auth-constraint>
		<role-name>SOMEROLE</role-name>
	</auth-constraint>
	<user-data-constraint>
		<transport-guarantee>NONE</transport-guarantee>
	</user-data-constraint>
</security-constraint>
<login-config>
	<auth-method>FORM</auth-method>
	<form-login-config>
		<form-login-page>/login.jspx</form-login-page>
		<form-error-page>/login.jspx</form-error-page>
	</form-login-config>
</login-config>
<security-role>
	<role-name>SOMEROLE</role-name>
</security-role>

in which the login.jspx page looks as follows:

<jsp:root xmlns:jsp="http://java.sun.com/JSP/Page" version="2.0" xmlns:c="http://java.sun.com/jsp/jstl/core">
    <jsp:output omit-xml-declaration="true" doctype-root-element="HTML"
                doctype-system="http://www.w3.org/TR/html4/loose.dtd"
                doctype-public="-//W3C//DTD HTML 4.01 Transitional//EN"/>
    <jsp:directive.page contentType="text/html;charset=utf-8"/>
    <html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
        <title>login</title>
    </head>
    <body><c:choose>
        <c:when test="${sessionScope.loginAttempts == null}">
            <c:set var="loginAttempts" scope="session" value="0"/>
        </c:when>
        <c:when test="${sessionScope.loginAttempts > 2}">
            <table>
                <tr>
                    <td colspan="2">
                        <hr/>
                    </td>
                </tr>
                <tr>
                    <td colspan="2">
                        <h2>To bad. No Entry!</h2>
                    </td>
                </tr>
                <tr>
                    <td colspan="2">
                        <hr/>
                    </td>
                </tr>
            </table>
            <c:set var="stop" scope="page" value="${true}"/>
        </c:when>
        <c:when test="${sessionScope.loginAttempts != null}">
            <c:set var="loginAttempts" scope="session" value="${sessionScope.loginAttempts + 1}"/>
        </c:when>
    </c:choose><c:if test="${!stop}">
        <form action="j_security_check" method="post">
            <table>
                <tr>
                    <td colspan="2">
                        <hr/>
                    </td>
                </tr>
                <tr>
                    <td colspan="2">
                        <h1>Enter your login data.</h1>
                    </td>
                </tr>
                <tr>
                    <td class="textbold">Your Username:</td>
                    <td>
                        <input class="textfield" type="text" name="j_username"/>
                    </td>
                </tr>
                <tr>
                    <td class="textbold">Your Password:</td>
                    <td>
                        <input class="textfield" type="password" name="j_password"/>
                    </td>
                </tr>
                <tr>
                    <td class="textboldred">
                        <c:if test="${sessionScope.loginAttempts > 0}">Wrong Login.</c:if>
                    </td>
                    <td>
                        <input class="button" type="submit" name="logon" value="Login"/>
                    </td>
                </tr>
                <tr>
                    <td colspan="2">
                        <hr/>
                    </td>
                </tr>
            </table>
        </form>
    </c:if></body>
    </html>
</jsp:root>

To keep the set-up simple we create additional users and groups in the embedded LDAP:

• Click Security Realms, myrealm and click on the Users and Groups tab.
• First create a new Group, for example, users.
• Next create a new User, for example, someone with password welcome1 and add this user to the users group.
• The appropriate mappings to the role SOMEROLE will be made in the deployment override weblogic.xml.

With the security view and the defined users and groups in place, the security configuration is finished. More information on WebLogic security and Java security in general can be found in the post Securing the WebLogic Server.

Deployment plan

When the deployment plan is created we have the following directory structure:

/example
	/app
		loadtest.ear
	/plan
		Plan.xml
	/plan/PersonModel.jar/META-INF
		weblogic-ejb-jar.xml
	/plan/PersonUI.war/WEB-INF
		weblogic.xml

Note that the /plan/PersonUI.war/WEB-INF directory is created automatically. The /plan/PersonModel.jar/META-INF directory must be created by hand (the same goes for the weblogic-ejb-jar.xml file). The contents of the weblogic.xml file look as follows:

<weblogic-web-app xmlns="http://xmlns.oracle.com/weblogic/weblogic-web-app" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/weblogic/weblogic-web-app http://xmlns.oracle.com/weblogic/weblogic-web-app/1.2/weblogic-web-app.xsd">
	<security-role-assignment>
		<role-name>SOMEROLE</role-name>
		<principal-name>users</principal-name>
	</security-role-assignment>
	<session-descriptor>
		<persistent-store-type>replicated_if_clustered</persistent-store-type>
	</session-descriptor>
</weblogic-web-app>

The security-role-assignment maps a role to a principle in this case the users group. Note that the application does not put anything critical in the HTTP session that needs to be replicated. Apart from the login information the application is stateless. In this case we can leave out the persistent-store-type option and decide to go for a share nothing architecture. The only thing that is lost when a server crashes is the login information, with the result that users needs to login again. If we really want to approach a linear scalability this is a serious option to consider.

The enterprise beans can be configured by using, for example,

<weblogic-ejb-jar xmlns="http://xmlns.oracle.com/weblogic/weblogic-ejb-jar" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/weblogic/weblogic-ejb-jar http://xmlns.oracle.com/weblogic/weblogic-ejb-jar/1.1/weblogic-ejb-jar.xsd">
	<weblogic-enterprise-bean>
		<ejb-name>Company</ejb-name>
		<enable-call-by-reference>True</enable-call-by-reference>
		<jndi-name>ejb/Company</jndi-name>
		<dispatch-policy>CompanyWorkManager</dispatch-policy>
	</weblogic-enterprise-bean>
	<work-manager>
		<name>CompanyWorkManager</name>
		<response-time-request-class>
			<name>CompanyResponseClass</name>
			<goal-ms>300</goal-ms>
		</response-time-request-class>
		<max-threads-constraint>
			<name>CompanyMaxThread</name>
			<pool-name>DataSource</pool-name>
		</max-threads-constraint>
		<min-threads-constraint>
			<name>CompanyMinThread</name>
			<count>20</count>
		</min-threads-constraint>
		<capacity>
			<name>CompanyCapacity</name>
			<count>100</count>
		</capacity>
		<work-manager-shutdown-trigger>
			<max-stuck-thread-time>600</max-stuck-thread-time>
			<stuck-thread-count>10</stuck-thread-count>
		</work-manager-shutdown-trigger>
	</work-manager>
</weblogic-ejb-jar>

Information with respect to work managers and configuring enterprise beans can be found in the post Configuring Enterprise Beans in WebLogic.

Class loading

Note that we have used JSTL in the login.jspx to this end we have packaged the jstl-api-1.2.jar and jstl-impl-1.2.jar into the WEB-INF/lib directory of the ear file. Class loading is discussed in detail in the posts Classloading: Making Hibernate work on WebLogic and Classloading and Application Packaging.

Now that we have everything needed for application in place, we can update the deployment, i.e.,

• Click deployment, select the application and click update.
• Choose redeploy this application, set the appropriate source path and deployment plan path and click finish.
• Now, we can click active changes.

Testing

We can test the environment by using the The Grinder. The following script calls the application and when it needs to authenticate, a username and password are send to the appropriate resource:

from net.grinder.script.Grinder import grinder
from net.grinder.script import Test
from net.grinder.plugin.http import HTTPRequest
from HTTPClient import NVPair

protectedResourceTest = Test(1, "Request resource")
authenticationTest = Test(2, "POST to login.jspx")
class TestRunner:
	def __call__(self):
		request = protectedResourceTest.wrap(HTTPRequest(url="http://172.31.0.106:7777/LoadTest/testservlet"))
		result = request.GET()
		result = maybeAuthenticate(result)
		result = request.GET()

def maybeAuthenticate(lastResult):
	if lastResult.statusCode == 401 or lastResult.text.find("login.jspx") != -1:
		grinder.logger.output("Challenged, authenticating")
		authenticationFormData = ( NVPair("j_username", "someone"), NVPair("j_password", "welcome1"))
		request = authenticationTest.wrap(HTTPRequest(url="%s/j_security_check" % lastResult.originalURI))
		return request.POST(authenticationFormData)

To get an idea of how the environment is performing we run a JRockit flight recording by using JRockit Mission Control which has the WebLogic plug-in enabled. More information regarding JRockit Mission Control can be found in the post Fast, Faster, JRockit. Steps to set-up the WebLogic plug-in in JRockit Mission Control can be found in the post Fun with JRockit. The WebLogic console also provides the diagnostics with respect to how the data source and the messaging is performing. More information regarding WebLogic diagnostics can be found in the post Performing Diagnostics in a WebLogic environment.

Start the test and create a flight recording by using the profile normal template and let it run for 30 minutes.

Results

Let us first look at some diagnostics collected by WebLogic. To this end, open the admin console, click deployments and click the monitoring tab. This page displays monitoring information for all applications deployed to the domain. The Web Applications tab shows the average, high, and total number of sessions:

/LoadTest -	Active Server Count: 2, Maximum Sessions on Any Server: 35968, Total Sessions: 143869

The JMS tab displays monitoring information for all JMS destinations (note that the application only sends a message when a new person is added):

JMSServer1!JMSServer1.TemporaryQueue0 - Bytes Received Count: 18770, Messages Received Count: 590
JMSServer2!JMSServer2.TemporaryQueue0 - Bytes Received Count: 18770, Messages Received Count: 590
SystemModule!JMSServer1@DistributedQueue - Bytes Received Count: 425387, Messages Received Count: 1388
SystemModule!JMSServer2@DistributedQueue - Bytes Received Count: 427521, Messages Received Count: 1395

The EJB (stateless and message-driven) tab displays monitoring information for all the Enterprise JavaBeans (EJBs):

Company	example - Pooled Beans Current Count: 4, Access Total Count: 71933, Transactions Committed Total Count: 71926, Transactions Rolled Back Total Count: 7
CompanyMDB example - Access Total Count: 2783, Processed Message Count: 2783, Transactions Committed Total Count: 2783

The JDBC tab displays monitoring information for all JDBC data sources:

DataSource Server1 - Active Connections High Count: 2, Connection Delay Time: 176, PrepStmt Cache Access Count: 41186, Reserve Request Count (cummulative running count of requests for a connection): 36349, (cumulative running count of requests for a connection that had to wait before getting a connection) 0
DataSource Server2 - Active Connections High Count: 2, Connection Delay Time: 176, PrepStmt Cache Access Count: 39286, Reserve Request Count (cummulative running count of requests for a connection): 36341, (cumulative running count of requests for a connection that had to wait before getting a connection) 0

The workload tab shows statistics for the Work Managers, constraints, and policies that are configured for application deployments:

default Server1 - Pending Requests: 0, Completed Requests: 109292
default Server2 - Pending Requests: 0, Completed Requests: 109297

To see if there were any hick-ups due to garbage collections, we can use the created flight recordings:

One thing to note is that the garbage collections did not run at same time:

• server1: 12:19:38 815ms – 12:19:39 005ms
• server2: 12:20:01 788ms – 12:20:02 044ms

In general, JVM instances running on the same machine will typically not run the garbage collection at the same time. This means that we will have a JVM available to process application requests on other available CPUs. This is an advantage of vertical scaling that leads to a higher application throughput.

To get some insight what effect the garbage collection had on the application we use the events environment of the flight recording. We enable the following events:

• Java application – Java blocked
• Java virtual machine – GC – garbage collection
• WebLogic Server – EJB – EJB business method invocation
• WebLogic Server – JDBC – JDBC statement execute
• WebLogic Server – Servlet – Servlet invocation

The thread group ‘thread group for queue: weblogic.socket.Muxer’ contain the muxer threads. As a native muxer is used and we have 2 CPUs available, the number of threads is equal to #CPUs + 1 = 3. These threads are always showing this behavior, i.e., one thread at a time is active; picking requests of the sockets and put it in the execute queue. Here it will be picked up by an execute thread from the thread group ‘pooled threads’, which will process it, i.e., the servlet invocation (light green), the EJB business method invocation (blue) and the JDBC statement execution (purple).

Looking at the statistics, we can conclude that WebLogic (and, of course, JRockit) did a fine job.

References

[1] Administration Console Online Help.
[2] WebLogic Server Documentation.