In this post we show a step-by-step example of how to set-up a high available tuned Java EE environment. We start with installing a Java Virtual Machine (in this case HotSpot) and GlassFish Server. We continue with configuring a domain and tune the Java Virtual Machine for the admin server (and the default environment). Subsequently, we show how to set-up a standalone instance on which we deploy an application that uses JBoss RichFaces and Coherence, show how to set-up security for the application and create a deployment plan that is used to configure the application. Next, we show the steps involved in setting up a cluster and configure a load balancer. Subsequently, we show how to create resources needed for Java messaging and database communication. Once the resources are configured, we deploy a Java EE application that uses these resources. Finally, when everything is in-place we create a benchmark to test the application.

Install GlassFish

We first create a user (on the operating system in this case RedHat Linux), for example, oracle. To create a user run the following commands under the root user:

• useradd oracle
• passwd magic11g

This creates a /home/oracle directory. To delete the user we can use: userdel -r oracle the option -r removes the /home/oracle directory.

Next, we adjust the /etc/hosts file. Use /sbin/ifconfig to retrieve the IP address, for example, eth0 – inet addr:172.31.0.149:

127.0.0.1       localhost.localdomain localhost
::1             localhost6.localdomain6 localhost6
172.31.0.149    edu-glf-rh

The name edu-glf-rh is our DNS-name that was mapped to 127.0.0.1, which gets an IP from DHCP (usually in server environments we have to make sure we have a fixed IP address).

Adjust the number of open file descriptors. As root add the following to /etc/security/limits.conf:

oracle soft nofile 8192
oracle hard nofile 8192

Execute the ulimit -u 8192 command. As we are using the Apache HTTP Server we need to install the following libraries:

yum install gcc
yum install gcc-c++

Log out and log in again, but now as the user oracle.

The following software versions will be used in the installation:

• HotSpot JVM – jdk-6u24-linux-x64.bin
• GlassFish Server – ogs-3.1.1-unix-ml.sh

To install HotSpot we follow these steps:

• Run in the /home/oracle directory the file jdk-6u24-linux-x64.bin (when necessary adjust the execute permissions, chmod ug=rwx <filename>).

To install GlassFish we follow these steps:

• Open a command shell and define JAVA_HOME, i.e., export JAVA_HOME=/home/oracle/jdk1.6.0_24.
• Start the installer (./ogs-3.1.1-unix-ml.sh).
• Click next in the welcome screen.
• Choose custom installation and click next.
• Choose the install only option and click next.
• Specify the install directory: /home/oracle/glassfish3 (which is the default) and click next.
• Choose the installed JDK (which is selected by default, if not then specify the directory /home/oracle/jdk1.6.0_24).
• Click next, deselect the update tool and click next.
• Click install and when the installation is finished click exit.

Configure a domain

A GlassFish domain is a logical grouping of server instances that are controlled through a domain administration server. When creating a domain our first step is to set-up the domain administration server. To this end we run the installer (./ogs-3.1.1-unix-ml.sh) again:

• Click next in the welcome screen.
• Choose custom installation and click next.
• Choose configure an existing installation and click next.
• Specify the install directory: /home/oracle/glassfish3 and click next.
• Choose create a server domain and click next.
• Enter the following parameters:
  • domain name: base_domain
  • admin port: 4848
  • http port: 8080
  • username: glassfish
  • password: magic11g
  • reenter password: magic11g
  • deselect start domain after creation
• Click next (domain is being configured).
• Click exit.

Note that the domain administration server is used to configure, manage and monitor servers in a domain. The domain administration server is a GlassFish server instance with extra applications deployed on it that provide administrative capabilities. Other GlassFish server instances (standalone instances and clustered instances) also contain extra applications that the domain administration server uses to send information to them. It further maintains an XML repository (domain.xml) in the ${DOMAIN_HOME}/config directory.

One thing to note is that the domain administration server is not clusterable and when it goes down, we cannot administer our domain. In general, we can just restart the admin server. But what if the machine on which the admin server runs fails? In this case we have to make sure that the files the domain administration server relies on (the admin server directory and the domain configuration files) are available on another machine, such that domain administration server can be restarted on this machine. In GlassFish terms a machine is represented by a node, i.e., a node is a host on which GlassFish Server software has been installed. For each machine that contains server instances, a host must exist. By creating a server domain a node by the name of localhost-<domain-name&tgt; is created automatically.

Apply a patch

When using the Apache HTTP Server and in particular mod_jk as a front-end for the GlassFish Server a NullPointerException is thrown. To this end we have to replace the grizzly-utils.jar in the directory /home/oracle/glassfish3/glassfish/modules (the old one can be removed) by a patched one that can be obtained here.

Getting acquainted

The asadmin utility (present in the /home/oracle/glassfish3/bin directory) can be used for administrative tasks as well as the lifecycle of the GlassFish Server. Note that GlassFish also provides an admin console that can be used. To start the domain we can use:

To stop the domain we can use:

[sourcecode language="text" wraplines="false"
./asadmin stop-domain base_domain

Waiting for the domain to stop ......
Command stop-domain executed successfully.

When the domain administration server is started the admin console can be reached at: http://hostname:4848. As mentioned before the admin console offers an alternative to the asadmin utility in order to configure a GlassFish domain.

The following gives some examples on how to retrieve various sorts of information by using the asadmin utility:

• ./asadmin list-commands - shows all the available commands.
• ./asadmin help <command> - shows the help for a specific command.
• ./asadmin version - shows the GlassFish Server version.
• ./asadmin list-applications [--type <type>] - shows the deployed applications.
• ./asadmin list-containers - shows the application containers.
• ./asadmin list-modules - shows all the available modules.
• ./asadmin show-component-status <application-name> - shows the status of a deployed application.
• ./asadmin uptime - show the time the domain administration server is running.
• ./asadmin list-jvm-options - shows the used JVM options.
• ./asadmin generate-jvm-report [--type summary] - generate a JVM report that contains information about threads, classes, memory and loggers for the domain administration server.
• ./asadmin list-threadpools <server-name> - shows the existing thread pools.

GlassFish maintains one or more thread pools. When GlassFish receives a request, the request is mapped to a request thread from a specific thread pool. The request subsequently processes the request and returns the result. This means we have to make sure that the resources are plenty. For example, when request threads use connections from a connection pool and there are more requests threads asking for connections than there are connections in the connection pool, we get some contention, i.e., a request thread has to wait when a connection becomes available. Note that this adds time to the response a user is waiting for, thus we have to make sure enough resources are available.

The domain configuration is maintained in the domain.xml file, located in the /home/oracle/glassfish3/glassfish/domains/base_domain/config directory The used Java version is maintained in the asenv.conf file (AS_JAVA=/home/oracle/jdk1.6.0_24), located in the /home/oracle/glassfish3/glassfish/config directory.

Tune the JVM

JVM parameters can be added by using the admin console:

• Open the admin console (http://hostname:4848).
• Click on server (Admin Server) and click on configuration: server-config.
• Click on the link JVM Settings and subsequently on the JVM options tab to add or adjust parameters.
• To make the changes active, the server needs to be restarted.

One thing to note is that GlassFish uses a so-called configuration on which a standalone server or cluster of servers is based. When creating a server the option is offered to choose a configuration (which is usually the default-config). In the same manner as described above we can adjust the JVM parameters of the default configuration, which can than be used as a basis for all the servers in the domain.

When using GlassFish (or any other application server for that matter) it is beneficial to tune for application throughput. GlassFish is a multi-threaded environment and to let this run as smoothly as possible we need to give the threads as much resources as possible, hence the choice for the throughput optimization configuration. In the post Tune the JVM that runs Coherence Revisited a detailed discussion is given on how to set up the JVM parameters if we want to tune the HotSpot to optimize for throughput, for example, we can use:

-server -Xms1024m -Xmx1024m -XX:NewRatio=2 -XX:+UseParallelGC -XX:ParallelGCThreads=2 -XX:MaxGCPauseMillis=200 -XX:GCTimeRatio=19 -XX:+UseParallelOldGC -XX:+UseTLAB -XX:LargePageSizeInBytes=2048k -XX:+UseLargePages

in which,

• -server: selects the JIT compiler.
• -Xms: sets the initial heap size.
• -Xmx: sets the maximum heap size.
• -XX:NewRatio=N: sets the young generation to heap size / (1 + N).
• -XX:+UseParallelGC: selects the parallel collector for minor collections.
• -XX:ParallelGCThreads: sets the number of garbage collector threads, i.e., the number of CPUs to be used when collecting garbage.
• -XX:MaxGCPauseMillis: sets the maximum pause time goal.
• -XX:GCTimeRatio=N: sets the throughput goal that is measured in terms of the time spent doing garbage collection versus the time spent outside of garbage collection.
• -XX:+UseParallelOldGC: selects the parallel collector for major collections.
• -XX:+UseTLAB: enables thread-local object allocation.
• -XX:LargePageSizeInBytes: sets the large page size used for the Java heap.
• -XX:+UseLargePages: use large page memory. Note that when the large pages option is enabled, additional operating system configuration is necessary. The steps involved are discussed in detail in the post Tune the JVM that runs Coherence in particular the section Call profiling and large pages.

We only have talked about tuning the JVM, note that there can be additional resources that need to be tuned.

Standalone instance that runs JBoss RichFaces and uses Coherence

In the post JBoss RichFaces, Facelets and Coherence on WebLogic we already showed how to build an application that uses JBoss RichFaces and integrate Coherence. When using GlassFish the JBoss configuration has to be adjusted a little, i.e., in the faces-config.xml we have to configure a org.ajax4jsf.application.AjaxViewHandler:

<faces-config xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-facesconfig_1_2.xsd" version="1.2">
    <application>
        <message-bundle>userinterface.resources.messagebundle</message-bundle>
        <locale-config>
            <supported-locale>en</supported-locale>
            <supported-locale>nl</supported-locale>
        </locale-config>
        <view-handler>org.ajax4jsf.application.AjaxViewHandler</view-handler>
    </application>
	...
</faces-config>

in the web.xml we have to configure the org.ajax4jsf.VIEW_HANDLERS property in we want to use facelets:

<web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" version="2.5">
    <context-param>
        <param-name>org.ajax4jsf.VIEW_HANDLERS</param-name>
        <param-value>com.sun.facelets.FaceletViewHandler</param-value>
    </context-param>
    <context-param>
        <param-name>javax.faces.DEFAULT_SUFFIX</param-name>
        <param-value>.xhtml</param-value>
    </context-param>
    <context-param>
        <param-name>javax.faces.STATE_SAVING_METHOD</param-name>
        <param-value>server</param-value>
    </context-param>
    <context-param>
        <param-name>org.richfaces.SKIN</param-name>
        <param-value>blueSky</param-value>
    </context-param>
    <context-param>
        <param-name>org.richfaces.CONTROL_SKINNING</param-name>
        <param-value>enable</param-value>
    </context-param>
    <filter>
        <filter-name>RichFaces Filter</filter-name>
        <filter-class>org.ajax4jsf.Filter</filter-class>
    </filter>
    <filter-mapping>
        <filter-name>RichFaces Filter</filter-name>
        <servlet-name>Faces Servlet</servlet-name>
        <dispatcher>REQUEST</dispatcher>
        <dispatcher>FORWARD</dispatcher>
        <dispatcher>INCLUDE</dispatcher>
    </filter-mapping>
    <servlet>
        <servlet-name>Faces Servlet</servlet-name>
        <servlet-class>javax.faces.webapp.FacesServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
    </servlet>
    <servlet-mapping>
        <servlet-name>Faces Servlet</servlet-name>
        <url-pattern>/faces/*</url-pattern>
    </servlet-mapping>
    <listener>
        <listener-class>com.sun.faces.config.ConfigureListener</listener-class>
    </listener>
    <listener>
        <listener-class>userinterface.listeners.ContextListener</listener-class>
    </listener>
    <security-constraint>
        <web-resource-collection>
            <web-resource-name>All</web-resource-name>
            <url-pattern>/faces/*</url-pattern>
        </web-resource-collection>
        <auth-constraint>
            <role-name>MANAGER</role-name>
            <role-name>EMPLOYEE</role-name>
        </auth-constraint>
        <user-data-constraint>
            <transport-guarantee>NONE</transport-guarantee>
        </user-data-constraint>
    </security-constraint>
    <login-config>
        <auth-method>FORM</auth-method>
        <form-login-config>
            <form-login-page>/login.jspx</form-login-page>
            <form-error-page>/login.jspx</form-error-page>
        </form-login-config>
    </login-config>
    <security-role>
        <role-name>MANAGER</role-name>
    </security-role>
    <security-role>
        <role-name>EMPLOYEE</role-name>
    </security-role>
</web-app>

To create a standalone instance we can use the following steps:

• In the admin console, click on standalone instances and click new.
• Enter the following parameters:
  • instance name: app-server1
  • node: localhost-base_domain
  • configuration: default config (make a copy of the selected configuration)
• Click OK.

When using asadmin we can use: ./asadmin create-instance --node localhost-base_domain app-server1. By leaving out --config a copy of the default-config configuration will be created by the name of app-server1-config. By using ./asadmin delete-instance app-server1 the instance can be deleted when necessary.

To start the server:

• In the admin console, click on standalone instances, select app-server1 and click start

Note that the listen port information can be found in the app-server1, properties environment. The HTTP listen port is set by using the HTTP_LISTENER_PORT property, for example, 28080.

To start and stop standalone instances by using asadmin, we can respectively use: ./asadmin start-instance app-server1 and ./asadmin stop-instance app-server1.

To deploy the application, we first create a new directory ${GLASSFISH_HOME}/deploy/application. In this directory we copy our WAR file. To deploy applications by using the admin console we can follow these steps:

• Click on standalone instances and on app-server1.
• Click on the applications tab and click deploy.
• Enter the following parameters:
  • location: choose local packaged file..., click browse files and navigate to the war file.
  • type: web application
  • context root: application
  • application name: application
  • status: enabled
  • targets: app-server1
• Click OK.

To deploy an application by using asadmin we can use: ./asadmin deploy --name application --contextroot application --enabled=true --target app-server1 /home/oracle/glassfish3/deploy/application/application.war. When the application is successfully deployed it can be reached at: http://hostname:28080/application/faces/overview.xhtml.

Logging

The admin server log files are located in the ${DOMAIN_HOME}/logs directory. The server instance log files are located in the ${NODE_HOME}/<server-instance-name>/logs directory (in our case ${NODE_HOME} is equal to /home/oracle/glassfish3/glassfish/nodes/localhost-base_domain and server-instance-name is app-server1).

By using the admin console log files can be viewed as follows:

• Click on standalone instances and choose a particular server listed in the table and click view log files.

Using the Apache HTTP Server as a front-end

General information about the Apache HTTP Server can be found here. When we want to use the Apache HTTP Server as a front-end and in particular mod_jk we have to enable the JK listener for the server in question. In order to accomplish this we follow these steps:

• In the admin console, click on standalone instances and choose app-server1.
• Click on the configuration link app-server1-config.
• Click on the network listeners link and choose http-listener-1.
• Enable the JK Listener option.

We also need to increase the max thread pool size of the servers for which we have configured the JK listener. The default is rather low, namely 5. We have to make the max thread pool size equal to the MaxClients directive configured on the Apache HTTP Server (usually it is something like MaxClients 150) otherwise threads will pile up in the GlassFish Server. To adjust the max thread pool size, we can follow these steps:

• In the admin console, open the tree configurations, app-server1-config, thread pools, http-thread-pool.
• Adjust the max thread pool size attribute and set this to 150.
• Click save.
• Restart the server.

Note that it is handy when we plan to add more servers to do this for the default configuration as well. Information about which thread pool is being used by a particular listener can be found by clicking: configurations, app-server1-config, network config, network listeners, http-listener-1. The thread pool attribute contains the thread pool that is used by the listener.

To install the Apache HTTP Server we can follow these steps:

• Unpack the httpd-2.2.21.tar.gz file:
  • gzip -d httpd-2.2.21.tar.gz
  • tar xvf httpd-2.2.21.tar
  • cd httpd-2.2.21
• The next step is to configure:
  • ./configure --prefix=/home/oracle/glassfish3/apache
• Next, compile the various parts for the Apache HTTP Server by using:
  • make
• To install the Apache HTTP Server we use:
  • make install
• Open the httpd.conf (/home/oracle/glassfish3/apache/conf) file and adjust the following directives:
  • Listen 7777
  • ServerName ip-address
• To test the set-up, start the Apache HTTP Server:
  • Navigate to /home/oracle/glassfish3/apache/bin.
  • Run: ./apachectl -k start (To stop the Apache HTTP Server we can use: ./apachectl -k stop).
  • Open a browser en type the following URL: http://<hostname>:7777.

To install the mod_jk module we can follow these steps:

• Unpack the tomcat-connectors-1.2.32-src.tar.gz file:
  • gzip -d tomcat-connectors-1.2.32-src.tar.gz
  • tar xvf tomcat-connectors-1.2.32-src.tar
  • cd tomcat-connectors-1.2.32-src/native
• The next step is to configure the module:
  • ./configure --with-apxs=/home/oracle/glassfish3/apache/bin/apxs
• Compile:
  • make
• In the native directory, a directory apache-2.0 is created. Here a mod_jk.so file is present. Copy this file to the directory: /home/oracle/glassfish3/apache/modules.

To configure the mod_jk module, create a new file mod_jk.conf in the directory /home/oracle/glassfish3/apache/conf. An example configuration looks as follows:

LoadModule jk_module /home/oracle/glassfish3/apache/modules/mod_jk.so

JkWorkersFile /home/oracle/glassfish3/apache/conf/glassfish-workers.properties

# where to put the log files for the jk module
JkLogFile /home/oracle/glassfish3/apache/logs/mod_jk.log

# the log level [debug|info|error]
JkLogLevel info

# log format
JkLogStampFormat "[%a %b %d %H:%M:%S %Y]"

# options to indicate to send SSL KEY SIZE
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories

# set the request log format
JkRequestLogFormat "%w %V %T"

# send requests that contain the following to glassfish
JkMount /application/* app-server1-worker

Next, we need to define the worker file that is specified by JkWorkersFile directive. In this case, we create a file called glassfish-workers.properties in the directory /home/oracle/glassfish3/apache/conf. An example worker configuration is the following:

# define a worker that uses ajp13
worker.list=app-server1-worker

# set properties for the worker
worker.app-server1-worker.type=ajp13
worker.app-server1-worker.host=172.31.0.149 (which is the IP-address where the GlassFish server is running)
worker.app-server1-worker.port=28080 (which is HTTP listen port of the GlassFish Server)

To let the Apache HTTP Server pick up the configuration we add the following to the httpd.conf file:

# put it near the end of the file where all the other includes are present
# mod_jk configuration
Include conf/mod_jk.conf

Restart the HTTP Server (./apachectl -k stop and ./apachectl -k start). The application can now be reached at: http://hostname:7777/application/faces/overview.xhtml. Note that by using the JkMount directive, we tell mod_jk to forward requests to a configured back-end server in our case: 172.31.0.149:28080.

Class loading

Next to the bootstrap classloader we have the following classloaders in a GlassFish environment:

• Extension (domain-dir/lib/ext) - contains the so-called optional packages.
• Public API (glassfish/modules) - jars needed for the JavaEE runtime, that can be used by deployed applications.
• Common (domain-dir/lib) - usually contains jars that contain database drivers.
• Applib (domain-dir/lib/applibs) - application specific class loading, loads classes that are specified during a deployment (classes that are loaded by this class loader overwrite the same classes loaded by the preceding class loaders).
• Archive - loads the classes contained by a WAR or an EAR file.

To experiment a little, say we exclude Coherence from our WAR deployment file. In this case we need to specify the coherence.jar in one of the classloaders. A good choice here is to let it be loaded by the Applib class loader, i.e., not all applications deployed will use Coherence. The class loaders (Extension, Public API and Common) load classes for the general environment. For example, if we want to configure connection pools for an Oracle database a good place for the Oracle database drivers is the Common class loader. Thus to let Coherence be loaded per application and the Oracle database drivers be generally loaded, we put the jars in the following directories:

${GLASSFISH_HOME}
	/glassfish
		/domains/base_domain
			/lib
				/applibs
					coherence.jar
				ojdbc6.jar
		/nodes/localhost-base_domain
			/lib
				/applibs
					coherence.jar
				ojdbc6.jar

To deploy the WAR file excluding Coherence, we can use the following steps:

• In the admin console, click on applications and click deploy.
• Enter the following parameters:
  • Choose Local Packaged File... and browse to the war file
  • type: web application
  • context root: application
  • application name: application
  • status: enabled
  • libraries: coherence.jar (here jar files can be referenced that are present in the ${NODE_HOME}/app-server1/lib/applibs directory)
  • targets: app-server1
• Click OK.

Security and deployment plans

To create user and groups in GlassFish, we follow these steps:

• In the admin console, open the tree configurations, app-server1-config, security.
• Click on realm and click file.
• Click manage users and subsequently new.
• Enter the following parameters:
  • user id: manager
  • group list: managers
  • new password: welcome1
  • confirm new password: welcome1
• Click OK.
• Create another user (employee) that belongs to the group employees.

By using asadmin we can check our configuration, for example,

./asadmin list-auth-realms app-server1
admin-realm
file
certificate

./asadmin list-file-users --authrealmname file app-server1
employee
manager

./asadmin list-file-groups --authrealmname file app-server1
file-group : managers
file-group : employees

In order to map the roles defined in the application to users or groups defined in the security realm, we need to use a deployment plan. When we have an enterprise application that has the following structure:

application.ear
	/META-INF
		application.xml
	EJB.jar
		/META-INF
			ejb-jar.xml
			persistence.xml
	WEB.war
		/WEB-INF
			web.xml

we can use the following deployment plan structure (note that deployment plans are packaged as jars):

plan.jar
	glassfish-application.xml (provides the override for application.xml)
	EJB.jar.glassfish-ejb-jar.xml (provides the override for ejb-jar.xml)
	WEB.war.glassfish-web.xml (provides the override for web.xml)

When deploying just a WAR file (as in our case), we only need the glassfish-web.xml file, i.e.,

application-deployment-plan.jar
	glassfish-web.xml

To map the application roles to groups in the security realm we can use the following as the contents for the glassfish-web.xml deployment override:

<!DOCTYPE glassfish-web-app PUBLIC "-//GlassFish.org//DTD GlassFish Application Server 3.1 Servlet 3.0//EN" "http://glassfish.org/dtds/glassfish-web-app_3_0-1.dtd">
<glassfish-web-app>
	<security-role-mapping>
        <role-name>MANAGER</role-name>
		<!--principal-name>manager</principal-name-->
        <group-name>managers</group-name>
    </security-role-mapping>
    <security-role-mapping>
        <role-name>EMPLOYEE</role-name>
        <group-name>employees</group-name>
    </security-role-mapping>
</glassfish-web-app>

The jar file for the deployment plan can be created by using, for example, jar cvf application-deployment-plan.jar glassfish-web.xml.

To deploy an application that uses a deployment plan, we need to use the asadmin command-line utility. First, we undeploy the application:

./asadmin list-applications --long=true app-server1

./asadmin disable --target app-server1 application

./asadmin undeploy --target app-server1 application

The command-line to deploy the application and the deployment plan is as follows:

./asadmin deploy --contextroot application --name application
    --deploymentplan /home/oracle/glassfish3/deploy/application/application-deployment-plan.jar
    --enabled=true --libraries coherence.jar
    --target app-server1 /home/oracle/glassfish3/deploy/application/application.war

To check if the configuration works open a browser and enter the URL for the application (http://hostname:7777/application/faces/overview.xhtml) and login by using the created credentials.

Clustering

In the example below we will create a cluster that is present on one host. In the case, we are dealing with a cluster that spans multiple hosts, the administrative operations executed by the domain administration server need a means to communicate with a remote host. When the instance on the remote host is already running the domain administration server can communicate directly. On the other hand when the instance needs to be started, communication must be with SSH (secure shell). To set-up SSH we can follow these steps:

• Edit the file /etc/ssh/sshd_config and set the parameters StrictModes and PubkeyAuthentication to yes.
• To determine if the SSH daemon (sshd) is running we can use: /sbin/service sshd status.
• /sbin/service sshd start can be used to start the SSH daemon.
• To test the set-up we can use: ssh -l user-name host-name, for example, ssh -l oracle 172.31.0.149.

When the cluster spans multiple hosts, we can use asadmin install-node host-list to install the GlassFish software on the other hosts. By using asadmin list-nodes all the nodes in the domain are displayed (including the type and hostname). The host type can be either ssh or config. A config node does not allow remote communication an ssh typed node does. To test if a particular node is reachable we can use: asadmin ping-node-ssh node-name.

GlassFish clusters use multicast to retrieve server health information (heartbeat messages). To check if multicast is available in our environment we can use:

./asadmin validate-multicast
Will use port 2048
Will use address 228.9.3.1
Will use bind interface null
Will use wait period 2,000 (in milliseconds)
Listening for data...
Sending message with content "edu-glf-rh" every 2,000 milliseconds
Received data from edu-glf-rh (loopback)
Exiting after 20 seconds. To change this timeout, use the --timeout command line option.

To create a cluster we can follow these steps:

• In the admin console, click cluster and subsequently new.
• Enter the following parameters:
  • name: app-cluster
  • configuration: default-config (make a copy)
  • add two servers: app-cluster-server1 en app-cluster-server2 (load balance weight - 100, node - localhost-base_domain)
• Click OK.

Adjust the network listener:

• In the admin console open the tree configurations, app-cluster-config, network config, network listeners and click http-listener-1.
• Enable the JK Listener option.
• Click save.

To start the cluster we can use either the admin console (clusters, select app-cluster and click start cluster) or the asadmin command-line utility:

./asadmin start-cluster --verbose=true app-cluster
start-instance app-cluster-server2
start-instance app-cluster-server1
The command start-instance executed successfully for: app-cluster-server2 app-cluster-server1
Command start-cluster executed successfully.

To retrieve the listen ports:

• In the admin console click clusters, app-cluster and the instances tab.
• Select a server instance and click on the properties tab (HTTP_LISTENER_PORT contains the listen port).

To retrieve properties by means of asadmin, we can use:

./asadmin get servers.server.app-cluster-server1.system-property.HTTP_LISTENER_PORT
servers.server.app-cluster-server1.system-property.HTTP_LISTENER_PORT.name=HTTP_LISTENER_PORT
servers.server.app-cluster-server1.system-property.HTTP_LISTENER_PORT.value=28082

./asadmin get servers.server.app-cluster-server2.system-property.HTTP_LISTENER_PORT
servers.server.app-cluster-server2.system-property.HTTP_LISTENER_PORT.name=HTTP_LISTENER_PORT
servers.server.app-cluster-server2.system-property.HTTP_LISTENER_PORT.value=28081

By using the following commands we can retrieve information regarding health, multicast and node information:

./asadmin list-clusters
app-cluster running

./asadmin get-health app-cluster
app-cluster-server1 started since Thu Dec 01 09:51:23 CET 2011
app-cluster-server2 started since Thu Dec 01 09:51:23 CET 2011

./asadmin get clusters.cluster.app-cluster
clusters.cluster.app-cluster.config-ref=app-cluster-config
clusters.cluster.app-cluster.gms-bind-interface-address=${GMS-BIND-INTERFACE-ADDRESS-app-cluster}
clusters.cluster.app-cluster.gms-enabled=true
clusters.cluster.app-cluster.gms-multicast-address=228.9.132.110
clusters.cluster.app-cluster.gms-multicast-port=29361
clusters.cluster.app-cluster.name=app-cluster

./asadmin list-nodes --long=true
NODE NAME                TYPE      NODE HOST    INSTALL DIRECTORY          REFERENCED BY
localhost-base_domain    CONFIG    localhost    /home/oracle/glassfish3    app-server1, app-cluster-server2, app-cluster-server1

Scaling

We have two scaling options: vertical and horizontal scaling. Vertical scaling relates to adding more CPUs to a machine. To better utilize the server hardware we can add more GlassFish instances to the machine that could lead to increased application throughput. To determine if this is indeed the case we need to benchmark. Benchmarking for scalability is about measuring resource utilization. Good scalability means that service levels can be maintained while the workload is increased. If an application does not scale well, it is not fully utilizing the hardware. Consequently, throughput will degrade. Ideally, a linear load increase should lead to a linear degradation in service levels and performance. Linear scalabilty can be approached when so-called share nothing clusters are used. The nodes provide the same functionality and know nothing about other nodes in the cluster (no HTTP session replication). In this case, the computing ability of the cluster increases almost linearly as more nodes are added to the cluster, if the back-end information systems, such as a database, are powerful enough.

Applications that 'share nothing' are usually sharing state through the database. The application-tier can scale as far as when the database becomes a bottleneck. In general, relying on a single shared resource will eventually cause contention for that resource and thus limit the scalability. Caching is a good resolution. When we cache data at the application-tier we avoid calls to the database (and also avoid relational data to object data conversions). When using a cluster of server instances we need to maintain multiple caches; this is not a problem for read-only data but for read/write data it is. Caching solutions, such as Coherence, provide different kind of caching, i.e., replicated and partitioned. Replicated does not scale well when cache writes are involved as the data needs to be replicated across all the nodes in the grid. A partioned cache, on the other hand, scales very well when cache writes are involved as data ownership is spread throughout the cluster (the system automatically rebalances the data when the number of nodes in the grid changes - we do not need to decide on how to partition the data, it comes out of the box). Another plus is that access to the cache means at most one network trip, this in order to maintain linear scalability. An optimization on read-access can be made when data can be obtained locally (sticky access) in this case a hybrid solution such as the near cache can be applied.

Horizontal scaling relates to adding more machines to the environment, which gives a failover capability that we cannot get with vertical scaling. A good approach is to combine both scaling techniques to obtain better CPU utilization and failover capability.

To add an extra machine to our cluster we can use the following steps:

• Configure ssh
  • yum install openssh-server
  • yum install openssh-clients
  • chkconfig sshd on
  • service sshd start
  • Test is ssh is working from the other host

    [glassfish@glassfish1 bin]$ ssh glassfish@glassfish2.com
    The authenticity of host 'glassfish2.com (192.168.1.160)' can't be established.
    RSA key fingerprint is a6:cd:73:87:2f:f3:96:1e:16:34:23:a8:51:5e:69:97.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added 'glassfish2.com,192.168.1.160' (RSA) to the list of known hosts.
    glassfish@glassfish2.com's password:
    [glassfish@glassfish2 ~]$ exit
    logout
    Connection to glassfish2.com closed.
    [glassfish@glassfish1 bin]$ ssh -l glassfish glassfish2.com
    glassfish@glassfish2.com's password:
    Last login: Fri Mar 29 13:42:46 2013 from glassfish1.com
    [glassfish@glassfish2 ~]$
    

• Install the JVM and GlassFish on both machines.
• Create a domain on host GlassFish1.
• Start the domain (asadmin start-domain base_domain).
• Enable remote administration (asadmin enable-secure-admin).
• Restart the domain (asadmin stop-domain base_domain and asadmin start-domain base_domain).
• Check if the remote administration is working, for example by using

  [glassfish@glassfish2 bin]$ ./asadmin --host glassfish1.com --port 4848 list-instances
  [
  [
    Version: V3
    Subject: CN=glassfish1.com, OU=GlassFish, O=Oracle Corporation, L=Santa Clara, ST=California, C=US
    Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11
  
    Key:  Sun RSA public key, 2048 bits
    modulus: 23240487021714962896424948633607969649221492541901259278251354022217316690568190613146558081413942287422551011017205757573869498943009098808161810308175849714018499282054244250620744466891501116668156575322374884073512287758514196369461357936242418926377510601714431282318816590915749319224521312843683532542085392111709672784400132041769359346688114343332995853786148444245754014425308575464844915206464756572993994428879805996793446709630188793718353028158966848655646660605205712353399783441346022509600162844812962400421138671723296014926498127250905535360169590799349586873306206634176842832235319451743899186797
    public exponent: 65537
    Validity: [From: Fri Mar 29 13:07:29 CET 2013,
                 To: Mon Mar 27 14:07:29 CEST 2023]
    Issuer: CN=glassfish1.com, OU=GlassFish, O=Oracle Corporation, L=Santa Clara, ST=California, C=US
    SerialNumber: [    41f25b98]
  
  Certificate Extensions: 1
  [1]: ObjectId: 2.5.29.14 Criticality=false
  SubjectKeyIdentifier [
  KeyIdentifier [
  0000: C0 69 98 7C 3A FB 96 61   D2 42 35 74 12 8C 6F A0  .i..:..a.B5t..o.
  0010: DB D1 0B 09                                        ....
  ]
  ]
  
  ]
    Algorithm: [SHA256withRSA]
    Signature:
  0000: AC 75 86 D6 43 24 0B AB   52 0F C5 BC 4A 38 6E 76  .u..C$..R...J8nv
  0010: A8 A5 E3 20 34 AE 13 3A   45 AA 27 AB 93 30 89 1A  ... 4..:E.'..0..
  0020: A8 96 33 B1 BB 6A E7 16   44 20 A0 2D 82 0A 0E 42  ..3..j..D .-...B
  0030: CF 95 DB 86 66 75 17 7F   F0 0B D5 F7 50 19 42 C1  ....fu......P.B.
  0040: F3 03 BD 76 18 B2 42 BD   CA B6 CF 32 83 3C CF C3  ...v..B....2.<..
  0050: EE BD 5D B9 A3 9E 4E E3   B9 4D E7 C4 6F 0F C0 AA  ..]...N..M..o...
  0060: 0F 84 54 0F 5C 66 9A CC   96 69 77 34 31 4A 27 6B  ..T.\f...iw41J'k
  0070: 6D 8B 7B A7 19 95 62 B0   56 7A 0F 7E 8D EA ED E4  m.....b.Vz......
  0080: D8 3D 76 CD BB 59 D8 5C   EC 5E 70 9D 67 D9 48 94  .=v..Y.\.^p.g.H.
  0090: 39 66 74 4A A5 62 A5 33   19 38 96 22 46 8A CC FD  9ftJ.b.3.8."F...
  00A0: C3 D6 79 46 29 30 6D 0B   3B E9 A9 9E 39 26 7E F1  ..yF)0m.;...9&..
  00B0: A3 11 8D 86 4B 74 10 FD   EA 3C 22 39 F7 16 93 5F  ....Kt...<"9..._
  00C0: 8C 37 62 E7 DB 02 8E 02   ED 49 48 DC 53 92 D3 59  .7b......IH.S..Y
  00D0: 22 21 EF AE DE 12 8C BD   31 0A C1 FB 5F BD 32 4A  "!......1..._.2J
  00E0: A9 E3 7C 32 9C FB B8 E4   60 F9 DF D7 7E 19 AC D3  ...2....`.......
  00F0: 3D D3 31 BC 06 7A C9 01   B1 AC 6A 38 F8 E8 5E EA  =.1..z....j8..^.
  
  ]
  Do you trust the above certificate [y|N] -->y
  Enter admin user name>  glassfish
  Enter admin password for user "glassfish">
  Nothing to list.
  Command list-instances executed successfully.
  

• Create a cluster, and create two instances, one on the local node and one on the remote host
  • In the admin console, click nodes, new. Enter a name and select SSH as the type. Next enter the hostname in the node host field, and set the SSH settings, i.e., select password as the SSH user authentication and set the SSH user name (the user under which the GlassFish Server is installed) and SSH password belonging to the user.
  • Click cluster, new. Enter a name and select a configuration.
  • Next click the created cluster, click the instances tab and click new. Enter a name and select a host. To create an instance we can also use asadmin --host glassfish1.com --port 4848 create-local-instance --cluster test-cluster test-server-glassfish2.

    [glassfish@glassfish2 bin]$ ./asadmin --host glassfish1.com --port 4848 create-local-instance --cluster test-cluster test-server-glassfish2
    Enter admin user name>  glassfish
    Enter admin password for user "glassfish">
    Rendezvoused with DAS on glassfish1.com:4848.
    Port Assignments for server instance test-server-glassfish2:
    JMX_SYSTEM_CONNECTOR_PORT=28686
    JMS_PROVIDER_PORT=27676
    HTTP_LISTENER_PORT=28080
    ASADMIN_LISTENER_PORT=24848
    JAVA_DEBUGGER_PORT=29009
    IIOP_SSL_LISTENER_PORT=23820
    IIOP_LISTENER_PORT=23700
    OSGI_SHELL_TELNET_PORT=26666
    HTTP_SSL_LISTENER_PORT=28181
    IIOP_SSL_MUTUALAUTH_PORT=23920
    Command create-local-instance executed successfully.
    

Start the servers to test if it is working

Configure load balancer

The load balancer decides how to dispatch requests to the back end server instances. A load balancer needs to do tasks such as distributing requests, health checking and session binding. As these are simple jobs it is rare that a load balancer will become a bottleneck. To load balance requests, we are going to use mod_jk. To configure mod_jk, open the mod_jk.conf file and add the following contents:

LoadModule jk_module /home/oracle/glassfish3/apache/modules/mod_jk.so

JkWorkersFile /home/oracle/glassfish3/apache/conf/glassfish-workers.properties

# where to put the log files for the jk module
JkLogFile /home/oracle/glassfish3/apache/logs/mod_jk.log

# the log level [debug|info|error]
JkLogLevel info

# log format
JkLogStampFormat "[%a %b %d %H:%M:%S %Y]"

# options to indicate to send SSL KEY SIZE
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories

# set the request log format
JkRequestLogFormat "%w %V %T"

# send requests that contain the following to glassfish
JkMount /application/* app-server1-worker

JkMount /LoadTest/* loadbalancer

The workers (glassfish-workers.properties) are defined as follows:

# define a worker that uses ajp13
worker.list=app-server1-worker,app-cluster-server1-worker,app-cluster-server2-worker,loadbalancer

# set properties for the worker
worker.app-server1-worker.type=ajp13
worker.app-server1-worker.host=172.31.0.149
worker.app-server1-worker.port=28080

# set properties for the app-cluster-server1-worker
worker.app-cluster-server1-worker.type=ajp13
worker.app-cluster-server1-worker.host=172.31.0.149
worker.app-cluster-server1-worker.port=28082
worker.app-cluster-server1-worker.lbfactor=1
worker.app-cluster-server1-worker.socket_keepalive=1
worker.app-cluster-server1-worker.socket_timeout=300

# set properties for the app-cluster-server2-worker
worker.app-cluster-server2-worker.type=ajp13
worker.app-cluster-server2-worker.host=172.31.0.149
worker.app-cluster-server2-worker.port=28081
worker.app-cluster-server2-worker.lbfactor=1
worker.app-cluster-server2-worker.socket_keepalive=1
worker.app-cluster-server2-worker.socket_timeout=300

# set properties for the loadbalancer
worker.loadbalancer.type=lb
worker.loadbalancer.balance_workers=app-cluster-server1-worker,app-cluster-server2-worker

We can also choose to use the GlassFish Load Balancer. We can use the same steps to configure Apache as presented above (./configure --with-mpm=worker --with-included-apr --prefix=apache_install_path which also enables multithreading support). Once Apache is installed we can use the load balancer configurer (that can be downloaded here). Run java -jar glassfish-lbconfigurator-3_1_1.jar to start the configuration and follow the instructions on the screen. When the configuration is finished we need to edit the loadbalancer.xml.example (located in the ${APACHE_HOME}/conf directory and rename it to loadbalancer.xml). An example configuration looks as follows

<!DOCTYPE loadbalancer PUBLIC "-//GlassFish.org//DTD GlassFish Application Server 3.1//EN" "glassfish-loadbalancer_1_3.dtd">
<loadbalancer>
    <cluster name="test-cluster"  policy="round-robin">
        <instance  name="test-server-glassfish1" enabled="true" disable-timeout-in-minutes="60" listeners="http://192.168.1.155:28080"  weight="100"/>
        <instance  name="test-server-glassfish2" enabled="true" disable-timeout-in-minutes="60" listeners="http://192.168.1.160:28080"  weight="100"/>
        <web-module context-root="SessionTest" enabled="true" disable-timeout-in-minutes="60" error-url="sun-http-lberror.html" />
        <health-checker url="/" interval-in-seconds="10" timeout-in-seconds="30" />
    </cluster>
    <property name="reload-poll-interval-in-seconds" value="60"/>
    <property name="response-timeout-in-seconds" value="30"/>
    <property name="https-routing" value="false"/>
    <property name="require-monitor-data" value="false"/>
    <property name="active-healthcheck-enabled" value="false"/>
    <property name="number-healthcheck-retries" value="3"/>
    <property name="rewrite-location" value="true"/>
</loadbalancer>

Note that when using the GlassFish Load Balancer we do not need to enable the JK Listener on the Network Listener.

Configure resources

The application we will deploy to the cluster uses Java Persistence and Java Messaging, i.e., we have the following persistence.xml file

<persistence xmlns="http://java.sun.com/xml/ns/persistence" version="1.0">
    <persistence-unit name="PersonPersistenceUnit">
        <provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>
        <jta-data-source>jdbc/exampleDS</jta-data-source>
        <class>model.entities.Person</class>
    </persistence-unit>
</persistence>

Note that EclipseLink is the default persistent provider on GlassFish, so no further jar files need to be added to the classpath. We have the following stateless enterprise bean:

package model.logic;

import model.entities.Person;

import javax.annotation.PostConstruct;
import javax.annotation.PreDestroy;
import javax.annotation.Resource;
import javax.ejb.Stateless;
import javax.jms.*;
import javax.persistence.EntityManager;
import javax.persistence.PersistenceContext;

@Stateless(name = "Company", mappedName = "ejb/Company")
public class CompanyBean implements Company {

    @PersistenceContext(unitName = "PersonPersistenceUnit")
    private EntityManager entityManager;

    @Resource(name = "jms/ConnectionFactory", mappedName = "jms/ConnectionFactory", type = ConnectionFactory.class)
    private ConnectionFactory connectionFactory;

    @Resource(name = "jms/CompanyQueue", mappedName = "jms/CompanyQueue", type = Queue.class)
    private Queue destination;

    private Connection connection = null;
    private Session session = null;
    private MessageProducer messageProducer = null;

    @PostConstruct
    public void init() {
        try {
            connection = connectionFactory.createConnection();
            session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
            messageProducer = session.createProducer(destination);
        } catch (JMSException e) {
            if (connection != null) {
                try {
                    connection.close();
                } catch (JMSException f) {
                    e.printStackTrace();
                }
            }
        }
    }

    @PreDestroy
    public void release() {
        try {
            if (connection != null) {
                connection.close();
            }
        } catch (JMSException e) {
            e.printStackTrace();
        }
    }

    public void insertPerson(Person person) {
        if (findPerson(person.getSofinummer()) == null) {
            entityManager.persist(person);
            sendMessage(person);
        } else {
            updatePerson(person);
        }
    }

    public void removePerson(Integer sofinummer) {
        Person person = findPerson(sofinummer);
        if (person != null) {
            entityManager.remove(entityManager.merge(person));
            sendMessage(person);
        }
    }

    public void updatePerson(Person person) {
        entityManager.merge(person);
    }

    private Person findPerson(Integer sofinummer) {
        return entityManager.find(Person.class, sofinummer);
    }

    private void sendMessage(Person person) {
        try {
            ObjectMessage message = session.createObjectMessage();
            message.setObject(person);
            messageProducer.send(message);
        } catch (JMSException e) {
            e.printStackTrace();
        }
    }
}

and the following message-driven bean:

package model.logic;

import model.entities.Person;

import javax.ejb.ActivationConfigProperty;
import javax.ejb.MessageDriven;
import javax.jms.*;

@MessageDriven(mappedName = "jms/CompanyQueue", activationConfig = {
        @ActivationConfigProperty(propertyName = "acknowledgeMode", propertyValue = "Auto-acknowledge"),
        @ActivationConfigProperty(propertyName = "destination", propertyValue = "CompanyQueue"),
        @ActivationConfigProperty(propertyName = "destinationType", propertyValue = "javax.jms.Queue")
})
public class CompanyMDB implements MessageListener {

    public void onMessage(Message message) {
        if (message instanceof ObjectMessage) {
            ObjectMessage objectMessage = (ObjectMessage) message;
            try {
                System.out.println("RECEIVED OBJECT MESSAGE " + objectMessage.getObject());
            } catch (JMSException e) {
                e.printStackTrace();
            }
        }

        if (message instanceof TextMessage) {
            TextMessage textMessage = (TextMessage) message;
            try {
                System.out.println("RECEIVED TEXT MESSAGE " + textMessage.getText());
            } catch (JMSException e) {
                e.printStackTrace();
            }
        }
    }
}

Three resources need thus to be configured:

• JDBC data source with JNDI-name jdbc/exampleDS.
• JMS connection factory with JNDI-name jms/ConnectionFactory.
• JMS queue with JNDI-name jms/CompanyQueue.

To create a data source we can follow these steps:

• In de admin console, open the tree resources, JDBC and click JDBC connection pools.
• Click new and enter the following parameters:
  • pool name: LoadTestConnectionPool
  • resource type: javax.sql.DataSource
  • database vendor: Oracle
• Click next and enter the following parameters:
  • url: jdbc:oracle:thin:@hostname:1521:SID
  • password: schema password
  • user: schema user
• Click finish.
• Click in the connection pool table on LoadTestConnectionPool and click ping.
• Open the tree resources, JDBC and click JDBC resources.
• Click new and enter the following parameters:
  • JNDI name: jdbc/exampleDS
  • pool name: LoadTestConnectionPool
  • status: enabled
  • targets: app-cluster

To create the JMS resources we can follow these steps:

• Open the tree resources, JMS resources and click on connection factories.
• Click new and enter the following parameters:
  • pool name: jms/ConnectionFactory
  • resource type: javax.jms.QueueConnectionFactory
  • targets: app-cluster
• Click OK.
• Click destination resources and new.
• Enter the following parameters:
  • JNDI name: jms/CompanyQueue
  • physical destination name: CompanyQueue
  • resource type: javax.jms.Queue
  • targets: app-cluster
• Click OK.

Now, that the resources are in place we can deploy the application. Create a new directory ${GLASSFISH_HOME}/deploy/loadtest and copy the EAR file to this directory. To deploy the application we can follow these steps:

• In the admin console click on the applications link and click deploy.
• Enter the following parameters:
  • Choose Local Packaged File... and browse to the EAR file
  • type: enterprise application
  • status: enabled
  • java web start: disabled
  • targets: app-cluster
• Click OK.

To test the deployment enter the following URL: http://hostname:7777/LoadTest/testservlet. Check in the database if a person has been added. In the server log file (${NODE_HOME}/app-server1/logs) we can check if the JMS system has processed the message: 2011-11-30T14:13:55.601+0100 | INFO | oracle-glassfish3.1.1 | javax.enterprise.system.std.com.sun.enterprise.server.logging | _ThreadID=19; _ThreadName=Thread-2; | RECEIVED OBJECT MESSAGE ifdfcjecgh9d 1911.

Monitoring

To get an idea how the application is performing, we can enable some monitoring:

• In the admin console, click configuration, app-cluster-config, monitoring.
• Enable monitoring for the web container, thread pool, JMS service, JDBC connection pool and EJB container components, for example, by setting the monitoring level to high.
• Click clusters, app-cluster and select one of the server instances.

Request statistics can be found in the applications and server environments. Statistics regarding the JDBC connection pool can be found in the resources environment. Statistics for the JMS resources can be found the JMS physical destinations environment of the cluster (statistics column in the destinations table).

Start and stop scripts

An example of a script that starts the environment is the following:

#!/bin/bash

GLASSFISH_HOME="/home/oracle/glassfish3"
export GLASSFISH_HOME

APACHE_HOME="${GLASSFISH_HOME}/apache/"
export APACHE_HOME

start_apache() {
	echo "Starting Apache HTTP Server"
	${APACHE_HOME}/bin/apachectl -k start
}

start_glassfish_das() {
	if [ -z "$1" ]
	then
		echo "no parameters, starting glassfish DAS for domain: base_domain"
		${GLASSFISH_HOME}/bin/asadmin start-domain base_domain
	else
		echo "starting glassfish DAS for domain: $1"
		${GLASSFISH_HOME}/bin/asadmin start-domain $1
	fi
}

start_glassfish_instance() {
	if [ -z "$1" ]
	then
		echo "no parameters, starting glassfish instance: app-server1"
		${GLASSFISH_HOME}/bin/asadmin start-instance app-server1
	else
		echo "starting glassfish instance: $1"
		${GLASSFISH_HOME}/bin/asadmin start-instance $1
	fi
}

start_glassfish_cluster() {
	if [ -z "$1" ]
	then
		echo "no parameters, starting glassfish cluster: app-cluster"
		${GLASSFISH_HOME}/bin/asadmin start-cluster --verbose=true app-cluster
	else
		echo "starting glassfish cluster: $1"
		${GLASSFISH_HOME}/bin/asadmin start-cluster --verbose=true $1
	fi
}

start_glassfish_das base_domain

start_glassfish_instance app-server1

start_glassfish_cluster app-cluster

start_apache

An example to stop the environment is the following:

#!/bin/bash

GLASSFISH_HOME="/home/oracle/glassfish3"
export GLASSFISH_HOME

APACHE_HOME="${GLASSFISH_HOME}/apache/"
export APACHE_HOME

stop_apache() {
	echo "Stopping Apache HTTP Server"
	${APACHE_HOME}/bin/apachectl -k stop
}

stop_glassfish_das() {
	if [ -z "$1" ]
	then
		echo "no parameters, stopping glassfish DAS for domain: base_domain"
		${GLASSFISH_HOME}/bin/asadmin stop-domain base_domain
	else
		echo "stopping glassfish das DAS domain: $1"
		${GLASSFISH_HOME}/bin/asadmin stop-domain $1
	fi
}

stop_glassfish_instance() {
	if [ -z "$1" ]
	then
		echo "no parameters, stopping glassfish instance: app-server1"
		${GLASSFISH_HOME}/bin/asadmin stop-instance app-server1
	else
		echo "stopping glassfish instance: $1"
		${GLASSFISH_HOME}/bin/asadmin stop-instance $1
	fi
}

stop_glassfish_cluster() {
	if [ -z "$1" ]
	then
		echo "no parameters, stopping glassfish cluster: app-cluster"
		${GLASSFISH_HOME}/bin/asadmin stop-cluster --verbose=true app-cluster
	else
		echo "stopping glassfish cluster: $1"
		${GLASSFISH_HOME}/bin/asadmin stop-cluster --verbose=true $1
	fi
}

stop_apache

stop_glassfish_cluster app-cluster

stop_glassfish_instance app-server1

stop_glassfish_das base_domain

Load test

We can test the environment by using the The Grinder. The following script calls the application:

from net.grinder.script.Grinder import grinder
from net.grinder.script import Test
from net.grinder.plugin.http import HTTPRequest

test1 = Test(1, "Request resource")
request1 = test1.wrap(HTTPRequest())

class TestRunner:
    def __call__(self):
        result = request1.GET("http://172.31.0.149:7777/LoadTest/testservlet")

Edit the grinder.properties (${GRINDER_HOME}/examples) file such that the script is used, for example,

# The file name of the script to run. Relative paths are evaluated from the directory containing the properties file.
grinder.script = test.py

# The number of worker processes each agent should start.
grinder.processes = 1

# The number of worker threads each worker process should start.
grinder.threads = 1

# The number of runs each worker process will perform. When using the console set this to 0, i.e., the runs are controlled by the console.
grinder.runs = 0

Use the following scripts to start a test:

setGrinderEnv.sh

#!/bin/sh
GRINDERPATH=/home/oracle/grinder-3.4
export GRINDERPATH

JAVA_HOME=/home/oracle/jdk1.6.0_24
export JAVA_HOME

USER_MEM_ARGS="-server -Xms512m -Xmx512m -XX:NewRatio=2 -XX:+UseParallelGC -XX:ParallelGCThreads=2 -XX:MaxGCPauseMillis=200 -XX:GCTimeRatio=19 -XX:+UseParallelOldGC"
export USER_MEM_ARGS
GRINDERPROPERTIES=${GRINDERPATH}/examples/grinder.properties
export GRINDERPROPERTIES
CLASSPATH=${GRINDERPATH}/lib/grinder.jar:${CLASSPATH}
export CLASSPATH

PATH=${JAVA_HOME}/bin:${PATH}
export PATH

startConsole.sh

#!/bin/sh
source setGrinderEnv.sh
java ${USER_MEM_ARGS} -cp ${CLASSPATH} net.grinder.Console

startAgent.sh

#!/bin/sh
source setGrinderEnv.sh
java ${USER_MEM_ARGS} -cp ${CLASSPATH} net.grinder.Grinder ${GRINDERPROPERTIES}

First start the startConsole.sh script and then start the startAgent.sh script. The agent will now wait for a start action from the console (click the action tab and then choose start processes).

Use the GlassFish admin console to see how the requests are handled. By using jvisualvm or jconsole the JVM can be monitored.

References

[1] GlassFish Server Documentation