Coherence*Web is an HTTP session management module dedicated to managing session state in clustered environments. Built on top of Coherence, Coherence*Web: brings Coherence data grid’s data scalability, availability, reliability, and performance to in-memory session management and storage; allows storage of session data outside of the Java EE application server, freeing application server heap space and enabling server restarts without session data loss; enables session sharing and management across different Web applications, domains and application servers. More information can be found in the post Setting-up a WebLogic Cluster that uses Coherence.

ActiveCache is employed by applications running on WebLogic Server and provides replicated and distributed caching services that make an application’s data available to servers in a Coherence data cluster. ActiveCache provides direct access by applications to data caches, either through resource injection or component-based JNDI lookup, and lets us display, monitor, create, and configure Coherence clusters using the WebLogic Server Administration Console and WLST.

So far the commercial side of this post. Now let us first set-up an application that puts some data into the HttpSession. To this end we will use the following servlet

package userinterface.servlets;

import model.entities.Klant;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.servlet.http.HttpSession;
import java.io.IOException;
import java.io.PrintWriter;
import java.util.Enumeration;
import java.util.Random;

public class TestServlet extends HttpServlet {

    private Random generator;

    @Override
    public void init() throws ServletException {
        generator = (Random)getServletContext().getAttribute("generator");
    }

    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        HttpSession session = request.getSession(false);

        // generate a random client
        Klant klant = createKlant();

        // insert a client
        session.setAttribute(klant.getKlantnummer().toString(), klant);
        if (generator.nextDouble() < 0.001) {
            // remove a client
            session.removeAttribute(generateKlantNummer().toString());
        } else {
            // find a client by ID
            session.getAttribute(generateKlantNummer().toString());
        }

        PrintWriter writer = response.getWriter();
        Enumeration<String> clientids = session.getAttributeNames();
        while (clientids.hasMoreElements()) {
            writer.println(session.getAttribute(clientids.nextElement()));
        }
        writer.flush();
        writer.close();
    }

    private Klant createKlant() {
        int klantnummer = generateKlantNummer();

        Klant klant = new Klant();
        klant.setKlantnummer(klantnummer);
        klant.setNaam("Middleware" + klantnummer);
        klant.setAdres("Magic");
        klant.setStad("Pune");
        klant.setProvincie("IN");
        klant.setPostcode("1234AB");
        klant.setGebied(1);
        klant.setTelefoonnummer("123-4567");
        klant.setReputatieNummer(1);
        klant.setKredietlimiet(Math.rint(generator.nextDouble() * 5000.0));
        klant.setCommentaar(Long.toString(Math.abs(generator.nextLong()), 36));

        return klant;
    }

    private Integer generateKlantNummer() {
        return generator.nextInt(10000);
    }
}

and the following web configuration

<web-app ...>
    <listener>
        <listener-class>userinterface.listeners.ContextListener</listener-class>
    </listener>
    <servlet>
        <servlet-name>TestServlet</servlet-name>
        <servlet-class>userinterface.servlets.TestServlet</servlet-class>
    </servlet>
    <servlet-mapping>
        <servlet-name>TestServlet</servlet-name>
        <url-pattern>/test</url-pattern>
    </servlet-mapping>
    <security-constraint>
        <web-resource-collection>
            <web-resource-name>All</web-resource-name>
            <url-pattern>/test</url-pattern>
        </web-resource-collection>
        <auth-constraint>
            <role-name>MANAGER</role-name>
            <role-name>EMPLOYEE</role-name>
        </auth-constraint>
        <user-data-constraint>
            <transport-guarantee>NONE</transport-guarantee>
        </user-data-constraint>
    </security-constraint>
    <login-config>
        <auth-method>FORM</auth-method>
        <form-login-config>
            <form-login-page>/login.jsp</form-login-page>
            <form-error-page>/login.jsp</form-error-page>
        </form-login-config>
    </login-config>
    <security-role>
        <role-name>MANAGER</role-name>
    </security-role>
    <security-role>
        <role-name>EMPLOYEE</role-name>
    </security-role>
</web-app>

which secures the servlet URL (and enables us to set-up a session). Refer to the posts WebLogic 12c in Action and Deploy WebLogic12c to Multiple Machines for more information (and WLST scripts) on setting-up a WebLogic cluster. To deploy the application, we create the following directory structure

/testactivecache
	/app
		TestActiveCache.war
	/plan
		Plan.xml
		/WEB-INF
			weblogic.xml

Here weblogic.xml has the following contents

<weblogic-web-app ...>
	<security-role-assignment>
		<role-name>EMPLOYEE</role-name>
		<principal-name>employees</principal-name>
	</security-role-assignment>
	<security-role-assignment>
		<role-name>MANAGER</role-name>
		<principal-name>managers</principal-name>
	</security-role-assignment>
	<session-descriptor>
		<persistent-store-type>replicated_if_clustered</persistent-store-type>
	</session-descriptor>
</weblogic-web-app>

in which we map the EMPLOYEE and MANAGER roles (defined in the application) to groups (employees and managers) created in the WebLogic console. We set the persistent-store-type to replicated_if_clustered, i.e., if the Web application is deployed on a clustered server, the in-effect persistent-store-type will be replicated.

Let us test the session fail-over. We start with two servers in the cluster and issue requests as two different users. By using the cluster, monitoring environment in the WebLogic Admin console, we can see that cluster-server1 is the holder of the primary sessions and cluster-server2 holds the secundairy sessions

Next, we shutdown cluster-server1

Issue some requests (and notice that the session has failed-over to cluster-server2) and we can continue working as if nothing has happened. Bring up cluster-server1 again

Now cluster-server2 is the holder of the primary sessions and cluster-server1 holds the secundairy sessions. Note that the secundairy sessions are present after the user has issued some requests, i.e., the session is replicated when setAttribute has been called on HttpSession. In the application server logging the following is observed

# SHUTDOWN CLUSTER-SERVER1
####<Mar 7, 2013 12:59:37 PM CET> <Info> <Cluster> <middleware-magic.com> <cluster-server1> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <weblogic> <> <94ba3ca995633592:-60f7582f:13d44ab2061:-8000-0000000000000401> <1362657577726> <BEA-000103> <Disconnecting from cluster loadtest-cluster> 

####<Mar 7, 2013 12:59:37 PM CET> <Info> <Cluster> <middleware-magic.com> <cluster-server2> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1362657577103> <BEA-000144> <Managed Server cluster-server1 has been suspended or shut down.> 

# START CLUSTER_SERVER1
####<Mar 7, 2013 1:04:21 PM CET> <Notice> <Cluster> <middleware-magic.com> <cluster-server1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1362657861115> <BEA-000197> <Listening for announcements from cluster using unicast cluster messaging> 

####<Mar 7, 2013 1:04:21 PM CET> <Notice> <Cluster> <middleware-magic.com> <cluster-server1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1362657861125> <BEA-000133> <Waiting to synchronize with other running members of loadtest-cluster.> 

####<Mar 7, 2013 1:04:29 PM CET> <Info> <Cluster> <middleware-magic.com> <cluster-server1> <weblogic.cluster.MessageReceiver> <<WLS Kernel>> <> <> <1362657869754> <BEA-003107> <Lost 2 unicast message(s).> 

####<Mar 7, 2013 1:04:29 PM CET> <Info> <Cluster> <middleware-magic.com> <cluster-server1> <[STANDBY] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1362657869781> <BEA-000111> <Adding cluster-server2 with ID -5268211843454898694S:middleware-magic.com:[9002,9002,-1,-1,-1,-1,-1]:base_domain:cluster-server2 to cluster: loadtest-cluster view.> 

####<Mar 7, 2013 1:04:29 PM CET> <Notice> <Cluster> <middleware-magic.com> <cluster-server1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1362657869787> <BEA-000142> <Trying to download cluster JNDI tree from server cluster-server2.> 

####<Mar 7, 2013 1:04:29 PM CET> <Info> <Cluster> <middleware-magic.com> <cluster-server1> <[STANDBY] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1362657869806> <BEA-000128> <Updating -5268211843454898694S:middleware-magic.com:[9002,9002,-1,-1,-1,-1,-1]:base_domain:cluster-server2 in the cluster.> 

####<Mar 7, 2013 1:04:29 PM CET> <Notice> <Cluster> <middleware-magic.com> <cluster-server1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1362657869936> <BEA-000164> <Synchronized cluster JNDI tree from server cluster-server2.> 

####<Mar 7, 2013 1:04:30 PM CET> <Notice> <Cluster> <middleware-magic.com> <cluster-server1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1362657870214> <BEA-000162> <Starting "async" replication service with remote cluster address "null">

####<Mar 7, 2013 1:04:30 PM CET> <Info> <Cluster> <middleware-magic.com> <cluster-server2> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1362657870415> <BEA-000111> <Adding cluster-server1 with ID 7719031716195535281S:middleware-magic.com:[9001,9001,-1,-1,-1,-1,-1]:base_domain:cluster-server1 to cluster: loadtest-cluster view.> 

####<Mar 7, 2013 1:04:31 PM CET> <Info> <Cluster> <middleware-magic.com> <cluster-server2> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1362657871227> <BEA-000128> <Updating 7719031716195535281S:middleware-magic.com:[9001,9001,-1,-1,-1,-1,-1]:base_domain:cluster-server1 in the cluster.> 

####<Mar 7, 2013 1:04:31 PM CET> <Info> <Cluster> <middleware-magic.com> <cluster-server1> <[STANDBY] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1362657871758> <BEA-000128> <Updating -5268211843454898694S:middleware-magic.com:[9002,9002,-1,-1,-1,-1,-1]:base_domain:cluster-server2 in the cluster.>

Now let us configure the application such that it will use Coherence*Web to store the session information (such that the session data will be off loaded to separate JVMs and not interfere with the application). We start with configuring a Coherence Cluster by using the WebLogic Admin console. The Coherence Cluster has the following configuration

<!-- snippet from config.xml -->
<coherence-cluster-system-resource>
	<name>coherence-session-cluster</name>
    <target>loadtest-cluster</target>
    <descriptor-file-name>coherence/coherence-session-cluster/coherence-session-cluster-coherence.xml</descriptor-file-name>
</coherence-cluster-system-resource>

<!-- coherence-session-cluster-coherence.xml -->
<weblogic-coherence ...>
	<name>coherence-session-cluster</name>
	<coherence-cluster-params>
		<unicast-listen-address>localhost</unicast-listen-address>
		<unicast-listen-port>8088</unicast-listen-port>
		<unicast-port-auto-adjust>true</unicast-port-auto-adjust>
		<multicast-listen-address>231.1.1.1</multicast-listen-address>
		<multicast-listen-port>7777</multicast-listen-port>
		<coherence-cluster-well-known-addresses>
			<coherence-cluster-well-known-address>
				<name>middleware-magic</name>
				<listen-address>middleware-magic.com</listen-address>
				<listen-port>8088</listen-port>
			</coherence-cluster-well-known-address>
		</coherence-cluster-well-known-addresses>
	</coherence-cluster-params>
</weblogic-coherence>

Note that when a well-known-address has been confgured, the multicast settings are not used. Next, we need to configure a Coherence Server. The Coherence Server has the following configuration

<coherence-server>
    <name>coherence-session-server</name>
    <machine>machine1</machine>
    <coherence-cluster-system-resource>coherence-session-cluster</coherence-cluster-system-resource>
    <unicast-listen-address>localhost</unicast-listen-address>
    <unicast-listen-port>8088</unicast-listen-port>
    <unicast-port-auto-adjust>true</unicast-port-auto-adjust>
    <coherence-server-start>
		<java-vendor>Oracle</java-vendor>
		<java-home>/home/weblogic/jrockit-jdk1.6.0_29-R28.2.2-4.1.0</java-home>
		<class-path>/home/weblogic/weblogic12.1.1/installation/coherence_3.7/lib/coherence.jar:/home/weblogic/weblogic12.1.1/installation/modules/features/weblogic.server.modules.coherence.server_12.1.1.0.jar:/home/weblogic/weblogic12.1.1/installation/coherence_3.7/lib/coherence-web.jar</class-path>
		<arguments>-jrockit -Xms512m -Xmx512m -Xns128m -XXkeepAreaRatio:25 -Xgc:pausetime -XpauseTarget:200ms -Dtangosol.coherence.cacheconfig=session-cache-config.xml -Dtangosol.coherence.session.localstorage=true -Dtangosol.coherence.management=all -Dtangosol.coherence.management.remote=true</arguments>
    </coherence-server-start>
</coherence-server>

Note the class path entries. The first is logical, the second is needed such that the Coherence Server can be started using the node manager, the third is needed as it contains the Coherence cache conifguration file (in this case session-cache-config.xml). In the JVM arguments we set the system property tangosol.coherence.cacheconfig to session-cache-config.xml. This tells Coherence which cache configuration to use. We also set the system property tangosol.coherence.session.localstorage to true, as within the cache configuration the storage is turned off

...
<distributed-scheme>
	<scheme-name>session-base</scheme-name>
	<service-name>DistributedSessions</service-name>
	<thread-count>0</thread-count>
	<lease-granularity>member</lease-granularity>
	<local-storage system-property="tangosol.coherence.session.localstorage">false</local-storage>
	<partition-count>257</partition-count>
	<backup-count>1</backup-count>
	<backup-storage>
		<type>on-heap</type>
	</backup-storage>
	<request-timeout>30s</request-timeout>
	<backing-map-scheme>
		<local-scheme>
			<scheme-ref>unlimited-local</scheme-ref>
		</local-scheme>
	</backing-map-scheme>
	<autostart>true</autostart>
</distributed-scheme>
...

Next we deploy the following shared libraries

<library>
    <name>coherence-web-spi#1.0.0.0@1.0.0.0</name>
    <target>loadtest-cluster</target>
    <module-type>war</module-type>
    <source-path>/home/weblogic/weblogic12.1.1/installation/coherence_3.7/lib/coherence-web-spi.war</source-path>
    <security-dd-model>DDOnly</security-dd-model>
</library>
<library>
    <name>coherence#3.7.1.1@3.7.1.1</name>
    <target>loadtest-cluster</target>
    <module-type xsi:nil="true"></module-type>
    <source-path>/home/weblogic/weblogic12.1.1/installation/coherence_3.7/lib/coherence.jar</source-path>
    <security-dd-model>DDOnly</security-dd-model>
</library>
<library>
    <name>active-cache#1.0@1.0</name>
    <target>loadtest-cluster</target>
    <module-type xsi:nil="true"></module-type>
    <source-path>/home/weblogic/weblogic12.1.1/installation/wlserver_12.1/common/deployable-libraries/active-cache-1.0.jar</source-path>
    <security-dd-model>DDOnly</security-dd-model>
    <staging-mode>nostage</staging-mode>
</library>

Note that for the ActiveCache library we have chosen nostage as the staging mode, i.e., in the WebLogic Admin console on the deployment settings page, choose the ‘I will make the deployment accessible from the following location’. The reason for this, is that the jar file only contains a MANIFEST.MF with the following class path setting

Class-Path: ../../../modules/features/weblogic.server.modules.coherence.integration_12.1.1.0.jar

The weblogic.server.modules.coherence.integration_12.1.1.0.jar file in its MANIFEST.MF refers to

Class-Path: ../com.oracle.core.coherence.integration_2.0.0.0.jar

If we would choose a staging method the references would break, and ActiveCache will not work. To let the application refer to the deployed ActiveCache and Coherence libraries, we add the following to the MANIFEST.MF file

Extension-List: ActiveCache Coherence
ActiveCache-Extension-Name: active-cache
ActiveCache-Specification-Version: 1.0
ActiveCache-Implementation-Version: 1.0
Coherence-Extension-Name: coherence
Coherence-Specification-Version: 3.7.1.1
Coherence-Implementation-Version: 3.7.1.1

We also included a tangosol-coherence-override.xml file in the WEB-INF/classes directory of the application, with the following contents

<coherence ...>
    <cluster-config>
        <member-identity>
            <cluster-name system-property="tangosol.coherence.cluster">coherence-session-cluster</cluster-name>
        </member-identity>
        <unicast-listener>
            <well-known-addresses>
                <socket-address id="1">
                    <address>middleware-magic.com</address>
                    <port>8088</port>
                </socket-address>
            </well-known-addresses>
        </unicast-listener>
    </cluster-config>
</coherence>

Here, we configure the cluster-name, such that the application will only join the cluster with name coherence-session-cluster, which we also used in the configuration of the Coherence Cluster in the WebLogic Admin console. We also have configured a well-known-address. By default, Coherence uses a multicast protocol to discover other nodes when forming a cluster. If multicast networking is undesirable, or unavailable in the environment, the Well Known Addresses feature may be used to eliminate the need for multicast traffic. When in use the cluster is configured with a relatively small list of nodes which are allowed to start the cluster, and which are likely to remain available over the cluster lifetime. There is no requirement for all WKA nodes to be simultaneously active at any point in time. This list is used by all other nodes to find their way into the cluster without the use of multicast, thus at least one node that is configured as a well-known node must be running for other nodes to be able to join.

To deploy the application we again use the following structure

/testactivecache
	/app
		TestActiveCache.war
	/plan
		Plan.xml
		/WEB-INF
			weblogic.xml

in which weblogic.xml has the following contents

<weblogic-web-app ...>
	<security-role-assignment>
		<role-name>EMPLOYEE</role-name>
		<principal-name>employees</principal-name>
	</security-role-assignment>
	<security-role-assignment>
		<role-name>MANAGER</role-name>
		<principal-name>managers</principal-name>
	</security-role-assignment>
	<library-ref>
		<library-name>coherence-web-spi</library-name>
		<specification-version>1.0.0.0</specification-version>
		<implementation-version>1.0.0.0</implementation-version>
		<exact-match>true</exact-match>
	</library-ref>
	<coherence-cluster-ref>
		<coherence-cluster-name>coherence-session-cluster</coherence-cluster-name>
	</coherence-cluster-ref>
</weblogic-web-app>

Here we refer to the Coherence*Web shared library, such that it will be merged on runtime with our application, as it contains, next to the necessary jars, an important configuration in its web.xml

<context-param>
	<param-name>coherence-web-sessions-enabled</param-name>
    <param-value>true</param-value>
</context-param>

which enables Coherence web sessions for the application. To test the set-up, deploy the application, issue some requests as different users

shutdown a server (in this case cluster-server2)

As we have turned on the MBean management in Coherence (-Dtangosol.coherence.management=all) we can use an MBean browser to see what is going on

What is fun is that it is even possible to shut down all the WebLogic servers in the cluster and still be able to retrieve the session

<Mar 7, 2013 2:08:57 PM> <INFO> <NodeManager> <Starting Coherence server with command line: /home/weblogic/jrockit-jdk1.6.0_29-R28.2.2-4.1.0/bin/java -Dtangosol.coherence.member=coherence-session-server -Dtangosol.coherence.wka=middleware-magic.com -Dtangosol.coherence.wka.port=8088 -Dtangosol.coherence.clusterport=7777 -Dtangosol.coherence.clusteraddress=231.1.1.1 -Dtangosol.coherence.localport=8088 -Dtangosol.coherence.localhost=localhost -Dtangosol.coherence.cluster=coherence-session-cluster -Djava.class.path=/home/weblogic/weblogic12.1.1/installation/coherence_3.7/lib/coherence.jar:/home/weblogic/weblogic12.1.1/installation/modules/features/weblogic.server.modules.coherence.server_12.1.1.0.jar:/home/weblogic/weblogic12.1.1/installation/coherence_3.7/lib/coherence-web.jar -jrockit -Xms512m -Xmx512m -Xns128m -XXkeepAreaRatio:25 -Xgc:pausetime -XpauseTarget:200ms -Dtangosol.coherence.cacheconfig=session-cache-config.xml -Dtangosol.coherence.session.localstorage=true -Dtangosol.coherence.management=all -Dtangosol.coherence.management.remote=true -Dweblogic.RootDirectory=/home/weblogic/weblogic12.1.1/configuration/domains/base_domain weblogic.nodemanager.server.provider.WeblogicCacheServer >
<Mar 7, 2013 2:08:57 PM> <INFO> <NodeManager> <Working directory is '/home/weblogic/weblogic12.1.1/configuration/domains/base_domain'>
<Mar 7, 2013 2:08:57 PM> <INFO> <NodeManager> <Server output log file is '/home/weblogic/weblogic12.1.1/configuration/domains/base_domain/servers_coherence/coherence-session-server/logs/coherence-session-server.out'>
2013-03-07 14:08:58.464/1.196 Oracle Coherence 3.7.1.1 <Info> (thread=Main Thread, member=n/a): Loaded operational configuration from "jar:file:/home/weblogic/weblogic12.1.1/installation/coherence_3.7/lib/coherence.jar!/tangosol-coherence.xml"
2013-03-07 14:08:58.489/1.221 Oracle Coherence 3.7.1.1 <Info> (thread=Main Thread, member=n/a): Loaded operational overrides from "jar:file:/home/weblogic/weblogic12.1.1/installation/coherence_3.7/lib/coherence.jar!/tangosol-coherence-override-dev.xml"
2013-03-07 14:08:58.489/1.221 Oracle Coherence 3.7.1.1 <D5> (thread=Main Thread, member=n/a): Optional configuration override "/tangosol-coherence-override.xml" is not specified
2013-03-07 14:08:58.494/1.226 Oracle Coherence 3.7.1.1 <D5> (thread=Main Thread, member=n/a): Optional configuration override "/custom-mbeans.xml" is not specified

Oracle Coherence Version 3.7.1.1 Build 28901
 Grid Edition: Development mode
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.

2013-03-07 14:08:59.345/2.077 Oracle Coherence GE 3.7.1.1 <Info> (thread=Main Thread, member=n/a): Loaded Reporter configuration from "jar:file:/home/weblogic/weblogic12.1.1/installation/coherence_3.7/lib/coherence.jar!/reports/report-group.xml"
2013-03-07 14:09:00.016/2.748 Oracle Coherence GE 3.7.1.1 <D4> (thread=Main Thread, member=n/a): TCMP bound to /192.168.1.150:8088 using SystemSocketProvider
2013-03-07 14:09:03.440/6.172 Oracle Coherence GE 3.7.1.1 <Info> (thread=Cluster, member=n/a): Created a new cluster "coherence-session-cluster" with Member(Id=1, Timestamp=2013-03-07 14:09:00.083, Address=192.168.1.150:8088, MachineId=65081, Location=site:,machine:middleware-magic,process:8431,member:coherence-session-server, Role=WeblogicWeblogicCacheServer, Edition=Grid Edition, Mode=Development, CpuCount=4, SocketCount=1) UID=0xC0A801960000013D44F80E33FE391F98
2013-03-07 14:09:03.477/6.209 Oracle Coherence GE 3.7.1.1 <Info> (thread=Main Thread, member=n/a): Started cluster Name=coherence-session-cluster

WellKnownAddressList(Size=1,
  WKA{Address=192.168.1.150, Port=8088}
  )

MasterMemberSet(
  ThisMember=Member(Id=1, Timestamp=2013-03-07 14:09:00.083, Address=192.168.1.150:8088, MachineId=65081, Location=site:,machine:middleware-magic,process:8431,member:coherence-session-server, Role=WeblogicWeblogicCacheServer)
  OldestMember=Member(Id=1, Timestamp=2013-03-07 14:09:00.083, Address=192.168.1.150:8088, MachineId=65081, Location=site:,machine:middleware-magic,process:8431,member:coherence-session-server, Role=WeblogicWeblogicCacheServer)
  ActualMemberSet=MemberSet(Size=1
    Member(Id=1, Timestamp=2013-03-07 14:09:00.083, Address=192.168.1.150:8088, MachineId=65081, Location=site:,machine:middleware-magic,process:8431,member:coherence-session-server, Role=WeblogicWeblogicCacheServer)
    )
  MemberId|ServiceVersion|ServiceJoined|MemberState
    1|3.7.1|2013-03-07 14:09:03.445|JOINED
  RecycleMillis=1200000
  RecycleSet=MemberSet(Size=0
    )
  )

TcpRing{Connections=[]}
IpMonitor{AddressListSize=0}

2013-03-07 14:09:03.596/6.328 Oracle Coherence GE 3.7.1.1 <D5> (thread=Invocation:Management, member=1): Service Management joined the cluster with senior service member 1
2013-03-07 14:09:03.682/6.414 Oracle Coherence GE 3.7.1.1 <D5> (thread=ReplicatedCache:ReplicatedSessionsMisc, member=1): Service ReplicatedSessionsMisc joined the cluster with senior service member 1
2013-03-07 14:09:03.872/6.604 Oracle Coherence GE 3.7.1.1 <D5> (thread=DistributedCache:DistributedSessions, member=1): Service DistributedSessions joined the cluster with senior service member 1
2013-03-07 14:09:03.914/6.646 Oracle Coherence GE 3.7.1.1 <Info> (thread=Main Thread, member=1):
Services
  (
  ClusterService{Name=Cluster, State=(SERVICE_STARTED, STATE_JOINED), Id=0, Version=3.7.1, OldestMemberId=1}
  InvocationService{Name=Management, State=(SERVICE_STARTED), Id=1, Version=3.1, OldestMemberId=1}
  ReplicatedCache{Name=ReplicatedSessionsMisc, State=(SERVICE_STARTED), Id=2, Version=3.0, OldestMemberId=1}
  PartitionedCache{Name=DistributedSessions, State=(SERVICE_STARTED), LocalStorage=enabled, PartitionCount=257, BackupCount=1, AssignedPartitions=257, BackupPartitions=0}
  )

Started DefaultCacheServer...

2013-03-07 14:10:48.607/111.339 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member(Id=2, Timestamp=2013-03-07 14:10:48.395, Address=192.168.1.150:8090, MachineId=65081, Location=site:,machine:middleware-magic,process:8311, Role=WeblogicServer) joined Cluster with senior member 1
2013-03-07 14:10:48.856/111.589 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member 2 joined Service Management with senior member 1
2013-03-07 14:10:48.903/111.635 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member(Id=3, Timestamp=2013-03-07 14:10:48.702, Address=192.168.1.150:8092, MachineId=65081, Location=site:,machine:middleware-magic,process:8308, Role=WeblogicServer) joined Cluster with senior member 1
2013-03-07 14:10:49.360/112.092 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member 3 joined Service Management with senior member 1
2013-03-07 14:10:56.837/119.569 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member 3 joined Service DistributedSessions with senior member 1
2013-03-07 14:10:56.858/119.590 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member 2 joined Service DistributedSessions with senior member 1
# MBEAN BROWSER IS STARTED
[INFO ][mgmnt  ] Local JMX connector started
# CLUSTER-SERVER 2 IS SHUTDOWN
2013-03-07 14:23:26.704/869.436 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member 3 left service Management with senior member 1
2013-03-07 14:23:26.710/869.442 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member 3 left service DistributedSessions with senior member 1
2013-03-07 14:23:26.731/869.463 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): TcpRing disconnected from Member(Id=3, Timestamp=2013-03-07 14:10:48.702, Address=192.168.1.150:8092, MachineId=65081, Location=site:,machine:middleware-magic,process:8308, Role=WeblogicServer) due to a peer departure; removing the member.
2013-03-07 14:23:26.733/869.465 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member(Id=3, Timestamp=2013-03-07 14:23:26.733, Address=192.168.1.150:8092, MachineId=65081, Location=site:,machine:middleware-magic,process:8308, Role=WeblogicServer) left Cluster with senior member 1
# CLUSTER-SERVER 2 IS STARTED AGAIN
2013-03-07 14:43:17.466/2060.198 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member(Id=4, Timestamp=2013-03-07 14:43:17.261, Address=192.168.1.150:8092, MachineId=65081, Location=site:,machine:middleware-magic,process:9145, Role=WeblogicServer) joined Cluster with senior member 1
2013-03-07 14:43:17.908/2060.640 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member 4 joined Service Management with senior member 1
2013-03-07 14:43:20.149/2062.881 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member 4 joined Service DistributedSessions with senior member 1
# CLUSTER-SERVER 1 AND CLUSTER-SERVER 2 ARE SHUTDOWN
2013-03-07 14:46:16.667/2239.399 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member 2 left service Management with senior member 1
2013-03-07 14:46:16.672/2239.404 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member 4 left service Management with senior member 1
2013-03-07 14:46:16.680/2239.412 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member 2 left service DistributedSessions with senior member 1
2013-03-07 14:46:16.687/2239.419 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member 4 left service DistributedSessions with senior member 1
2013-03-07 14:46:16.711/2239.443 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): TcpRing disconnected from Member(Id=4, Timestamp=2013-03-07 14:43:17.261, Address=192.168.1.150:8092, MachineId=65081, Location=site:,machine:middleware-magic,process:9145, Role=WeblogicServer) due to a peer departure; removing the member.
2013-03-07 14:46:16.711/2239.443 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member(Id=4, Timestamp=2013-03-07 14:46:16.711, Address=192.168.1.150:8092, MachineId=65081, Location=site:,machine:middleware-magic,process:9145, Role=WeblogicServer) left Cluster with senior member 1
2013-03-07 14:46:16.712/2239.444 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): TcpRing connection to Member(Id=2, Timestamp=2013-03-07 14:10:48.395, Address=192.168.1.150:8090, MachineId=65081, Location=site:,machine:middleware-magic,process:8311, Role=WeblogicServer) refused (Connection refused); removing the member.
2013-03-07 14:46:16.712/2239.444 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member(Id=2, Timestamp=2013-03-07 14:46:16.712, Address=192.168.1.150:8090, MachineId=65081, Location=site:,machine:middleware-magic,process:8311, Role=WeblogicServer) left Cluster with senior member 1
# CLUSTER-SERVER 1 AND CLUSTER-SERVER 2 ARE STARTED AGAIN
2013-03-07 14:48:12.673/2355.405 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member(Id=3, Timestamp=2013-03-07 14:48:12.472, Address=192.168.1.150:8090, MachineId=65081, Location=site:,machine:middleware-magic,process:9402, Role=WeblogicServer) joined Cluster with senior member 1
2013-03-07 14:48:13.124/2355.856 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member 3 joined Service Management with senior member 1
2013-03-07 14:48:15.067/2357.799 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member(Id=5, Timestamp=2013-03-07 14:48:14.866, Address=192.168.1.150:8092, MachineId=65081, Location=site:,machine:middleware-magic,process:9411, Role=WeblogicServer) joined Cluster with senior member 1
2013-03-07 14:48:15.482/2358.214 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member 5 joined Service Management with senior member 1
2013-03-07 14:48:15.594/2358.326 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member 3 joined Service DistributedSessions with senior member 1
2013-03-07 14:48:17.851/2360.583 Oracle Coherence GE 3.7.1.1 <D5> (thread=Cluster, member=1): Member 5 joined Service DistributedSessions with senior member 1

When cluster-server1 and cluster-server2 are up again, we can access the application with the session still available. Not very useful, as the application cannot be reached when all the servers that it runs on are down, but good to know that Coherence holds on to the data as long as the data does not expire. Very powerful stuff!

References

[1] Coherence Documentation.
[2] User’s Guide for Coherence*Web.
[3] Using ActiveCache.