In this post we are going to conduct a little experiment in which we are going to use different cache configurations and draw some conclusions on the performance where possible. Coherence is extremely pluggable, i.e., we start with a basic set-up and configuration wise we will plug in different cache configurations.

Introduction

Coherence has three layers:

• Named cache – seen by the application as a cloud-like structure that provides access to the data stored in the cache.
• Cache service – responsible for the cache data distribution and retrieval. Can be replicated (data is replicated to all the nodes) or partitioned (data is partitioned across all the nodes). Two data structures are available: near-cache and continuous-query-cache. These consist of two tiers, which usually consist of a local cache in the front and a partitioned cache in the back. Note that the continuous query cache populates the front cache based on a query and that the near cache populates the front cache when items are requested.
• Backing map – responsible for the data storage. Note that in the case of a partitioned cache an additional backup storage map exists in order to ensure no data is lost if a certain node fails. Some examples are:
  • Local cache – stores all the data on the heap (fastest access). It is used as a backing map in the replicated and partitioned cache and as the front cache for the near-cache and continuous-query-cache.
  • External cache – stores the data off-heap (more storage capacity but at the cost of lesser performance).
    • NIO memory manager – stores the data in memory but outside the Java heap.
    • NIO file manager – stores the data in memory mapped files, which affects performance.
  • Read-write cache – has an internal cache (usually a local cache) and a cache loader to load data from an external source or a cache store that adds the ability to update data in an external source.

The following tests will be run:

• Distributed Cache
  • Backing map – local cache
  • Backing map – external cache nio memory
  • Backing map – external cache nio file
  • Backing map – ram journal backed by a flash journal (elastic data feature)
  • Backing map – flash journal
  • Backing map – read-write cache with the internal cache configured as a local cache
• Near cache
  • front – local cache
  • back – distributed cache with a local cache as backing map.

Basic set-up

We have one entity

package model.entities;

import com.tangosol.io.ExternalizableLite;
import com.tangosol.io.pof.PofReader;
import com.tangosol.io.pof.PofWriter;
import com.tangosol.io.pof.PortableObject;
import com.tangosol.util.ExternalizableHelper;

import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;

public class Klant implements ExternalizableLite {

    private Integer klantnummer;
    private String naam;
    private String adres;
    private String stad;
    private String provincie;
    private String postcode;
    private Integer gebied;
    private String telefoonnummer;
    private Integer reputatieNummer;
    private Double kredietlimiet;
    private String commentaar;

    public Klant() {
    }

    public Integer getKlantnummer() {
        return klantnummer;
    }

    public void setKlantnummer(Integer klantnummer) {
        this.klantnummer = klantnummer;
    }

    public String getNaam() {
        return naam;
    }

    public void setNaam(String naam) {
        this.naam = naam;
    }

    public String getAdres() {
        return adres;
    }

    public void setAdres(String adres) {
        this.adres = adres;
    }

    public String getStad() {
        return stad;
    }

    public void setStad(String stad) {
        this.stad = stad;
    }

    public String getProvincie() {
        return provincie;
    }

    public void setProvincie(String provincie) {
        this.provincie = provincie;
    }

    public String getPostcode() {
        return postcode;
    }

    public void setPostcode(String postcode) {
        this.postcode = postcode;
    }

    public Integer getGebied() {
        return gebied;
    }

    public void setGebied(Integer gebied) {
        this.gebied = gebied;
    }

    public String getTelefoonnummer() {
        return telefoonnummer;
    }

    public void setTelefoonnummer(String telefoonnummer) {
        this.telefoonnummer = telefoonnummer;
    }

    public Integer getReputatieNummer() {
        return reputatieNummer;
    }

    public void setReputatieNummer(Integer reputatieNummer) {
        this.reputatieNummer = reputatieNummer;
    }

    public Double getKredietlimiet() {
        return kredietlimiet;
    }

    public void setKredietlimiet(Double kredietlimiet) {
        this.kredietlimiet = kredietlimiet;
    }

    public String getCommentaar() {
        return commentaar;
    }

    public void setCommentaar(String commentaar) {
        this.commentaar = commentaar;
    }

    @Override
    public boolean equals(Object object) {
        if (this == object) {
            return true;
        }

        if (object == null) {
            return false;
        }

        if (!(object instanceof Klant)) {
            return false;
        }

        Klant klant = (Klant) object;
        return klantnummer.equals(klant.getKlantnummer());
    }

    @Override
    public int hashCode() {
        if (klantnummer != null) {
            return 37 * klantnummer.hashCode();
        } else {
            return System.identityHashCode(this);
        }
    }

    @Override
    public String toString() {
        return naam + ", " + adres;
    }

    public void readExternal(PofReader reader) throws IOException {
        setKlantnummer(reader.readInt(0));
        setNaam(reader.readString(1));
        setAdres(reader.readString(2));
        setStad(reader.readString(3));
        setProvincie(reader.readString(4));
        setPostcode(reader.readString(5));
        setGebied(reader.readInt(6));
        setTelefoonnummer(reader.readString(7));
        setReputatieNummer(reader.readInt(8));
        setKredietlimiet(reader.readDouble(9));
        setCommentaar(reader.readString(10));
    }

    public void writeExternal(PofWriter writer) throws IOException {
        if (getKlantnummer() != null) {
            writer.writeInt(0, getKlantnummer());
        }
        if (getNaam() != null) {
            writer.writeString(1, getNaam());
        }
        if (getAdres() != null) {
            writer.writeString(2, getAdres());
        }
        if (getStad() != null) {
            writer.writeString(3, getStad());
        }
        if (getProvincie() != null) {
            writer.writeString(4, getProvincie());
        }
        if (getPostcode() != null) {
            writer.writeString(5, getPostcode());
        }
        if (getGebied() != null) {
            writer.writeInt(6, getGebied());
        }
        if (getTelefoonnummer() != null) {
            writer.writeString(7, getTelefoonnummer());
        }
        if (getReputatieNummer() != null) {
            writer.writeInt(8, getReputatieNummer());
        }
        if (getKredietlimiet() != null) {
            writer.writeDouble(9, getKredietlimiet());
        }
        if (getCommentaar() != null) {
            writer.writeString(10, getCommentaar());
        }
    }

    public void readExternal(DataInput dataInput) throws IOException {
        setKlantnummer(dataInput.readInt());
        setNaam(ExternalizableHelper.readSafeUTF(dataInput));
        setAdres(ExternalizableHelper.readSafeUTF(dataInput));
        setStad(ExternalizableHelper.readSafeUTF(dataInput));
        setProvincie(ExternalizableHelper.readSafeUTF(dataInput));
        setPostcode(ExternalizableHelper.readSafeUTF(dataInput));
        setGebied(dataInput.readInt());
        setTelefoonnummer(ExternalizableHelper.readSafeUTF(dataInput));
        setReputatieNummer(dataInput.readInt());
        setKredietlimiet(dataInput.readDouble());
        setCommentaar(ExternalizableHelper.readSafeUTF(dataInput));
    }

    public void writeExternal(DataOutput dataOutput) throws IOException {
        if (getKlantnummer() != null) {
            dataOutput.writeInt(getKlantnummer());
        }
        if (getNaam() != null) {
            ExternalizableHelper.writeSafeUTF(dataOutput, getNaam());
        }
        if (getAdres() != null) {
            ExternalizableHelper.writeSafeUTF(dataOutput, getAdres());
        }
        if (getStad() != null) {
            ExternalizableHelper.writeSafeUTF(dataOutput, getStad());
        }
        if (getProvincie() != null) {
            ExternalizableHelper.writeSafeUTF(dataOutput, getProvincie());
        }
        if (getPostcode() != null) {
            ExternalizableHelper.writeSafeUTF(dataOutput, getPostcode());
        }
        if (getGebied() != null) {
            dataOutput.writeInt(getGebied());
        }
        if (getTelefoonnummer() != null) {
            ExternalizableHelper.writeSafeUTF(dataOutput, getTelefoonnummer());
        }
        if (getReputatieNummer() != null) {
            dataOutput.writeInt(getReputatieNummer());
        }
        if (getKredietlimiet() != null) {
            dataOutput.writeDouble(getKredietlimiet());
        }
        if (getCommentaar() != null) {
            ExternalizableHelper.writeSafeUTF(dataOutput, getCommentaar());
        }
    }
}

Here, we have implemented PortableObject and ExternalizableLite. We perform basic cache operations (get, put and remove) for which we define the following interface

package model.logic;

import java.io.Serializable;
import java.util.List;

public interface GenericDAO<T, ID extends Serializable> {
    public void addEntity(ID id, T entity);

    public void removeEntity(ID id);

    public T findEntity(ID id);

    public List<T> findEntities();
}

The interface can be implemented as follows:

package model.logic;

import com.tangosol.net.CacheFactory;
import com.tangosol.net.NamedCache;
import com.tangosol.util.ValueExtractor;
import com.tangosol.util.extractor.ChainedExtractor;
import com.tangosol.util.filter.EqualsFilter;

import java.io.Serializable;
import java.lang.reflect.ParameterizedType;
import java.lang.reflect.Type;
import java.util.ArrayList;
import java.util.List;
import java.util.Set;

public abstract class GenericCoherenceDAO<T, ID extends Serializable> implements GenericDAO<T, ID> {

    private Class<T> persistentClass;
    private NamedCache namedCache;

    public GenericCoherenceDAO() {
        Type type = getClass().getGenericSuperclass();
        if (type instanceof ParameterizedType) {
            ParameterizedType parameterizedType = (ParameterizedType) type;
            setPersistentClass((Class<T>) parameterizedType.getActualTypeArguments()[0]);
        } else {
            System.out.println("Not an instance of parameterized type: " + type);
        }

        setNamedCache(CacheFactory.getCache(getPersistentClass().getName()));
    }

    public Class<T> getPersistentClass() {
        return persistentClass;
    }

    public void setPersistentClass(Class<T> persistentClass) {
        this.persistentClass = persistentClass;
    }

    public NamedCache getNamedCache() {
        return namedCache;
    }

    public void setNamedCache(NamedCache namedCache) {
        this.namedCache = namedCache;
    }

    public void addEntity(ID id, T entity) {
        getNamedCache().put(id, entity);
    }

    public void removeEntity(ID id) {
        getNamedCache().remove(id);
    }

    public T findEntity(ID id) {
        return (T) getNamedCache().get(id);
    }

    public List<T> findEntities() {
        ValueExtractor extractor = new ChainedExtractor("getClass.getName");
        Set keys = getNamedCache().keySet(new EqualsFilter(extractor, getPersistentClass().getName()));
        return new ArrayList<T>(getNamedCache().getAll(keys).values());
    }
}

Specific entity functionality is implemented in entity specific DAOs, for example,

package model.logic;

import model.entities.Klant;
import java.util.Set;

public interface KlantDAO extends GenericDAO<Klant, Integer> {
    public Set<Klant> getKlantData();
}

Which is implemented as follows

package model.logic;

import com.tangosol.net.cache.ContinuousQueryCache;
import com.tangosol.util.Filter;
import com.tangosol.util.QueryHelper;
import model.entities.Klant;
import java.util.Set;

public class KlantDAOBean extends GenericCoherenceDAO<Klant, Integer> implements KlantDAO {

    private ContinuousQueryCache continuousQueryCache;

    public KlantDAOBean() {
        Filter filter = QueryHelper.createFilter("kredietlimiet between 4000.0 and 12000.0");
        continuousQueryCache = new ContinuousQueryCache(getNamedCache(), filter);
    }

    public Set<Klant> getKlantData() {
        return continuousQueryCache.entrySet();
    }
}

To test the environment we will use the following:

package model.test;

import model.entities.Klant;
import model.logic.KlantDAO;
import model.logic.KlantDAOBean;

import java.util.Random;

public class Test {

    private Random generator = new Random();

    public static void main(String[] args) {
        Test test = new Test();

        KlantDAO klantDAO = new KlantDAOBean();

        test.doRandomReadWriteTest(klantDAO);
    }

    private void doRandomReadWriteTest(KlantDAO klantDAO) {
        while (true) {
            // generate a random client
            Klant klant = createKlant();
            // insert or update a client - an update is issued when the client already exists
            klantDAO.addEntity(klant.getKlantnummer(), klant);
            if (generator.nextDouble() < 0.001) {
                // remove a client
                klantDAO.removeEntity(generateKlantNummer());
                // get all client data
                klantDAO.findEntities();
                // get data from the continuous query cache
                klantDAO.getKlantData();
            } else {
                // find a client by ID
                klantDAO.findEntity(generateKlantNummer());
            }
        }
    }

    private Klant createKlant() {
        int klantnummer = generateKlantNummer();

        Klant klant = new Klant();
        klant.setKlantnummer(klantnummer);
        klant.setNaam("Middleware" + klantnummer);
        klant.setAdres("Magic");
        klant.setStad("Pune");
        klant.setProvincie("IN");
        klant.setPostcode("1234AB");
        klant.setGebied(1);
        klant.setTelefoonnummer("123-4567");
        klant.setReputatieNummer(1);
        klant.setKredietlimiet(Math.rint(generator.nextDouble() * 5000.0));
        klant.setCommentaar(Long.toString(Math.abs(generator.nextLong()), 36));

        return klant;
    }

    private Integer generateKlantNummer() {
        int klantnummer = generator.nextInt(10000);
        if (klantnummer == 0 || (klantnummer >= 100 && klantnummer <= 109)) {
            return 42;
        } else {
            return klantnummer;
        }
    }
}

We further create two start scripts. One to start a default cache server and another to start the test.

#!/bin/sh
# coherence options
COHERENCE_MANAGEMENT_OPTIONS="-Dtangosol.coherence.management=all -Dtangosol.coherence.management.remote=true"
COHERENCE_OPTIONS="-Dtangosol.coherence.cacheconfig=tryout-cache-config.xml ${COHERENCE_MANAGEMENT_OPTIONS}"
export COHERENCE_OPTIONS

JAVA_HOME="/home/oracle/documents/jrrt-4.0.1-1.6.0"
#JAVA_HOME="/home/oracle/documents/jdk1.6.0_24"
export JAVA_HOME
MEM_ARGS="-jrockit -Xms1024m -Xmx1024m -Xns256m -XXkeepAreaRatio:25 -Xgc:pausetime -XpauseTarget:200ms -XX:+UseCallProfiling"
#MEM_ARGS="-server -Xmx1024m -Xms1024 -Xmn256m -XX:SurvivorRatio=8 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=31 -XX:PermSize=128m -XX:MaxPermSize=128m -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:+CMSIncrementalPacing -XX:CMSIncrementalDutyCycleMin=0 -XX:CMSIncrementalDutyCycle=10"
export MEM_ARGS

CLASSPATH="/home/oracle/temp/TryOut/lib/Coherence/coherence-hibernate.jar:/home/oracle/temp/TryOut/lib/Coherence/commonj.jar:/home/oracle/temp/TryOut/lib/Coherence/coherence.jar:/home/oracle/temp/TryOut/lib/Coherence/coherence-work.jar:/home/oracle/temp/TryOut/lib/Database/ojdbc6.jar:/home/oracle/temp/TryOut/lib/Hibernate/jta.jar:/home/oracle/temp/TryOut/lib/Hibernate/ehcache-1.2.3.jar:/home/oracle/temp/TryOut/lib/Hibernate/hibernate3.jar:/home/oracle/temp/TryOut/lib/Hibernate/commons-logging-1.0.4.jar:/home/oracle/temp/TryOut/lib/Hibernate/dom4j-1.6.1.jar:/home/oracle/temp/TryOut/lib/Hibernate/cglib-2.1.3.jar:/home/oracle/temp/TryOut/lib/Hibernate/commons-collections-2.1.1.jar:/home/oracle/temp/TryOut/lib/Hibernate/asm.jar:/home/oracle/temp/TryOut/lib/Hibernate/c3p0-0.9.1.jar:/home/oracle/temp/TryOut/lib/Hibernate/antlr-2.7.6.jar:/home/oracle/temp/TryOut/lib/Hibernate/asm-attrs.jar:/home/oracle/temp/TryOut/out/artifacts/test/test.jar"
export CLASSPATH

# start the default cache server
${JAVA_HOME}/bin/java ${MEM_ARGS} ${COHERENCE_OPTIONS} com.tangosol.net.DefaultCacheServer

#!/bin/sh
# coherence options
COHERENCE_MANAGEMENT_OPTIONS="-Dtangosol.coherence.management=all -Dtangosol.coherence.management.remote=true"
COHERENCE_OPTIONS="-Dtangosol.coherence.cacheconfig=tryout-cache-config.xml"
export COHERENCE_OPTIONS

JAVA_HOME="/home/oracle/documents/jrrt-4.0.1-1.6.0"
#JAVA_HOME="/home/oracle/documents/jdk1.6.0_24"
export JAVA_HOME
MEM_ARGS="-jrockit -Xms1024m -Xmx1024m -Xns256m -XXkeepAreaRatio:25 -Xgc:pausetime -XpauseTarget:200ms -XX:+UseCallProfiling"
#MEM_ARGS="-server -Xmx1024m -Xms1024 -Xmn256m -XX:SurvivorRatio=8 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=31 -XX:PermSize=128m -XX:MaxPermSize=128m -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:+CMSIncrementalPacing -XX:CMSIncrementalDutyCycleMin=0 -XX:CMSIncrementalDutyCycle=10"
export MEM_ARGS

CLASSPATH="/home/oracle/temp/TryOut/lib/Coherence/coherence-hibernate.jar:/home/oracle/temp/TryOut/lib/Coherence/commonj.jar:/home/oracle/temp/TryOut/lib/Coherence/coherence.jar:/home/oracle/temp/TryOut/lib/Coherence/coherence-work.jar:/home/oracle/temp/TryOut/lib/Database/ojdbc6.jar:/home/oracle/temp/TryOut/lib/Hibernate/jta.jar:/home/oracle/temp/TryOut/lib/Hibernate/ehcache-1.2.3.jar:/home/oracle/temp/TryOut/lib/Hibernate/hibernate3.jar:/home/oracle/temp/TryOut/lib/Hibernate/commons-logging-1.0.4.jar:/home/oracle/temp/TryOut/lib/Hibernate/dom4j-1.6.1.jar:/home/oracle/temp/TryOut/lib/Hibernate/cglib-2.1.3.jar:/home/oracle/temp/TryOut/lib/Hibernate/commons-collections-2.1.1.jar:/home/oracle/temp/TryOut/lib/Hibernate/asm.jar:/home/oracle/temp/TryOut/lib/Hibernate/c3p0-0.9.1.jar:/home/oracle/temp/TryOut/lib/Hibernate/antlr-2.7.6.jar:/home/oracle/temp/TryOut/lib/Hibernate/asm-attrs.jar:/home/oracle/temp/TryOut/out/artifacts/test/test.jar"
export CLASSPATH

# start the test
${JAVA_HOME}/bin/java ${MEM_ARGS} ${COHERENCE_OPTIONS} model.test.Test

Distributed cache

A distributed, or partitioned cache, is a collection of data that is distributed across a number of nodes such that exactly one node in the cluster is responsible for a piece of data. In this case the size of the cache and the processing power associated with the management of the cache can grow linearly with the size of the cluster. Operations against data in the cache involve at most one other server. Note that the distributed cache allows for a backup to be configured, in which case any cluster node can fail without data loss.

Local cache

First we will see if there is any difference as to which serialization mechanism is used. In this case we will run two tests. One that uses ExternalizableLite and one that use the Portable Object Format (POF). As a backing map we will use a local cache, i.e., we will use the following cache configuration:

<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd">
    <!--defaults>
        <serializer>
            <instance>
                <class-name>com.tangosol.io.pof.ConfigurablePofContext</class-name>
                <init-params>
                    <init-param>
                        <param-type>String</param-type>
                        <param-value>tryout-pof-config.xml</param-value>
                    </init-param>
                </init-params>
            </instance>
        </serializer>
    </defaults-->
    <caching-scheme-mapping>
        <cache-mapping>
            <cache-name>*</cache-name>
            <scheme-name>distributed-cache</scheme-name>
            <init-params>
                <init-param>
                    <param-name>back-size-limit</param-name>
                    <param-value>250M</param-value>
                </init-param>
            </init-params>
        </cache-mapping>
    </caching-scheme-mapping>
    <caching-schemes>
        <distributed-scheme>
            <scheme-name>distributed-cache</scheme-name>
            <service-name>DistributedCache</service-name>
            <thread-count>5</thread-count>
            <backup-count>1</backup-count>
            <backing-map-scheme>
                <local-scheme>
                    <scheme-ref>local-backing-map</scheme-ref>
                </local-scheme>
            </backing-map-scheme>
            <autostart>true</autostart>
        </distributed-scheme>
        <local-scheme>
            <scheme-name>local-backing-map</scheme-name>
            <eviction-policy>hybrid</eviction-policy>
            <high-units>{back-size-limit 0}</high-units>
            <unit-calculator>binary</unit-calculator>
            <expiry-delay>{back-expiry-delay 1h}</expiry-delay>
        </local-scheme>
    </caching-schemes>
</cache-config>

Note that for ExternalizableLite no extra configuration is needed. When we want to use POF we have to enable the defaults section. The POF configuration file looks as follows:

<pof-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xmlns="http://xmlns.oracle.com/coherence/coherence-pof-config"
            xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-pof-config coherence-pof-config.xsd">
    <user-type-list>
        <include>coherence-pof-config.xml</include>
        <user-type>
            <type-id>1001</type-id>
            <class-name>model.entities.Klant</class-name>
        </user-type>
    </user-type-list>
    <allow-interfaces>true</allow-interfaces>
    <allow-subclasses>true</allow-subclasses>
</pof-config>

Each test runs for 30 minutes and we will use the JRockit flight recording as the analysis tool.

Some statistics from the MBean browser

here is not much difference performance wise between the two serializable mechanisms. As we are also setting up a read-write cache, which has a bug when using POF that is solved in Coherence 3.7.1, we will choose ExternalizableLite as the serialization mechanism.

External cache

We start with the following configuration. This configures a distributed cache that uses an off-heap backing map.

<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd">
    <caching-scheme-mapping>
        <cache-mapping>
            <cache-name>*</cache-name>
            <scheme-name>distributed-cache</scheme-name>
            <init-params>
                <init-param>
                    <param-name>back-size-limit</param-name>
                    <param-value>250M</param-value>
                </init-param>
            </init-params>
        </cache-mapping>
    </caching-scheme-mapping>
    <caching-schemes>
        <distributed-scheme>
            <scheme-name>distributed-cache</scheme-name>
            <service-name>DistributedCache</service-name>
            <thread-count>5</thread-count>
            <backup-count>1</backup-count>
            <backup-storage>
                <type>off-heap</type>
                <initial-size>1MB</initial-size>
                <maximum-size>50MB</maximum-size>
            </backup-storage>
            <backing-map-scheme>
                <partitioned>true</partitioned>
                <external-scheme>
                    <scheme-ref>external-backing-map</scheme-ref>
                </external-scheme>
            </backing-map-scheme>
            <autostart>true</autostart>
        </distributed-scheme>
        <external-scheme>
            <scheme-name>external-backing-map</scheme-name>
            <nio-memory-manager>
                <initial-size>1MB</initial-size>
                <maximum-size>50MB</maximum-size>
            </nio-memory-manager>
            <high-units>{back-size-limit 0}</high-units>
            <unit-calculator>binary</unit-calculator>
            <expiry-delay>{back-expiry-delay 1h}</expiry-delay>
        </external-scheme>
    </caching-schemes>
</cache-config>

To configure a distributed cache that uses a file-mapped backing map we can use the following:

<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd">
    <caching-scheme-mapping>
        <cache-mapping>
            <cache-name>*</cache-name>
            <scheme-name>distributed-cache</scheme-name>
            <init-params>
                <init-param>
                    <param-name>back-size-limit</param-name>
                    <param-value>250M</param-value>
                </init-param>
            </init-params>
        </cache-mapping>
    </caching-scheme-mapping>
    <caching-schemes>
        <distributed-scheme>
            <scheme-name>distributed-cache</scheme-name>
            <service-name>DistributedCache</service-name>
            <thread-count>5</thread-count>
            <backup-count>1</backup-count>
            <backup-storage>
                <type>file-mapped</type>
                <initial-size>1MB</initial-size>
                <maximum-size>50MB</maximum-size>
                <directory>/home/oracle/temp/backup</directory>
            </backup-storage>
            <backing-map-scheme>
                <partitioned>true</partitioned>
                <external-scheme>
                    <scheme-ref>external-backing-map</scheme-ref>
                </external-scheme>
            </backing-map-scheme>
            <autostart>true</autostart>
        </distributed-scheme>
        <external-scheme>
            <scheme-name>external-backing-map</scheme-name>
            <nio-file-manager>
                <initial-size>1MB</initial-size>
                <maximum-size>50MB</maximum-size>
                <directory>/home/oracle/temp/storage</directory>
            </nio-file-manager>
            <high-units>{back-size-limit 0}</high-units>
            <unit-calculator>binary</unit-calculator>
            <expiry-delay>{back-expiry-delay 1h}</expiry-delay>
        </external-scheme>
    </caching-schemes>
</cache-config>

Elastic data

Background information on the elastic data feature can be found here. To configure a distributed cache that uses a ram journal backed by a flash journal we can use the following:

<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd">
    <caching-scheme-mapping>
        <cache-mapping>
            <cache-name>*</cache-name>
            <scheme-name>distributed-cache</scheme-name>
        </cache-mapping>
    </caching-scheme-mapping>
    <caching-schemes>
        <distributed-scheme>
            <scheme-name>distributed-cache</scheme-name>
            <service-name>DistributedCache</service-name>
            <thread-count>5</thread-count>
            <backup-count>1</backup-count>
            <backing-map-scheme>
                <ramjournal-scheme/>
				<!--flashjournal-scheme/-->
            </backing-map-scheme>
            <autostart>true</autostart>
        </distributed-scheme>
    </caching-schemes>
</cache-config>

The journaling behavior is configured by using the tangosol-coherence-override.xml file, for example,

<coherence xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xmlns="http://xmlns.oracle.com/coherence/coherence-operational-config"
           xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-operational-config coherence-operational-config.xsd">
    <cluster-config>
        <journaling-config>
            <ramjournal-manager>
                <maximum-value-size>16KB</maximum-value-size>
                <maximum-size system-property="tangosol.coherence.ramjournal.size">25%</maximum-size>
            </ramjournal-manager>
            <flashjournal-manager>
                <maximum-value-size>64KB</maximum-value-size>
                <maximum-file-size>2048KB</maximum-file-size>
                <block-size>256KB</block-size>
                <maximum-pool-size>16MB</maximum-pool-size>
                <directory>/home/oracle/temp/journaling</directory>
                <async-limit>16MB</async-limit>
            </flashjournal-manager>
        </journaling-config>
    </cluster-config>
</coherence>

The RAM journal manager contains the configuration for a RAM journal that manages memory buffers for journal-based storage in-memory. Note that a RAM journal always uses a flash journal to store large objects that is also used as an overflow when the amount of total memory is reached.

• maximum value size – binary values stored in the RAM journal that exceed the maximum value are delegated to the flash journal.
• maximum size – the maximum amount of RAM.
• maximum file size – maximum of the underlying journal files.
• block size – the size of the write buffer (a multiple of the disk’s optimal size and a power of two).
• maximum pool size – size of the buffer pool.
• directory – directory where the journal files are placed.
• async limit – maximum of the backlog (the amount of data that has yet to be persisted.

Read-write cache

To configure a distributed cache that uses a read-write cache with the internal cache configured as a local cache we can use the following:

<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd">
    <caching-scheme-mapping>
        <cache-mapping>
            <cache-name>*</cache-name>
            <scheme-name>distributed-read-write-cache</scheme-name>
            <init-params>
                <init-param>
                    <param-name>back-size-limit</param-name>
                    <param-value>250M</param-value>
                </init-param>
            </init-params>
        </cache-mapping>
    </caching-scheme-mapping>
    <caching-schemes>
        <distributed-scheme>
            <scheme-name>distributed-read-write-cache</scheme-name>
            <service-name>DistributedReadWriteCache</service-name>
            <thread-count>5</thread-count>
            <backup-count>1</backup-count>
            <backup-count-after-writebehind>0</backup-count-after-writebehind>
            <backing-map-scheme>
                <read-write-backing-map-scheme>
                    <scheme-ref>read-write-backing-map</scheme-ref>
                </read-write-backing-map-scheme>
            </backing-map-scheme>
            <autostart>true</autostart>
        </distributed-scheme>
        <read-write-backing-map-scheme>
            <scheme-name>read-write-backing-map</scheme-name>
            <internal-cache-scheme>
                <local-scheme>
                    <scheme-ref>local-backing-map</scheme-ref>
                </local-scheme>
            </internal-cache-scheme>
            <cachestore-scheme>
                <class-scheme>
                    <class-name>com.tangosol.coherence.hibernate.HibernateCacheStore</class-name>
                    <init-params>
                        <init-param>
                            <param-type>String</param-type>
                            <param-value>{cache-name}</param-value>
                        </init-param>
                    </init-params>
                </class-scheme>
            </cachestore-scheme>
            <write-delay>{write-delay 10s}</write-delay>
            <write-batch-factor>{write-batch-factor 0.75}</write-batch-factor>
            <write-requeue-threshold>{write-requeue-threshold 128}</write-requeue-threshold>
            <refresh-ahead-factor>{refresh-ahead-factor 0.1}</refresh-ahead-factor>
        </read-write-backing-map-scheme>
        <local-scheme>
            <scheme-name>local-backing-map</scheme-name>
            <eviction-policy>hybrid</eviction-policy>
            <high-units>{back-size-limit 0}</high-units>
            <unit-calculator>binary</unit-calculator>
            <expiry-delay>{back-expiry-delay 1h}</expiry-delay>
        </local-scheme>
    </caching-schemes>
</cache-config>

Results

Some statistics from the MBean browser:

The numbers were to be expected, i.e., the in-memory variants are performing better than the file-based ones.

Near cache

To configure a near cache with a local cache as the front cache and a distributed cache with a local cache as the back cache we can use the following:

<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd">
    <caching-scheme-mapping>
        <cache-mapping>
            <cache-name>*</cache-name>
            <scheme-name>near-cache</scheme-name>
            <init-params>
                <init-param>
                    <param-name>front-size-limit</param-name>
                    <param-value>1000</param-value>
                </init-param>
                <init-param>
                    <param-name>back-size-limit</param-name>
                    <param-value>250M</param-value>
                </init-param>
            </init-params>
        </cache-mapping>
    </caching-scheme-mapping>
    <caching-schemes>
        <near-scheme>
            <scheme-name>near-cache</scheme-name>
            <front-scheme>
                <local-scheme>
                    <high-units>{front-size-limit 0}</high-units>
                    <expiry-delay>{front-expiry-delay 1m}</expiry-delay>
                </local-scheme>
            </front-scheme>
            <back-scheme>
                <distributed-scheme>
                    <scheme-ref>distributed-cache</scheme-ref>
                </distributed-scheme>
            </back-scheme>
            <invalidation-strategy>none</invalidation-strategy>
            <autostart>true</autostart>
        </near-scheme>
        <distributed-scheme>
            <scheme-name>distributed-cache</scheme-name>
            <service-name>DistributedCache</service-name>
            <thread-count>5</thread-count>
            <backup-count>1</backup-count>
            <backing-map-scheme>
                <local-scheme>
                    <scheme-ref>local-backing-map</scheme-ref>
                </local-scheme>
            </backing-map-scheme>
            <autostart>true</autostart>
        </distributed-scheme>
        <local-scheme>
            <scheme-name>local-backing-map</scheme-name>
            <eviction-policy>hybrid</eviction-policy>
            <high-units>{back-size-limit 0}</high-units>
            <unit-calculator>binary</unit-calculator>
            <expiry-delay>{back-expiry-delay 1h}</expiry-delay>
        </local-scheme>
    </caching-schemes>
</cache-config>

Results

Some garbage collection statistics from the flight recordings:

The following are the hot packages(classes):

• com.tangosol.util (ExternalizableHelper) – used when objects are deserialized/serialized which is happening a lot as we are constantly getting object from and putting objects into the cache.
• com.tangosol.io (AbstractByteArrayReadBuffer and ByteArrayWriteBuffer) – used when objects are deserialized/serialized.
• com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor (PacketPublisher and PacketReceiver) – used when there are two or more nodes in the cluster.
• com.tangosol.coherence.component.util.daemon.queueProcessor.server.grid.partionedService (PartionedCache)
• with the near cache com.tangosol.net.cache (OldCache)
• with POF com.tangosol.io.pof (PofBufferReader and PofBufferWriter) – used when objects are deserialized/serialized when POF is enabled.

There is nothing out of the ordinary, but at least we have some nice examples on how the set-up difference cache configurations.

References

[1] Introduction to Caches.
[2] Implementing Storage and Backing Maps.
[3] Caching Data Sources.