oracle coherence tools

This blog post is going to cover the new Oracle Tools project hosted on the Coherence Comunity GitHub site and how to use this to help test your applications, specifically Java and Oracle Coherence applications. For anyone who has been using the Oracle Coherence Incubator you will already be familiar with some of this post and I have blogged about the Incubator runtime package in the past here.

There have been a lot of changes around the Oracle Coherence Incubator over the last year thanks to a lot of work by the guys at Oracle. The biggest positive is that everything is now properly open sourced on GitHub. You can take the code and do whatever you like with it but more importantly it makes keeping up to date with changes much easier and it makes contributing changes and fixes back to the project a lot easier too. If you look at the Coherence Comunity on GitHub you will see it is made up of a number of different repositories. In this post I am going to talk about just one of those, the Oracle Tools project. If you do not want to get the source you can just pull the jar files from the central Maven repository as the releases of Oracle Tools are all published to Maven under com.oracle.tools.

After some discussions a while back it was clear that there was a lot of code in the Oracle Coherence Incubator Commons runtime package that would be useful for development other applications that may have nothing to do with Oracle Coherence. The runtime package allowed you to control any external process or virtualised Java process but being part of the Incubator dragged in the dependency on Coherence. A decision was made to refactor all of this code out of the Incubator and into its own project and so Oracle Tools was born.

If you look at the project you will see that it is pretty much all of the code from the old Incubator runtime package refactored into various modules. On the subject of modules, all of the projects on GitHub are now proper Maven projects so there should be no more headaches trying to figure out how to build them. I’m not getting into the pros and cons of build tools, Maven was chosen as the build tool and that is the way it is.

So, on with the meat of the blog post…

The Oracle Tools Modules

Oracle Tools is made up of the following modules:

  • oracle-tools-core
  • oracle-tools-runtime
  • oracle-tools-testing-support
  • oracle-tools-runtime-tests
  • oracle-tools-coherence

The functionality provided by Oracle Tools right now is pretty much all geared around running “processes” and supporting easy testing of your applications and as you would expect specifically testing clustered applications made up of a number of processes. This means that typically to use Oracle Tools only your tests need a dependency on the Oracle Tools artefacts, your application code does not. I think this is quite an important point, I know a number of teams that are somehow scared of using the Incubator code in their application even though it is open source and after all, who does not already use open source code in their applications. But for those of a nervous disposition, you can add Oracle Tools as a test dependancy without impacting your production code. Another point regarding dependencies is that although the Oracle Tools Coherence module is built against the current version of Coherence and has a dependdency on it as far as I know it will work fine with older versions so you should be fine using Oracle Tools even if your project is on an older version of Coherence (or your project is like mine where you have clients who may take your client API and run it with old versions).

For Maven users these are the dependencies you want.

  1. <dependency>
  2.     <groupId>com.oracle.tools</groupId>
  3.     <artifactId>oracle-tools-core</artifactId>
  4.     <version>${oracle.tools.revision}</version>
  5. </dependency>
  6. <dependency>
  7.     <groupId>com.oracle.tools</groupId>
  8.     <artifactId>oracle-tools-runtime</artifactId>
  9.     <version>${oracle.tools.revision}</version>
  10. </dependency>
  11. <dependency>
  12.     <groupId>com.oracle.tools</groupId>
  13.     <artifactId>oracle-tools-testing-support</artifactId>
  14.     <version>${oracle.tools.revision}</version>
  15. </dependency>
  16. <dependency>
  17.     <groupId>com.oracle.tools</groupId>
  18.     <artifactId>oracle-tools-coherence</artifactId>
  19.     <version>${oracle.tools.revision}</version>
  20. </dependency>
<dependency>
    <groupId>com.oracle.tools</groupId>
    <artifactId>oracle-tools-core</artifactId>
    <version>${oracle.tools.revision}</version>
</dependency>
<dependency>
    <groupId>com.oracle.tools</groupId>
    <artifactId>oracle-tools-runtime</artifactId>
    <version>${oracle.tools.revision}</version>
</dependency>
<dependency>
    <groupId>com.oracle.tools</groupId>
    <artifactId>oracle-tools-testing-support</artifactId>
    <version>${oracle.tools.revision}</version>
</dependency>
<dependency>
    <groupId>com.oracle.tools</groupId>
    <artifactId>oracle-tools-coherence</artifactId>
    <version>${oracle.tools.revision}</version>
</dependency>

Where ${oracle.tools.revision} is whatever the latest version is.

Deferred – Waiting for conditions to be met

The main functionality in the oracle-tools-core module is the deferred package. This code provides a way to reference objects that may or may not yet exist but probably will at some point in the future. Thinking about Coherence, this could typically be something like an Extend Proxy Service that may not yet exist but at some point will and at some other future point will be started and listening for connections. Used in combination with the oracle-tools-test-support module that provides deferred assertions it becomes very easy to write tests cases based on waiting for something to happen.

Still keeping with the Coherence example, as that is what I mostly work with, you can imagine that using the deferred functionality can be useful in tests to make sure that the pre-conditions of the tests have been met, for example, wait until the cluster has started and has the correct number of members then wait until the proxy is running, once those conditions are met, then the tests can start. Now you could write that code yourself using various combinations of waits or sleeps, catching exceptions when services are not present etc, but that is all now nicely wrapped away.

Typically, waiting for a cluster to be started and have a specific number of nodes is as simple as this

  1. assertThat(eventually(invoking(cluster).getClusterSize()), is(3));
assertThat(eventually(invoking(cluster).getClusterSize()), is(3));

The code above will wait for a default time of 30 seconds for the condition to be met, otherwise an exception is thrown and the test would fail. If you want to wait for a different amount of time then you can pass other parameters, for example to wait for five minutes…

  1. assertThat(eventually(invoking(cluster).getClusterSize(), 5, TimeUnit.MINUTES), is(3));
assertThat(eventually(invoking(cluster).getClusterSize(), 5, TimeUnit.MINUTES), is(3));

As you can see, this makes waiting for conditions to be met in your tests very easy and can be used in a number of other situations, for example waiting for events, messages etc…

The deferred package contains a number of different classes that allow you to work with various types of deferred values. From a Java and Coherence perspective, one of the most useful is the deferred JMX functionality that allows you to obtain values from JMX for a process but instead of you having to write code to wait for the process to start, wait for the JMX server to start and wait for the MBean to be available, that is all taken care of. For example, to wait for the proxy service to start on the extend proxy cluster member you can write this in your test…

  1. Int nodeId = proxyMember.getLocalMemberId();
  2. ObjectName objectName = new ObjectName("Coherence:type=Service,name=ExtendTcpProxyService,nodeId=" + nodeId));
  3. assertThat(eventually(invoking(proxyMember).getMBeanAttribute(objectName, "Running", Boolean.class)), is(true));
Int nodeId = proxyMember.getLocalMemberId();
ObjectName objectName = new ObjectName("Coherence:type=Service,name=ExtendTcpProxyService,nodeId=" + nodeId));
assertThat(eventually(invoking(proxyMember).getMBeanAttribute(objectName, "Running", Boolean.class)), is(true));

You will see these assertions used in the examples below.

Runtime – Process Control

As its name suggests the oracle-tools-runtime module contains the code from the old Coherence incubator’s runtime package, in this case the non-Coherence specific code. As I have mentioned in a previous post the Runtime functionality is all about making it easy to configure and control proceses that you may need as part of your application testing. These processes can be Java or non-Java and can be externally forked processes, or if they are Java, they can be isolated, virtualized processes inside the same JVM as your application tests. The runtime package is similar to what I previously blogged about with a few package and class name changes and new functionality.

The runtime package is based on the concept of Schema that define a process and builders that take a schema and realize a running process. Oracle Tools comes with schema for various types of processes, Java, non-Java and Coherence and builders for external processes and virtualized java processes.

I will run through a few simple examples to show how easy this is to use.

Simple External Processes

To run a simple external process you will need to have a dependency on the jar files from the oracle-tool-core and oracle-tools-runtime modules; so no Coherence dependencies yet.

This is a very basic example of running an external executable. For this example I wanted to do something pretty generic that would work anywhere. I work at home on a Mac Book Pro, in the office on Windows and I know some of you work on various flavours of Linux desktop so I chose to run the Java executable. It is a bit pointless in terms of doing anything useful, but it shows the functionality.

  1. package com.thegridman.oracle.tools.examples;
  2.  
  3. import com.oracle.tools.runtime.SimpleApplication;
  4. import com.oracle.tools.runtime.SimpleApplicationBuilder;
  5. import com.oracle.tools.runtime.SimpleApplicationSchema;
  6. import com.oracle.tools.runtime.console.SystemApplicationConsole;
  7.  
  8. public class RunSomethingForked
  9. {
  10.     public static void main(String[] args) throws Exception
  11.     {
  12.         SimpleApplicationSchema schema =
  13.                 new SimpleApplicationSchema("java")
  14.                     .setArgument("-version");
  15.  
  16.         SimpleApplicationBuilder builder
  17.                 = new SimpleApplicationBuilder();
  18.  
  19.         SimpleApplication application
  20.                 = builder.realize(schema, "Java", new SystemApplicationConsole());
  21.  
  22.         int exitCode = application.waitFor();
  23.     }
  24. }
package com.thegridman.oracle.tools.examples;

import com.oracle.tools.runtime.SimpleApplication;
import com.oracle.tools.runtime.SimpleApplicationBuilder;
import com.oracle.tools.runtime.SimpleApplicationSchema;
import com.oracle.tools.runtime.console.SystemApplicationConsole;

public class RunSomethingForked
{
    public static void main(String[] args) throws Exception
    {
        SimpleApplicationSchema schema =
                new SimpleApplicationSchema("java")
                    .setArgument("-version");

        SimpleApplicationBuilder builder
                = new SimpleApplicationBuilder();

        SimpleApplication application
                = builder.realize(schema, "Java", new SystemApplicationConsole());

        int exitCode = application.waitFor();
    }
}

The interesting bit is lines 12 – 22.

  • On line 12 we create an instance of a SimpleApplicationSchema which defines the process we want to run. This is the most simple type of process that requires as a minimum just the name of the executable to run; in this case “java”. We then call setArgument to pass in the command line argument -version that will cause the Java executable to print out the version information then exit.
  • On line 16 we create an instance of the SimpleApplicationBuilder that will realize a running process from a given schema. As the name suggests SimpleApplicationBuilder can only build processes based on a SimpleApplicationSchema.
  • On line 19 we call realize on the SimpleApplicationBuilder to build a running instance of a process based on our SimpleApplicationSchema. There are various overloaded versions of the realize method. At a minimum you need to pass a SimpleApplicationSchema. You also have the option to pass a process name and finally the version we have used where we pass in a name and an instance of an ApplicationConsole, in this case a SystemApplicationConsole.
    An ApplicationConsole is used to capture the output of the process and send it somewhere useful. In this case the SystemApplicationConsole captures the process output and sends it to the stdout of our parent process. There are a few built in implementations for sending the output to stdout or stderr or just ignoring it but it is easy enough to add your own if you want to do anything special.
  • Finally on line 22 we wait for the process to exit and capture it’s exit code.

If we run the code above we see something like this displayed on the console (this image is from running inside IntelliJ)

Run Something Forked

This is the captured output of our process that has been formatted by the ApplicationConsole. The format of a line starts with [application-name:output-type:pid] where application-name is the name of the application that we passed to the builder’s realize method. The output-type is either out for the process stdout stream or err for the process stderr stream. Finally pid is the O/S PID of the process. This is then followed by a line number and colon. So [Java:err:37401] 1: is line 1 of the stderr stream of our process. We gave the process the name “Java” and the O/S assigned it a PID of 37401.
As you can see the first two err lines are typically what you would expect from running java -version. The two lines containing (terminated) are from the Oracle Tools framework to signify the end of the process output for that stream, so there will be one for stderr and one for stdout.

So what we have ended up with is basically the same as running java -version from a command line in our current working directory.

That is the most basic example of running a process. Using SimplaApplicationSchema we can configure the process executable, its command line arguments, working directory and environment variables. This is pretty much all you would need to run any externally forked process, after all it is no less than you get with Java’s own ProcessBuilder. Oracle Tools though has some specialisations of the ApplicationSchema that make it easier to configure Java applications and Coherence applications.

Non-Coherence Java Processes

To run an external non-Coherence java process you will need to have a dependency on the jar files from the oracle-tool-core and oracle-tools-runtime modules; so still no Coherence dependencies yet.

When working with Java processes you now have two options on how to run them. You can run them externally, i.e. as a forked process as we did above; or you can run them in a virtualised sandbox inside the same process. Running internally has advantages, especially when testing, in that you are not left with various forked processes still running if your parent process dies.

To configure a Java process instead of SimpleApplicationSchema we can use SimpleJavaApplicationSchema which has a number of useful methods to configure settings specific to Java processes. At a minimum we need to create an instance of SimpleJavaApplicationSchema and specify a class name to run. We then have the option to specify a whole host of other options such as class path, system properties, JVM options and JMX configuration.

External Java Process

First we will run an example of a simple Java class. As with our first example above I wanted to run something that everyone will have on their system without me having to use a custom class. In this case given my site is pretty much all about Oracle Coherence I assume you have a Coherence jar file somewhere so I am going to run com.tangosol.net.DefaultCacheServer. I know this is a Coherence class and this section of the blog is supposed to be running a non-Coherence Java process, so we will have to pretend for a while.

  1. package com.thegridman.oracle.tools.examples;
  2.  
  3. import com.oracle.tools.runtime.console.SystemApplicationConsole;
  4. import com.oracle.tools.runtime.java.ExternalJavaApplicationBuilder;
  5. import com.oracle.tools.runtime.java.JavaApplication;
  6. import com.oracle.tools.runtime.java.SimpleJavaApplicationSchema;
  7.  
  8. import java.io.File;
  9.  
  10. public class RunJavaClassForked
  11. {
  12.     public static void main(String[] args) throws Exception
  13.     {
  14.         String classpath = "/m2/repository/com/oracle/coherence/coherence/3.7.1.7/coherence-3.7.1.7.jar";
  15.  
  16.         SimpleJavaApplicationSchema schema =
  17.                   new SimpleJavaApplicationSchema("com.tangosol.net.DefaultCacheServer", classpath)
  18.                     .setArgument("coherence-cache-config.xml")
  19.                     .setOption("-Xmx256m")
  20.                     .setSystemProperty("tangosol.coherence.clusterport", "12345")
  21.                     .setWorkingDirectory(new File("/Users/jonathanknight"));
  22.  
  23.         ExternalJavaApplicationBuilder builder =
  24.                 new ExternalJavaApplicationBuilder();
  25.  
  26.         JavaApplication application =
  27.                 builder.realize(schema, "DCS", new SystemApplicationConsole());
  28.  
  29.         Thread.sleep(10000);
  30.         application.destroy();
  31.     }
  32. }
package com.thegridman.oracle.tools.examples;

import com.oracle.tools.runtime.console.SystemApplicationConsole;
import com.oracle.tools.runtime.java.ExternalJavaApplicationBuilder;
import com.oracle.tools.runtime.java.JavaApplication;
import com.oracle.tools.runtime.java.SimpleJavaApplicationSchema;

import java.io.File;

public class RunJavaClassForked
{
    public static void main(String[] args) throws Exception
    {
        String classpath = "/m2/repository/com/oracle/coherence/coherence/3.7.1.7/coherence-3.7.1.7.jar";

        SimpleJavaApplicationSchema schema =
                  new SimpleJavaApplicationSchema("com.tangosol.net.DefaultCacheServer", classpath)
                    .setArgument("coherence-cache-config.xml")
                    .setOption("-Xmx256m")
                    .setSystemProperty("tangosol.coherence.clusterport", "12345")
                    .setWorkingDirectory(new File("/Users/jonathanknight"));

        ExternalJavaApplicationBuilder builder =
                new ExternalJavaApplicationBuilder();

        JavaApplication application =
                builder.realize(schema, "DCS", new SystemApplicationConsole());

        Thread.sleep(10000);
        application.destroy();
    }
}

As with the previous example the interesting bit is lines 14 – 30.

  • The first thing we need to do on lines 14 is work out what the class path will be for running our class. In this case we need a coherence.jar on the class path and on my Mac Book Pro this lives in a Maven repository.
  • On line 16 and 17 we create our SimpleJavaApplicationSchema that configures the java application, we specify the class to run, in this case com.tangosol.net.DefaultCacheServer and the class path
    At this point we have nothing else you need to configure to run DefaultCacheServer, as it will happily start up with no more configuration at all but that would not be much of an example so we will set some other options
  • On line 18 we set an argument that will be passed to the class’s main method – in this case coherence-cache-config.xml as DefaultCacheServer allows you to pass the name of the cache configuration file as the first argument..
    On line 19 we set a JVM option to set the maximum heap size to 256 MB.
    On line 20 we set a the multi-cast cluster port using the relevant system property
    On line 21 we set the working directory to my home directory.
  • On line 23 we create our application builder, in this case we want to run a forked external Java process so we create an instance of ExternalJavaApplicationBuilder
  • On line 26 we realize a running instance of our JConsole application, which we have named “DCS” and we are using a SystemApplicationConsole to capture the output
  • On line 29 we sleep for 10 seconds and on line 30 we destroy (kill) the process (there is no real need to sleep, but this example would run so fast otherwise the process would be killed before anything was output). This is different to the previous example where we waited for the application to finish by calling waitFor(). DefaultCacheServer is a server application that does not end until we somehow shut it down it. If we used waitFor() as in the previous example then this code would block until we shut down the DefaultCacheServer. The code we have here will force the DefaultCacheServer to close after 10 seconds.

So if you run the code above you will see in the console output DefaultCacheServer start then after 10 seconds die as the process is killed.
When running an external Java process like the example above the class being run must have a public static void main(String[] args) method as with any Java class being run from the command line. In effect what we have just run above would equate to a command line of…
java -Xmx256m -tangosol.coherence.clusterport=12345 -cp=/m2/repository/com/oracle/coherence/coherence/3.7.1.7/coherence-3.7.1.7.jar com.tangosol.net.DefaultCacheServer coherence-cache-config.xml

While DefaultCacheServer is running, if you look at the processes running on your machine you should see an new Java process that is DefaultCacheServer and the PID should match the PID in the console output.

You can see that when we relized the process we recieved an instance of JavaApplication instead of the SimpleApplication in the previous examples. The JavaApplication class has a number of Java specific methods on it.

  • You can obtain the System properties that were used to start the process. These are just the properties used at start up and do not reflect any changes that might be made to the process system properties while it is running.
  • You can also interact with the process JMX server and perform JMX queries. This can be useful for verifying application state if the process you are running supports JMX
  • Internal Java Process

    A feature of Oracle Tools is that it allows you to run Java processes in a virtualized sandbox inside the parent process. This sandbox isolates the applications class path, system properties and JMX server from the parent so for most purposes it can be treated like a separate application.

    There are a few differences to be aware of between external and virtualized processes.

    • You are not restricted to using the usual Java main method to start the virtualized process, you can use any method with the caveat that the method must not block.
    • It is not possible for the destroy method to kill the process in the same way that it does with an external process. To stop the virtualized process you can specify a stop method that will be called on the class to programatically shut it down.

    As an example we will run the same DefaultCacheServer again but this time as an internal virtualized process.

    1. package com.thegridman.oracle.tools.examples;
    2.  
    3. import com.oracle.tools.runtime.console.SystemApplicationConsole;
    4. import com.oracle.tools.runtime.java.JavaApplication;
    5. import com.oracle.tools.runtime.java.SimpleJavaApplicationSchema;
    6. import com.oracle.tools.runtime.java.VirtualizedJavaApplicationBuilder;
    7.  
    8. import java.io.File;
    9.  
    10. public class RunJavaClassInProcess
    11. {
    12.     public static void main(String[] args) throws Exception
    13.     {
    14.         String classpath = "/m2/repository/com/oracle/coherence/coherence/3.7.1.7/coherence-3.7.1.7.jar";
    15.  
    16.         SimpleJavaApplicationSchema schema =
    17.                   new SimpleJavaApplicationSchema("com.tangosol.net.DefaultCacheServer", classpath)
    18.                       .setStartMethodName("startDaemon")
    19.                       .setStopMethodName("shutdown")
    20.                       .setSystemProperty("tangosol.coherence.cacheconfig", "coherence-cache-config.xml")
    21.                       .setOption("-Xmx256m")
    22.                       .setSystemProperty("tangosol.coherence.clusterport", "12345")
    23.                       .setWorkingDirectory(new File("/Users/jonathanknight"));
    24.  
    25.         VirtualizedJavaApplicationBuilder builder =
    26.                 new VirtualizedJavaApplicationBuilder();
    27.  
    28.         JavaApplication application =
    29.                 builder.realize(schema, "DCS", new SystemApplicationConsole());
    30.  
    31.         Thread.sleep(10000);
    32.         application.destroy();
    33.     }
    34. }
    package com.thegridman.oracle.tools.examples;
    
    import com.oracle.tools.runtime.console.SystemApplicationConsole;
    import com.oracle.tools.runtime.java.JavaApplication;
    import com.oracle.tools.runtime.java.SimpleJavaApplicationSchema;
    import com.oracle.tools.runtime.java.VirtualizedJavaApplicationBuilder;
    
    import java.io.File;
    
    public class RunJavaClassInProcess
    {
        public static void main(String[] args) throws Exception
        {
            String classpath = "/m2/repository/com/oracle/coherence/coherence/3.7.1.7/coherence-3.7.1.7.jar";
    
            SimpleJavaApplicationSchema schema =
                      new SimpleJavaApplicationSchema("com.tangosol.net.DefaultCacheServer", classpath)
                          .setStartMethodName("startDaemon")
                          .setStopMethodName("shutdown")
                          .setSystemProperty("tangosol.coherence.cacheconfig", "coherence-cache-config.xml")
                          .setOption("-Xmx256m")
                          .setSystemProperty("tangosol.coherence.clusterport", "12345")
                          .setWorkingDirectory(new File("/Users/jonathanknight"));
    
            VirtualizedJavaApplicationBuilder builder =
                    new VirtualizedJavaApplicationBuilder();
    
            JavaApplication application =
                    builder.realize(schema, "DCS", new SystemApplicationConsole());
    
            Thread.sleep(10000);
            application.destroy();
        }
    }

    You can see that the code is almost identical

  • On line 18 we call setStartMethodName to specify the name of the method to call to start the process. In this case we do not call the DefaultCacheServer.main method as this is a blocking method that will not return, so instead we use DefaultCacheServer.startDaemon
  • On line 19 we call setStopMethodName to set the method to call to stop the process, in this case we call DefaultCacheServer.shutdown.
  • On line 20 we have change how we set the cache configuration file. The DefaultCacheServer.startDaemon does not take any arguments so we cannot use the setArgument method so instead we set the relevant system property.
  • Line 25 is where we now create a VirtualizedJavaApplicationBuilder to build our application as an internal virtualized process instead of external as in the previous example.
  • The rest of the code is identical to the previous example. If we run the code you can see exactly the same output in the console as previously with the exception that there is no PID shown at the start of each line. This is because there is no external process and if you look at the processes running on your machine while the code runs you will see not other processes are spawned.

    Coherence Processes

    As we saw above it is possible to start a Coherence cache server by treating it as just another Java class, which it obviously is, but there is an easier way and more functionaly rich way if we use the Coherence specific parts of Oracle Tools. To run a Coherence process you will need to have a dependency on the jar files from the oracle-tool-core, oracle-tools-runtime and oracle-tools-coherence modules as well as providing a coherence.jar file. As Coherence applications are Java processes you have the same options regarding whether you run externally or in a virtualized process. Oracle Tools also provides a cluster builder that allows you to control a whole cluster of processes at once.

    To create Coherence cluster member processes instead of using SimpleJavaApplicationSchema we use ClusterMemberSchema. This class has all of the same functionality as the previous two schema classes we have looked at, so you can set all of the normal process stuff, and Java specific properties, but it also has a host of Coherence specific settings too. For example, instead of setting the cache configuration using the setSystemProperty method as we did above, and having to remember to type the correct property name, we can just use ClusterMemberSchema.setCacheConfigURI instead.

    So, on with a few examples…

    External Coherence Process

    In this example we will run a single cache server process, the same as the previous example but using ClusterMemberSchema.

    1. package com.thegridman.oracle.tools.examples;
    2.  
    3. import com.oracle.tools.runtime.coherence.ClusterMember;
    4. import com.oracle.tools.runtime.coherence.ClusterMemberSchema;
    5. import com.oracle.tools.runtime.console.SystemApplicationConsole;
    6. import com.oracle.tools.runtime.java.ExternalJavaApplicationBuilder;
    7.  
    8. import java.io.File;
    9.  
    10. import static com.oracle.tools.deferred.DeferredAssert.assertThat;
    11. import static com.oracle.tools.deferred.DeferredHelper.eventually;
    12. import static com.oracle.tools.deferred.DeferredHelper.invoking;
    13. import static org.hamcrest.CoreMatchers.is;
    14.  
    15. public class RunCacheServerForked
    16. {
    17.     public static void main(String[] args) throws Exception
    18.     {
    19.         String classpath = "/m2/repository/com/oracle/coherence/coherence/3.7.1.7/coherence-3.7.1.7.jar";
    20.  
    21.         ClusterMemberSchema schema = new ClusterMemberSchema("com.tangosol.net.DefaultCacheServer", classpath)
    22.                 .setCacheConfigURI("coherence-cache-config.xml")
    23.                 .setPofEnabled(true)
    24.                 .setPofConfigURI("pof-config.xml")
    25.                 .setClusterPort(12345)
    26.                 .setStorageEnabled(true)
    27.                 .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
    28.                 .setOption("-Xmx256m")
    29.                 .setWorkingDirectory(new File("/Users/jonathanknight"));
    30.  
    31.         ExternalJavaApplicationBuilder<ClusterMember, ClusterMemberSchema> builder =
    32.                     new ExternalJavaApplicationBuilder<ClusterMember, ClusterMemberSchema>();
    33.  
    34.         ClusterMember member = builder.realize(schema, "Data", new SystemApplicationConsole());
    35.  
    36.         assertThat(eventually(invoking(member).getClusterSize()), is(1));
    37.  
    38.         Thread.sleep(10000);
    39.         member.destroy();
    40.     }
    41. }
    package com.thegridman.oracle.tools.examples;
    
    import com.oracle.tools.runtime.coherence.ClusterMember;
    import com.oracle.tools.runtime.coherence.ClusterMemberSchema;
    import com.oracle.tools.runtime.console.SystemApplicationConsole;
    import com.oracle.tools.runtime.java.ExternalJavaApplicationBuilder;
    
    import java.io.File;
    
    import static com.oracle.tools.deferred.DeferredAssert.assertThat;
    import static com.oracle.tools.deferred.DeferredHelper.eventually;
    import static com.oracle.tools.deferred.DeferredHelper.invoking;
    import static org.hamcrest.CoreMatchers.is;
    
    public class RunCacheServerForked
    {
        public static void main(String[] args) throws Exception
        {
            String classpath = "/m2/repository/com/oracle/coherence/coherence/3.7.1.7/coherence-3.7.1.7.jar";
    
            ClusterMemberSchema schema = new ClusterMemberSchema("com.tangosol.net.DefaultCacheServer", classpath)
                    .setCacheConfigURI("coherence-cache-config.xml")
                    .setPofEnabled(true)
                    .setPofConfigURI("pof-config.xml")
                    .setClusterPort(12345)
                    .setStorageEnabled(true)
                    .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
                    .setOption("-Xmx256m")
                    .setWorkingDirectory(new File("/Users/jonathanknight"));
    
            ExternalJavaApplicationBuilder<ClusterMember, ClusterMemberSchema> builder =
                        new ExternalJavaApplicationBuilder<ClusterMember, ClusterMemberSchema>();
    
            ClusterMember member = builder.realize(schema, "Data", new SystemApplicationConsole());
    
            assertThat(eventually(invoking(member).getClusterSize()), is(1));
    
            Thread.sleep(10000);
            member.destroy();
        }
    }

    So, very similar to our previous examples, we create a schema and use a builder to realize the running process. This time though things are a bit more Coherence specific.

  • In lines 21 – 29 we set up our ClusterMemberSchema with the configuration we require. You can see that we have used various custom methods to set things like the cache configuration, enable POF, set the POF configuration and set the JMX mode. This is much easier than using properties.
  • In line 31 we create the same ExternalJavaApplicationBuilder that we used previously to create an externally forked process.
  • In line 34 we realize our process using the builder, we name it “Data” and use a SystemApplicationConsole to capture the output. You will see that the process created is an instance of ClusterMember which has other useful methods on.
  • In line 36 we use one of these methods ClusterMember.getClusterSize() to tell us the size of the cluster (we expect it to eventually be one once the process starts up). We use a deferred assertion, which I covered above, to basically assert that the process starts properly within a reasonable time.
  • Finally on lines 38 and 39 we wait for 10 seconds then kill the process (again we don’t really need to sleep, and would in a normal test, but this allows the exmaple time to do something before it is killed).


  • When we realize the Coherence process the builder returns an instance of ClusterMember which is a Coherence specific version of the previous JavaApplication. We can still get all of the same functionality as the JavaProcess but we also get some extra Coherence methods.

  • We can get the cluster size, in our example this was one, but if running more nodes we would get a different value. The getClusterSize() method uses JMX to communicate the the JMX server of the process and interrogate the Coherence Cluster MBean. Obviously the MBean Server may take a while to start along with the process so that is why we checked the value using a deferred assertion.
  • You can obtain the local member ID of the process. This is the id that this node has been assigned in the cluster and again is obtained via a JMX call. This value is then useful when constructing ObjectName values to query other Coherence MBeans for this process.
  • Internal Coherence Process

    Running our Coherence process as a virtualized process is nothing more than change the builder used.

    1. package com.thegridman.oracle.tools.examples;
    2.  
    3. import com.oracle.tools.runtime.coherence.ClusterMember;
    4. import com.oracle.tools.runtime.coherence.ClusterMemberSchema;
    5. import com.oracle.tools.runtime.console.SystemApplicationConsole;
    6. import com.oracle.tools.runtime.java.VirtualizedJavaApplicationBuilder;
    7.  
    8. import java.io.File;
    9.  
    10. import static com.oracle.tools.deferred.DeferredAssert.assertThat;
    11. import static com.oracle.tools.deferred.DeferredHelper.eventually;
    12. import static com.oracle.tools.deferred.DeferredHelper.invoking;
    13. import static org.hamcrest.CoreMatchers.is;
    14.  
    15. public class RunCacheServerInProcess
    16. {
    17.     public static void main(String[] args) throws Exception
    18.     {
    19.         String classpath = "/m2/repository/com/oracle/coherence/coherence/3.7.1.7/coherence-3.7.1.7.jar";
    20.  
    21.         ClusterMemberSchema schema = new ClusterMemberSchema("com.tangosol.net.DefaultCacheServer", classpath)
    22.                 .setCacheConfigURI("coherence-cache-config.xml")
    23.                 .setPofEnabled(true)
    24.                 .setPofConfigURI("pof-config.xml")
    25.                 .setClusterPort(12345)
    26.                 .setStorageEnabled(true)
    27.                 .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
    28.                 .setOption("-Xmx256m")
    29.                 .setWorkingDirectory(new File("/Users/jonathanknight"));
    30.  
    31.         VirtualizedJavaApplicationBuilder<ClusterMember, ClusterMemberSchema> builder =
    32.                     new VirtualizedJavaApplicationBuilder<ClusterMember, ClusterMemberSchema>();
    33.  
    34.         ClusterMember member = builder.realize(schema, "Data", new SystemApplicationConsole());
    35.  
    36.         assertThat(eventually(invoking(member).getClusterSize()), is(1));
    37.  
    38.         Thread.sleep(10000);
    39.         member.destroy();
    40.     }
    41. }
    package com.thegridman.oracle.tools.examples;
    
    import com.oracle.tools.runtime.coherence.ClusterMember;
    import com.oracle.tools.runtime.coherence.ClusterMemberSchema;
    import com.oracle.tools.runtime.console.SystemApplicationConsole;
    import com.oracle.tools.runtime.java.VirtualizedJavaApplicationBuilder;
    
    import java.io.File;
    
    import static com.oracle.tools.deferred.DeferredAssert.assertThat;
    import static com.oracle.tools.deferred.DeferredHelper.eventually;
    import static com.oracle.tools.deferred.DeferredHelper.invoking;
    import static org.hamcrest.CoreMatchers.is;
    
    public class RunCacheServerInProcess
    {
        public static void main(String[] args) throws Exception
        {
            String classpath = "/m2/repository/com/oracle/coherence/coherence/3.7.1.7/coherence-3.7.1.7.jar";
    
            ClusterMemberSchema schema = new ClusterMemberSchema("com.tangosol.net.DefaultCacheServer", classpath)
                    .setCacheConfigURI("coherence-cache-config.xml")
                    .setPofEnabled(true)
                    .setPofConfigURI("pof-config.xml")
                    .setClusterPort(12345)
                    .setStorageEnabled(true)
                    .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
                    .setOption("-Xmx256m")
                    .setWorkingDirectory(new File("/Users/jonathanknight"));
    
            VirtualizedJavaApplicationBuilder<ClusterMember, ClusterMemberSchema> builder =
                        new VirtualizedJavaApplicationBuilder<ClusterMember, ClusterMemberSchema>();
    
            ClusterMember member = builder.realize(schema, "Data", new SystemApplicationConsole());
    
            assertThat(eventually(invoking(member).getClusterSize()), is(1));
    
            Thread.sleep(10000);
            member.destroy();
        }
    }

    You can see the code above is identical to the previous example except that it uses a VirtualizedJavaApplicationBuilder and runs the Coherence node as a virtualized process. We still have all the same functioanlity as the previous external process.

    External Coherence Cluster

    Typically when working with and testing Coherence application we are not working with a single process but a whole cluster. It is uaual in an accepance or integration test to run a couple of storage nodes and maybe, if you require it, an extend proxy node. This is where Oracle Tools makes it easier by allowing you to build and control a whole cluster together.

    1. package com.thegridman.oracle.tools.examples;
    2.  
    3. import com.oracle.tools.deferred.DeferredAssert;
    4. import com.oracle.tools.runtime.coherence.Cluster;
    5. import com.oracle.tools.runtime.coherence.ClusterBuilder;
    6. import com.oracle.tools.runtime.coherence.ClusterMember;
    7. import com.oracle.tools.runtime.coherence.ClusterMemberSchema;
    8. import com.oracle.tools.runtime.console.SystemApplicationConsole;
    9. import com.oracle.tools.runtime.java.ExternalJavaApplicationBuilder;
    10. import com.oracle.tools.runtime.network.AvailablePortIterator;
    11.  
    12. import java.io.File;
    13.  
    14. import static com.oracle.tools.deferred.DeferredHelper.eventually;
    15. import static com.oracle.tools.deferred.DeferredHelper.invoking;
    16. import static org.hamcrest.CoreMatchers.is;
    17.  
    18. public class RunClusterForked
    19. {
    20.     public static void main(String[] args) throws Exception
    21.     {
    22.         AvailablePortIterator ports = new AvailablePortIterator(40000);
    23.  
    24.         ClusterMemberSchema storage
    25.             = new ClusterMemberSchema()
    26.                     .setCacheConfigURI("coherence-cache-config.xml")
    27.                     .setPofEnabled(true)
    28.                     .setPofConfigURI("pof-config.xml")
    29.                     .setClusterPort(12345)
    30.                     .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
    31.                     .setJMXPort(ports)
    32.                     .setOption("-Xmx256m")
    33.                     .setWorkingDirectory(new File("/Users/jonathanknight"))
    34.                     .setStorageEnabled(true);
    35.  
    36.         ClusterMemberSchema extend
    37.             = new ClusterMemberSchema()
    38.                     .setCacheConfigURI("coherence-cache-config.xml")
    39.                     .setPofEnabled(true)
    40.                     .setPofConfigURI("pof-config.xml")
    41.                     .setClusterPort(12345)
    42.                     .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
    43.                     .setJMXPort(ports)
    44.                     .setOption("-Xmx256m")
    45.                     .setWorkingDirectory(new File("/Users/jonathanknight"))
    46.                     .setStorageEnabled(false)
    47.                     .setSystemProperty("tangosol.coherence.extend.enabled", true)
    48.                     .setSystemProperty("tangosol.coherence.extend.port", ports);
    49.  
    50.         ExternalJavaApplicationBuilder<ClusterMember,ClusterMemberSchema> builder
    51.                 = new ExternalJavaApplicationBuilder<ClusterMember,ClusterMemberSchema>();
    52.  
    53.         ClusterBuilder clusterBuilder = new ClusterBuilder();
    54.         clusterBuilder.addBuilder(builder, storage, "Data", 2);
    55.         clusterBuilder.addBuilder(builder, extend, "Proxy", 1);
    56.  
    57.         Cluster cluster = clusterBuilder.realize(new SystemApplicationConsole());
    58.  
    59.         DeferredAssert.assertThat(eventually(invoking(cluster).getClusterSize()), is(3));
    60.  
    61.         cluster.destroy();
    62.     }
    63. }
    package com.thegridman.oracle.tools.examples;
    
    import com.oracle.tools.deferred.DeferredAssert;
    import com.oracle.tools.runtime.coherence.Cluster;
    import com.oracle.tools.runtime.coherence.ClusterBuilder;
    import com.oracle.tools.runtime.coherence.ClusterMember;
    import com.oracle.tools.runtime.coherence.ClusterMemberSchema;
    import com.oracle.tools.runtime.console.SystemApplicationConsole;
    import com.oracle.tools.runtime.java.ExternalJavaApplicationBuilder;
    import com.oracle.tools.runtime.network.AvailablePortIterator;
    
    import java.io.File;
    
    import static com.oracle.tools.deferred.DeferredHelper.eventually;
    import static com.oracle.tools.deferred.DeferredHelper.invoking;
    import static org.hamcrest.CoreMatchers.is;
    
    public class RunClusterForked
    {
        public static void main(String[] args) throws Exception
        {
            AvailablePortIterator ports = new AvailablePortIterator(40000);
    
            ClusterMemberSchema storage
                = new ClusterMemberSchema()
                        .setCacheConfigURI("coherence-cache-config.xml")
                        .setPofEnabled(true)
                        .setPofConfigURI("pof-config.xml")
                        .setClusterPort(12345)
                        .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
                        .setJMXPort(ports)
                        .setOption("-Xmx256m")
                        .setWorkingDirectory(new File("/Users/jonathanknight"))
                        .setStorageEnabled(true);
    
            ClusterMemberSchema extend
                = new ClusterMemberSchema()
                        .setCacheConfigURI("coherence-cache-config.xml")
                        .setPofEnabled(true)
                        .setPofConfigURI("pof-config.xml")
                        .setClusterPort(12345)
                        .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
                        .setJMXPort(ports)
                        .setOption("-Xmx256m")
                        .setWorkingDirectory(new File("/Users/jonathanknight"))
                        .setStorageEnabled(false)
                        .setSystemProperty("tangosol.coherence.extend.enabled", true)
                        .setSystemProperty("tangosol.coherence.extend.port", ports);
    
            ExternalJavaApplicationBuilder<ClusterMember,ClusterMemberSchema> builder
                    = new ExternalJavaApplicationBuilder<ClusterMember,ClusterMemberSchema>();
    
            ClusterBuilder clusterBuilder = new ClusterBuilder();
            clusterBuilder.addBuilder(builder, storage, "Data", 2);
            clusterBuilder.addBuilder(builder, extend, "Proxy", 1);
    
            Cluster cluster = clusterBuilder.realize(new SystemApplicationConsole());
    
            DeferredAssert.assertThat(eventually(invoking(cluster).getClusterSize()), is(3));
    
            cluster.destroy();
        }
    }

    If you run the code above you will see in the console output that there are three Coherence nodes started and all three should form a cluster. As soon as the assertion has determined that all three nodes have joined the cluster, the cluster is destroyed. If you are quick enough you should see the three external processes fire up on your machine.

    Based on what we have already covered it should be easy to follow what the code is doing.

    • One new things is line 22, where we create an AvailablePortIterator. This allows us to obtain free port numbers without having to hard code them in our tests and risk clashing or in use port numbers. The ports will be dynamically assigned as required when applications are realized from schema that use the iterator. We have a few places in the code where we assign ports, lines 31 and 43 where we assign JMX ports and line 48 where we set the extend port.
    • Lines 24 – 34 create the schema that configures a storage node
    • Lines 36 – 48 create the schema that configures an extend proxy node
    • Lines 50 and 51 create the ExternalJavaApplicationBuilder as we want to fork the cluster as external processes
    • Line 53 is the new part where we create our ClusterBuilder that will realize and control the whole cluster of processes
    • Line 54 we tell the ClusterBuilder we want to realize two instances of processes from the storage node schema, the processes should be realized using the ExternalJavaApplicationBuilder and we will name the processes with the prefix “Data”. When the processes are realized they will actually be named Data-n where n is the instance number from zero to the number of required instance.
    • Line 55 we tell the ClusterBuilder we want to realize a single instance of a process using the proxy schema that we will name “Proxy” and realize with the ExternalJavaApplicationBuilder
    • Line 57 is where we actually realize the running cluster using the ClusterBuilder and obtain an instance of a Cluster. We use a SystemApplicationConsole to capture the output of all of the processes to stdout
    • Line 59 we wait for the result of cluster.getClusterSize() to be three, the expected cluster size
    • Line 61 we destroy the cluster, which will destroy all of the processes

    Being able to control and group all the various members of the cluster together makes the code a bit tider than having everything separate.

    Internal Coherence Cluster

    As with the previous internal example if we want to run all the members of the cluster as virtualized processes all we need to do is change a single line in the code. We would change line 50 to create a VirtualizedJavaApplicationBuilder and that would be it, the whole thing would run just the same. I will not bother repeating all the code again.

    Testing With Oracle Tools

    To finish off the examples here is an example test class that uses what I have covered above to start a Coherence cluster then run a test method that interacts with the cluster. This example uses JUnit but you should easily be able to convert it if you use another test framework.

    1. package com.thegridman.oracle.tools.examples;
    2.  
    3. import com.oracle.tools.deferred.DeferredAssert;
    4. import com.oracle.tools.runtime.coherence.Cluster;
    5. import com.oracle.tools.runtime.coherence.ClusterBuilder;
    6. import com.oracle.tools.runtime.coherence.ClusterMember;
    7. import com.oracle.tools.runtime.coherence.ClusterMemberSchema;
    8. import com.oracle.tools.runtime.console.SystemApplicationConsole;
    9. import com.oracle.tools.runtime.java.VirtualizedJavaApplicationBuilder;
    10. import com.oracle.tools.runtime.network.AvailablePortIterator;
    11. import com.tangosol.net.CacheFactory;
    12. import com.tangosol.net.CacheFactoryBuilder;
    13. import com.tangosol.net.ConfigurableCacheFactory;
    14. import com.tangosol.net.NamedCache;
    15. import com.tangosol.run.xml.XmlDocument;
    16. import com.tangosol.run.xml.XmlElement;
    17. import com.tangosol.run.xml.XmlHelper;
    18. import com.tangosol.run.xml.XmlValue;
    19. import org.junit.After;
    20. import org.junit.AfterClass;
    21. import org.junit.Assert;
    22. import org.junit.Before;
    23. import org.junit.BeforeClass;
    24. import org.junit.Test;
    25.  
    26. import javax.management.ObjectName;
    27. import java.io.File;
    28. import java.util.List;
    29. import java.util.Properties;
    30.  
    31. import static com.oracle.tools.deferred.DeferredHelper.eventually;
    32. import static com.oracle.tools.deferred.DeferredHelper.invoking;
    33. import static org.hamcrest.CoreMatchers.is;
    34. import static org.hamcrest.CoreMatchers.notNullValue;
    35.  
    36. public class ClusterTest
    37. {
    38.     private static Cluster cluster;
    39.  
    40.     private ConfigurableCacheFactory clientCacheFactory;
    41.  
    42.     @BeforeClass
    43.     public static void startCluster() throws Exception
    44.     {
    45.         AvailablePortIterator ports = new AvailablePortIterator(40000);
    46.  
    47.         ClusterMemberSchema storage = createStorageNodeSchema(ports);
    48.         ClusterMemberSchema extend =  createExtendProxySchema(ports);
    49.  
    50.         VirtualizedJavaApplicationBuilder<ClusterMember,ClusterMemberSchema> builder
    51.                 = new VirtualizedJavaApplicationBuilder<ClusterMember,ClusterMemberSchema>();
    52.  
    53.         ClusterBuilder clusterBuilder = new ClusterBuilder();
    54.         clusterBuilder.addBuilder(builder, storage, "Data", 2);
    55.         clusterBuilder.addBuilder(builder, extend, "Proxy", 1);
    56.  
    57.         cluster = clusterBuilder.realize(new SystemApplicationConsole());
    58.     }
    59.  
    60.     @Before
    61.     public void setupTest() throws Exception
    62.     {
    63.         // Assert the cluster is ready
    64.         Assert.assertThat(cluster, is(notNullValue()));
    65.         DeferredAssert.assertThat(eventually(invoking(cluster).getClusterSize()), is(3));
    66.  
    67.         // Assert the Proxy Service is running
    68.         assertServiceIsRunning("Proxy-0", "TcpProxyService");
    69.  
    70.         // Get the extend port the proxy is using
    71.         ClusterMember proxyNode = cluster.getApplication("Proxy-0");
    72.         String extendPort = proxyNode.getSystemProperty("tangosol.coherence.extend.port");
    73.  
    74.         // Create the client properties
    75.         Properties properties = new Properties(System.getProperties());
    76.         properties.setProperty("tangosol.coherence.extend.port", extendPort);
    77.  
    78.         // Create the client Cache Factory
    79.         clientCacheFactory = createClientCacheFactory("client-cache-config.xml", properties);
    80.     }
    81.  
    82.     @Test
    83.     public void shouldPutDataIntoCache() throws Exception
    84.     {
    85.         NamedCache cache = clientCacheFactory.ensureCache("dist-test", null);
    86.         cache.put("Key-1", "Value-1");
    87.         Assert.assertThat((String)cache.get("Key-1"), is("Value-1"));
    88.     }
    89.  
    90.     @After
    91.     public void shutdownClient()
    92.     {
    93.         if (clientCacheFactory != null)
    94.         {
    95.             CacheFactory.getCacheFactoryBuilder().release(clientCacheFactory);
    96.             clientCacheFactory = null;
    97.         }
    98.     }
    99.  
    100.     @AfterClass
    101.     public static void stopCluster()
    102.     {
    103.         if (cluster != null)
    104.         {
    105.             cluster.destroy();
    106.             cluster = null;
    107.         }
    108.     }
    109.  
    110.     public static ClusterMemberSchema createStorageNodeSchema(AvailablePortIterator ports) {
    111.         return createCommonSchema(ports)
    112.                     .setStorageEnabled(true);
    113.     }
    114.  
    115.     public static ClusterMemberSchema createExtendProxySchema(AvailablePortIterator ports) {
    116.         return createCommonSchema(ports)
    117.                     .setStorageEnabled(false)
    118.                     .setSystemProperty("tangosol.coherence.extend.enabled", true)
    119.                     .setSystemProperty("tangosol.coherence.extend.port", ports);
    120.     }
    121.  
    122.     public static ClusterMemberSchema createCommonSchema(AvailablePortIterator ports) {
    123.         return new ClusterMemberSchema()
    124.                 .setCacheConfigURI("coherence-cache-config.xml")
    125.                 .setPofEnabled(true)
    126.                 .setPofConfigURI("pof-config.xml")
    127.                 .setClusterPort(12345)
    128.                 .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
    129.                 .setJMXPort(ports)
    130.                 .setOption("-Xmx256m")
    131.                 .setWorkingDirectory(new File("/Users/jonathanknight"));
    132.     }
    133.  
    134.     private void assertServiceIsRunning(String memberName, String serviceName) throws Exception
    135.     {
    136.         Assert.assertThat(cluster, is(notNullValue()));
    137.         ClusterMember member = cluster.getApplication(memberName);
    138.         int nodeId = member.getLocalMemberId();
    139.         ObjectName objectName = new ObjectName(String.format("Coherence:type=Service,name=%s,nodeId=%d", serviceName, nodeId));
    140.         DeferredAssert.assertThat(eventually(invoking(member).getMBeanAttribute(objectName, "Running", Boolean.class)), is(true));
    141.     }
    142.  
    143.     private ConfigurableCacheFactory createClientCacheFactory(String cacheConfigurationURI, Properties properties)
    144.     {
    145.         XmlDocument clientConfigXml = XmlHelper.loadFileOrResource(cacheConfigurationURI, "Client");
    146.         replacePropertiesInXml(clientConfigXml, "system-property", properties);
    147.  
    148.         CacheFactoryBuilder cacheFactoryBuilder = CacheFactory.getCacheFactoryBuilder();
    149.         cacheFactoryBuilder.setCacheConfiguration(cacheConfigurationURI, null, clientConfigXml);
    150.         return cacheFactoryBuilder.getConfigurableCacheFactory(cacheConfigurationURI, null);
    151.     }
    152.  
    153.     @SuppressWarnings("unchecked")
    154.     private void replacePropertiesInXml(XmlElement xml, String propertyAttributeName, Properties properties)
    155.     {
    156.         XmlValue attribute = xml.getAttribute(propertyAttributeName);
    157.         if (attribute != null)
    158.         {
    159.             xml.setAttribute(propertyAttributeName, null);
    160.             try
    161.             {
    162.                 String propertyValue = properties.getProperty(attribute.getString());
    163.                 if (propertyValue != null)
    164.                 {
    165.                     xml.setString(propertyValue);
    166.                 }
    167.             }
    168.             catch (Exception _ignored)
    169.             {
    170.                 // ignored on purpose
    171.             }
    172.         }
    173.  
    174.         for (XmlElement child : (List<XmlElement>) xml.getElementList())
    175.         {
    176.             replacePropertiesInXml(child, propertyAttributeName, properties);
    177.         }
    178.     }
    179. }
    package com.thegridman.oracle.tools.examples;
    
    import com.oracle.tools.deferred.DeferredAssert;
    import com.oracle.tools.runtime.coherence.Cluster;
    import com.oracle.tools.runtime.coherence.ClusterBuilder;
    import com.oracle.tools.runtime.coherence.ClusterMember;
    import com.oracle.tools.runtime.coherence.ClusterMemberSchema;
    import com.oracle.tools.runtime.console.SystemApplicationConsole;
    import com.oracle.tools.runtime.java.VirtualizedJavaApplicationBuilder;
    import com.oracle.tools.runtime.network.AvailablePortIterator;
    import com.tangosol.net.CacheFactory;
    import com.tangosol.net.CacheFactoryBuilder;
    import com.tangosol.net.ConfigurableCacheFactory;
    import com.tangosol.net.NamedCache;
    import com.tangosol.run.xml.XmlDocument;
    import com.tangosol.run.xml.XmlElement;
    import com.tangosol.run.xml.XmlHelper;
    import com.tangosol.run.xml.XmlValue;
    import org.junit.After;
    import org.junit.AfterClass;
    import org.junit.Assert;
    import org.junit.Before;
    import org.junit.BeforeClass;
    import org.junit.Test;
    
    import javax.management.ObjectName;
    import java.io.File;
    import java.util.List;
    import java.util.Properties;
    
    import static com.oracle.tools.deferred.DeferredHelper.eventually;
    import static com.oracle.tools.deferred.DeferredHelper.invoking;
    import static org.hamcrest.CoreMatchers.is;
    import static org.hamcrest.CoreMatchers.notNullValue;
    
    public class ClusterTest
    {
        private static Cluster cluster;
    
        private ConfigurableCacheFactory clientCacheFactory;
    
        @BeforeClass
        public static void startCluster() throws Exception
        {
            AvailablePortIterator ports = new AvailablePortIterator(40000);
    
            ClusterMemberSchema storage = createStorageNodeSchema(ports);
            ClusterMemberSchema extend =  createExtendProxySchema(ports);
    
            VirtualizedJavaApplicationBuilder<ClusterMember,ClusterMemberSchema> builder
                    = new VirtualizedJavaApplicationBuilder<ClusterMember,ClusterMemberSchema>();
    
            ClusterBuilder clusterBuilder = new ClusterBuilder();
            clusterBuilder.addBuilder(builder, storage, "Data", 2);
            clusterBuilder.addBuilder(builder, extend, "Proxy", 1);
    
            cluster = clusterBuilder.realize(new SystemApplicationConsole());
        }
    
        @Before
        public void setupTest() throws Exception
        {
            // Assert the cluster is ready
            Assert.assertThat(cluster, is(notNullValue()));
            DeferredAssert.assertThat(eventually(invoking(cluster).getClusterSize()), is(3));
    
            // Assert the Proxy Service is running
            assertServiceIsRunning("Proxy-0", "TcpProxyService");
    
            // Get the extend port the proxy is using
            ClusterMember proxyNode = cluster.getApplication("Proxy-0");
            String extendPort = proxyNode.getSystemProperty("tangosol.coherence.extend.port");
    
            // Create the client properties
            Properties properties = new Properties(System.getProperties());
            properties.setProperty("tangosol.coherence.extend.port", extendPort);
    
            // Create the client Cache Factory
            clientCacheFactory = createClientCacheFactory("client-cache-config.xml", properties);
        }
    
        @Test
        public void shouldPutDataIntoCache() throws Exception
        {
            NamedCache cache = clientCacheFactory.ensureCache("dist-test", null);
            cache.put("Key-1", "Value-1");
            Assert.assertThat((String)cache.get("Key-1"), is("Value-1"));
        }
    
        @After
        public void shutdownClient()
        {
            if (clientCacheFactory != null)
            {
                CacheFactory.getCacheFactoryBuilder().release(clientCacheFactory);
                clientCacheFactory = null;
            }
        }
    
        @AfterClass
        public static void stopCluster()
        {
            if (cluster != null)
            {
                cluster.destroy();
                cluster = null;
            }
        }
    
        public static ClusterMemberSchema createStorageNodeSchema(AvailablePortIterator ports) {
            return createCommonSchema(ports)
                        .setStorageEnabled(true);
        }
    
        public static ClusterMemberSchema createExtendProxySchema(AvailablePortIterator ports) {
            return createCommonSchema(ports)
                        .setStorageEnabled(false)
                        .setSystemProperty("tangosol.coherence.extend.enabled", true)
                        .setSystemProperty("tangosol.coherence.extend.port", ports);
        }
    
        public static ClusterMemberSchema createCommonSchema(AvailablePortIterator ports) {
            return new ClusterMemberSchema()
                    .setCacheConfigURI("coherence-cache-config.xml")
                    .setPofEnabled(true)
                    .setPofConfigURI("pof-config.xml")
                    .setClusterPort(12345)
                    .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
                    .setJMXPort(ports)
                    .setOption("-Xmx256m")
                    .setWorkingDirectory(new File("/Users/jonathanknight"));
        }
    
        private void assertServiceIsRunning(String memberName, String serviceName) throws Exception
        {
            Assert.assertThat(cluster, is(notNullValue()));
            ClusterMember member = cluster.getApplication(memberName);
            int nodeId = member.getLocalMemberId();
            ObjectName objectName = new ObjectName(String.format("Coherence:type=Service,name=%s,nodeId=%d", serviceName, nodeId));
            DeferredAssert.assertThat(eventually(invoking(member).getMBeanAttribute(objectName, "Running", Boolean.class)), is(true));
        }
    
        private ConfigurableCacheFactory createClientCacheFactory(String cacheConfigurationURI, Properties properties)
        {
            XmlDocument clientConfigXml = XmlHelper.loadFileOrResource(cacheConfigurationURI, "Client");
            replacePropertiesInXml(clientConfigXml, "system-property", properties);
    
            CacheFactoryBuilder cacheFactoryBuilder = CacheFactory.getCacheFactoryBuilder();
            cacheFactoryBuilder.setCacheConfiguration(cacheConfigurationURI, null, clientConfigXml);
            return cacheFactoryBuilder.getConfigurableCacheFactory(cacheConfigurationURI, null);
        }
    
        @SuppressWarnings("unchecked")
        private void replacePropertiesInXml(XmlElement xml, String propertyAttributeName, Properties properties)
        {
            XmlValue attribute = xml.getAttribute(propertyAttributeName);
            if (attribute != null)
            {
                xml.setAttribute(propertyAttributeName, null);
                try
                {
                    String propertyValue = properties.getProperty(attribute.getString());
                    if (propertyValue != null)
                    {
                        xml.setString(propertyValue);
                    }
                }
                catch (Exception _ignored)
                {
                    // ignored on purpose
                }
            }
    
            for (XmlElement child : (List<XmlElement>) xml.getElementList())
            {
                replacePropertiesInXml(child, propertyAttributeName, properties);
            }
        }
    }

    We will go through the class bit by bit

    @BeforeClass Method – Starting the Cluster

    The obvious place to start the cluster for a test class is before anything else runs, so in this case we use a method annotated with @BeforeClass I have called obviously enough startCluster().

    1. @BeforeClass
    2. public static void startCluster() throws Exception
    3. {
    4.     AvailablePortIterator ports = new AvailablePortIterator(40000);
    5.  
    6.     ClusterMemberSchema storage = createStorageNodeSchema(ports);
    7.     ClusterMemberSchema extend =  createExtendProxySchema(ports);
    8.  
    9.     VirtualizedJavaApplicationBuilder<ClusterMember,ClusterMemberSchema> builder
    10.             = new VirtualizedJavaApplicationBuilder<ClusterMember,ClusterMemberSchema>();
    11.  
    12.     ClusterBuilder clusterBuilder = new ClusterBuilder();
    13.     clusterBuilder.addBuilder(builder, storage, "Data", 2);
    14.     clusterBuilder.addBuilder(builder, extend, "Proxy", 1);
    15.  
    16.     cluster = clusterBuilder.realize(new SystemApplicationConsole());
    17. }
    @BeforeClass
    public static void startCluster() throws Exception
    {
        AvailablePortIterator ports = new AvailablePortIterator(40000);
    
        ClusterMemberSchema storage = createStorageNodeSchema(ports);
        ClusterMemberSchema extend =  createExtendProxySchema(ports);
    
        VirtualizedJavaApplicationBuilder<ClusterMember,ClusterMemberSchema> builder
                = new VirtualizedJavaApplicationBuilder<ClusterMember,ClusterMemberSchema>();
    
        ClusterBuilder clusterBuilder = new ClusterBuilder();
        clusterBuilder.addBuilder(builder, storage, "Data", 2);
        clusterBuilder.addBuilder(builder, extend, "Proxy", 1);
    
        cluster = clusterBuilder.realize(new SystemApplicationConsole());
    }

    The startCluster() method should be easy to understand as it is pretty identical to the last example. We have extracted out the schema creation into two other methods createStorageNodeSchema and createExtendProxySchema. If you look at these methods you will see we have a further optimisation where we extract creation of all the common schema settings into another method createCommonSchema. This meas there is less code and is obviously less error prone.

    @Before – Verify The Cluster is Ready & Setup the Test

    Before we run any test methods we want to make sure that the cluster is available, and also in our case we are going to connect over Extend, so we want to make sure the proxy is running. On top of that we will also set up the client Cache Factory that the tests will use to connect to the cluster. It makes sense to do this in one common method rather than every test method (even though in this example we only have one test method). All of the code I have put in the @Before method could just have easily been put in the @BeforeClass method if you only want to do it once for the whole test class (or even somewhere once for a suite).

    1. @Before
    2. public void setupTest() throws Exception
    3. {
    4.     // Assert the cluster is ready
    5.     Assert.assertThat(cluster, is(notNullValue()));
    6.     DeferredAssert.assertThat(eventually(invoking(cluster).getClusterSize()), is(3));
    7.  
    8.     // Assert the Proxy Service is running
    9.     assertServiceIsRunning("Proxy-0", "TcpProxyService");
    10.  
    11.     // Get the extend port the proxy is using
    12.     ClusterMember proxyNode = cluster.getApplication("Proxy-0");
    13.     String extendPort = proxyNode.getSystemProperty("tangosol.coherence.extend.port");
    14.  
    15.     // Create the client properties
    16.     Properties properties = new Properties(System.getProperties());
    17.     properties.setProperty("tangosol.coherence.extend.port", extendPort);
    18.  
    19.     // Create the client Cache Factory
    20.     clientCacheFactory = createClientCacheFactory("client-cache-config.xml", properties);
    21. }
    @Before
    public void setupTest() throws Exception
    {
        // Assert the cluster is ready
        Assert.assertThat(cluster, is(notNullValue()));
        DeferredAssert.assertThat(eventually(invoking(cluster).getClusterSize()), is(3));
    
        // Assert the Proxy Service is running
        assertServiceIsRunning("Proxy-0", "TcpProxyService");
    
        // Get the extend port the proxy is using
        ClusterMember proxyNode = cluster.getApplication("Proxy-0");
        String extendPort = proxyNode.getSystemProperty("tangosol.coherence.extend.port");
    
        // Create the client properties
        Properties properties = new Properties(System.getProperties());
        properties.setProperty("tangosol.coherence.extend.port", extendPort);
    
        // Create the client Cache Factory
        clientCacheFactory = createClientCacheFactory("client-cache-config.xml", properties);
    }

    So, in our @Before method the first thing we do is assert we actually have a cluster.
    Next we assert that the cluster size is three, using a deferred assertion the same as we have done in other examples.
    Next we assert that the proxy service is running. For this we use a utility method that checks via JMX that the service is running.

    1. private void assertServiceIsRunning(String memberName, String serviceName) throws Exception
    2. {
    3.     Assert.assertThat(cluster, is(notNullValue()));
    4.     ClusterMember member = cluster.getApplication(memberName);
    5.     int nodeId = member.getLocalMemberId();
    6.     ObjectName objectName = new ObjectName(String.format("Coherence:type=Service,name=%s,nodeId=%d", serviceName, nodeId));
    7.     DeferredAssert.assertThat(eventually(invoking(member).getMBeanAttribute(objectName, "Running", Boolean.class)), is(true));
    8. }
    private void assertServiceIsRunning(String memberName, String serviceName) throws Exception
    {
        Assert.assertThat(cluster, is(notNullValue()));
        ClusterMember member = cluster.getApplication(memberName);
        int nodeId = member.getLocalMemberId();
        ObjectName objectName = new ObjectName(String.format("Coherence:type=Service,name=%s,nodeId=%d", serviceName, nodeId));
        DeferredAssert.assertThat(eventually(invoking(member).getMBeanAttribute(objectName, "Running", Boolean.class)), is(true));
    }

    This is quite a useful method and I may at some point feed it back into the Oracle Tool project.
    The method takes two parameters, the name of the member the service is on and the name of the service. If you remember our proxy member will be called Proxy-0 as we told the ClusterBuilder to use the prefix “Proxy” for the extend proxy member and there is only a single instance which will be instance zero. The second parameter is the name of the service; this name comes from the cache configuration file. We are using the coherence-cache-config.xml file from inside the coherence jar file and the name of the proxy service in this file is TcpProxyService.

    • The first thing we do is assert we have a cluster, this is better than just throwing a NullPointerException later.
    • Next we get the proxy member from the cluster using the specified member name.
    • Now we have the member we need to get its cluster node ID, which Oracle Tools does via JMX.
    • Once we have the node ID we can create the JMX ObjectName for the service. All service names in Coherence follow the same naming pattern so this is easy to do. There is a method on the member called getMBeanAttribute that will retunr the value of a given MBean attribute. We use this to get the value of the “Running” attribute of the service. We wrap this in a deferred assertion to give the service a reasonable time to start up.

    Now we know the proxy is started we can configure our Cache Factory that the tests will use. If you look at the code of the test method you will see that the NamedCache is not obtained using the static CacheFactory.getCache method but from a real instance of a ConfigurableCacheFactory. Personally I would avoid using the static methods in Coherence to do things as much as you can. Using statics makes tests hard to control and makes things hard to mock. I know there are mock frameworks that let you mock static calls but doing this is like worshiping the devil, so better not to use statics in the first place. So, how do we create our client cache factory and tell it the ports for the proxy…

  • First we need to know the extend port. If you remember we have not hard coded this, but instead we used an AvailablePortIterator to pick a port when the process is realized. You will rember I covered earlier that the Java process and hence ClusterMember instance has a method on it to obtain the values of System properties used to realize the process and this is how we get the port used.
    1. ClusterMember proxyNode = cluster.getApplication("Proxy-0");
    2. String extendPort = proxyNode.getSystemProperty("tangosol.coherence.extend.port");
    ClusterMember proxyNode = cluster.getApplication("Proxy-0");
    String extendPort = proxyNode.getSystemProperty("tangosol.coherence.extend.port");
  • Now we have the port we can use another useful utility method createClientCacheFactory to create the client Cache Factory. This method takes the name of the cache configuration file to use and a set of properties to use to replace any properties in the XML from the configuration file – in our case the extend port.
    1. // Create the client properties
    2. Properties properties = new Properties(System.getProperties());
    3. properties.setProperty("tangosol.coherence.extend.port", extendPort);
    4.  
    5. // Create the client Cache Factory
    6. clientCacheFactory = createClientCacheFactory("client-cache-config.xml", properties);
    // Create the client properties
    Properties properties = new Properties(System.getProperties());
    properties.setProperty("tangosol.coherence.extend.port", extendPort);
    
    // Create the client Cache Factory
    clientCacheFactory = createClientCacheFactory("client-cache-config.xml", properties);

    So in our setup method we create the Properties adding the extend port property, then call the createClientCacheFactory method with the client configuration name and the properties.

    A very brief description of the createClientCacheFactory method…

    1. private ConfigurableCacheFactory createClientCacheFactory(String cacheConfigurationURI, Properties properties)
    2. {
    3.     XmlDocument clientConfigXml = XmlHelper.loadFileOrResource(cacheConfigurationURI, "Client");
    4.     replacePropertiesInXml(clientConfigXml, "system-property", properties);
    5.  
    6.     CacheFactoryBuilder cacheFactoryBuilder = CacheFactory.getCacheFactoryBuilder();
    7.     cacheFactoryBuilder.setCacheConfiguration(cacheConfigurationURI, null, clientConfigXml);
    8.     return cacheFactoryBuilder.getConfigurableCacheFactory(cacheConfigurationURI, null);
    9. }
    private ConfigurableCacheFactory createClientCacheFactory(String cacheConfigurationURI, Properties properties)
    {
        XmlDocument clientConfigXml = XmlHelper.loadFileOrResource(cacheConfigurationURI, "Client");
        replacePropertiesInXml(clientConfigXml, "system-property", properties);
    
        CacheFactoryBuilder cacheFactoryBuilder = CacheFactory.getCacheFactoryBuilder();
        cacheFactoryBuilder.setCacheConfiguration(cacheConfigurationURI, null, clientConfigXml);
        return cacheFactoryBuilder.getConfigurableCacheFactory(cacheConfigurationURI, null);
    }

    First we load the XML for the specified configuration URI. When we have the XML we replace any system properties in it using another utility method. That method should be easy enough for you to work out what it does. We then get the CacheFactoryBuilder using one of the few static Coherence methods used. If we wanted to we could have just created a new instance of the relevant concrete ConfigurableCacheFactory but I am going to use the builder. Once we have the builder we register the configuration XML we created then obtain a ConfigurableCacheFactory from that.

    So, now we have the client Cache factory we are ready to run the test.

    @Test – The Actual Test Method

    In this class we only have one test method that does something very simple.

    1. @Test
    2. public void shouldPutDataIntoCache() throws Exception
    3. {
    4.     NamedCache cache = clientCacheFactory.ensureCache("dist-test", null);
    5.     cache.put("Key-1", "Value-1");
    6.     Assert.assertThat((String)cache.get("Key-1"), is("Value-1"));
    7. }
    @Test
    public void shouldPutDataIntoCache() throws Exception
    {
        NamedCache cache = clientCacheFactory.ensureCache("dist-test", null);
        cache.put("Key-1", "Value-1");
        Assert.assertThat((String)cache.get("Key-1"), is("Value-1"));
    }

    As I’ve already mentioned, we have a concrete ConfigurableCacheFactory so we use the ConfigurableCacheFactory.ensureCache method to get our cache instances. All the test does is put a value into the cache and then asserts the same value comes back with a get call.

    @After – Cleaning Up the Test

    After each test has run I have decided I want to clean up the client before the @Before method runs again.

    1. @After
    2. public void shutdownClient()
    3. {
    4.     if (clientCacheFactory != null)
    5.     {
    6.         CacheFactory.getCacheFactoryBuilder().release(clientCacheFactory);
    7.         clientCacheFactory = null;
    8.     }
    9. }
    @After
    public void shutdownClient()
    {
        if (clientCacheFactory != null)
        {
            CacheFactory.getCacheFactoryBuilder().release(clientCacheFactory);
            clientCacheFactory = null;
        }
    }

    All we are doing in our clean up is deregistering the Cache Factory and configuration from the static CacheFactoryBuilder.

    @AfterClass – Destroy The Cluster

    Finally after all of our tests have run we want to stop the cluster so we have an @AfterClass annotated method that does just that.

    1. @AfterClass
    2. public static void stopCluster()
    3. {
    4.     if (cluster != null)
    5.     {
    6.         cluster.destroy();
    7.         cluster = null;
    8.     }
    9. }
    @AfterClass
    public static void stopCluster()
    {
        if (cluster != null)
        {
            cluster.destroy();
            cluster = null;
        }
    }

    And that is it, a full test class example.

    Testing Tips

    So, you have seen how easy it is to define different types of processes and control them from within code. This makes it simple to add to your automated functional test suite as part of a CI build. The project I work on for my current client uses this functionality and it works very well. It is possible to run a wide mixture of different processes, Java and non-Java, internal and external and mix and match if you really wanted to. There are a few tips and things to be aware of when using the above techniques to test Coherence applications so I’ve noted a few of them below.

    Start Your Cluster Once

    Typically a project will only have a single Coherence cluster so it is a very good idea to start and configure you test cluster once at the beginning of the functional test suite. Starting and stopping a cluster for every test class can add significant amounts of time to your test run. How you do this depends on your test framework, for example JUnit provides Rules that can be used to make sure the cluster has started. TestNG provides a @BeforeSuite method that you can use in a super-class of all your functional tests.

    Wait for the cluster to be ready before starting the tests – this is important for a stable test suite. If you just called realize on all your builders to fire up a cluster then immediately carried on into your tests methods the cluster may or may not actually be ready for the tests meaning tests may intermittently fail – the worst kind of test failure. With the deferred assertions mentioned previously it is easy to wait for all the cluster members to have joined the cluster, wait for the extend proxy to start etc. Again using my current client’s project as an example this application even has an internal MBean that monitors the application status and sets an MBean attribute when the application has been fully initialised and is ready to start accepting client requests. We then do a deferred assertion on this MBean attribute in the test suite to wait until the application is ready before continuing on to the tests.

    The start it once rule really applies to starting up any part of your test suite that takes some time. In my current project we start a Coherence cluster, embedded database and JMS server at the start of our test suites and tear it all down at the end and run over 25,000 tests in a shade over 25 minutes. The test run would take forever if we performed the initialisation too often.

    Memory Usage

    When using virtualized processes in your tests – which is the most sensible way to make sure everything is killed off when the tests finish – you need to make sure you have a big enough JVM heap to run everything so make sure your test harness is configured with a big heap. My current project is quite complex and runs a lot of things in a single JVM and we use a 1024MB heap. More important though is Perm Gen size as each virtualized process is isolated by ClassLoader so will increase Perm Gen usage as classes will be loaded multiple times. You can set Perm Space with a JVM argument like this

    -XX:MaxPermSize=256M

    The exact size will depend on your use case but we typically run with 256M or 300M.

    You might also want to try lowering the thread stack size for tests as this will reduce heap usage as long as you do not have any methods with very deep recursive calls. Running a number of Coherence cluster members in a single JVM can use a lot of threads so having a lower stack size can be a big help. Stack size is set using a JVM parameter like this

    -Xss64k

    Again, you might need a different value but we typically drop the stack down to 64k.

    So that’s it, not too complicated and very useful functionality. It is well worth looking at Oracle Tools to see how it can help simplify testing of applications, especially clustered applications. It is all open source so if it does not quite do what you want it is easy to change and feedback on new requirements is always welcome.

    Enjoy…

  • Tagged with:
     

    One Response to Oracle Coherence Testing with Oracle Tools

    1. […] The Grid Man > Coherence > Coherence .Net Testing with Oracle Tools Oracle Coherence Testing with Oracle Tools […]

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code lang=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre lang="" extra="">