Add Functionality To Oracle Coherence Services

This post is going to be about how to add functionality to Oracle Coherence services using the cache configuration and my new favourite Coherence 12.1.2 feature – custom namespaces. I had been playing around with this while on holiday in Turkey – well, there is only so much laying on the beach you can put up with, then with a 10 hour delay at Istanbul airport (thank you Pegasus Airlines) I had plenty of time to write it all up. This is quite a long post but if you want a decent example of what you can do with a custom namespace it should be useful.

There are various bits of functionality in a Coherence Service that cannot be configured using the cache configuration file, for example it is possible to add a ServiceListener to a Service, but there is not a way to do this in configuration. A service listener might be useful if you have application logic that relies on knowing that a particular service is up and running; we could poll for the state but a listener that is called back on is always preferable to polling. In the project I am working on for my current client we rely on knowing services have started and partitions have been allocated before we allow other processing to start.

Now I realise that for most people this might not be a big deal, after all you can quite easily add something like a service listener in code and I have done exactly that in the past. What I usually end up with though is a custom main class that either extends or calls DefaultCacheServer to start my cluster and then does any custom configuration. This is fine, but there are occasions where I do not really want to override the main class as I want to stick with DefaultCacheServer or whatever other main class Coherence might use, for example I might be writing a new GAR package that runs inside WebLogic.

As usual all the code from this blog is in my Blog GitHub repository under the coherence-twelve-one-two module.

Adding a Service Listener

OK, to continue the example mentioned above, say I want to add a ServiceListener to one of my services. If this was a built in Coherence feature it might look something like the in the configuration…

  1.    
  2. <distributed-scheme>
  3.     <scheme-name>my-distributed-scheme</scheme-name>
  4.     <service-name>DistributedCache</service-name>
  5.     <backing-map-scheme>
  6.         <local-scheme/>
  7.     </backing-map-scheme>
  8.     <autostart>true</autostart>
  9.     <service-listeners>
  10.         <service-listener>
  11.             <instance>
  12.                 <class-name>com.thegridman.coherence.service.LoggingServiceListener</class-name>
  13.             </instance>
  14.         </service-listener>
  15.     </service-listeners>
  16. </distributed-scheme>
    
<distributed-scheme>
    <scheme-name>my-distributed-scheme</scheme-name>
    <service-name>DistributedCache</service-name>
    <backing-map-scheme>
        <local-scheme/>
    </backing-map-scheme>
    <autostart>true</autostart>
    <service-listeners>
        <service-listener>
            <instance>
                <class-name>com.thegridman.coherence.service.LoggingServiceListener</class-name>
            </instance>
        </service-listener>
    </service-listeners>
</distributed-scheme>

You can see we have added a new <service-listeners> element at the end of the <distributed-scheme> which in turn contains one or more <service-listener> elements. Inside the <service-listener> element is the standard Coherence <instance> element which creates the actual ServiceListener instance.

Obviously Coherence does not really allow this configuration so we need to add it using the custom namespace functionality. There are two ways we could do this: Option 1, we override the default processing for the default namespace which will allow us to write the configuration exactly as it is above. Option 2, we introduce a new namespace handler and add the service listener using a custom namespace and element name prefix.

I have chosen to go with option 2, the reason being that this will allow me to also have a custom XSD file that goes along with my namespace handler which will validate the XML, both at runtime and also in my IDE at development time. I also think it is just better to do this sort of thing as a custom namespace as it makes it obvious in the configuration file what is standard Coherence and what is extra that we have added. So our configuration might now look something like this, with our custom namespace handler declared with a prefix of ex and a class name of com.thegridman.coherence.config.ExtensionsNamespaceHandler

  1. <cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  2.              xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
  3.              xmlns:ex="class://com.thegridman.coherence.config.ExtensionsNamespaceHandler"
  4.              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd
  5.                  class://com.thegridman.coherence.config.ExtensionsNamespaceHandler coherence-extended-config.xsd">
<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
              xmlns:ex="class://com.thegridman.coherence.config.ExtensionsNamespaceHandler"
              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd
                  class://com.thegridman.coherence.config.ExtensionsNamespaceHandler coherence-extended-config.xsd">

…and our <distributed-scheme>now looks like this, with the service listener elements using the new namespace prefix.

  1. <distributed-scheme>
  2.     <scheme-name>my-distributed-scheme</scheme-name>
  3.     <service-name>DistributedCache</service-name>
  4.     <backing-map-scheme>
  5.         <local-scheme/>
  6.     </backing-map-scheme>
  7.     <autostart>true</autostart>
  8.     <ex:service-listeners>
  9.         <ex:service-listener>
  10.             <instance>
  11.                 <class-name>com.thegridman.coherence.service.LoggingServiceListener</class-name>
  12.             </instance>
  13.         </ex:service-listener>
  14.     </ex:service-listeners>
  15. </distributed-scheme>
<distributed-scheme>
    <scheme-name>my-distributed-scheme</scheme-name>
    <service-name>DistributedCache</service-name>
    <backing-map-scheme>
        <local-scheme/>
    </backing-map-scheme>
    <autostart>true</autostart>
    <ex:service-listeners>
        <ex:service-listener>
            <instance>
                <class-name>com.thegridman.coherence.service.LoggingServiceListener</class-name>
            </instance>
        </ex:service-listener>
    </ex:service-listeners>
</distributed-scheme>

So how do we make this all work and what parts do we need? To answer that we will have a quick look at how service configuration is currently processed.

Default Service Configuration & Building

All of the standard Coherence cache configuration is processed by a built in namespace handler called CacheConfigNamespaceHandler. One of the things that this namespace handler does is register an ElementProcessor for each of the XML elements that define services, for example <distributed-scheme>, <replicated-scheme>, <invocation-scheme> etc…

Default Service Config Processing

The diagram above shows a simplified view of the classes involved (for clarity, and to make the diagram a manageable size, I have not included all of the possible services). You can see that the ElementProcessor that processes the XML elements is the same for each service, it just hands off the actual building of the service to a ServiceBuilder instance that is specific to the service being configured and built. For the <distributed-scheme> element the builder is the DistributedScheme class and there is a corresponding class for each service type.

Extending Service Configuration & Building

In the current release of Coherence (12.1.2) the above classes are not really very extensible. As far as I can tell there are no plug in points that allow additional processing to be easily added. If we just coded the standard namespace handler, element processors and builders for the new XML elements we want to add to create a service listener, we would find that our custom element processors do not get called. Even though the Coherence cache configuration XSD file allows XML from other namespaces to be added inside the <distributed-scheme> element it is just ignored when the services are created.

The only way to have our XML (and any other custom namespace XML) processed is to override a few of the classes above.

Extended Service Config Processing

The diagram above shows another simplified view of the classes we will need to create to be able to extend the configuration of the various services. For simplicity it only shows the classes for processing the <distributed-scheme> element but all the other services would work the same way.

Working across from the left, the first class we need will be the custom namespace handler ExtensionsNamespaceHandler. As well as registering the ElementProcessor instances for any new XML elements we add, such as the <service-listeners> element it will also override the registered ElementProcessor for the the <distributed-scheme> element in the default namespace handler CacheConfigNamespaceHandler.

The next new class is the ExtendedServiceBuilderProcessor class. This class will extend the regular ServiceBuilderProcessor class and be responsible for making sure any XML from addition namespaces is processed for the service.

The ExtendedDisributedScheme class extends the DistributedScheme class and will build the service and apply any extensions to the service.

The ExtendedDisributedScheme class will apply the extensions by making use of the ServiceExtension classes that will be created by the ExtendedServiceBuilderProcessor class, by processing any foreign namespace XML, and then passed to the ExtendedDisributedScheme instance. The reason the code for extending the service, for example adding service listeners, will be in ServiceExtension classes is that this code is common to all the services we might want to extend, so rather than duplicating the code in each service specific scheme class we extend it is better in a helper classes.

Coding The Extension Classes

So lets actually do some code and write the classes we need. I will start with the classes above, as these are all pretty generic, and then add in the classes for the service listener later.

The ExtensionsNamespaceHandler Class

All custom namespaces will need a namespace handler, logically enough ours will be called ExtensionsNamespaceHandler. As I said above, this class will be responsible for registering the ElementProcessor instances of our custom XML elements and also for overriding the default processors in the default CacheConfigNamespaceHandler for the services we are extending.

How do we override the default processors? All the element processors for the standard cache configuration are registered in the CacheConfigNamespaceHandler so to override them we need to get the instance of this handler and re-register our own ElementProcessor classes. Each namespace handler class has an onStartNamespace method that Coherence will call at the start of processing the XML configuration file, so this is where we can add the code in our custom namespace handler to do the overrides.

The onStartNamespace method is passed four parameters, the first of which is the ProcessingContext and it is this context that can give us the instance of the CacheConfigNamespaceHandler. The processing context has a method context.getNamespaceHandler(URI) that will return the namespace handler instance for a given namespace URI. The namespace URI for the default Coherence configuration is class://com.tangosol.coherence.config.xml.CacheConfigNamespaceHandler, that is class:// followed by the fully qualified class name of the CacheConfigNamespaceHandler class. Once we have the instance of the CacheConfigNamespaceHandler class we can call the registerProcessor on it to register our own ElementProcessor classes to override the defaults.

  1. package com.thegridman.coherence.config;
  2.  
  3. import com.tangosol.config.xml.AbstractNamespaceHandler;
  4. import com.tangosol.config.xml.ProcessingContext;
  5. import com.tangosol.run.xml.XmlElement;
  6.  
  7. import java.net.URI;
  8.  
  9. /**
  10.  * @author Jonathan Knight
  11.  */
  12. public class ExtensionsNamespaceHandler extends AbstractNamespaceHandler
  13. {
  14.     public ExtensionsNamespaceHandler()
  15.     {
  16.     }
  17.  
  18.     @SuppressWarnings("unchecked")
  19.     @Override
  20.     public void onStartNamespace(ProcessingContext context, XmlElement element, String prefix, URI uri)
  21.     {
  22.         URI coherenceURI = URI.create("class://" + CacheConfigNamespaceHandler.class.getCanonicalName());
  23.         AbstractNamespaceHandler coherenceNamespaceHandler
  24.                 = (AbstractNamespaceHandler) context.getNamespaceHandler(coherenceURI);
  25.  
  26.         coherenceNamespaceHandler.registerProcessor(
  27.                 "distributed-scheme",
  28.                 new ExtendedServiceBuilderProcessor(ExtendedDistributedScheme.class));
  29.         coherenceNamespaceHandler.registerProcessor(
  30.                 "invocation-scheme",
  31.                 new ExtendedServiceBuilderProcessor(ExtendedInvocationScheme.class));
  32.         coherenceNamespaceHandler.registerProcessor(
  33.                 "optimistic-scheme",
  34.                 new ExtendedServiceBuilderProcessor(ExtendedOptimisticScheme.class));
  35.         coherenceNamespaceHandler.registerProcessor(
  36.                 "proxy-scheme",
  37.                 new ExtendedServiceBuilderProcessor(ExtendedProxyScheme.class));
  38.         coherenceNamespaceHandler.registerProcessor(
  39.                 "remote-cache-scheme",
  40.                 new ExtendedServiceBuilderProcessor(ExtendedRemoteCacheScheme.class));
  41.         coherenceNamespaceHandler.registerProcessor(
  42.                 "remote-invocation-scheme",
  43.                 new ExtendedServiceBuilderProcessor(ExtendedRemoteInvocationScheme.class));
  44.         coherenceNamespaceHandler.registerProcessor(
  45.                 "replicated-scheme",
  46.                 new ExtendedServiceBuilderProcessor(ExtendedReplicatedScheme.class));
  47.         coherenceNamespaceHandler.registerProcessor(
  48.                 "transactional-scheme",
  49.                 new ExtendedServiceBuilderProcessor(ExtendedTransactionalScheme.class));
  50.     }
  51.  
  52. }
package com.thegridman.coherence.config;

import com.tangosol.config.xml.AbstractNamespaceHandler;
import com.tangosol.config.xml.ProcessingContext;
import com.tangosol.run.xml.XmlElement;

import java.net.URI;

/**
 * @author Jonathan Knight
 */
public class ExtensionsNamespaceHandler extends AbstractNamespaceHandler
{
    public ExtensionsNamespaceHandler()
    {
    }

    @SuppressWarnings("unchecked")
    @Override
    public void onStartNamespace(ProcessingContext context, XmlElement element, String prefix, URI uri)
    {
        URI coherenceURI = URI.create("class://" + CacheConfigNamespaceHandler.class.getCanonicalName());
        AbstractNamespaceHandler coherenceNamespaceHandler
                = (AbstractNamespaceHandler) context.getNamespaceHandler(coherenceURI);

        coherenceNamespaceHandler.registerProcessor(
                "distributed-scheme",
                new ExtendedServiceBuilderProcessor(ExtendedDistributedScheme.class));
        coherenceNamespaceHandler.registerProcessor(
                "invocation-scheme",
                new ExtendedServiceBuilderProcessor(ExtendedInvocationScheme.class));
        coherenceNamespaceHandler.registerProcessor(
                "optimistic-scheme",
                new ExtendedServiceBuilderProcessor(ExtendedOptimisticScheme.class));
        coherenceNamespaceHandler.registerProcessor(
                "proxy-scheme",
                new ExtendedServiceBuilderProcessor(ExtendedProxyScheme.class));
        coherenceNamespaceHandler.registerProcessor(
                "remote-cache-scheme",
                new ExtendedServiceBuilderProcessor(ExtendedRemoteCacheScheme.class));
        coherenceNamespaceHandler.registerProcessor(
                "remote-invocation-scheme",
                new ExtendedServiceBuilderProcessor(ExtendedRemoteInvocationScheme.class));
        coherenceNamespaceHandler.registerProcessor(
                "replicated-scheme",
                new ExtendedServiceBuilderProcessor(ExtendedReplicatedScheme.class));
        coherenceNamespaceHandler.registerProcessor(
                "transactional-scheme",
                new ExtendedServiceBuilderProcessor(ExtendedTransactionalScheme.class));
    }

}

The code above is for our ExtensionsNamespaceHandlerclass. You can see the onStartNamespace where we obtain the CacheConfigNamespaceHandler instance. We then call its registerProcessor to register our overrides for the different elements.

Right now we have not written any custom XML elements that we need to register but we will add the <service-listeners> element later.

The ExtendedServiceBuilderProcessor Class

The next class we need to write is the ExtendedServiceBuilderProcessor class. This will be responsible for creating our ServiceExtension classes from the XML configuration and passing them to the service scheme class for use when the service is built. This class is not specific to any particular service type, just like the default ServiceBuilderProcessor class that it extends so it will be used for all of our services. The purpose of us overriding the class is so that we can make sure that any XML from custom namespaces get correctly processed when the configuration file is processed and when services are created so we need to override the process method.

  1. package com.thegridman.coherence.config;
  2.  
  3. import com.tangosol.coherence.config.builder.ServiceBuilder;
  4. import com.tangosol.coherence.config.xml.processor.ServiceBuilderProcessor;
  5. import com.tangosol.config.ConfigurationException;
  6. import com.tangosol.config.xml.ProcessingContext;
  7. import com.tangosol.run.xml.XmlElement;
  8.  
  9. import java.util.ArrayList;
  10. import java.util.List;
  11. import java.util.Map;
  12.  
  13. /**
  14.  * @author Jonathan Knight
  15.  */
  16. public class ExtendedServiceBuilderProcessor extends ServiceBuilderProcessor
  17. {
  18.     public ExtendedServiceBuilderProcessor(Class clzToRealize)
  19.     {
  20.         super(clzToRealize);
  21.     }
  22.  
  23.     @SuppressWarnings("unchecked")
  24.     @Override
  25.     public T process(ProcessingContext context, XmlElement element) throws ConfigurationException
  26.     {
  27.         T builder = super.process(context, element);
  28.         if (!(builder instanceof ExtendedServiceScheme))
  29.         {
  30.             return builder;
  31.         }
  32.        
  33.         Map map = context.processForeignElementsOf(element);
  34.         List extensions = new ArrayList<>(map.size());
  35.         for (Object value : map.values())
  36.         {
  37.             if (value instanceof ServiceExtension)
  38.             {
  39.                 extensions.add((ServiceExtension) value);
  40.             }
  41.         }
  42.         ((ExtendedServiceScheme)builder).setExtensions(extensions);
  43.  
  44.         return builder;
  45.     }
  46.  
  47. }
package com.thegridman.coherence.config;

import com.tangosol.coherence.config.builder.ServiceBuilder;
import com.tangosol.coherence.config.xml.processor.ServiceBuilderProcessor;
import com.tangosol.config.ConfigurationException;
import com.tangosol.config.xml.ProcessingContext;
import com.tangosol.run.xml.XmlElement;

import java.util.ArrayList;
import java.util.List;
import java.util.Map;

/**
 * @author Jonathan Knight
 */
public class ExtendedServiceBuilderProcessor extends ServiceBuilderProcessor
{
    public ExtendedServiceBuilderProcessor(Class clzToRealize)
    {
        super(clzToRealize);
    }

    @SuppressWarnings("unchecked")
    @Override
    public T process(ProcessingContext context, XmlElement element) throws ConfigurationException
    {
        T builder = super.process(context, element);
        if (!(builder instanceof ExtendedServiceScheme))
        {
            return builder;
        }
        
        Map map = context.processForeignElementsOf(element);
        List extensions = new ArrayList<>(map.size());
        for (Object value : map.values())
        {
            if (value instanceof ServiceExtension)
            {
                extensions.add((ServiceExtension) value);
            }
        }
        ((ExtendedServiceScheme)builder).setExtensions(extensions);

        return builder;
    }

}

You can see in the code for the ExtendedServiceBuilderProcessor that we have extended the default ServiceBuilderProcessor class and overridden the process method. In the process method the first thing we do is to call process on the super class, which will return a configured instance of the service scheme class that will build the service. The type of the class returned is the same as the clzToRealize constructor parameter. Next we check to see if the builder returned is an instance of our new ExtendedServiceScheme interface; if it is not we just return the builder. If the builder is an instance of ExtendedServiceScheme then is means we need to process any foreign namespace XML to find any ServiceExtension instances and pass them to the builder.

The ExtendedServiceScheme Interface

The ExtendedServiceScheme interface is a simple interface with one method. All of the classes that we write to extend the built in service builder scheme classes will implement this interface so that any ServiceExtension instances can be passed to them.

  1. package com.thegridman.coherence.config;
  2.  
  3. import java.util.List;
  4.  
  5. /**
  6.  * @author Jonathan Knight
  7.  */
  8. public interface ExtendedServiceScheme
  9. {
  10.     void setExtensions(List extensions);
  11. }
package com.thegridman.coherence.config;

import java.util.List;

/**
 * @author Jonathan Knight
 */
public interface ExtendedServiceScheme
{
    void setExtensions(List extensions);
}

The ServiceExtension Interface

The ServiceExtension interface will be implemented by any class we write that knows how to apply extensions to a service. In our first example we are going to write an implementation that adds ServiceListener instances to a service.

  1. package com.thegridman.coherence.config;
  2.  
  3. import com.tangosol.config.expression.ParameterResolver;
  4. import com.tangosol.net.Cluster;
  5. import com.tangosol.net.Service;
  6.  
  7. /**
  8.  * @author Jonathan Knight
  9.  */
  10. public interface ServiceExtension
  11. {
  12.     void extendService(Service service, ParameterResolver resolver, ClassLoader loader, Cluster cluster);
  13. }
package com.thegridman.coherence.config;

import com.tangosol.config.expression.ParameterResolver;
import com.tangosol.net.Cluster;
import com.tangosol.net.Service;

/**
 * @author Jonathan Knight
 */
public interface ServiceExtension
{
    void extendService(Service service, ParameterResolver resolver, ClassLoader loader, Cluster cluster);
}

The interface has a single method that is passed the service to have the extensions applied.

The ExtendedDistributedScheme Class

 The next class we need is the builder for the DistributedCacheService which is an extension of the built in DistributedScheme class. Sticking to our theme for class names we will call it ExtendedDistributedScheme.

  1. package com.thegridman.coherence.config;
  2.  
  3. import com.tangosol.coherence.config.scheme.DistributedScheme;
  4. import com.tangosol.config.expression.ParameterResolver;
  5. import com.tangosol.net.Cluster;
  6. import com.tangosol.net.Service;
  7.  
  8. import java.util.List;
  9.  
  10. /**
  11.  * @author Jonathan Knight
  12.  */
  13. public class ExtendedDistributedScheme extends DistributedScheme implements ExtendedServiceScheme
  14. {
  15.     private List<ServiceExtension> extensions;
  16.  
  17.     public ExtendedDistributedScheme()
  18.     {
  19.     }
  20.  
  21.     @Override
  22.     public void setExtensions(List<ServiceExtension> extensions)
  23.     {
  24.         this.extensions = extensions;
  25.     }
  26.  
  27.     @Override
  28.     public Service realizeService(ParameterResolver resolver, ClassLoader loader, Cluster cluster)
  29.     {
  30.         Service service = super.realizeService(resolver, loader, cluster);
  31.         if (service.isRunning())
  32.         {
  33.             return service;
  34.         }
  35.  
  36.         if (extensions != null)
  37.         {
  38.             for (ServiceExtension extension : extensions)
  39.             {
  40.                 extension.extendService(service, resolver, loader, cluster);
  41.             }
  42.         }
  43.         return service;
  44.     }
  45. }
package com.thegridman.coherence.config;

import com.tangosol.coherence.config.scheme.DistributedScheme;
import com.tangosol.config.expression.ParameterResolver;
import com.tangosol.net.Cluster;
import com.tangosol.net.Service;

import java.util.List;

/**
 * @author Jonathan Knight
 */
public class ExtendedDistributedScheme extends DistributedScheme implements ExtendedServiceScheme
{
    private List<ServiceExtension> extensions;

    public ExtendedDistributedScheme()
    {
    }

    @Override
    public void setExtensions(List<ServiceExtension> extensions)
    {
        this.extensions = extensions;
    }

    @Override
    public Service realizeService(ParameterResolver resolver, ClassLoader loader, Cluster cluster)
    {
        Service service = super.realizeService(resolver, loader, cluster);
        if (service.isRunning())
        {
            return service;
        }

        if (extensions != null)
        {
            for (ServiceExtension extension : extensions)
            {
                extension.extendService(service, resolver, loader, cluster);
            }
        }
        return service;
    }
}

This class has to implement ExtendedServiceScheme so that it can be passed the list of ServiceExtension instances by the ExtendedServiceBuilder that we coded above.

As this class is an instance of ServiceBuilder the work of creating the Service is done in the realizeService method, so this is the method we need to override. In our realizeService method we first call the super class’s realizeService method to actually build the service. If the service returned by the super class is already running then there is nothing for us to do as the service has already been configured by a previous call to the builder so we just return the service.

If the service has is not running then we need to apply any ServiceExtension instances if any have been added to this scheme. We just iterate over the list of ServiceExtension instances calling the extendService method on each one.

The Other Extended Scheme Classes

All of the other services that we want to extend will follow the same pattern as the ExtendedDistributedScheme above. So for a replicated scheme we extend ReplicatedScheme with a class called ExtendedReplicatedScheme and the code in it will be identical to the code we wrote in our ExtendedDistributedScheme class. The same for all the other service schemes InvocationScheme, OptimisticScheme, ProxyScheme, RemoteCacheScheme, RemoteInvocationScheme and TransactionalScheme.

That is all of the generic classes for extending services and these will now work for any service extensions that we want to add.

Adding the ServiceListener Extensions

Now we have our service extension framework we can use it for our original requirement of adding ServiceListener instances in the cache configuration file. If we look again at the XML configuration we want to use we will be able to see what classes we need to create.

  1. <distributed-scheme>
  2.     <scheme-name>my-distributed-scheme</scheme-name>
  3.     <service-name>DistributedCache</service-name>
  4.     <backing-map-scheme>
  5.         <local-scheme/>
  6.     </backing-map-scheme>
  7.     <autostart>true</autostart>
  8.     <ex:service-listeners>
  9.         <ex:service-listener>
  10.             <instance>
  11.                 <class-name>com.thegridman.coherence.service.LoggingServiceListener</class-name>
  12.             </instance>
  13.         </ex:service-listener>
  14.     </ex:service-listeners>
  15. </distributed-scheme>
<distributed-scheme>
    <scheme-name>my-distributed-scheme</scheme-name>
    <service-name>DistributedCache</service-name>
    <backing-map-scheme>
        <local-scheme/>
    </backing-map-scheme>
    <autostart>true</autostart>
    <ex:service-listeners>
        <ex:service-listener>
            <instance>
                <class-name>com.thegridman.coherence.service.LoggingServiceListener</class-name>
            </instance>
        </ex:service-listener>
    </ex:service-listeners>
</distributed-scheme>

We already have our ExtensionsNamespaceHandler so we will need ElementProcessor classes for the <service-listeners> and <service-listener> elements. We are actually going to implement these as more than two classes as we want to get some reuse for later examples in the post.

The ServiceListenerBuilder Class

The <service-listener> element does not contain any other XML that we need to process in an ElementProcessor. The only XML that the <service-listener> element will contain is an <instance> element which is processed by the Coherence namespace handler. This means that to handle the <service-listener> element we are going to use a built in ElementProcessor called CustomizableBuilderProcessor which takes the class name of our builder class as its constructor parameter. Our builder looks like this…

  1. package com.thegridman.coherence.config;
  2.  
  3. import com.tangosol.coherence.config.ParameterList;
  4. import com.tangosol.coherence.config.builder.BuilderCustomization;
  5. import com.tangosol.coherence.config.builder.ParameterizedBuilder;
  6. import com.tangosol.coherence.config.builder.ParameterizedBuilderHelper;
  7. import com.tangosol.config.expression.ParameterResolver;
  8. import com.tangosol.util.ServiceListener;
  9.  
  10. /**
  11.  * @author Jonathan Knight
  12.  */
  13. public class ServiceListenerBuilder
  14.         implements ParameterizedBuilder<ServiceListener>, BuilderCustomization<ServiceListener>
  15. {
  16.     private ParameterizedBuilder<ServiceListener> builder;
  17.  
  18.     public ServiceListenerBuilder()
  19.     {
  20.     }
  21.  
  22.     @Override
  23.     public ServiceListener realize(ParameterResolver resolver, ClassLoader loader, ParameterList listParameters)
  24.     {
  25.         loader = loader == null ? getClass().getClassLoader() : loader;
  26.  
  27.         if (!ParameterizedBuilderHelper.realizes(builder, ServiceListener.class, resolver, loader))
  28.         {
  29.           throw new IllegalArgumentException(
  30.                   "Unable to build an ServiceListener based on the specified class: "
  31.                   + String.valueOf(builder));
  32.         }
  33.  
  34.         return builder.realize(resolver, loader, listParameters);
  35.     }
  36.  
  37.     @Override
  38.     public ParameterizedBuilder<ServiceListener> getCustomBuilder()
  39.     {
  40.         return this.builder;
  41.     }
  42.  
  43.     @Override
  44.     public void setCustomBuilder(ParameterizedBuilder<ServiceListener> builder)
  45.     {
  46.         this.builder = builder;
  47.     }
  48. }
package com.thegridman.coherence.config;

import com.tangosol.coherence.config.ParameterList;
import com.tangosol.coherence.config.builder.BuilderCustomization;
import com.tangosol.coherence.config.builder.ParameterizedBuilder;
import com.tangosol.coherence.config.builder.ParameterizedBuilderHelper;
import com.tangosol.config.expression.ParameterResolver;
import com.tangosol.util.ServiceListener;

/**
 * @author Jonathan Knight
 */
public class ServiceListenerBuilder
        implements ParameterizedBuilder<ServiceListener>, BuilderCustomization<ServiceListener>
{
    private ParameterizedBuilder<ServiceListener> builder;

    public ServiceListenerBuilder()
    {
    }

    @Override
    public ServiceListener realize(ParameterResolver resolver, ClassLoader loader, ParameterList listParameters)
    {
        loader = loader == null ? getClass().getClassLoader() : loader;

        if (!ParameterizedBuilderHelper.realizes(builder, ServiceListener.class, resolver, loader))
        {
          throw new IllegalArgumentException(
                  "Unable to build an ServiceListener based on the specified class: "
                  + String.valueOf(builder));
        }

        return builder.realize(resolver, loader, listParameters);
    }

    @Override
    public ParameterizedBuilder<ServiceListener> getCustomBuilder()
    {
        return this.builder;
    }

    @Override
    public void setCustomBuilder(ParameterizedBuilder<ServiceListener> builder)
    {
        this.builder = builder;
    }
}

The class is pretty straight forward and implements two built in interfaces. If you look at the XML configuration you will see that the actual ServiceListener implementation is defined by an instance element (or in theory could be defined by a similar element from another custom namespace that will build a ServiceListener). All our ServiceListenerBuilder needs to do is handle the inner builder and use it to get the listener. The Coherence configuration framework will call the setCustomBuilder method on our ServiceListenerBuilder class and pass to it the inner builder defined by the the <instance> element. The realize method of our ServiceListenerBuilder checks to make sure the inner builder actually builds a ServiceListener and if not throws an exception. If it does, then the inner builder’s realize method is called to get the ServiceListener which is then returned.

The <service-listeners> ElementProcessor Code

The code to process the <service-listeners> element is also quite simple. In effect all it has to do is process the set of inner <service-listener> elements in its process method and return a class that implements ServiceExtension which will be passed to our extended service scheme classes as we explained above.

This pattern of code where we have an ElementProcessor that just handles a list of inner elements who’s ElementProcessor just returns a builder of a particular type is quite common, so we will first write a generic class to handle this processing then we will extend it to work for handling a ServiceListener.

The ListOfInstancesProcessor Class

As our ElementProcessor will handle a list of instances of a specific type I have called it ListOfInstancesProcessor. The code is below…

  1. package com.thegridman.coherence.config;
  2.  
  3. import com.tangosol.config.ConfigurationException;
  4. import com.tangosol.config.xml.ElementProcessor;
  5. import com.tangosol.config.xml.ProcessingContext;
  6. import com.tangosol.run.xml.XmlElement;
  7.  
  8. import java.util.ArrayList;
  9. import java.util.List;
  10.  
  11. /**
  12.  * @author Jonathan Knight
  13.  */
  14. public class ListOfInstancesProcessor<R extends ListOfInstancesProcessor.ListOfInstancesScheme,T>
  15. implements ElementProcessor<R>
  16. {
  17.     private final Class<R> classToRealize;
  18.     private final Class<T> type;
  19.  
  20.     public ListOfInstancesProcessor(Class<R> classToRealize, Class<T> type)
  21.     {
  22.         this.classToRealize = classToRealize;
  23.         this.type = type;
  24.     }
  25.  
  26.     @SuppressWarnings("unchecked")
  27.     @Override
  28.     public R process(ProcessingContext context, XmlElement element) throws ConfigurationException
  29.     {
  30.         List<XmlElement> children = (List<XmlElement>) element.getElementList();
  31.         ArrayList<T> instances = new ArrayList<>(children.size());
  32.         for (XmlElement child : children)
  33.         {
  34.             try
  35.             {
  36.                 T value = (T) context.processElement(child);
  37.                 if (!type.isAssignableFrom(value.getClass()))
  38.                 {
  39.                     throw new ClassCastException();
  40.                 }
  41.                 instances.add(value);
  42.             }
  43.             catch (ClassCastException e)
  44.             {
  45.                 throw new ConfigurationException("Element " + child.getName()
  46.                                                  + " does not produce an instance of "
  47.                                                  + type,
  48.                                                  "Check the configuration", e);
  49.             }
  50.         }
  51.  
  52.         R scheme;
  53.         try
  54.         {
  55.             scheme = classToRealize.newInstance();
  56.             scheme.setInstances(instances);
  57.         }
  58.         catch (Exception e)
  59.         {
  60.             throw new ConfigurationException("Error creating class " + classToRealize,
  61.                                              "Make sure class " + classToRealize +
  62.                                              " is a valid class with public no-arg constructor", e);
  63.         }
  64.         return scheme;
  65.     }
  66.  
  67.     public static interface ListOfInstancesScheme<T>
  68.     {
  69.         void setInstances(List<T> instances);
  70.     }
  71. }
package com.thegridman.coherence.config;

import com.tangosol.config.ConfigurationException;
import com.tangosol.config.xml.ElementProcessor;
import com.tangosol.config.xml.ProcessingContext;
import com.tangosol.run.xml.XmlElement;

import java.util.ArrayList;
import java.util.List;

/**
 * @author Jonathan Knight
 */
public class ListOfInstancesProcessor<R extends ListOfInstancesProcessor.ListOfInstancesScheme,T> 
implements ElementProcessor<R>
{
    private final Class<R> classToRealize;
    private final Class<T> type;

    public ListOfInstancesProcessor(Class<R> classToRealize, Class<T> type)
    {
        this.classToRealize = classToRealize;
        this.type = type;
    }

    @SuppressWarnings("unchecked")
    @Override
    public R process(ProcessingContext context, XmlElement element) throws ConfigurationException
    {
        List<XmlElement> children = (List<XmlElement>) element.getElementList();
        ArrayList<T> instances = new ArrayList<>(children.size());
        for (XmlElement child : children)
        {
            try
            {
                T value = (T) context.processElement(child);
                if (!type.isAssignableFrom(value.getClass()))
                {
                    throw new ClassCastException();
                }
                instances.add(value);
            }
            catch (ClassCastException e)
            {
                throw new ConfigurationException("Element " + child.getName()
                                                 + " does not produce an instance of "
                                                 + type,
                                                 "Check the configuration", e);
            }
        }

        R scheme;
        try
        {
            scheme = classToRealize.newInstance();
            scheme.setInstances(instances);
        }
        catch (Exception e)
        {
            throw new ConfigurationException("Error creating class " + classToRealize,
                                             "Make sure class " + classToRealize +
                                             " is a valid class with public no-arg constructor", e);
        }
        return scheme;
    }

    public static interface ListOfInstancesScheme<T>
    {
        void setInstances(List<T> instances);
    }
}

The constructor takes two Class parameters. The first is the Class of the value that will be returned from the process method. The second is the class of the value that will be returned from processing the child XML elements of the element passed to this ElementProcessor's process method.

The process method is where all the work is done. First we iterate over each child XmlElement of the XmlElement passed to the process method. For each child element we process it and get the value produced (usually as a result of the child elements ElementProcessor being called). We check to make sure that the returned value is of the expected type (the second constructor parameter) and if not we throw an exception. If the type is correct we add the value to a list of instances created from the child elements. When all the child elements have been processed we create an instance of the class specified as the return type of this ListOfInstancesProcessor (the first constructor parameter) which must have a public no-args constructor and implement the ListOfInstancesProcessor.ListOfInstancesScheme interface. We then pass the list of instances to this class’s setInstances method and return.

The ServiceListenersScheme Class

Now we have a generic class for dealing with a list of child elements we can write the class that will handle the specifics of managing a list of ServiceListener instances. We will call this class ServiceListenersScheme and it will implement the ServiceExtension class as it needs to be able to apply the ServiceListener instances to a Service and it will implement the ListOfInstancesProcessor.ListOfInstancesScheme interface we just described above as it will be used by the ListOfInstancesProcessor.

The code for our ServiceListenersScheme class is shown below…

  1. package com.thegridman.coherence.config;
  2.  
  3. import com.tangosol.config.expression.ParameterResolver;
  4. import com.tangosol.net.Cluster;
  5. import com.tangosol.net.Service;
  6. import com.tangosol.util.ServiceListener;
  7.  
  8. import java.util.List;
  9.  
  10. /**
  11.  * @author Jonathan Knight
  12.  */
  13. public class ServiceListenersScheme
  14.         implements ServiceExtension, ListOfInstancesProcessor.ListOfInstancesScheme<ServiceListenerBuilder>
  15. {
  16.     private List<ServiceListenerBuilder> instances;
  17.  
  18.     public ServiceListenersScheme()
  19.     {
  20.     }
  21.  
  22.     @Override
  23.     public void setInstances(List<ServiceListenerBuilder> instances)
  24.     {
  25.         this.instances = instances;
  26.     }
  27.  
  28.     @Override
  29.     public void extendService(Service service, ParameterResolver resolver, ClassLoader loader, Cluster cluster)
  30.     {
  31.         if (instances == null || instances.isEmpty())
  32.         {
  33.             return;
  34.         }
  35.  
  36.         for (ServiceListenerBuilder builder : instances)
  37.         {
  38.             ServiceListener listener = builder.realize(resolver, loader, null);
  39.             service.addServiceListener(listener);
  40.         }
  41.     }
  42. }
package com.thegridman.coherence.config;

import com.tangosol.config.expression.ParameterResolver;
import com.tangosol.net.Cluster;
import com.tangosol.net.Service;
import com.tangosol.util.ServiceListener;

import java.util.List;

/**
 * @author Jonathan Knight
 */
public class ServiceListenersScheme
        implements ServiceExtension, ListOfInstancesProcessor.ListOfInstancesScheme<ServiceListenerBuilder>
{
    private List<ServiceListenerBuilder> instances;

    public ServiceListenersScheme()
    {
    }

    @Override
    public void setInstances(List<ServiceListenerBuilder> instances)
    {
        this.instances = instances;
    }

    @Override
    public void extendService(Service service, ParameterResolver resolver, ClassLoader loader, Cluster cluster)
    {
        if (instances == null || instances.isEmpty())
        {
            return;
        }

        for (ServiceListenerBuilder builder : instances)
        {
            ServiceListener listener = builder.realize(resolver, loader, null);
            service.addServiceListener(listener);
        }
    }
}

The setInstances method is the implementation of the ListOfInstancesProcessor.ListOfInstancesScheme interface and will be passed the list of ServiceListenerBuilder instances processed by the ListOfInstancesProcessor.

The extendService method is the implementation of the ServiceExtension interface and will add the ServiceListener instances to the Service. All the method does is loop over the list of ServiceListener builder instances and calls the realize method of each to obtain an instance of a ServiceListener. It then adds each of these listeners to the service.

Registering the ElementProcessors

Now that we have all of our classes the last thing to do is register the new ElementProcessors in our custom namespace handler so that the XML elements are correctly processed. We do this in the constructor of the ExtensionsNamespaceHandler class like this…

  1. public ExtensionsNamespaceHandler()
  2. {
  3.     registerProcessor("service-listener",
  4.             new CustomizableBuilderProcessor<>(ServiceListenerBuilder.class));
  5.     registerProcessor("service-listeners",
  6.             new ListOfInstancesProcessor<>(ServiceListenersScheme.class, ServiceListenerBuilder.class));
  7. }
public ExtensionsNamespaceHandler()
{
    registerProcessor("service-listener", 
            new CustomizableBuilderProcessor<>(ServiceListenerBuilder.class));
    registerProcessor("service-listeners", 
            new ListOfInstancesProcessor<>(ServiceListenersScheme.class, ServiceListenerBuilder.class));
}

And that is all we need to do; we can now add ServiceListener instances to a service in the cache configuration file.

The Example ServiceListener Classes

Two final classes to finish off the example are the ServiceListener implementations. We have one class, MuliplexingServiceListener that multiplexes the three methods of the ServiceListener interface down to a single onEvent method. The second class we create is called LoggingServiceListener which extends MuliplexingServiceListener and in its onEvent method just prints the event to the Coherence log.

The MuliplexingServiceListener Class
  1. package com.thegridman.coherence.service;
  2.  
  3. import com.tangosol.util.Base;
  4. import com.tangosol.util.ServiceEvent;
  5. import com.tangosol.util.ServiceListener;
  6.  
  7. /**
  8.  * @author Jonathan Knight
  9.  */
  10. public abstract class MuliplexingServiceListener extends Base implements ServiceListener
  11. {
  12.     protected MuliplexingServiceListener()
  13.     {
  14.     }
  15.  
  16.     @Override
  17.     public void serviceStarted(ServiceEvent serviceEvent)
  18.     {
  19.         onEvent(serviceEvent);
  20.     }
  21.  
  22.     @Override
  23.     public void serviceStarting(ServiceEvent serviceEvent)
  24.     {
  25.         onEvent(serviceEvent);
  26.     }
  27.  
  28.     @Override
  29.     public void serviceStopping(ServiceEvent serviceEvent)
  30.     {
  31.         onEvent(serviceEvent);
  32.     }
  33.  
  34.     @Override
  35.     public void serviceStopped(ServiceEvent serviceEvent)
  36.     {
  37.         onEvent(serviceEvent);
  38.     }
  39.  
  40.     protected abstract void onEvent(ServiceEvent serviceEvent);
  41. }
package com.thegridman.coherence.service;

import com.tangosol.util.Base;
import com.tangosol.util.ServiceEvent;
import com.tangosol.util.ServiceListener;

/**
 * @author Jonathan Knight
 */
public abstract class MuliplexingServiceListener extends Base implements ServiceListener
{
    protected MuliplexingServiceListener()
    {
    }

    @Override
    public void serviceStarted(ServiceEvent serviceEvent)
    {
        onEvent(serviceEvent);
    }

    @Override
    public void serviceStarting(ServiceEvent serviceEvent)
    {
        onEvent(serviceEvent);
    }

    @Override
    public void serviceStopping(ServiceEvent serviceEvent)
    {
        onEvent(serviceEvent);
    }

    @Override
    public void serviceStopped(ServiceEvent serviceEvent)
    {
        onEvent(serviceEvent);
    }

    protected abstract void onEvent(ServiceEvent serviceEvent);
}
The LoggingServiceListener Class
  1. package com.thegridman.coherence.service;
  2.  
  3. import com.tangosol.util.ServiceEvent;
  4.  
  5. /**
  6.  * @author Jonathan Knight
  7.  */
  8. public class LoggingServiceListener extends MuliplexingServiceListener
  9. {
  10.     @Override
  11.     protected void onEvent(ServiceEvent serviceEvent)
  12.     {
  13.         log(serviceEvent);
  14.     }
  15. }
package com.thegridman.coherence.service;

import com.tangosol.util.ServiceEvent;

/**
 * @author Jonathan Knight
 */
public class LoggingServiceListener extends MuliplexingServiceListener
{
    @Override
    protected void onEvent(ServiceEvent serviceEvent)
    {
        log(serviceEvent);
    }
}

Running An Example

If we now run an instance of DefaultCacheServer with the XML configuration below…

  1. <?xml version="1.0"?>
  2. <cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  3.              xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
  4.              xmlns:ex="class://com.thegridman.coherence.config.ExtensionsNamespaceHandler"
  5.              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd
  6.                                  class://com.thegridman.coherence.config.ExtensionsNamespaceHandler coherence-extended-config.xsd">
  7.  
  8.     <defaults>
  9.         <serializer>pof</serializer>
  10.     </defaults>
  11.  
  12.     <caching-scheme-mapping>
  13.         <cache-mapping>
  14.             <cache-name>dist-*</cache-name>
  15.             <scheme-name>my-distributed-scheme</scheme-name>
  16.         </cache-mapping>
  17.     </caching-scheme-mapping>
  18.  
  19.     <caching-schemes>
  20.  
  21.     <distributed-scheme>
  22.         <scheme-name>my-distributed-scheme</scheme-name>
  23.         <service-name>DistributedCache</service-name>
  24.         <backing-map-scheme>
  25.             <local-scheme/>
  26.         </backing-map-scheme>
  27.         <autostart>true</autostart>
  28.         <ex:service-listeners>
  29.             <ex:service-listener>
  30.                 <instance>
  31.                     <class-name>com.thegridman.coherence.service.LoggingServiceListener</class-name>
  32.                 </instance>
  33.             </ex:service-listener>
  34.         </ex:service-listeners>
  35.     </distributed-scheme>
  36.  
  37.     </caching-schemes>
  38.  
  39. </cache-config>
<?xml version="1.0"?>
<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
              xmlns:ex="class://com.thegridman.coherence.config.ExtensionsNamespaceHandler"
              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd
                                  class://com.thegridman.coherence.config.ExtensionsNamespaceHandler coherence-extended-config.xsd">

    <defaults>
        <serializer>pof</serializer>
    </defaults>

    <caching-scheme-mapping>
        <cache-mapping>
            <cache-name>dist-*</cache-name>
            <scheme-name>my-distributed-scheme</scheme-name>
        </cache-mapping>
    </caching-scheme-mapping>

    <caching-schemes>

    <distributed-scheme>
        <scheme-name>my-distributed-scheme</scheme-name>
        <service-name>DistributedCache</service-name>
        <backing-map-scheme>
            <local-scheme/>
        </backing-map-scheme>
        <autostart>true</autostart>
        <ex:service-listeners>
            <ex:service-listener>
                <instance>
                    <class-name>com.thegridman.coherence.service.LoggingServiceListener</class-name>
                </instance>
            </ex:service-listener>
        </ex:service-listeners>
    </distributed-scheme>

    </caching-schemes>

</cache-config>

…when the service is started we will see something like the following message in the Coherence log output

  1. 2013-09-15 12:49:17.238/3.426 Oracle Coherence GE 12.1.2.0.0  (thread=DistributedCache:EventDispatcher, member=1):
  2. ServiceEvent{STARTED com.tangosol.coherence.component.util.safeService.safeCacheService.SafeDistributedCacheService}
2013-09-15 12:49:17.238/3.426 Oracle Coherence GE 12.1.2.0.0  (thread=DistributedCache:EventDispatcher, member=1):
ServiceEvent{STARTED com.tangosol.coherence.component.util.safeService.safeCacheService.SafeDistributedCacheService}

so we can see that our LoggingServiceListener did indeed get added to the DistributedCache service and received the Started ServiceEvent.

The Service User Context

I suspect this is quite an under used feature of Coherence Services, but it is possible to add an application defined user context value to a service. This can be any object you like and is entirely under your control. It is not clustered at all, so for example, if you add a value as the user context to a Distributed Cache Service on one node of the cluster that value is not replicated to other instances of the same service on other cluster members. The service has two methods…

  1. public Object getUserContext();
  2.  
  3. public void setUserContext(Object o);
public Object getUserContext();

public void setUserContext(Object o);

If you’ve been paying attention I suspect you can see where I am going next, yes, I want to add a user context value (or values) to a Coherence Service by putting them in the cache configuration file using a custom namespace as we did above with service listeners. Now that we have a framework for extending the service configuration we should be able to reuse a lot of the code we wrote above.

In order to make the user context value useful we may want to be able to add multiple application specific values as context values. One way to do this would be to use a Map as the user context value then we could add named values to the map and get them out again. Instead of a Map though I am going to use one of the classes from Coherence called a ResourceRegistry to which you add named resources of particular types.

Here is an example of the type of configuration that I want to be able to add to the cache configuration file…

  1. <distributed-scheme>
  2.     <scheme-name>my-distributed-scheme</scheme-name>
  3.     <service-name>DistributedCache</service-name>
  4.     <thread-count>5</thread-count>
  5.     <backing-map-scheme>
  6.         <local-scheme/>
  7.     </backing-map-scheme>
  8.     <autostart>true</autostart>
  9.     <ex:user-context-resources>
  10.         <ex:user-context-resource>
  11.             <ex:resource-name>test-resource</ex:resource-name>
  12.             <ex:resource-type>com.thegridman.coherence.config.MyTestResource</ex:resource-type>
  13.             <instance>
  14.                 <class-name>com.thegridman.coherence.config.MyTestResource</class-name>
  15.             </instance>
  16.         </ex:user-context-resource>
  17.     </ex:user-context-resources>
  18. </distributed-scheme>
<distributed-scheme>
    <scheme-name>my-distributed-scheme</scheme-name>
    <service-name>DistributedCache</service-name>
    <thread-count>5</thread-count>
    <backing-map-scheme>
        <local-scheme/>
    </backing-map-scheme>
    <autostart>true</autostart>
    <ex:user-context-resources>
        <ex:user-context-resource>
            <ex:resource-name>test-resource</ex:resource-name>
            <ex:resource-type>com.thegridman.coherence.config.MyTestResource</ex:resource-type>
            <instance>
                <class-name>com.thegridman.coherence.config.MyTestResource</class-name>
            </instance>
        </ex:user-context-resource>
    </ex:user-context-resources>
</distributed-scheme>

You can see that there is a custom <user-context-resources> element that can contain multiple <user-context-resource> elements (although the XML above only contains a single <user-context-resource> element). Inside the <user-context-resource> element are the following…

  • The <resource-name> element that contains the name of the resource.
  • The <resource-type> element that contains the class name of the resource (which may be different to the actual class of the resource as it may be say an interface class that the resource implements.
  • An <instance> element that is a standard Coherence element for defining instances of classes, in this case a simple test class.

Sticking to the patterns we used for service listeners above we will need the following classes:

  • A NamedResource class that will hold the name, type and resource value to be added to the ResourceRegistry user context.
  • A NamedResourceBuilder class that knows how to build a NamedResource.
  • A NamedResourceProcessor class to process the <user-context-resource> XML element and return a NamedResourceBuilder.
  • A NamedResourceScheme class that will be our ServiceExtension implementation that knows how to add the various named resources to a Service’s user context.

The NamedResource Class

The NamedResource class is just a holder class for a resource that will be added to a Service’s user context. As we are using a ResourceRegistry as the user context, each resource we add needs to have a name and a type as well as a value.

  1. package com.thegridman.coherence.config;
  2.  
  3. /**
  4.  * @author Jonathan Knight
  5.  */
  6. public class NamedResource
  7. {
  8.     private String name;
  9.     private Class resourceClass;
  10.     private Object resource;
  11.  
  12.     public NamedResource(String name, Class resourceClass, Object resource)
  13.     {
  14.         this.name = name;
  15.         this.resource = resource;
  16.         this.resourceClass = (resourceClass != null) ? resourceClass : resource.getClass();
  17.     }
  18.  
  19.     public String getName()
  20.     {
  21.         return name;
  22.     }
  23.  
  24.     public Class getResourceClass()
  25.     {
  26.         return resourceClass;
  27.     }
  28.  
  29.     public Object getResource()
  30.     {
  31.         return resource;
  32.     }
  33. }
package com.thegridman.coherence.config;

/**
 * @author Jonathan Knight
 */
public class NamedResource
{
    private String name;
    private Class resourceClass;
    private Object resource;

    public NamedResource(String name, Class resourceClass, Object resource)
    {
        this.name = name;
        this.resource = resource;
        this.resourceClass = (resourceClass != null) ? resourceClass : resource.getClass();
    }

    public String getName()
    {
        return name;
    }

    public Class getResourceClass()
    {
        return resourceClass;
    }

    public Object getResource()
    {
        return resource;
    }
}

The NamedResource is just a simple holder for the three attributes we require.

The NamedResourceBuilder Class

We next need a builder class that can build a NamedResource from the elements of the XML configuration; this will be a simple implementation of the built in ParameterizedBuilder class.

  1. package com.thegridman.coherence.config;
  2.  
  3. import com.tangosol.coherence.config.ParameterList;
  4. import com.tangosol.coherence.config.builder.ParameterizedBuilder;
  5. import com.tangosol.coherence.config.builder.ParameterizedBuilderHelper;
  6. import com.tangosol.config.annotation.Injectable;
  7. import com.tangosol.config.expression.Expression;
  8. import com.tangosol.config.expression.Parameter;
  9. import com.tangosol.config.expression.ParameterResolver;
  10. import com.tangosol.config.expression.Value;
  11. import com.tangosol.util.Base;
  12.  
  13. /**
  14.  * @author Jonathan Knight
  15.  */
  16. public class NamedResourceBuilder implements ParameterizedBuilder<NamedResource>
  17. {
  18.     private String resourceName;
  19.  
  20.     private String resourceClassName;
  21.  
  22.     private Expression resourceExpression;
  23.  
  24.     public String getResourceName()
  25.     {
  26.         return resourceName;
  27.     }
  28.  
  29.     @Injectable
  30.     public void setResourceName(String resourceName)
  31.     {
  32.         this.resourceName = resourceName;
  33.     }
  34.  
  35.     public String getResourceType()
  36.     {
  37.         return resourceClassName;
  38.     }
  39.  
  40.     @Injectable
  41.     public void setResourceType(String resourceClassName)
  42.     {
  43.         this.resourceClassName = resourceClassName;
  44.     }
  45.  
  46.     public Expression getResourceExpression()
  47.     {
  48.         return resourceExpression;
  49.     }
  50.  
  51.     public void setResourceExpression(Expression resourceExpression)
  52.     {
  53.         this.resourceExpression = resourceExpression;
  54.     }
  55.  
  56.     @Override
  57.     public NamedResource realize(ParameterResolver resolver, ClassLoader loader, ParameterList parameters)
  58.     {
  59.         try
  60.         {
  61.             Class<?> resourceClass = null;
  62.  
  63.             if (resourceClassName != null && !resourceClassName.isEmpty())
  64.             {
  65.                 resourceClass = loader.loadClass(resourceClassName);
  66.             }
  67.  
  68.             Object resource = null;
  69.             if (resourceExpression != null)
  70.             {
  71.                 resource = resourceExpression.evaluate(resolver);
  72.             }
  73.  
  74.             if (resource instanceof Value)
  75.             {
  76.                 resource = ((Value)resource).get();
  77.             }
  78.             else if (resource instanceof ParameterizedBuilder)
  79.             {
  80.                 ParameterizedBuilder builder = (ParameterizedBuilder) resource;
  81.                 if (resourceClass != null
  82.                     && !ParameterizedBuilderHelper.realizes(builder, resourceClass, resolver, loader))
  83.                 {
  84.                   throw new IllegalArgumentException("The specified resource class [" +
  85.                                                      resourceClass + "] is not built by: "
  86.                                                      + String.valueOf(builder));
  87.                 }
  88.                 resource = builder.realize(resolver, loader, parameters);
  89.             }
  90.  
  91.             return new NamedResource(resourceName, resourceClass, resource);
  92.         }
  93.         catch (ClassNotFoundException e)
  94.         {
  95.             throw Base.ensureRuntimeException(e);
  96.         }
  97.     }
  98. }
package com.thegridman.coherence.config;

import com.tangosol.coherence.config.ParameterList;
import com.tangosol.coherence.config.builder.ParameterizedBuilder;
import com.tangosol.coherence.config.builder.ParameterizedBuilderHelper;
import com.tangosol.config.annotation.Injectable;
import com.tangosol.config.expression.Expression;
import com.tangosol.config.expression.Parameter;
import com.tangosol.config.expression.ParameterResolver;
import com.tangosol.config.expression.Value;
import com.tangosol.util.Base;

/**
 * @author Jonathan Knight
 */
public class NamedResourceBuilder implements ParameterizedBuilder<NamedResource>
{
    private String resourceName;

    private String resourceClassName;

    private Expression resourceExpression;

    public String getResourceName()
    {
        return resourceName;
    }

    @Injectable
    public void setResourceName(String resourceName)
    {
        this.resourceName = resourceName;
    }

    public String getResourceType()
    {
        return resourceClassName;
    }

    @Injectable
    public void setResourceType(String resourceClassName)
    {
        this.resourceClassName = resourceClassName;
    }

    public Expression getResourceExpression()
    {
        return resourceExpression;
    }

    public void setResourceExpression(Expression resourceExpression)
    {
        this.resourceExpression = resourceExpression;
    }

    @Override
    public NamedResource realize(ParameterResolver resolver, ClassLoader loader, ParameterList parameters)
    {
        try
        {
            Class<?> resourceClass = null;

            if (resourceClassName != null && !resourceClassName.isEmpty())
            {
                resourceClass = loader.loadClass(resourceClassName);
            }

            Object resource = null;
            if (resourceExpression != null)
            {
                resource = resourceExpression.evaluate(resolver);
            }

            if (resource instanceof Value)
            {
                resource = ((Value)resource).get();
            }
            else if (resource instanceof ParameterizedBuilder)
            {
                ParameterizedBuilder builder = (ParameterizedBuilder) resource;
                if (resourceClass != null
                    && !ParameterizedBuilderHelper.realizes(builder, resourceClass, resolver, loader))
                {
                  throw new IllegalArgumentException("The specified resource class [" +
                                                     resourceClass + "] is not built by: "
                                                     + String.valueOf(builder));
                }
                resource = builder.realize(resolver, loader, parameters);
            }

            return new NamedResource(resourceName, resourceClass, resource);
        }
        catch (ClassNotFoundException e)
        {
            throw Base.ensureRuntimeException(e);
        }
    }
}

There are three fields in the builder to hold the three things we need to build a NamedResource, each of them has a setter method annotated with @Injectable so that Coherence will inject the required values when processing the XML configuration.

  • First is the resourceName field with the setter method setResourceName which will be injected from the contents of the <resource-name> XML element.
  • Next is the resourceClassName field with the setter method setResourceType which will be injected from the con tents of the <resource-type> XML element.
  • Finally is the resourceExpression field with the setter method setresourceExpression which will not be injected automatically but will be set by the NamedResourceProcessor using whatever is the value of the last child XML element within the <user-context-resource> element.

You will notice in the realize method that if after evaluating the resourceExpression the value of the resource is an instance of either a ParameterizedBuilder or a Value then the resource value is further evaluated.

If the last XML element in the <user-context-resource> is an <instance> element (as in our example XML) then the resource will evaluate to a ParamterizedBuilder. We then need to call the realize method on this builder to actually build the resource.

By also allowing the resource expression to evaluate to a Value we can do this in the XML configuration…

  1. <ex:user-context-resource>
  2.     <ex:resource-name>replicated-test-cache</ex:resource-name>
  3.     <ex:resource-type>com.tangosol.net.NamedCache</ex:resource-type>
  4.     <init-param>
  5.         <param-name>{cache-ref}</param-name>
  6.         <param-value>replicated-test</param-value>
  7.     </init-param>
  8. </ex:user-context-resource>
<ex:user-context-resource>
    <ex:resource-name>replicated-test-cache</ex:resource-name>
    <ex:resource-type>com.tangosol.net.NamedCache</ex:resource-type>
    <init-param>
        <param-name>{cache-ref}</param-name>
        <param-value>replicated-test</param-value>
    </init-param>
</ex:user-context-resource>

…that is, we can use an <init-param> element to define the resource. Doing this allows us to use a cache reference as our resource, as in the example above where we use a reference to the replicated-test cache, or any other valid <init-param> values such as a scheme-reference to reference another scheme or service, or we could just have a literal value.

The NamedResourceProcessor Class

The NamedResourceProcessor class is the ElementProcessor for the <user-context-resource> element.

  1. package com.thegridman.coherence.config;
  2.  
  3. import com.tangosol.config.ConfigurationException;
  4. import com.tangosol.config.expression.Expression;
  5. import com.tangosol.config.expression.LiteralExpression;
  6. import com.tangosol.config.xml.ElementProcessor;
  7. import com.tangosol.config.xml.ProcessingContext;
  8. import com.tangosol.run.xml.XmlElement;
  9.  
  10. import java.util.Map;
  11.  
  12. /**
  13.  * @author Jonathan Knight
  14.  */
  15. public class NamedResourceProcessor implements ElementProcessor<NamedResourceBuilder>
  16. {
  17.  
  18.     @SuppressWarnings("unchecked")
  19.     @Override
  20.     public NamedResourceBuilder process(ProcessingContext context, XmlElement element) throws ConfigurationException
  21.     {
  22.         NamedResourceBuilder builder = new NamedResourceBuilder();
  23.         context.inject(builder, element);
  24.  
  25.         Map<String,?> map = context.processRemainingElementsOf(element);
  26.         if (map.size() > 0)
  27.         {
  28.             Object value = map.values().iterator().next();
  29.             Expression expression = (value instanceof Expression)
  30.                     ? (Expression) value : new LiteralExpression(value);
  31.             builder.setResourceExpression(expression);
  32.         }
  33.  
  34.         return builder;
  35.     }
  36. }
package com.thegridman.coherence.config;

import com.tangosol.config.ConfigurationException;
import com.tangosol.config.expression.Expression;
import com.tangosol.config.expression.LiteralExpression;
import com.tangosol.config.xml.ElementProcessor;
import com.tangosol.config.xml.ProcessingContext;
import com.tangosol.run.xml.XmlElement;

import java.util.Map;

/**
 * @author Jonathan Knight
 */
public class NamedResourceProcessor implements ElementProcessor<NamedResourceBuilder>
{

    @SuppressWarnings("unchecked")
    @Override
    public NamedResourceBuilder process(ProcessingContext context, XmlElement element) throws ConfigurationException
    {
        NamedResourceBuilder builder = new NamedResourceBuilder();
        context.inject(builder, element);

        Map<String,?> map = context.processRemainingElementsOf(element);
        if (map.size() > 0)
        {
            Object value = map.values().iterator().next();
            Expression expression = (value instanceof Expression) 
                    ? (Expression) value : new LiteralExpression(value);
            builder.setResourceExpression(expression);
        }

        return builder;
    }
}

This is a simple ElementProcessor implementation. In the process method we first create an instance of NamedResourceBuilder and then we inject the values from the XML configuration into the builder (this would be the resource name and resource type). Then we get the ProcessingContext to process any remaining XML elements, which will be returned in a Map. We are only interested in the values of the map and there should only be a single value. This will be our resource value Expression so we set this into the builder. Finally we return the NamedResourceBuilder.

The NamedResourceScheme Class

The last class we require is the NamedResourceScheme class. Just like the ServiceListenersScheme class we wryte earlier, this class implements the ServiceExtension interface so that it can extend a Service by setting the user context and also it implements the ListOfInstancesProcessor.ListOfInstancesScheme interface so that it can be passed the list of NamedResourceBuilder instances created from the <user-context-resource> elements.

  1. package com.thegridman.coherence.config;
  2.  
  3. import com.tangosol.coherence.config.builder.ParameterizedBuilder;
  4. import com.tangosol.config.expression.ParameterResolver;
  5. import com.tangosol.net.Cluster;
  6. import com.tangosol.net.Service;
  7. import com.tangosol.util.ResourceRegistry;
  8. import com.tangosol.util.SimpleResourceRegistry;
  9.  
  10. import java.util.List;
  11.  
  12. /**
  13.  * @author Jonathan Knight
  14.  */
  15. public class NamedResourcesScheme
  16.         implements ServiceExtension, ListOfInstancesProcessor.ListOfInstancesScheme<NamedResourceBuilder>
  17. {
  18.     private List<NamedResourceBuilder> instances;
  19.  
  20.     public NamedResourcesScheme()
  21.     {
  22.     }
  23.  
  24.     @Override
  25.     public void setInstances(List<NamedResourceBuilder> instances)
  26.     {
  27.         this.instances = instances;
  28.     }
  29.  
  30.     @Override
  31.     public void extendService(Service service, ParameterResolver resolver, ClassLoader loader, Cluster cluster)
  32.     {
  33.         if (instances == null || instances.isEmpty())
  34.         {
  35.             return;
  36.         }
  37.  
  38.         Object userContext = service.getUserContext();
  39.         ResourceRegistry registry;
  40.         if (userContext != null && userContext instanceof ResourceRegistry)
  41.         {
  42.             registry = (ResourceRegistry) userContext;
  43.         }
  44.         else
  45.         {
  46.             registry = new SimpleResourceRegistry();
  47.         }
  48.  
  49.         for (ParameterizedBuilder<NamedResource> builder : instances)
  50.         {
  51.             NamedResource resource = builder.realize(resolver, loader, null);
  52.             registry.registerResource(resource.getResourceClass(), resource.getName(), resource.getResource());
  53.         }
  54.  
  55.         service.setUserContext(registry);
  56.     }
  57. }
package com.thegridman.coherence.config;

import com.tangosol.coherence.config.builder.ParameterizedBuilder;
import com.tangosol.config.expression.ParameterResolver;
import com.tangosol.net.Cluster;
import com.tangosol.net.Service;
import com.tangosol.util.ResourceRegistry;
import com.tangosol.util.SimpleResourceRegistry;

import java.util.List;

/**
 * @author Jonathan Knight
 */
public class NamedResourcesScheme
        implements ServiceExtension, ListOfInstancesProcessor.ListOfInstancesScheme<NamedResourceBuilder>
{
    private List<NamedResourceBuilder> instances;

    public NamedResourcesScheme()
    {
    }

    @Override
    public void setInstances(List<NamedResourceBuilder> instances)
    {
        this.instances = instances;
    }

    @Override
    public void extendService(Service service, ParameterResolver resolver, ClassLoader loader, Cluster cluster)
    {
        if (instances == null || instances.isEmpty())
        {
            return;
        }

        Object userContext = service.getUserContext();
        ResourceRegistry registry;
        if (userContext != null && userContext instanceof ResourceRegistry)
        {
            registry = (ResourceRegistry) userContext;
        }
        else
        {
            registry = new SimpleResourceRegistry();
        }

        for (ParameterizedBuilder<NamedResource> builder : instances)
        {
            NamedResource resource = builder.realize(resolver, loader, null);
            registry.registerResource(resource.getResourceClass(), resource.getName(), resource.getResource());
        }

        service.setUserContext(registry);
    }
}

The setInstances method is the implementation of the ListOfInstancesProcessor.ListOfInstancesScheme interface. This method will be passed the list of NamedResourceBuilder instances to use to build the resources.

The extendService method is the implementation of the ServiceExtension interface. If there are no NamedResourceBuilder instances this method just returns straight away. If there are instances then first the code checks to see if the user context of the Service is already a ResourceRegistry and if it is then the same registry is re-used otherwise a new registry is created. The NamedResouceBuilder instances are then iterated over to build each NamedResource. The resource is then added to the registry for the service.

Registering the User Context Element Processors

Now we have all of the classes built we just need to register the ElementProcessor instances for our custom namespace elements in our namespace handler as we did for the ServiceListener elements. Our ExtensionsNamespaceHandler Class’s constructor now looks like this…

  1. public ExtensionsNamespaceHandler()
  2. {
  3.     registerProcessor("service-listener", new CustomizableBuilderProcessor<>(ServiceListenerBuilder.class));
  4.     registerProcessor("service-listeners",
  5.                       new ListOfInstancesProcessor<>(ServiceListenersScheme.class, ServiceListenerBuilder.class));
  6.     registerProcessor("user-context-resource", new NamedResourceProcessor());
  7.     registerProcessor("user-context-resources",
  8.                       new ListOfInstancesProcessor<>(NamedResourcesScheme.class, NamedResourceBuilder.class));
  9. }
public ExtensionsNamespaceHandler()
{
    registerProcessor("service-listener", new CustomizableBuilderProcessor<>(ServiceListenerBuilder.class));
    registerProcessor("service-listeners", 
                      new ListOfInstancesProcessor<>(ServiceListenersScheme.class, ServiceListenerBuilder.class));
    registerProcessor("user-context-resource", new NamedResourceProcessor());
    registerProcessor("user-context-resources", 
                      new ListOfInstancesProcessor<>(NamedResourcesScheme.class, NamedResourceBuilder.class));
}

…with the addition of the two new processors for the <user-context-resource> and <user-context-resources> elements.

A Configurable Service User Context Example

Now everything is done we can finish with a full blown example of how we can set a whole range of values into the user context for a service.

Here is an example cache configuration file called usercontext-extensions-cache-config.xml

  1. <?xml version="1.0"?>
  2. <cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  3.              xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
  4.              xmlns:ex="class://com.thegridman.coherence.config.ExtensionsNamespaceHandler"
  5.              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd
  6.                                  class://com.thegridman.coherence.config.ExtensionsNamespaceHandler coherence-extended-config.xsd">
  7.  
  8.     <defaults>
  9.         <serializer>pof</serializer>
  10.     </defaults>
  11.  
  12.     <caching-scheme-mapping>
  13.         <cache-mapping>
  14.             <cache-name>dist-*</cache-name>
  15.             <scheme-name>my-distributed-scheme</scheme-name>
  16.         </cache-mapping>
  17.  
  18.         <cache-mapping>
  19.             <cache-name>replicated-*</cache-name>
  20.             <scheme-name>my-replicated-scheme</scheme-name>
  21.         </cache-mapping>
  22.     </caching-scheme-mapping>
  23.  
  24.     <caching-schemes>
  25.  
  26.         <distributed-scheme>
  27.             <scheme-name>my-distributed-scheme</scheme-name>
  28.             <service-name>DistributedCache</service-name>
  29.             <thread-count>5</thread-count>
  30.             <backing-map-scheme>
  31.                 <local-scheme/>
  32.             </backing-map-scheme>
  33.             <autostart>true</autostart>
  34.             <ex:user-context-resources>
  35.                 <ex:user-context-resource>
  36.                     <ex:resource-name>test-resource</ex:resource-name>
  37.                     <ex:resource-type>com.thegridman.coherence.config.MyTestResource</ex:resource-type>
  38.                     <instance>
  39.                         <class-name>com.thegridman.coherence.config.MyTestResource</class-name>
  40.                         <init-params>
  41.                             <init-param>
  42.                                 <param-type>String</param-type>
  43.                                 <param-value>from instance element</param-value>
  44.                             </init-param>
  45.                         </init-params>
  46.                     </instance>
  47.                 </ex:user-context-resource>
  48.                 <ex:user-context-resource>
  49.                     <ex:resource-name>replicated-test-cache</ex:resource-name>
  50.                     <ex:resource-type>com.tangosol.net.NamedCache</ex:resource-type>
  51.                     <init-param>
  52.                         <param-type>{cache-ref}</param-type>
  53.                         <param-value>replicated-test</param-value>
  54.                     </init-param>
  55.                 </ex:user-context-resource>
  56.                 <ex:user-context-resource>
  57.                     <ex:resource-name>another-test</ex:resource-name>
  58.                     <ex:resource-type>com.thegridman.coherence.config.MyTestResource</ex:resource-type>
  59.                     <init-param>
  60.                         <param-type>{scheme-ref}</param-type>
  61.                         <param-value>test-scheme</param-value>
  62.                     </init-param>
  63.                 </ex:user-context-resource>
  64.                 <ex:user-context-resource>
  65.                     <ex:resource-name>invocation</ex:resource-name>
  66.                     <ex:resource-type>com.tangosol.net.InvocationService</ex:resource-type>
  67.                     <init-param>
  68.                         <param-type>{scheme-ref}</param-type>
  69.                         <param-value>my-invocation-scheme</param-value>
  70.                     </init-param>
  71.                 </ex:user-context-resource>
  72.             </ex:user-context-resources>
  73.         </distributed-scheme>
  74.  
  75.         <class-scheme>
  76.             <scheme-name>test-scheme</scheme-name>
  77.             <class-name>com.thegridman.coherence.config.MyTestResource</class-name>
  78.             <init-params>
  79.                 <init-param>
  80.                     <param-type>String</param-type>
  81.                     <param-value>from class scheme</param-value>
  82.                 </init-param>
  83.             </init-params>
  84.         </class-scheme>
  85.  
  86.         <replicated-scheme>
  87.             <scheme-name>my-replicated-scheme</scheme-name>
  88.             <service-name>ReplicatedService</service-name>
  89.             <backing-map-scheme>
  90.                 <local-scheme/>
  91.             </backing-map-scheme>
  92.         </replicated-scheme>
  93.  
  94.         <invocation-scheme>
  95.             <scheme-name>my-invocation-scheme</scheme-name>
  96.             <service-name>MyInvocationService</service-name>
  97.             <thread-count>5</thread-count>
  98.             <autostart>true</autostart>
  99.         </invocation-scheme>
  100.     </caching-schemes>
  101.  
  102. </cache-config>
<?xml version="1.0"?>
<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
              xmlns:ex="class://com.thegridman.coherence.config.ExtensionsNamespaceHandler"
              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd
                                  class://com.thegridman.coherence.config.ExtensionsNamespaceHandler coherence-extended-config.xsd">

    <defaults>
        <serializer>pof</serializer>
    </defaults>

    <caching-scheme-mapping>
        <cache-mapping>
            <cache-name>dist-*</cache-name>
            <scheme-name>my-distributed-scheme</scheme-name>
        </cache-mapping>

        <cache-mapping>
            <cache-name>replicated-*</cache-name>
            <scheme-name>my-replicated-scheme</scheme-name>
        </cache-mapping>
    </caching-scheme-mapping>

    <caching-schemes>

        <distributed-scheme>
            <scheme-name>my-distributed-scheme</scheme-name>
            <service-name>DistributedCache</service-name>
            <thread-count>5</thread-count>
            <backing-map-scheme>
                <local-scheme/>
            </backing-map-scheme>
            <autostart>true</autostart>
            <ex:user-context-resources>
                <ex:user-context-resource>
                    <ex:resource-name>test-resource</ex:resource-name>
                    <ex:resource-type>com.thegridman.coherence.config.MyTestResource</ex:resource-type>
                    <instance>
                        <class-name>com.thegridman.coherence.config.MyTestResource</class-name>
                        <init-params>
                            <init-param>
                                <param-type>String</param-type>
                                <param-value>from instance element</param-value>
                            </init-param>
                        </init-params>
                    </instance>
                </ex:user-context-resource>
                <ex:user-context-resource>
                    <ex:resource-name>replicated-test-cache</ex:resource-name>
                    <ex:resource-type>com.tangosol.net.NamedCache</ex:resource-type>
                    <init-param>
                        <param-type>{cache-ref}</param-type>
                        <param-value>replicated-test</param-value>
                    </init-param>
                </ex:user-context-resource>
                <ex:user-context-resource>
                    <ex:resource-name>another-test</ex:resource-name>
                    <ex:resource-type>com.thegridman.coherence.config.MyTestResource</ex:resource-type>
                    <init-param>
                        <param-type>{scheme-ref}</param-type>
                        <param-value>test-scheme</param-value>
                    </init-param>
                </ex:user-context-resource>
                <ex:user-context-resource>
                    <ex:resource-name>invocation</ex:resource-name>
                    <ex:resource-type>com.tangosol.net.InvocationService</ex:resource-type>
                    <init-param>
                        <param-type>{scheme-ref}</param-type>
                        <param-value>my-invocation-scheme</param-value>
                    </init-param>
                </ex:user-context-resource>
            </ex:user-context-resources>
        </distributed-scheme>

        <class-scheme>
            <scheme-name>test-scheme</scheme-name>
            <class-name>com.thegridman.coherence.config.MyTestResource</class-name>
            <init-params>
                <init-param>
                    <param-type>String</param-type>
                    <param-value>from class scheme</param-value>
                </init-param>
            </init-params>
        </class-scheme>

        <replicated-scheme>
            <scheme-name>my-replicated-scheme</scheme-name>
            <service-name>ReplicatedService</service-name>
            <backing-map-scheme>
                <local-scheme/>
            </backing-map-scheme>
        </replicated-scheme>

        <invocation-scheme>
            <scheme-name>my-invocation-scheme</scheme-name>
            <service-name>MyInvocationService</service-name>
            <thread-count>5</thread-count>
            <autostart>true</autostart>
        </invocation-scheme>
    </caching-schemes>

</cache-config>

You can see that we have added a number of user context resources to the DistributedCache service.

  • First we have a resource created from an <instance> element with the name test-resource and the type com.thegridman.coherence.config.MyTestResource that takes a single String constructor argument.
  • Second we have a resource created from an <init-param> element that is named replicated-test-cachethat will be a reference to a cache called replicated-test.
  • Third we have another resource created from an <init-param> element that is named another-test and is a reference to a class-scheme.
  • Finally we have yet another resource from an <init-param> element that is named invocation and is a scheme reference to an InvocationService scheme.

If we then create a unit test class that uses the cache configuration file we can prove that all the user context resources are created correctly.

  1. package com.thegridman.coherence.config;
  2.  
  3. import com.oracle.tools.junit.AbstractTest;
  4. import com.tangosol.net.CacheFactory;
  5. import com.tangosol.net.ConfigurableCacheFactory;
  6. import com.tangosol.net.InvocationService;
  7. import com.tangosol.net.NamedCache;
  8. import com.tangosol.net.Service;
  9. import com.tangosol.util.ResourceRegistry;
  10. import org.junit.BeforeClass;
  11. import org.junit.Test;
  12.  
  13. import static org.hamcrest.CoreMatchers.is;
  14. import static org.hamcrest.CoreMatchers.notNullValue;
  15. import static org.junit.Assert.assertThat;
  16.  
  17. /**
  18.  * @author Jonathan Knight
  19.  */
  20. public class UserContextServiceExtensionsAcceptanceTest extends AbstractTest
  21. {
  22.     private static ConfigurableCacheFactory cacheFactory;
  23.  
  24.     @BeforeClass
  25.     public static void createCacheFactory()
  26.     {
  27.         System.setProperty("tangosol.coherence.localhost", "Jonathans-MacBook-Pro.local");
  28.         System.setProperty("tangosol.coherence.cacheconfig", "usercontext-extensions-cache-config.xml");
  29.  
  30.         cacheFactory = CacheFactory.getCacheFactoryBuilder().getConfigurableCacheFactory(null);
  31.     }
  32.  
  33.     @Test
  34.     public void shouldContainTestResourceFromInstance() throws Exception
  35.     {
  36.         Service service = cacheFactory.ensureService("DistributedCache");
  37.         ResourceRegistry registry = (ResourceRegistry) service.getUserContext();
  38.  
  39.         MyTestResource resource = registry.getResource(MyTestResource.class, "test-resource");
  40.         assertThat(resource, is(notNullValue()));
  41.         assertThat(resource.getParamValue(), is("from instance element"));
  42.     }
  43.  
  44.     @Test
  45.     public void shouldContainTestResourceFromSchemeRef() throws Exception
  46.     {
  47.         Service service = cacheFactory.ensureService("DistributedCache");
  48.         ResourceRegistry registry = (ResourceRegistry) service.getUserContext();
  49.  
  50.         MyTestResource resource = registry.getResource(MyTestResource.class, "another-test");
  51.         assertThat(resource, is(notNullValue()));
  52.         assertThat(resource.getParamValue(), is("from class scheme"));
  53.     }
  54.  
  55.     @Test
  56.     public void shouldContainCacheReference() throws Exception
  57.     {
  58.         Service service = cacheFactory.ensureService("DistributedCache");
  59.         ResourceRegistry registry = (ResourceRegistry) service.getUserContext();
  60.  
  61.         NamedCache replicatedCache = registry.getResource(NamedCache.class, "replicated-test-cache");
  62.         assertThat(replicatedCache, is(notNullValue()));
  63.         assertThat(replicatedCache.getCacheName(), is("replicated-test"));
  64.     }
  65.  
  66.     @Test
  67.     public void shouldContainInvocationService() throws Exception
  68.     {
  69.         Service service = cacheFactory.ensureService("DistributedCache");
  70.         ResourceRegistry registry = (ResourceRegistry) service.getUserContext();
  71.  
  72.         InvocationService invocationService = registry.getResource(InvocationService.class, "invocation");
  73.         assertThat(invocationService, is(notNullValue()));
  74.         assertThat(invocationService.getInfo().getServiceName(), is("MyInvocationService"));
  75.     }
  76.  
  77. }
package com.thegridman.coherence.config;

import com.oracle.tools.junit.AbstractTest;
import com.tangosol.net.CacheFactory;
import com.tangosol.net.ConfigurableCacheFactory;
import com.tangosol.net.InvocationService;
import com.tangosol.net.NamedCache;
import com.tangosol.net.Service;
import com.tangosol.util.ResourceRegistry;
import org.junit.BeforeClass;
import org.junit.Test;

import static org.hamcrest.CoreMatchers.is;
import static org.hamcrest.CoreMatchers.notNullValue;
import static org.junit.Assert.assertThat;

/**
 * @author Jonathan Knight
 */
public class UserContextServiceExtensionsAcceptanceTest extends AbstractTest
{
    private static ConfigurableCacheFactory cacheFactory;

    @BeforeClass
    public static void createCacheFactory()
    {
        System.setProperty("tangosol.coherence.localhost", "Jonathans-MacBook-Pro.local");
        System.setProperty("tangosol.coherence.cacheconfig", "usercontext-extensions-cache-config.xml");

        cacheFactory = CacheFactory.getCacheFactoryBuilder().getConfigurableCacheFactory(null);
    }

    @Test
    public void shouldContainTestResourceFromInstance() throws Exception
    {
        Service service = cacheFactory.ensureService("DistributedCache");
        ResourceRegistry registry = (ResourceRegistry) service.getUserContext();

        MyTestResource resource = registry.getResource(MyTestResource.class, "test-resource");
        assertThat(resource, is(notNullValue()));
        assertThat(resource.getParamValue(), is("from instance element"));
    }

    @Test
    public void shouldContainTestResourceFromSchemeRef() throws Exception
    {
        Service service = cacheFactory.ensureService("DistributedCache");
        ResourceRegistry registry = (ResourceRegistry) service.getUserContext();

        MyTestResource resource = registry.getResource(MyTestResource.class, "another-test");
        assertThat(resource, is(notNullValue()));
        assertThat(resource.getParamValue(), is("from class scheme"));
    }

    @Test
    public void shouldContainCacheReference() throws Exception
    {
        Service service = cacheFactory.ensureService("DistributedCache");
        ResourceRegistry registry = (ResourceRegistry) service.getUserContext();

        NamedCache replicatedCache = registry.getResource(NamedCache.class, "replicated-test-cache");
        assertThat(replicatedCache, is(notNullValue()));
        assertThat(replicatedCache.getCacheName(), is("replicated-test"));
    }

    @Test
    public void shouldContainInvocationService() throws Exception
    {
        Service service = cacheFactory.ensureService("DistributedCache");
        ResourceRegistry registry = (ResourceRegistry) service.getUserContext();

        InvocationService invocationService = registry.getResource(InvocationService.class, "invocation");
        assertThat(invocationService, is(notNullValue()));
        assertThat(invocationService.getInfo().getServiceName(), is("MyInvocationService"));
    }

}

If we run the test class above then all the tests should pass and should show each resource correctly created in the user context of the service.

The Power of Custom Namespaces

I think this shows just how useful custom namespaces can be for creating configurable extensions to the built in Coherence functionality. There are a few extensibility issues with the current implementation of the configuration framework which if addressed would make things like this easier without overriding too many built in classes.

Hopefully it has been useful as an aid to seeing just what you can do with namespaces in your own applications.

Enjoy…

Oracle Coherence Regular Expression Cache Mapping

RegEx

This blog post is going to be about Oracle Coherence regular expression cache mapping. Recently there was an e-mail exchange between Andrew Wilson, a good friend of mine, and the Oracle Coherence engineers on the subject of using Java regular expressions in cache mappings rather than just being limited to adding an asterisk suffix. Andrew suggested that Oracle could add this to the core Coherence product whereas the engineering team pointed out that there was probably enough infrastructure in Coherence to do it yourself. Well, always being up for a challenge, especially if it means writing code (or cycling), I thought I’d write it myself and do a blog post about it. After all I’d much rather the engineering team spent time on putting things in the core product that I can’t do myself.

Now, as with most things in Oracle Coherence you could approach this a number of ways depending on what version of Coherence you are using and how hacky you are comfortable with getting. For the purposes of this post I am going to write one approach using the more elegant custom namespace support in Oracle Coherence 12.1.2 and then I will cover a different approach for those of you yet to upgrade and still on 3.7.1 or earlier.

How Built-In Coherence Cache Mapping Works

To start with I’ going to go over how cache mappings work at the moment in Coherence so we can see the limitations and also understand the limitations we will introduce when we add regular expression support.

Currently in Coherence cache mappings you have two choices, you either put the exact cache name in the mapping or you put a name prefix ending with an asterisk into the mapping (or a third option, you just have an asterisk that matches everything). For example here are three mappings

  1. <cache-mapping>
  2.     <cache-name>schedule-cache</cache-name>
  3.     <scheme-name>schedule-scheme</scheme-name>
  4. </cache-mapping>
  5.  
  6. <cache-mapping>
  7.     <cache-name>replicated-*</cache-name>
  8.     <scheme-name>replicated-scheme</scheme-name>
  9. </cache-mapping>
  10.  
  11. <cache-mapping>
  12.     <cache-name>*</cache-name>
  13.     <scheme-name>distributed-scheme</scheme-name>
  14. </cache-mapping>
<cache-mapping>
    <cache-name>schedule-cache</cache-name>
    <scheme-name>schedule-scheme</scheme-name>
</cache-mapping>

<cache-mapping>
    <cache-name>replicated-*</cache-name>
    <scheme-name>replicated-scheme</scheme-name>
</cache-mapping>

<cache-mapping>
    <cache-name>*</cache-name>
    <scheme-name>distributed-scheme</scheme-name>
</cache-mapping>

If we had these mappings in our cache configuration file then the following rules apply:

  • Cache schedule-cache maps to the schedule-scheme cache scheme.
  • Caches starting with replicated- map to the replicated-scheme cache scheme, e.g. caches called replicated-stuff and replicated-things both match this mapping.
  • Any other cache names will match the default * mapping and map to the distributed-scheme cache scheme.

Mapping Order

The order that you put the mappings in the cache configuration file is important. Coherence applies the following rules.
1. Exact matches can go anywhere in the caching-scheme-mapping section of the configuration file and Coherence will use that mapping to map the matching cache name.
2. The default * mapping can also go anywhere in the caching-scheme-mapping section of the configuration file. When mapping a cache name if there is no exact match and no prefix match then the default mapping will be used if one has been specified.
3. If there is no exact match then the LAST prefix mapping that matches the cache name will be used. For that reason you should make sure your mappings are more generic at the top of the caching-scheme-mapping section of the configuration file and get more specific as you go down the file.

For example, suppose we have the following mappings…

  1. <cache-mapping>
  2.     <cache-name>distributed-*</cache-name>
  3.     <scheme-name>distributed-scheme</scheme-name>
  4. </cache-mapping>
  5.  
  6. <cache-mapping>
  7.     <cache-name>distributed-two-*</cache-name>
  8.     <scheme-name>distributed-scheme-two</scheme-name>
  9. </cache-mapping>
<cache-mapping>
    <cache-name>distributed-*</cache-name>
    <scheme-name>distributed-scheme</scheme-name>
</cache-mapping>

<cache-mapping>
    <cache-name>distributed-two-*</cache-name>
    <scheme-name>distributed-scheme-two</scheme-name>
</cache-mapping>

…then a cache called distributed-two-cache would map to the distributed-scheme-two scheme as the distributed-two-* mapping is the last one that matches the cache name. If we now reordered the mappings like this…

  1. <cache-mapping>
  2.     <cache-name>distributed-two-*</cache-name>
  3.     <scheme-name>distributed-scheme-two</scheme-name>
  4. </cache-mapping>
  5.  
  6. <cache-mapping>
  7.     <cache-name>distributed-*</cache-name>
  8.     <scheme-name>distributed-scheme</scheme-name>
  9. </cache-mapping>
<cache-mapping>
    <cache-name>distributed-two-*</cache-name>
    <scheme-name>distributed-scheme-two</scheme-name>
</cache-mapping>

<cache-mapping>
    <cache-name>distributed-*</cache-name>
    <scheme-name>distributed-scheme</scheme-name>
</cache-mapping>

…then the cache called distributed-two-cache would now map to the distributed-scheme scheme as the distributed-* mapping is the last one that matches the cache name. In this case the distributed-two-* mapping would never be used as the more generic distributed-* comes after it in the mappings list.

  1. <cache-mapping>
  2.     <cache-name>distributed-two-*</cache-name>
  3.     <scheme-name>distributed-scheme-two</scheme-name>
  4. </cache-mapping>
  5.  
  6. <cache-mapping>
  7.     <cache-name>distributed-*</cache-name>
  8.     <scheme-name>distributed-scheme</scheme-name>
  9. </cache-mapping>
<cache-mapping>
    <cache-name>distributed-two-*</cache-name>
    <scheme-name>distributed-scheme-two</scheme-name>
</cache-mapping>

<cache-mapping>
    <cache-name>distributed-*</cache-name>
    <scheme-name>distributed-scheme</scheme-name>
</cache-mapping>

Andrew’s Problem

Now you might wonder why we need regular expression mapping, after all, we have managed with the mapping functionality above for many years and versions of Coherence. The scenario Andrew gave was one where we have a system that has many caches, say a hundred. Of these caches some have an expiry of say 1 day and some an expiry of 2 days, but there is no common cache name prefix shared by caches with the same expiry so you end up with this…

  1. <caching-scheme-mapping>
  2.     <cache-mapping>
  3.         <cache-name>cache-one</cache-name>
  4.         <scheme-name>distributed-scheme-one-day</scheme-name>
  5.     </cache-mapping>
  6.  
  7.     <cache-mapping>
  8.         <cache-name>cache-one</cache-name>
  9.         <scheme-name>distributed-scheme-two-days</scheme-name>
  10.     </cache-mapping>
  11.  
  12.     <!-- Lots more cache mappings go here -->
  13.    
  14.     <cache-mapping>
  15.         <cache-name>cache-ninety-nine</cache-name>
  16.         <scheme-name>distributed-scheme-one-day</scheme-name>
  17.     </cache-mapping>
  18.    
  19.     <cache-mapping>
  20.         <cache-name>cache-one-hundred</cache-name>
  21.         <scheme-name>distributed-scheme-two-days</scheme-name>
  22.     </cache-mapping>    
  23. </caching-scheme-mapping>
  24.  
  25. <caching-schemes>
  26.     <distributed-scheme>
  27.         <scheme-name>distributed-scheme-one-day</scheme-name>
  28.         <scheme-ref>distributed-scheme</scheme-ref>
  29.         <backing-map-scheme>
  30.             <local-scheme>
  31.                 <expiry-delay>1d</expiry-delay>
  32.             </local-scheme>
  33.         </backing-map-scheme>
  34.     </distributed-scheme>
  35.  
  36.     <distributed-scheme>
  37.         <scheme-name>distributed-scheme-two-days</scheme-name>
  38.         <scheme-ref>distributed-scheme</scheme-ref>
  39.         <backing-map-scheme>
  40.             <local-scheme>
  41.                 <expiry-delay>2d</expiry-delay>
  42.             </local-scheme>
  43.         </backing-map-scheme>
  44.     </distributed-scheme>
  45.  
  46.     <distributed-scheme>
  47.         <scheme-name>distributed-scheme</scheme-name>
  48.         <service-name>DistributedService</service-name>
  49.         <backing-map-scheme>
  50.             <local-scheme/>
  51.         </backing-map-scheme>
  52.         <autostart>true</autostart>
  53.     </distributed-scheme>
  54. </caching-schemes>
<caching-scheme-mapping>
    <cache-mapping>
        <cache-name>cache-one</cache-name>
        <scheme-name>distributed-scheme-one-day</scheme-name>
    </cache-mapping>

    <cache-mapping>
        <cache-name>cache-one</cache-name>
        <scheme-name>distributed-scheme-two-days</scheme-name>
    </cache-mapping>

    <!-- Lots more cache mappings go here -->
    
    <cache-mapping>
        <cache-name>cache-ninety-nine</cache-name>
        <scheme-name>distributed-scheme-one-day</scheme-name>
    </cache-mapping>
    
    <cache-mapping>
        <cache-name>cache-one-hundred</cache-name>
        <scheme-name>distributed-scheme-two-days</scheme-name>
    </cache-mapping>    
</caching-scheme-mapping>

<caching-schemes>
    <distributed-scheme>
        <scheme-name>distributed-scheme-one-day</scheme-name>
        <scheme-ref>distributed-scheme</scheme-ref>
        <backing-map-scheme>
            <local-scheme>
                <expiry-delay>1d</expiry-delay>
            </local-scheme>
        </backing-map-scheme>
    </distributed-scheme>

    <distributed-scheme>
        <scheme-name>distributed-scheme-two-days</scheme-name>
        <scheme-ref>distributed-scheme</scheme-ref>
        <backing-map-scheme>
            <local-scheme>
                <expiry-delay>2d</expiry-delay>
            </local-scheme>
        </backing-map-scheme>
    </distributed-scheme>

    <distributed-scheme>
        <scheme-name>distributed-scheme</scheme-name>
        <service-name>DistributedService</service-name>
        <backing-map-scheme>
            <local-scheme/>
        </backing-map-scheme>
        <autostart>true</autostart>
    </distributed-scheme>
</caching-schemes>

The comment line !-- Lots more cache mappings go here -- would be where the other 96 cache mappings go, as you can imagine a maintenance nightmare.

Now if we have the option to use regular expression instead of just the * suffix then we have a lot more flexibility and the chance that we would greatly reduce the need for our 100 individual cache mappings.

Coherence 12.1.2 Implementation

The first implementation I am going to cover is for the latest 12.1.2 version of Coherence and will be built using a custom namespace. Using a namespace is the easiest way to extend Coherence to do something other than what it does out of the box as it is now supported in the core product so you are not having to override internal Coherence classes to hack your own functionality in.

I covered some of the namespaces functionality in a previous post Coherence 12.1.2 Custom Namespaces and Oracle NoSQL and there are also some videos on the Coherence YouTube Channel that cover this too Coherence Configuration Enhancements – Part 1 – 4 so I’m going to dive straight into the implementation rather than explain namespace support.

The RegExCacheMapping Class

The first thing we need is an implementation of the built in CacheMapping class that supports regular expressions. We will call this class RegExCacheMapping. The CacheMapping class is the class that is part of the configuration framework in 12.1.2 that holds details of a particular mapping and works out whether its mapping matches a given cache name. In this case it should be easy enough to extend this class to use a regular expression.

  1. package com.thegridman.coherence.regex;
  2.  
  3. import com.tangosol.coherence.config.CacheMapping;
  4. import com.tangosol.coherence.config.builder.NamedEventInterceptorBuilder;
  5. import com.tangosol.config.annotation.Injectable;
  6. import com.tangosol.config.expression.ParameterResolver;
  7.  
  8. import java.util.ArrayList;
  9. import java.util.List;
  10. import java.util.regex.Matcher;
  11. import java.util.regex.Pattern;
  12.  
  13. /**
  14.  * @author Jonathan Knight
  15.  */
  16. public class RegExCacheMapping extends CacheMapping
  17. {
  18.     private Pattern pattern;
  19.  
  20.     public RegExCacheMapping(String regEx, String schemeName)
  21.     {
  22.         super(regEx, schemeName);
  23.         pattern = Pattern.compile(regEx);
  24.     }
  25.  
  26.     @Override
  27.     public boolean isForCacheName(String sCacheName)
  28.     {
  29.         Matcher matcher = pattern.matcher(sCacheName);
  30.         return matcher.matches();
  31.     }
  32.  
  33.     @Override
  34.     public boolean usesWildcard()
  35.     {
  36.         return true;
  37.     }
  38.  
  39.     @Override
  40.     @Injectable("interceptors")
  41.     public void setEventInterceptorBuilders(List<NamedEventInterceptorBuilder> listBuilders)
  42.     {
  43.         List<NamedEventInterceptorBuilder> regExBuilders = new ArrayList<>(listBuilders.size());
  44.         for (NamedEventInterceptorBuilder builder : listBuilders)
  45.         {
  46.             regExBuilders.add(new RegExEventInterceptorBuilder(builder));
  47.         }
  48.         super.setEventInterceptorBuilders(regExBuilders);
  49.     }
  50. }
package com.thegridman.coherence.regex;

import com.tangosol.coherence.config.CacheMapping;
import com.tangosol.coherence.config.builder.NamedEventInterceptorBuilder;
import com.tangosol.config.annotation.Injectable;
import com.tangosol.config.expression.ParameterResolver;

import java.util.ArrayList;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

/**
 * @author Jonathan Knight
 */
public class RegExCacheMapping extends CacheMapping
{
    private Pattern pattern;

    public RegExCacheMapping(String regEx, String schemeName)
    {
        super(regEx, schemeName);
        pattern = Pattern.compile(regEx);
    }

    @Override
    public boolean isForCacheName(String sCacheName)
    {
        Matcher matcher = pattern.matcher(sCacheName);
        return matcher.matches();
    }

    @Override
    public boolean usesWildcard()
    {
        return true;
    }

    @Override
    @Injectable("interceptors")
    public void setEventInterceptorBuilders(List<NamedEventInterceptorBuilder> listBuilders)
    {
        List<NamedEventInterceptorBuilder> regExBuilders = new ArrayList<>(listBuilders.size());
        for (NamedEventInterceptorBuilder builder : listBuilders)
        {
            regExBuilders.add(new RegExEventInterceptorBuilder(builder));
        }
        super.setEventInterceptorBuilders(regExBuilders);
    }
}

The code is pretty simple.

  • The class extends com.tangosol.coherence.config.CacheMapping which it has to as it will be used in the same places in the configuration.
  • The constructor takes the same arguments as CacheMapping but in this case the first constructor parameter will be the regular expression rather than the cache name. We pass this up to the super class constructor and we also compile the instance of java.util.regex.Pattern ready for use later.
  • We override the usesWildcard() to always return true. In the super class this method would return true if the cache name has an * suffix, but a reg-ex is always a wild-card mapping.
  • The isForCacheName(String sCacheName) again is a straight forward override. As its name suggests this method returns true if this mapping is for the specified cache name. In our implementation we just see if the reg-ex we have matches the specified cache name.
  • Finally we have overridden the setEventInterceptorBuilders(List<NamedEventInterceptorBuilder> listBuilders) method; the reason for which I shall explain. In the cache configuration file in 12.1.2 we can specify a list of Interceptors that can intercept events on a cache. This can be done either in the service configuration or in the cache mapping configuration. If the interceptors are specified in the cache mapping then they only listen for events on caches who’s name matches the mapping – you should be able to see where this is going, the normal interceptor supports the rules for matching cache names using only an * suffix, it does not support regular expressions so for this we need to write our own implementation of the interceptor builder NamedEventInterceptorBuilder and also an implementation of the interceptor wrapper to be built. Our implementation of setEventInterceptorBuilders(List<NamedEventInterceptorBuilder> listBuilders) just loops over the list of normal NamedEventInterceptorBuilders that it is passed and converts them to RegExEventInterceptorBuilder which are then passed to the super class.

The RegExEventInterceptorBuilder Class

The sole purpose of the RegExEventInterceptorBuilder class is to build our interceptor wrapper class RegExEventInterceptor wrapper class instead of the default NamedEventInterceptor class.

  1. package com.thegridman.coherence.regex;
  2.  
  3. import com.tangosol.coherence.config.ParameterList;
  4. import com.tangosol.coherence.config.builder.NamedEventInterceptorBuilder;
  5. import com.tangosol.coherence.config.builder.ParameterizedBuilder;
  6. import com.tangosol.coherence.config.builder.ParameterizedBuilderHelper;
  7. import com.tangosol.config.expression.Parameter;
  8. import com.tangosol.config.expression.ParameterResolver;
  9. import com.tangosol.net.events.EventInterceptor;
  10.  
  11. /**
  12.  * @author Jonathan Knight
  13.  */
  14. public class RegExEventInterceptorBuilder extends NamedEventInterceptorBuilder
  15. {
  16.  
  17.     public RegExEventInterceptorBuilder(NamedEventInterceptorBuilder builder)
  18.     {
  19.         this.setName(builder.getName());
  20.         this.setRegistrationBehavior(builder.getRegistrationBehavior());
  21.         this.setOrder(builder.getOrder());
  22.         this.setCustomBuilder(builder.getCustomBuilder());
  23.     }
  24.  
  25.     @SuppressWarnings("unchecked")
  26.     public RegExEventInterceptor realize(ParameterResolver resolver, ClassLoader loader, ParameterList parameters)
  27.     {
  28.         loader = (loader == null) ? getClass().getClassLoader() : loader;
  29.  
  30.         ParameterizedBuilder builder = getCustomBuilder();
  31.  
  32.         if (!ParameterizedBuilderHelper.realizes(builder, EventInterceptor.class, resolver, loader))
  33.         {
  34.             throw new IllegalArgumentException("Unable to build an EventInterceptor based on the specified class: "
  35.                                                + String.valueOf(builder));
  36.         }
  37.  
  38.         EventInterceptor interceptor = (EventInterceptor) builder.realize(resolver, loader, parameters);
  39.  
  40.         Parameter serviceNameParam = resolver.resolve("service-name");
  41.         String serviceName = serviceNameParam == null ? null : serviceNameParam.evaluate(resolver).as(String.class);
  42.         Parameter cacheNameParam = resolver.resolve("cache-name");
  43.         String cacheName = cacheNameParam == null ? null : cacheNameParam.evaluate(resolver).as(String.class);
  44.  
  45.         return new RegExEventInterceptor(getName(), interceptor, cacheName,
  46.                                          serviceName, getOrder(), getRegistrationBehavior());
  47.     }
  48.  
  49. }
package com.thegridman.coherence.regex;

import com.tangosol.coherence.config.ParameterList;
import com.tangosol.coherence.config.builder.NamedEventInterceptorBuilder;
import com.tangosol.coherence.config.builder.ParameterizedBuilder;
import com.tangosol.coherence.config.builder.ParameterizedBuilderHelper;
import com.tangosol.config.expression.Parameter;
import com.tangosol.config.expression.ParameterResolver;
import com.tangosol.net.events.EventInterceptor;

/**
 * @author Jonathan Knight
 */
public class RegExEventInterceptorBuilder extends NamedEventInterceptorBuilder
{

    public RegExEventInterceptorBuilder(NamedEventInterceptorBuilder builder)
    {
        this.setName(builder.getName());
        this.setRegistrationBehavior(builder.getRegistrationBehavior());
        this.setOrder(builder.getOrder());
        this.setCustomBuilder(builder.getCustomBuilder());
    }

    @SuppressWarnings("unchecked")
    public RegExEventInterceptor realize(ParameterResolver resolver, ClassLoader loader, ParameterList parameters)
    {
        loader = (loader == null) ? getClass().getClassLoader() : loader;

        ParameterizedBuilder builder = getCustomBuilder();

        if (!ParameterizedBuilderHelper.realizes(builder, EventInterceptor.class, resolver, loader))
        {
            throw new IllegalArgumentException("Unable to build an EventInterceptor based on the specified class: "
                                               + String.valueOf(builder));
        }

        EventInterceptor interceptor = (EventInterceptor) builder.realize(resolver, loader, parameters);

        Parameter serviceNameParam = resolver.resolve("service-name");
        String serviceName = serviceNameParam == null ? null : serviceNameParam.evaluate(resolver).as(String.class);
        Parameter cacheNameParam = resolver.resolve("cache-name");
        String cacheName = cacheNameParam == null ? null : cacheNameParam.evaluate(resolver).as(String.class);

        return new RegExEventInterceptor(getName(), interceptor, cacheName,
                                         serviceName, getOrder(), getRegistrationBehavior());
    }

}

The main method of interest is the realize method that will build the interceptor. The code follows the standard patter for builder code that is going to delegate to another builder. In this case the actual interceptor class to be built is specified in the inner <instance> tag of the configuration. In our method the interceptor builder will be returned by the getCustomBuilder() method and we then use that to build the actual interceptor. Once we have this it is wrapped in our RegExEventInterceptor which will ensure the interceptor only receives events for the caches which names matching the reg-ex.

The RegExEventInterceptor Class

We need and implementation of NamedEventInterceptor which works with regular expressions and which is built by our RegExEventInterceptorBuilder so we created the RegExEventInterceptor class.

  1. package com.thegridman.coherence.regex;
  2.  
  3. import com.tangosol.net.events.Event;
  4. import com.tangosol.net.events.EventDispatcher;
  5. import com.tangosol.net.events.EventInterceptor;
  6. import com.tangosol.net.events.annotation.Interceptor;
  7. import com.tangosol.net.events.internal.NamedEventInterceptor;
  8. import com.tangosol.net.events.partition.PartitionedServiceDispatcher;
  9. import com.tangosol.net.events.partition.cache.PartitionedCacheDispatcher;
  10. import com.tangosol.util.RegistrationBehavior;
  11.  
  12. import java.util.regex.Matcher;
  13. import java.util.regex.Pattern;
  14.  
  15. /**
  16.  * @author Jonathan Knight
  17.  */
  18. public class RegExEventInterceptor<E extends Event<?>> extends NamedEventInterceptor<E>
  19. {
  20.     private Pattern pattern;
  21.  
  22.     public RegExEventInterceptor(String sName, EventInterceptor<E> interceptor,
  23.                                  String cacheName, String serviceName, Interceptor.Order order,
  24.                                  RegistrationBehavior behavior)
  25.     {
  26.         super(sName, interceptor, cacheName, serviceName, order, behavior);
  27.         pattern = Pattern.compile(cacheName);
  28.     }
  29.  
  30.     @Override
  31.     public boolean isAcceptable(EventDispatcher dispatcher)
  32.     {
  33.         String serviceName = getServiceName();
  34.         if (serviceName == null && pattern == null)
  35.         {
  36.             return true;
  37.         }
  38.  
  39.         String dispatcherServiceName = null;
  40.         String dispatcherCacheName = null;
  41.         if (dispatcher instanceof PartitionedCacheDispatcher)
  42.         {
  43.             PartitionedCacheDispatcher bmd = (PartitionedCacheDispatcher) dispatcher;
  44.             dispatcherServiceName = bmd.getBackingMapContext()
  45.                     .getManagerContext().getCacheService().getInfo().getServiceName();
  46.             dispatcherCacheName = bmd.getBackingMapContext().getCacheName();
  47.         }
  48.         else if ((dispatcher instanceof PartitionedServiceDispatcher))
  49.         {
  50.             dispatcherServiceName = ((PartitionedServiceDispatcher) dispatcher).getService()
  51.                     .getInfo().getServiceName();
  52.         }
  53.  
  54.         boolean match = serviceName == null || serviceName.equals(dispatcherServiceName);
  55.         if (match && dispatcherCacheName != null && pattern != null)
  56.         {
  57.             Matcher matcher = pattern.matcher(dispatcherCacheName);
  58.             return matcher.matches();
  59.         }
  60.  
  61.         return false;
  62.     }
  63. }
package com.thegridman.coherence.regex;

import com.tangosol.net.events.Event;
import com.tangosol.net.events.EventDispatcher;
import com.tangosol.net.events.EventInterceptor;
import com.tangosol.net.events.annotation.Interceptor;
import com.tangosol.net.events.internal.NamedEventInterceptor;
import com.tangosol.net.events.partition.PartitionedServiceDispatcher;
import com.tangosol.net.events.partition.cache.PartitionedCacheDispatcher;
import com.tangosol.util.RegistrationBehavior;

import java.util.regex.Matcher;
import java.util.regex.Pattern;

/**
 * @author Jonathan Knight
 */
public class RegExEventInterceptor<E extends Event<?>> extends NamedEventInterceptor<E>
{
    private Pattern pattern;

    public RegExEventInterceptor(String sName, EventInterceptor<E> interceptor,
                                 String cacheName, String serviceName, Interceptor.Order order, 
                                 RegistrationBehavior behavior)
    {
        super(sName, interceptor, cacheName, serviceName, order, behavior);
        pattern = Pattern.compile(cacheName);
    }

    @Override
    public boolean isAcceptable(EventDispatcher dispatcher)
    {
        String serviceName = getServiceName();
        if (serviceName == null && pattern == null)
        {
            return true;
        }

        String dispatcherServiceName = null;
        String dispatcherCacheName = null;
        if (dispatcher instanceof PartitionedCacheDispatcher)
        {
            PartitionedCacheDispatcher bmd = (PartitionedCacheDispatcher) dispatcher;
            dispatcherServiceName = bmd.getBackingMapContext()
                    .getManagerContext().getCacheService().getInfo().getServiceName();
            dispatcherCacheName = bmd.getBackingMapContext().getCacheName();
        }
        else if ((dispatcher instanceof PartitionedServiceDispatcher))
        {
            dispatcherServiceName = ((PartitionedServiceDispatcher) dispatcher).getService()
                    .getInfo().getServiceName();
        }

        boolean match = serviceName == null || serviceName.equals(dispatcherServiceName);
        if (match && dispatcherCacheName != null && pattern != null)
        {
            Matcher matcher = pattern.matcher(dispatcherCacheName);
            return matcher.matches();
        }

        return false;
    }
}

Again, another simple class where we just extend the default built in class NamedEventInterceptor and override the relevant methods. In this case we only need to override the isAcceptable(EventDispatcher dispatcher) which returns true if this interceptor should respond to events from the given dispatcher.

In our constructor we take the same parameters as the super class and pass them straight through. We then use the cacheName parameter as our regular expression which we compile into a java.util.regex.Pattern for use later in the isAcceptable method.

In the isAcceptable(EventDispatcher dispatcher) we are basically applying a few rules around the service name and cache name provided by the dispatcher then if the cache name from the dispatcher matches our regular expression we return true.

The RegExMappingNamespaceHandler Class

Now we have all of the parts we need to plug them into a NamespaceHandler so that we can use them in the cache configuration XML file. Our RegExMappingNamespaceHandler class is a very simple implementation of a NamespaceHandler.

  1. package com.thegridman.coherence.regex;
  2.  
  3. import com.tangosol.coherence.config.CacheMapping;
  4. import com.tangosol.config.ConfigurationException;
  5. import com.tangosol.config.xml.AbstractNamespaceHandler;
  6. import com.tangosol.config.xml.ElementProcessor;
  7. import com.tangosol.config.xml.ProcessingContext;
  8. import com.tangosol.config.xml.XmlSimpleName;
  9. import com.tangosol.run.xml.XmlElement;
  10.  
  11. /**
  12.  * @author Jonathan Knight
  13.  */
  14. public class RegExMappingNamespaceHandler extends AbstractNamespaceHandler
  15. {
  16.     public static final String XMLTAG_CACHE_MAPPING = "regex-cache-mapping";
  17.  
  18.     public RegExMappingNamespaceHandler()
  19.     {
  20.     }
  21.  
  22.     @XmlSimpleName(value = XMLTAG_CACHE_MAPPING)
  23.     public static class CacheMappingElementProcessor implements ElementProcessor<RegExCacheMapping>
  24.     {
  25.         @SuppressWarnings("unchecked")
  26.         @Override
  27.         public RegExCacheMapping process(ProcessingContext context, XmlElement element) throws ConfigurationException
  28.         {
  29.             String cacheNamePattern = context.getMandatoryProperty("cache-regex", String.class, element);
  30.             String schemeName = context.getMandatoryProperty("scheme-name", String.class, element);
  31.             RegExCacheMapping mapping = new RegExCacheMapping(cacheNamePattern, schemeName);
  32.  
  33.             context.inject(mapping, element);
  34.  
  35.             XmlElement copy = (XmlElement) element.clone();
  36.             copy.setName(element.getQualifiedName().getLocalName());
  37.             context.inject(mapping, copy);
  38.             context.addCookie(CacheMapping.class, mapping);
  39.             return mapping;
  40.         }
  41.     }
  42.  
  43. }
package com.thegridman.coherence.regex;

import com.tangosol.coherence.config.CacheMapping;
import com.tangosol.config.ConfigurationException;
import com.tangosol.config.xml.AbstractNamespaceHandler;
import com.tangosol.config.xml.ElementProcessor;
import com.tangosol.config.xml.ProcessingContext;
import com.tangosol.config.xml.XmlSimpleName;
import com.tangosol.run.xml.XmlElement;

/**
 * @author Jonathan Knight
 */
public class RegExMappingNamespaceHandler extends AbstractNamespaceHandler
{
    public static final String XMLTAG_CACHE_MAPPING = "regex-cache-mapping";

    public RegExMappingNamespaceHandler()
    {
    }

    @XmlSimpleName(value = XMLTAG_CACHE_MAPPING)
    public static class CacheMappingElementProcessor implements ElementProcessor<RegExCacheMapping>
    {
        @SuppressWarnings("unchecked")
        @Override
        public RegExCacheMapping process(ProcessingContext context, XmlElement element) throws ConfigurationException
        {
            String cacheNamePattern = context.getMandatoryProperty("cache-regex", String.class, element);
            String schemeName = context.getMandatoryProperty("scheme-name", String.class, element);
            RegExCacheMapping mapping = new RegExCacheMapping(cacheNamePattern, schemeName);

            context.inject(mapping, element);

            XmlElement copy = (XmlElement) element.clone();
            copy.setName(element.getQualifiedName().getLocalName());
            context.inject(mapping, copy);
            context.addCookie(CacheMapping.class, mapping);
            return mapping;
        }
    }

}

All that our RegExMappingNamespaceHandler class does is contain a static inner class CacheMappingElementProcessor that is the processor for the <regex-cache-mapping> element in our custom namespace. This class is annotated with @XmlSimpleName so Coherence will automatically register it for us.

Our CacheMappingElementProcessor inner class is also pretty simple, it just builds instances of our RegExCacheMapping class. First it reads the <cache-regex> and <scheme-name> elements from within the <regex-cache-mapping> element and passes the values to the constructor of the RegExCacheMapping. We then need to get the configuration framework to inject any other configuration elements that may be present, such as <init-params> or <interceptors>

Using Reg-Ex Cache Mapping In The XML Configuration

So now we have all the parts we can use it in our cache configuration file. Here is an example.

  1. <?xml version="1.0"?>
  2. <cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  3.               xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
  4.               xmlns:reg="class://com.thegridman.coherence.regex.RegExMappingNamespaceHandler"
  5.               xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd
  6.                                  class://com.thegridman.coherence.regex.RegExMappingNamespaceHandler regex-cache-config.xsd">
  7.  
  8.     <defaults>
  9.         <serializer>pof</serializer>
  10.     </defaults>
  11.  
  12.     <caching-scheme-mapping>
  13.  
  14.         <cache-mapping>
  15.             <cache-name>dist-jk</cache-name>
  16.             <scheme-name>distributed-scheme</scheme-name>
  17.             <interceptors>
  18.                 <interceptor>
  19.                     <instance>
  20.                         <class-name>com.thegridman.coherence.regex.MyInterceptor</class-name>
  21.                     </instance>
  22.                 </interceptor>
  23.             </interceptors>
  24.         </cache-mapping>
  25.  
  26.         <reg:regex-cache-mapping>
  27.             <reg:cache-regex>(.)*-test$</reg:cache-regex>
  28.             <reg:scheme-name>regex-one-scheme</reg:scheme-name>
  29.             <init-params>
  30.                 <init-param>
  31.                     <param-name>expiry</param-name>
  32.                     <param-value>2s</param-value>
  33.                 </init-param>
  34.             </init-params>
  35.         </reg:regex-cache-mapping>
  36.  
  37.         <reg:regex-cache-mapping>
  38.             <reg:cache-regex>^test-(.)*</reg:cache-regex>
  39.             <reg:scheme-name>regex-two-scheme</reg:scheme-name>
  40.         </reg:regex-cache-mapping>
  41.  
  42.         <reg:regex-cache-mapping>
  43.             <reg:cache-regex>(.)*-test-(.)*</reg:cache-regex>
  44.             <reg:scheme-name>regex-three-scheme</reg:scheme-name>
  45.             <init-params>
  46.                 <init-param>
  47.                     <param-name>expiry</param-name>
  48.                     <param-value>2s</param-value>
  49.                 </init-param>
  50.             </init-params>
  51.             <interceptors>
  52.                 <interceptor>
  53.                     <name>test-interceptor-3</name>
  54.                     <order>HIGH</order>
  55.                     <instance>
  56.                         <class-name>com.thegridman.coherence.regex.MyInterceptor</class-name>
  57.                     </instance>
  58.                 </interceptor>
  59.             </interceptors>
  60.         </reg:regex-cache-mapping>
  61.  
  62.     </caching-scheme-mapping>
  63.  
  64.     <caching-schemes>
  65.  
  66.         <distributed-scheme>
  67.             <scheme-name>regex-one-scheme</scheme-name>
  68.             <service-name>RegExOneDistributedService</service-name>
  69.             <backing-map-scheme>
  70.                 <local-scheme>
  71.                     <expiry-delay>{expiry 0}</expiry-delay>
  72.                 </local-scheme>
  73.             </backing-map-scheme>
  74.             <autostart>true</autostart>
  75.         </distributed-scheme>
  76.  
  77.         <distributed-scheme>
  78.             <scheme-name>regex-two-scheme</scheme-name>
  79.             <service-name>RegExTwoDistributedService</service-name>
  80.             <backing-map-scheme>
  81.                 <local-scheme>
  82.                     <expiry-delay>{expiry 0}</expiry-delay>
  83.                 </local-scheme>
  84.             </backing-map-scheme>
  85.             <autostart>true</autostart>
  86.         </distributed-scheme>
  87.  
  88.         <distributed-scheme>
  89.             <scheme-name>regex-three-scheme</scheme-name>
  90.             <service-name>RegExThreeDistributedService</service-name>
  91.             <backing-map-scheme>
  92.                 <local-scheme>
  93.                     <expiry-delay>{expiry 0}</expiry-delay>
  94.                 </local-scheme>
  95.             </backing-map-scheme>
  96.             <autostart>true</autostart>
  97.         </distributed-scheme>
  98.  
  99.         <distributed-scheme>
  100.             <scheme-name>distributed-scheme</scheme-name>
  101.             <service-name>DistributedService</service-name>
  102.             <backing-map-scheme>
  103.                 <local-scheme/>
  104.             </backing-map-scheme>
  105.             <autostart>true</autostart>
  106.         </distributed-scheme>
  107.  
  108.     </caching-schemes>
  109.  
  110. </cache-config>
<?xml version="1.0"?>
<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
               xmlns:reg="class://com.thegridman.coherence.regex.RegExMappingNamespaceHandler"
               xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd
                                  class://com.thegridman.coherence.regex.RegExMappingNamespaceHandler regex-cache-config.xsd">

    <defaults>
        <serializer>pof</serializer>
    </defaults>

    <caching-scheme-mapping>

        <cache-mapping>
            <cache-name>dist-jk</cache-name>
            <scheme-name>distributed-scheme</scheme-name>
            <interceptors>
                <interceptor>
                    <instance>
                        <class-name>com.thegridman.coherence.regex.MyInterceptor</class-name>
                    </instance>
                </interceptor>
            </interceptors>
        </cache-mapping>

        <reg:regex-cache-mapping>
            <reg:cache-regex>(.)*-test$</reg:cache-regex>
            <reg:scheme-name>regex-one-scheme</reg:scheme-name>
            <init-params>
                <init-param>
                    <param-name>expiry</param-name>
                    <param-value>2s</param-value>
                </init-param>
            </init-params>
        </reg:regex-cache-mapping>

        <reg:regex-cache-mapping>
            <reg:cache-regex>^test-(.)*</reg:cache-regex>
            <reg:scheme-name>regex-two-scheme</reg:scheme-name>
        </reg:regex-cache-mapping>

        <reg:regex-cache-mapping>
            <reg:cache-regex>(.)*-test-(.)*</reg:cache-regex>
            <reg:scheme-name>regex-three-scheme</reg:scheme-name>
            <init-params>
                <init-param>
                    <param-name>expiry</param-name>
                    <param-value>2s</param-value>
                </init-param>
            </init-params>
            <interceptors>
                <interceptor>
                    <name>test-interceptor-3</name>
                    <order>HIGH</order>
                    <instance>
                        <class-name>com.thegridman.coherence.regex.MyInterceptor</class-name>
                    </instance>
                </interceptor>
            </interceptors>
        </reg:regex-cache-mapping>

    </caching-scheme-mapping>

    <caching-schemes>

        <distributed-scheme>
            <scheme-name>regex-one-scheme</scheme-name>
            <service-name>RegExOneDistributedService</service-name>
            <backing-map-scheme>
                <local-scheme>
                    <expiry-delay>{expiry 0}</expiry-delay>
                </local-scheme>
            </backing-map-scheme>
            <autostart>true</autostart>
        </distributed-scheme>

        <distributed-scheme>
            <scheme-name>regex-two-scheme</scheme-name>
            <service-name>RegExTwoDistributedService</service-name>
            <backing-map-scheme>
                <local-scheme>
                    <expiry-delay>{expiry 0}</expiry-delay>
                </local-scheme>
            </backing-map-scheme>
            <autostart>true</autostart>
        </distributed-scheme>

        <distributed-scheme>
            <scheme-name>regex-three-scheme</scheme-name>
            <service-name>RegExThreeDistributedService</service-name>
            <backing-map-scheme>
                <local-scheme>
                    <expiry-delay>{expiry 0}</expiry-delay>
                </local-scheme>
            </backing-map-scheme>
            <autostart>true</autostart>
        </distributed-scheme>

        <distributed-scheme>
            <scheme-name>distributed-scheme</scheme-name>
            <service-name>DistributedService</service-name>
            <backing-map-scheme>
                <local-scheme/>
            </backing-map-scheme>
            <autostart>true</autostart>
        </distributed-scheme>

    </caching-schemes>

</cache-config>

We have declared our new namespace to use the reg: prefix mapped to namespace class://com.thegridman.coherence.regex.RegExMappingNamespaceHandler which is the name of our handler class.
In our example we can see a normal Coherence mapping on line 14 which maps to the exact cache name dist-jk.
On line 26 we have one of our new reg-ex mapping which maps to the expression (.)*-test$ – which is any cache name ending in -test. We specify the reg-ex in the <cache-regex> and scheme name in the <scheme-name> elements both prefixed with the namespace reg: prefix. You can see that this mapping also has init-params to pass through to the scheme, these are normal Coherence elements so do not need the reg: prefix.
On line 37 we have a second reg-ex mapping which uses the expression ^test-(.)* that maps to the regex-two-scheme scheme. This will match any cache name that starts with test- so is equivalent to the normal coherence test-* mapping.
On line 42 we have a third reg-ex mapping that uses the expression (.)*-test-(.)* to map caches to the regex-three-scheme scheme. This will match any cache name that contains -test- somewhere in it. This mapping also has init-params like the first one and also has interceptors specified.

Custom XSD File

The eagle eyed of you will have spotted that we have included a custom XSD file in the schemaLocation at the top of the configuration.

  1. <?xml version="1.0"?>
  2. <cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  3.               xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
  4.               xmlns:reg="class://com.thegridman.coherence.regex.RegExMappingNamespaceHandler"
  5.               xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd
  6.                                  class://com.thegridman.coherence.regex.RegExMappingNamespaceHandler regex-cache-config.xsd">
<?xml version="1.0"?>
<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
               xmlns:reg="class://com.thegridman.coherence.regex.RegExMappingNamespaceHandler"
               xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd
                                  class://com.thegridman.coherence.regex.RegExMappingNamespaceHandler regex-cache-config.xsd">

It is a good idea to create an XSD file to match you custom namespace as this allows the XML to be validated both at runtime and in the developers IDE and helps detect bugs earlier. The XSD for our new namespace looks like this

  1. <?xml version="1.0"?>
  2. <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
  3.             targetNamespace="class://com.thegridman.coherence.regex.RegExMappingNamespaceHandler"
  4.             xmlns="class://com.thegridman.coherence.regex.RegExMappingNamespaceHandler"
  5.             xmlns:coh="http://xmlns.oracle.com/coherence/coherence-cache-config"
  6.             elementFormDefault="qualified"
  7.             attributeFormDefault="unqualified"
  8.             version="1.2">
  9.  
  10.     <xsd:import namespace="http://xmlns.oracle.com/coherence/coherence-cache-config"
  11.                 schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd"/>
  12.  
  13.  
  14.     <xsd:element name="regex-cache-mapping" type="regex-cache-mapping-type"/>
  15.  
  16.  
  17.     <xsd:complexType name="regex-cache-mapping-type">
  18.         <xsd:sequence>
  19.             <xsd:element ref="cache-regex" />
  20.             <xsd:element ref="scheme-name" />
  21.             <xsd:element ref="coh:init-params" minOccurs="0" />
  22.             <xsd:element ref="coh:interceptors" minOccurs="0" />
  23.         </xsd:sequence>
  24.         <xsd:anyAttribute namespace="##other" processContents="lax"/>
  25.     </xsd:complexType>
  26.  
  27.     <xsd:element name="cache-regex" type="coh:coherence-string-type"/>
  28.  
  29.     <xsd:element name="scheme-name" type="coh:coherence-string-type"/>
  30.  
  31. </xsd:schema>
<?xml version="1.0"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
             targetNamespace="class://com.thegridman.coherence.regex.RegExMappingNamespaceHandler"
             xmlns="class://com.thegridman.coherence.regex.RegExMappingNamespaceHandler"
             xmlns:coh="http://xmlns.oracle.com/coherence/coherence-cache-config"
             elementFormDefault="qualified"
             attributeFormDefault="unqualified"
             version="1.2">

    <xsd:import namespace="http://xmlns.oracle.com/coherence/coherence-cache-config"
                 schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd"/>


    <xsd:element name="regex-cache-mapping" type="regex-cache-mapping-type"/>


    <xsd:complexType name="regex-cache-mapping-type">
        <xsd:sequence>
            <xsd:element ref="cache-regex" />
            <xsd:element ref="scheme-name" />
            <xsd:element ref="coh:init-params" minOccurs="0" />
            <xsd:element ref="coh:interceptors" minOccurs="0" />
        </xsd:sequence>
        <xsd:anyAttribute namespace="##other" processContents="lax"/>
    </xsd:complexType>

    <xsd:element name="cache-regex" type="coh:coherence-string-type"/>

    <xsd:element name="scheme-name" type="coh:coherence-string-type"/>

</xsd:schema>

So if you are using Coherence 12.1.2 that is all you need to do to use regular expression cache mappings. It is only a handful of classes and makes use of the correct way to extend Coherence functionality.

Coherence 3.7.1 Implementation

If you have not upgraded to 12.1.2 yet – what are you waiting for!!! Seriously though, I know a lot of projects have not upgraded. At my current client we are still on 3.7.1 and having a big debate about when to go to 12.1.2 and what it will involve.

So, getting regular expression mapping to work in 3.7.1 (and also earlier versions) is also very straight forward as we only need a single class but it is not as elegant as the 12.1.2 solution.

We will make the 3.7.1 version work with exactly the same XML configuration elements and a custom namespace just as the 12.1.2 version. In this case though we will not have all of the custom namespace code, just an XSD file.

Custom ConfigurableCacheFactory Class

In Coherence 3.7.1 and earlier there is no clever configuration framework like there is in 12.1.2. Basically the ConfigurableCacheFactory implementation just uses the cache configuration XML directly to work out what to do. This means all we need to do is create a custom ConfigurableCacheFactory by extending DefaultConfigurableCacheFactory and override two methods.

  1. package com.thegridman.coherence.regex;
  2.  
  3. import com.tangosol.net.DefaultConfigurableCacheFactory;
  4. import com.tangosol.run.xml.XmlElement;
  5. import com.tangosol.run.xml.XmlHelper;
  6.  
  7. import java.util.HashMap;
  8. import java.util.Iterator;
  9. import java.util.LinkedHashMap;
  10. import java.util.Map;
  11. import java.util.regex.Matcher;
  12. import java.util.regex.Pattern;
  13.  
  14. /**
  15.  * @author Jonathan Knight
  16.  */
  17. public class RegExConfigurableCacheFactory extends DefaultConfigurableCacheFactory
  18. {
  19.     public static final String NAMESPACE_URI = "http://xmlns.oracle.com/coherence/regex-coherence-cache-config";
  20.  
  21.     private Map<XmlElement,Pattern> cacheMappingPatterns;
  22.     private Map<String,XmlElement> exactCacheMappings;
  23.     private String namespacePrefix;
  24.  
  25.     public RegExConfigurableCacheFactory()
  26.     {
  27.     }
  28.  
  29.     public RegExConfigurableCacheFactory(String sPath)
  30.     {
  31.         super(sPath);
  32.     }
  33.  
  34.     public RegExConfigurableCacheFactory(String sPath, ClassLoader loader)
  35.     {
  36.         super(sPath, loader);
  37.     }
  38.  
  39.     public RegExConfigurableCacheFactory(XmlElement xmlConfig)
  40.     {
  41.         super(xmlConfig);
  42.     }
  43.  
  44.     public RegExConfigurableCacheFactory(XmlElement xmlConfig, ClassLoader loader)
  45.     {
  46.         super(xmlConfig, loader);
  47.     }
  48.  
  49.     @SuppressWarnings("unchecked")
  50.     @Override
  51.     public void setConfig(XmlElement xmlConfig)
  52.     {
  53.         super.setConfig(xmlConfig);
  54.         namespacePrefix = XmlHelper.getNamespacePrefix(xmlConfig, NAMESPACE_URI);
  55.         cacheMappingPatterns = new LinkedHashMap<>();
  56.         Iterator<XmlElement> regExMappingIterator =
  57.                 xmlConfig.getSafeElement("caching-scheme-mapping")
  58.                         .getElements(namespacePrefix + ":regex-cache-mapping");
  59.  
  60.         while (regExMappingIterator.hasNext())
  61.         {
  62.             XmlElement mappingElement = regExMappingIterator.next();
  63.             String regEx = mappingElement.getSafeElement(namespacePrefix + ":cache-regex").getString();
  64.             Pattern pattern = Pattern.compile(regEx);
  65.             cacheMappingPatterns.put(mappingElement, pattern);
  66.         }
  67.  
  68.         exactCacheMappings = new LinkedHashMap<>();
  69.         Iterator<XmlElement> mappingIterator =
  70.                 xmlConfig.getSafeElement("caching-scheme-mapping")
  71.                     .getElements("cache-mapping");
  72.  
  73.         while (mappingIterator.hasNext())
  74.         {
  75.             XmlElement mappingElement = mappingIterator.next();
  76.             String cacheName = mappingElement.getSafeElement("cache-name").getString();
  77.             exactCacheMappings.put(cacheName, mappingElement);
  78.         }
  79.     }
  80.  
  81.     @SuppressWarnings("unchecked")
  82.     @Override
  83.     public CacheInfo findSchemeMapping(String cacheName)
  84.     {
  85.         XmlElement match = null;
  86.         String schemeName = null;
  87.         if (exactCacheMappings.containsKey(cacheName))
  88.         {
  89.             match = exactCacheMappings.get(cacheName);
  90.             schemeName = match.getSafeElement("scheme-name").getString();
  91.         }
  92.         else
  93.         {
  94.             for (Map.Entry<XmlElement,Pattern> mappingEntry : cacheMappingPatterns.entrySet())
  95.             {
  96.                 Matcher matcher = mappingEntry.getValue().matcher(cacheName);
  97.                 if (matcher.matches())
  98.                 {
  99.                     match = mappingEntry.getKey();
  100.                     schemeName = match.getSafeElement(namespacePrefix + ":scheme-name").getString();
  101.                 }
  102.             }
  103.         }
  104.  
  105.         if (match == null)
  106.         {
  107.             throw new IllegalArgumentException("No scheme for cache: \"" + cacheName + '"');
  108.         }
  109.  
  110.         Map initParamMap = new HashMap();
  111.         Iterator<XmlElement> initParamIterator = match.getSafeElement("init-params").getElements("init-param");
  112.         while(initParamIterator.hasNext())
  113.         {
  114.             XmlElement initParamElement = initParamIterator.next();
  115.             String paramName = initParamElement.getSafeElement("param-name").getString();
  116.             if (paramName.length() > 0)
  117.             {
  118.                 String paramValue = initParamElement.getSafeElement("param-value").getString();
  119.                 initParamMap.put(paramName, paramValue);
  120.             }
  121.         }
  122.  
  123.         return new CacheInfo(cacheName, schemeName, initParamMap);
  124.     }
  125.  
  126. }
package com.thegridman.coherence.regex;

import com.tangosol.net.DefaultConfigurableCacheFactory;
import com.tangosol.run.xml.XmlElement;
import com.tangosol.run.xml.XmlHelper;

import java.util.HashMap;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

/**
 * @author Jonathan Knight
 */
public class RegExConfigurableCacheFactory extends DefaultConfigurableCacheFactory
{
    public static final String NAMESPACE_URI = "http://xmlns.oracle.com/coherence/regex-coherence-cache-config";

    private Map<XmlElement,Pattern> cacheMappingPatterns;
    private Map<String,XmlElement> exactCacheMappings;
    private String namespacePrefix;

    public RegExConfigurableCacheFactory()
    {
    }

    public RegExConfigurableCacheFactory(String sPath)
    {
        super(sPath);
    }

    public RegExConfigurableCacheFactory(String sPath, ClassLoader loader)
    {
        super(sPath, loader);
    }

    public RegExConfigurableCacheFactory(XmlElement xmlConfig)
    {
        super(xmlConfig);
    }

    public RegExConfigurableCacheFactory(XmlElement xmlConfig, ClassLoader loader)
    {
        super(xmlConfig, loader);
    }

    @SuppressWarnings("unchecked")
    @Override
    public void setConfig(XmlElement xmlConfig)
    {
        super.setConfig(xmlConfig);
        namespacePrefix = XmlHelper.getNamespacePrefix(xmlConfig, NAMESPACE_URI);
        cacheMappingPatterns = new LinkedHashMap<>();
        Iterator<XmlElement> regExMappingIterator =
                xmlConfig.getSafeElement("caching-scheme-mapping")
                        .getElements(namespacePrefix + ":regex-cache-mapping");

        while (regExMappingIterator.hasNext())
        {
            XmlElement mappingElement = regExMappingIterator.next();
            String regEx = mappingElement.getSafeElement(namespacePrefix + ":cache-regex").getString();
            Pattern pattern = Pattern.compile(regEx);
            cacheMappingPatterns.put(mappingElement, pattern);
        }

        exactCacheMappings = new LinkedHashMap<>();
        Iterator<XmlElement> mappingIterator =
                xmlConfig.getSafeElement("caching-scheme-mapping")
                    .getElements("cache-mapping");

        while (mappingIterator.hasNext())
        {
            XmlElement mappingElement = mappingIterator.next();
            String cacheName = mappingElement.getSafeElement("cache-name").getString();
            exactCacheMappings.put(cacheName, mappingElement);
        }
    }

    @SuppressWarnings("unchecked")
    @Override
    public CacheInfo findSchemeMapping(String cacheName)
    {
        XmlElement match = null;
        String schemeName = null;
        if (exactCacheMappings.containsKey(cacheName))
        {
            match = exactCacheMappings.get(cacheName);
            schemeName = match.getSafeElement("scheme-name").getString();
        }
        else
        {
            for (Map.Entry<XmlElement,Pattern> mappingEntry : cacheMappingPatterns.entrySet())
            {
                Matcher matcher = mappingEntry.getValue().matcher(cacheName);
                if (matcher.matches())
                {
                    match = mappingEntry.getKey();
                    schemeName = match.getSafeElement(namespacePrefix + ":scheme-name").getString();
                }
            }
        }

        if (match == null)
        {
            throw new IllegalArgumentException("No scheme for cache: \"" + cacheName + '"');
        }

        Map initParamMap = new HashMap();
        Iterator<XmlElement> initParamIterator = match.getSafeElement("init-params").getElements("init-param");
        while(initParamIterator.hasNext())
        {
            XmlElement initParamElement = initParamIterator.next();
            String paramName = initParamElement.getSafeElement("param-name").getString();
            if (paramName.length() > 0)
            {
                String paramValue = initParamElement.getSafeElement("param-value").getString();
                initParamMap.put(paramName, paramValue);
            }
        }

        return new CacheInfo(cacheName, schemeName, initParamMap);
    }

}

That is the code for the whole class. We have written the same constructors as DefaultConfigurableCacheFactory and overridden the setConfig(XmlElement xmlConfig) and findSchemeMapping(String cacheName) methods.

Our setConfig(XmlElement xmlConfig) method first calls the super.setConfig(XmlElement xmlConfig) method and then needs to go through the mappings defined in the configuration. As I said already, there is no clever configuration framework, we just need to look at the XML. First we create a Map of our reg-ex mappings in the cacheMappingPatterns field. The key of the map is the mapping XMLElement and the value of the map is the compiled reg-ex Pattern.

Next the setConfig(XmlElement xmlConfig) method will create a Map of the normal Coherence cache mappings stored in the exactCacheMappings field. This time the key of the map is the value of the <cache-name> element and the value is the mapping XMLElement.

The findSchemeMapping(String cacheName) is where we have to do the work to find the scheme mapping. First we check to see if there is an exact match to the specified cache name and if there is we use that XMLElement for the mapping. If there is no exact match we go through the reg-ex mappings to see if any of them match and if so use that XMLElement for the mapping. If neither match we throw an exception just as normal Coherence would.

If we find a match we create a CacheInfo and include any <init-params> in the mapping element. The CacheInfo is then returned from the method.

Override File

To use our custom ConfigurableCacheFactory class we need to tell Coherence about it. This is usually done by adding it to the tangosol-coherence-override.xml file like this.

  1. <coherence>
  2.     <configurable-cache-factory-config>
  3.         <class-name>com.thegridman.coherence.regex.RegExConfigurableCacheFactory</class-name>
  4.         <init-params>
  5.             <init-param>
  6.                 <param-type>java.lang.String</param-type>
  7.                 <param-value system-property="tangosol.coherence.cacheconfig">coherence-cache-config.xml</param-value>
  8.             </init-param>
  9.         </init-params>
  10.     </configurable-cache-factory-config>
  11. </coherence>
<coherence>
    <configurable-cache-factory-config>
        <class-name>com.thegridman.coherence.regex.RegExConfigurableCacheFactory</class-name>
        <init-params>
            <init-param>
                <param-type>java.lang.String</param-type>
                <param-value system-property="tangosol.coherence.cacheconfig">coherence-cache-config.xml</param-value>
            </init-param>
        </init-params>
    </configurable-cache-factory-config>
</coherence>

There are other ways of using a custom ConfigurableCacheFactory such as a custom builder or just new one up, but changing the tangosol-coherence-override.xml file is probably most common.

3.7.1 Custom XSD

We need to specify a custom XSD namespace declared in the cache configuration file to make this code work. The XSD file looks similar to the 12.1.2 version.

  1. <?xml version="1.0"?>
  2. <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
  3.             targetNamespace="http://xmlns.oracle.com/coherence/regex-coherence-cache-config"
  4.             xmlns="http://xmlns.oracle.com/coherence/regex-coherence-cache-config"
  5.             xmlns:coh="http://xmlns.oracle.com/coherence/coherence-cache-config"
  6.             elementFormDefault="qualified"
  7.             attributeFormDefault="unqualified"
  8.             version="1.2">
  9.  
  10.     <xsd:import namespace="http://xmlns.oracle.com/coherence/coherence-cache-config"
  11.                 schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd"/>
  12.  
  13.     <xsd:element name="regex-cache-mapping" type="regex-cache-mapping-type"/>
  14.  
  15.     <xsd:complexType name="regex-cache-mapping-type">
  16.         <xsd:sequence>
  17.             <xsd:element ref="cache-regex" />
  18.             <xsd:element ref="scheme-name" />
  19.             <xsd:element ref="coh:init-params" minOccurs="0" />
  20.         </xsd:sequence>
  21.         <xsd:anyAttribute namespace="##other" processContents="lax"/>
  22.     </xsd:complexType>
  23.  
  24.     <xsd:element name="cache-regex" type="coh:coherence-string-type"/>
  25.  
  26.     <xsd:element name="scheme-name" type="coh:coherence-string-type"/>
  27.  
  28. </xsd:schema>
<?xml version="1.0"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
             targetNamespace="http://xmlns.oracle.com/coherence/regex-coherence-cache-config"
             xmlns="http://xmlns.oracle.com/coherence/regex-coherence-cache-config"
             xmlns:coh="http://xmlns.oracle.com/coherence/coherence-cache-config"
             elementFormDefault="qualified"
             attributeFormDefault="unqualified"
             version="1.2">

    <xsd:import namespace="http://xmlns.oracle.com/coherence/coherence-cache-config"
                 schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd"/>

    <xsd:element name="regex-cache-mapping" type="regex-cache-mapping-type"/>

    <xsd:complexType name="regex-cache-mapping-type">
        <xsd:sequence>
            <xsd:element ref="cache-regex" />
            <xsd:element ref="scheme-name" />
            <xsd:element ref="coh:init-params" minOccurs="0" />
        </xsd:sequence>
        <xsd:anyAttribute namespace="##other" processContents="lax"/>
    </xsd:complexType>

    <xsd:element name="cache-regex" type="coh:coherence-string-type"/>

    <xsd:element name="scheme-name" type="coh:coherence-string-type"/>

</xsd:schema>

Using Reg-Ex Cache Mapping In The XML Configuration

Using the reg-ex cache mapping in a 3.7.1 cache configuration file is just like it was in 12.1.2 but with a slightly different namespace declaration at the top.

  1. <?xml version="1.0"?>
  2. <cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  3.               xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
  4.               xmlns:reg="http://xmlns.oracle.com/coherence/regex-coherence-cache-config"
  5.               xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd
  6.                                  http://xmlns.oracle.com/coherence/regex-coherence-cache-config regex-cache-config.xsd">
  7.  
  8.     <defaults>
  9.         <serializer>pof</serializer>
  10.     </defaults>
  11.  
  12.     <caching-scheme-mapping>
  13.  
  14.         <cache-mapping>
  15.             <cache-name>dist-test</cache-name>
  16.             <scheme-name>my-distributed-scheme</scheme-name>
  17.         </cache-mapping>
  18.  
  19.         <reg:regex-cache-mapping>
  20.             <reg:cache-regex>dist-(.)*</reg:cache-regex>
  21.             <reg:scheme-name>regex-scheme-one</reg:scheme-name>
  22.             <init-params>
  23.                 <init-param>
  24.                     <param-name>expiry</param-name>
  25.                     <param-value>1d</param-value>
  26.                 </init-param>
  27.                 <init-param>
  28.                     <param-name>test-param</param-name>
  29.                     <param-value>test-value</param-value>
  30.                 </init-param>
  31.             </init-params>
  32.         </reg:regex-cache-mapping>
  33.  
  34.         <reg:regex-cache-mapping>
  35.             <reg:cache-regex>(.)*-test$</reg:cache-regex>
  36.             <reg:scheme-name>regex-scheme-two</reg:scheme-name>
  37.         </reg:regex-cache-mapping>
  38.  
  39.         <reg:regex-cache-mapping>
  40.             <reg:cache-regex>dist-(.)*-test$</reg:cache-regex>
  41.             <reg:scheme-name>regex-scheme-three</reg:scheme-name>
  42.         </reg:regex-cache-mapping>
  43.  
  44.     </caching-scheme-mapping>
  45.  
  46.     <caching-schemes>
  47.  
  48.         <distributed-scheme>
  49.             <scheme-name>my-distributed-scheme</scheme-name>
  50.             <service-name>DistributedService</service-name>
  51.             <backing-map-scheme>
  52.                 <local-scheme/>
  53.             </backing-map-scheme>
  54.             <autostart>true</autostart>
  55.         </distributed-scheme>
  56.  
  57.         <distributed-scheme>
  58.             <scheme-name>regex-scheme-one</scheme-name>
  59.             <service-name>DistributedRegExOneService</service-name>
  60.             <backing-map-scheme>
  61.                 <local-scheme>
  62.                     <expiry-delay>{expiry 0}</expiry-delay>
  63.                 </local-scheme>
  64.             </backing-map-scheme>
  65.             <autostart>true</autostart>
  66.         </distributed-scheme>
  67.  
  68.         <distributed-scheme>
  69.             <scheme-name>regex-scheme-two</scheme-name>
  70.             <service-name>DistributedRegExTwoService</service-name>
  71.             <backing-map-scheme>
  72.                 <local-scheme/>
  73.             </backing-map-scheme>
  74.             <autostart>true</autostart>
  75.         </distributed-scheme>
  76.  
  77.         <distributed-scheme>
  78.             <scheme-name>regex-scheme-three</scheme-name>
  79.             <service-name>DistributedRegExThreeService</service-name>
  80.             <backing-map-scheme>
  81.                 <local-scheme/>
  82.             </backing-map-scheme>
  83.             <autostart>true</autostart>
  84.         </distributed-scheme>
  85.  
  86.     </caching-schemes>
  87.  
  88. </cache-config>
<?xml version="1.0"?>
<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
               xmlns:reg="http://xmlns.oracle.com/coherence/regex-coherence-cache-config"
               xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd
                                  http://xmlns.oracle.com/coherence/regex-coherence-cache-config regex-cache-config.xsd">

    <defaults>
        <serializer>pof</serializer>
    </defaults>

    <caching-scheme-mapping>

        <cache-mapping>
            <cache-name>dist-test</cache-name>
            <scheme-name>my-distributed-scheme</scheme-name>
        </cache-mapping>

        <reg:regex-cache-mapping>
            <reg:cache-regex>dist-(.)*</reg:cache-regex>
            <reg:scheme-name>regex-scheme-one</reg:scheme-name>
            <init-params>
                <init-param>
                    <param-name>expiry</param-name>
                    <param-value>1d</param-value>
                </init-param>
                <init-param>
                    <param-name>test-param</param-name>
                    <param-value>test-value</param-value>
                </init-param>
            </init-params>
        </reg:regex-cache-mapping>

        <reg:regex-cache-mapping>
            <reg:cache-regex>(.)*-test$</reg:cache-regex>
            <reg:scheme-name>regex-scheme-two</reg:scheme-name>
        </reg:regex-cache-mapping>

        <reg:regex-cache-mapping>
            <reg:cache-regex>dist-(.)*-test$</reg:cache-regex>
            <reg:scheme-name>regex-scheme-three</reg:scheme-name>
        </reg:regex-cache-mapping>

    </caching-scheme-mapping>

    <caching-schemes>

        <distributed-scheme>
            <scheme-name>my-distributed-scheme</scheme-name>
            <service-name>DistributedService</service-name>
            <backing-map-scheme>
                <local-scheme/>
            </backing-map-scheme>
            <autostart>true</autostart>
        </distributed-scheme>

        <distributed-scheme>
            <scheme-name>regex-scheme-one</scheme-name>
            <service-name>DistributedRegExOneService</service-name>
            <backing-map-scheme>
                <local-scheme>
                    <expiry-delay>{expiry 0}</expiry-delay>
                </local-scheme>
            </backing-map-scheme>
            <autostart>true</autostart>
        </distributed-scheme>

        <distributed-scheme>
            <scheme-name>regex-scheme-two</scheme-name>
            <service-name>DistributedRegExTwoService</service-name>
            <backing-map-scheme>
                <local-scheme/>
            </backing-map-scheme>
            <autostart>true</autostart>
        </distributed-scheme>

        <distributed-scheme>
            <scheme-name>regex-scheme-three</scheme-name>
            <service-name>DistributedRegExThreeService</service-name>
            <backing-map-scheme>
                <local-scheme/>
            </backing-map-scheme>
            <autostart>true</autostart>
        </distributed-scheme>

    </caching-schemes>

</cache-config>

You can see on lines 4 and 6 that the namespace declaration uses a proper http: URI rather than the class: prefix used in 12.1.2, other than that it is identical to use.

Caveats and Gotchas

There are a few things to bear in mind when using the two implementations above regardless of the Coherence version.

  • It is best not to mix reg-ex mappings with normal Coherence wild card mapping that end with an asterisk. In the 3.7.1 version Coherence wild cards are not supported anyway. It is easy enough to do the same thing as an asterisk in a regular expression anyway, just put (.)* at then end of the reg-ex, for example a Coherence mapping of replicated-* would be replicated-(.)* in reg-ex mapping.
  • If the reg-ex expressions get complicated it could be difficult to work out which might apply if a developer is reading the cache configuration file and it would also be hard to work out which reg-ex might be the more specific mapping when working out what order to put the mappings into the file.

Challenge Met!

There we go then, the challenge of implementing regular expression cache mapping has been met.
The code for this is all on GitHub. I have decided to create a repo for all my blog code here https://github.com/thegridman/Coherence-Blog-Stuff

– enjoy.

Coherence 12.1.2 – You Don’t Need That Cache Config XML File

No XML

As I alluded to in my previous blog on Coherence 12.1.2 Custom Namespaces and Oracle NoSQL, you can now run Coherence without requiring a cache configuration XML file. Just for some fun, this post is going to show a very quick demo of how to do this.

Coherence 12.1.2 has had a major refactoring of how the configuration files are processed. The new code uses a combination of classes that can handle XML (if you are familiar with the namespaces code from the older Coherence Incubator then this will look very similar), classes that know how to configure stuff called schemes and classes that build stuff, called builders. All of these work together to build a cache configuration and hence a cache factory. What this means though is that we can now bypass all the XML processing code and use the scheme and builder classes directly to build a cache factory. Once we have a cache factory, we could use it in client code, or we could pass it to a DefaultCacheServer to run a normal server cluster member.

The Cache Configuration File

For this demo I am going to run a cluster that would have the following cache configuration XML – but actually we will have no XML.

  1. <?xml version="1.0"?>
  2. <cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  3.               xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
  4.               xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd">
  5.  
  6.     <defaults>
  7.         <serializer>pof</serializer>
  8.     </defaults>
  9.  
  10.     <caching-scheme-mapping>
  11.  
  12.         <cache-mapping>
  13.             <cache-name>dist-*</cache-name>
  14.             <scheme-name>my-distributed-scheme</scheme-name>
  15.         </cache-mapping>
  16.  
  17.     </caching-scheme-mapping>
  18.  
  19.     <caching-schemes>
  20.  
  21.         <distributed-scheme>
  22.             <scheme-name>my-distributed-scheme</scheme-name>
  23.             <service-name>MyDistributedService</service-name>
  24.             <backing-map-scheme>
  25.               <read-write-backing-map-scheme>
  26.                 <internal-cache-scheme>
  27.                   <local-scheme>
  28.                       <expiry-delay>10s</expiry-delay>
  29.                   </local-scheme>
  30.                 </internal-cache-scheme>
  31.                 <cachestore-scheme>
  32.                   <class-scheme>
  33.                     <class-name>com.thegridman.coherence.noxml.NoXmlDemo$MyCacheStore</class-name>
  34.                       <init-params>
  35.                           <init-param>
  36.                               <param-type>java.lang.String</param-type>
  37.                               <param-value>{cache-name}</param-value>
  38.                           </init-param>
  39.                       </init-params>
  40.                   </class-scheme>
  41.                 </cachestore-scheme>
  42.               </read-write-backing-map-scheme>
  43.             </backing-map-scheme>
  44.         </distributed-scheme>
  45.  
  46.         <proxy-scheme>
  47.           <scheme-name>my-proxy-scheme</scheme-name>
  48.           <service-name>TcpProxyService</service-name>
  49.           <acceptor-config>
  50.             <tcp-acceptor>
  51.               <local-address>
  52.                 <address>localhost</address>
  53.                 <port>9099</port>
  54.               </local-address>
  55.             </tcp-acceptor>
  56.           </acceptor-config>
  57.           <autostart>true</autostart>
  58.         </proxy-scheme>
  59.  
  60.     </caching-schemes>
  61.  
  62. </cache-config>
<?xml version="1.0"?>
<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
               xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd">

    <defaults>
        <serializer>pof</serializer>
    </defaults>

    <caching-scheme-mapping>

        <cache-mapping>
            <cache-name>dist-*</cache-name>
            <scheme-name>my-distributed-scheme</scheme-name>
        </cache-mapping>

    </caching-scheme-mapping>

    <caching-schemes>

        <distributed-scheme>
            <scheme-name>my-distributed-scheme</scheme-name>
            <service-name>MyDistributedService</service-name>
            <backing-map-scheme>
              <read-write-backing-map-scheme>
                <internal-cache-scheme>
                  <local-scheme>
                      <expiry-delay>10s</expiry-delay>
                  </local-scheme>
                </internal-cache-scheme>
                <cachestore-scheme>
                  <class-scheme>
                    <class-name>com.thegridman.coherence.noxml.NoXmlDemo$MyCacheStore</class-name>
                      <init-params>
                          <init-param>
                              <param-type>java.lang.String</param-type>
                              <param-value>{cache-name}</param-value>
                          </init-param>
                      </init-params>
                  </class-scheme>
                </cachestore-scheme>
              </read-write-backing-map-scheme>
            </backing-map-scheme>
        </distributed-scheme>

        <proxy-scheme>
          <scheme-name>my-proxy-scheme</scheme-name>
          <service-name>TcpProxyService</service-name>
          <acceptor-config>
            <tcp-acceptor>
              <local-address>
                <address>localhost</address>
                <port>9099</port>
              </local-address>
            </tcp-acceptor>
          </acceptor-config>
          <autostart>true</autostart>
        </proxy-scheme>

    </caching-schemes>

</cache-config>

The configuration above is very simple, but this is just a demo, you can get as complicated as you like if you want to try this yourself.

Doing It All In Code…

To make it easier to understand we will start bottom up by creating the service configurations first.

The local-scheme

The first thing we need is the inner local-scheme for the distributed-scheme. If you look at the configuration above, all we have added to this local scheme is an expiry-delay of 10 seconds. Here is the code…

  1. LocalScheme localScheme = new LocalScheme();
  2. localScheme.setExpiryDelay(new LiteralExpression<Seconds>(new Seconds(10)));
LocalScheme localScheme = new LocalScheme();
localScheme.setExpiryDelay(new LiteralExpression<Seconds>(new Seconds(10)));

It couldn’t really be much simpler. If you look at the com.tangosol.coherence.config.scheme.LocalScheme class you will see that it has various methods on it for setting the different things you would normally add in the configuration file.

The cachestore-scheme

Next you will see that our distributed-scheme contains a read-write-backing-map that contains a CacheStore. The cache store class is called com.thegridman.coherence.noxml.MyCacheStore which is a simple cache store that does nothing but some logging for the purposes of our demo.The class could be any cache store at all, you just need to create an instance of it. Here is the code for the cache store…

  1. package com.thegridman.coherence.noxml;
  2.  
  3. import com.tangosol.net.cache.CacheStore;
  4.  
  5. import java.util.Collection;
  6. import java.util.Map;
  7.  
  8. /**
  9.  * @author Jonathan Knight
  10.  */
  11. public class MyCacheStore implements CacheStore
  12. {
  13.     private String cacheName;
  14.  
  15.     public MyCacheStore(String cacheName)
  16.     {
  17.         this.cacheName = cacheName;
  18.     }
  19.  
  20.     public void store(Object key, Object value)
  21.     {
  22.         System.err.println(cacheName + " store called key=" + key + " value=" + value);
  23.     }
  24.  
  25.     public void storeAll(Map entries)
  26.     {
  27.         System.err.println(cacheName + " storeAll called entries=" + entries);
  28.     }
  29.  
  30.     public Object load(Object key)
  31.     {
  32.         System.err.println(cacheName + " load called key=" + key);
  33.         return null;
  34.     }
  35.  
  36.     public Map loadAll(Collection keys)
  37.     {
  38.         System.err.println(cacheName + " loadAll called keys=" + keys);
  39.         return null;
  40.     }
  41.  
  42.     public void erase(Object key)
  43.     {
  44.         System.err.println(cacheName + " erase called key=" + key);
  45.     }
  46.  
  47.     public void eraseAll(Collection keys)
  48.     {
  49.         System.err.println(cacheName + " eraseAll called keys=" + keys);
  50.     }
  51. }
package com.thegridman.coherence.noxml;

import com.tangosol.net.cache.CacheStore;

import java.util.Collection;
import java.util.Map;

/**
 * @author Jonathan Knight
 */
public class MyCacheStore implements CacheStore
{
    private String cacheName;

    public MyCacheStore(String cacheName)
    {
        this.cacheName = cacheName;
    }

    public void store(Object key, Object value)
    {
        System.err.println(cacheName + " store called key=" + key + " value=" + value);
    }

    public void storeAll(Map entries)
    {
        System.err.println(cacheName + " storeAll called entries=" + entries);
    }

    public Object load(Object key)
    {
        System.err.println(cacheName + " load called key=" + key);
        return null;
    }

    public Map loadAll(Collection keys)
    {
        System.err.println(cacheName + " loadAll called keys=" + keys);
        return null;
    }

    public void erase(Object key)
    {
        System.err.println(cacheName + " erase called key=" + key);
    }

    public void eraseAll(Collection keys)
    {
        System.err.println(cacheName + " eraseAll called keys=" + keys);
    }
}

You can see from the sample XML above and from the cache store code that our cache store requires a constructor parameter, which is actually a Coherence configuration macro, in this case {cache-name} which will resolve to the name of the cache that the cache store is being created for.

So, how do we create all of that, we will start with the init-params part. Parameters are create as a ParameterList and each parameter is added to the list like this…

  1. ParameterList parameterList = new ResolvableParameterList();
  2. Expression<?> expression = new ParameterMacroExpression<>("{cache-name}", String.class);
  3. parameterList.add(new Parameter("cache-name", String.class, expression));
ParameterList parameterList = new ResolvableParameterList();
Expression<?> expression = new ParameterMacroExpression<>("{cache-name}", String.class);
parameterList.add(new Parameter("cache-name", String.class, expression));

First we create the ParameterList. Next we create the parameter, in this case it is a macro so we use a ParameterMacroExpression class whose constructor takes the macro text and Class type as parameters. Finally we add the expression to the ParameterList.

There are various other types of Expression class we could use, depending on what we wanted to do, any decent IDE should show you the classes that implement com.tangosol.config.expression.Expression and you can deduce what they do and how you could use them, for example, com.tangosol.config.expression.LiteralExpression is fairly obviously for literal values.

The read-write-backing-map-scheme

Now we have the local-scheme and cachestore-scheme parts we can use them to create the read-write-backing-map-scheme. You can probably see a pattern forming now, and yes indeed there is a scheme called com.tangosol.coherence.config.scheme.ReadWriteBackingMapScheme that will hold the configuration of our read-write-backing-map-scheme.

  1. ReadWriteBackingMapScheme readWriteBackingMapScheme = new ReadWriteBackingMapScheme();
  2. readWriteBackingMapScheme.setCacheStoreScheme(cacheStoreScheme);
  3. readWriteBackingMapScheme.setInternalScheme(localScheme);
ReadWriteBackingMapScheme readWriteBackingMapScheme = new ReadWriteBackingMapScheme();
readWriteBackingMapScheme.setCacheStoreScheme(cacheStoreScheme);
readWriteBackingMapScheme.setInternalScheme(localScheme);

Our read-write-backing-map-scheme is very simple, we just add the LocalScheme and CacheStoreScheme that we have already created.

The distributed-scheme

Finally we will create the distributed-scheme and yet again we use the relevant scheme class.

  1. DistributedScheme distributedScheme = new DistributedScheme();
  2. distributedScheme.setSchemeName("my-distributed-scheme");
  3. distributedScheme.setServiceName("MyDistributedService");
  4. distributedScheme.setXml(new SimpleElement("distributed-scheme"));
  5. distributedScheme.setAutoStart(true);
  6.  
  7. BackingMapScheme backingMapScheme = new BackingMapScheme();
  8. backingMapScheme.setInnerScheme(readWriteBackingMapScheme);
  9. distributedScheme.setBackingMapScheme(backingMapScheme);
DistributedScheme distributedScheme = new DistributedScheme();
distributedScheme.setSchemeName("my-distributed-scheme");
distributedScheme.setServiceName("MyDistributedService");
distributedScheme.setXml(new SimpleElement("distributed-scheme"));
distributedScheme.setAutoStart(true);

BackingMapScheme backingMapScheme = new BackingMapScheme();
backingMapScheme.setInnerScheme(readWriteBackingMapScheme);
distributedScheme.setBackingMapScheme(backingMapScheme);

On line 1, we create the DistributedScheme that will hold the configuration for our distributed-scheme and we set its scheme name and service name to the same value we would have used in the XML.
On line 4 is the first little lie of the blog post. I know I said “no XML” and I was kind of correct, we have no cache config XML, but some scheme classes still require an XmlElement for certain parameters, even if that XML is empty. So, line 4 just passes an empty XmlElement to the DistributedScheme.setXml method.
On line 5, we set the service to be auto-start, that is, when used in a cache server node, the service will be started automatically.

Now we have our DistributedScheme we need to add the ReadWriteBackingMapScheme we created above as the backing-map-scheme. Obviously we do this with a BackingMapScheme on lines 7 – 9.

So that is the cache schemes down. What about the proxy-scheme you ask, well “bear with…” we will come back to that later.

The caching-schemes

In our example we have only created a single scheme but we need to create the equivalent of the caching-schemes sec ion, which is a holder for all of our schemes. In this case Coherence has a class called ServiceSchemeRegistry which we create for this purpose and add our DistributedScheme to it.

  1. ServiceSchemeRegistry serviceSchemeRegistry = new ServiceSchemeRegistry();
  2. serviceSchemeRegistry.register(distributedScheme);
ServiceSchemeRegistry serviceSchemeRegistry = new ServiceSchemeRegistry();
serviceSchemeRegistry.register(distributedScheme);

The cache-mapping

With our cache schemes done, now we will a the cache mappings. In our example configuration file we only had a single mapping. Creating the cache mappings is very simple indeed.

  1. CacheMapping mapping = new CacheMapping("dist-*", "my-distributed-scheme");
CacheMapping mapping = new CacheMapping("dist-*", "my-distributed-scheme");

You just create an instance of the CacheMapping class that you set the cache-name and scheme-name parameters on.
Our example is very simple but you could be more complicated by adding int-params and interceptors to the cache mapping.

The cache-scheme-mapping

As with the cache schemes we need a holder for our cache mappings that represents the cache-scheme-mappings section of the configuration. Again Coherence has a class for that called CacheMappingRegistry for that purpose.

  1. CacheMappingRegistry mappingRegistry = new CacheMappingRegistry();
  2. mappingRegistry.register(mapping);
CacheMappingRegistry mappingRegistry = new CacheMappingRegistry();
mappingRegistry.register(mapping);

The Cache Configuration

Finally we have the parts we need to create the top level cache configuration class.

  1. CacheConfig config = new CacheConfig(new DefaultProcessingContext().getDefaultParameterResolver());
  2. config.setCacheMappingRegistry(mappingRegistry);
  3. config.setServiceSchemeRegistry(serviceSchemeRegistry);
CacheConfig config = new CacheConfig(new DefaultProcessingContext().getDefaultParameterResolver());
config.setCacheMappingRegistry(mappingRegistry);
config.setServiceSchemeRegistry(serviceSchemeRegistry);

Now we have a cache configuration we can use it to create a cache factory.

Create the ConfigurableCacheFactory

The implementation of ConfigurableCacheFactory that we are going to create is ExtensibleConfigurableCacheFactory, you could of course create a sub-class of this if you wanted to do more customisation.

There are a couple of things we need to create first is an instance of ExtensibleConfigurableCacheFactory.Dependencies which will hold the CacheConfig we created above…

  1. ExtensibleConfigurableCacheFactory.Dependencies dependencies =
  2.                 new ExtensibleConfigurableCacheFactory.DefaultDependencies(config);
ExtensibleConfigurableCacheFactory.Dependencies dependencies =
                new ExtensibleConfigurableCacheFactory.DefaultDependencies(config);

Next we need to add a few things to the ResourceRegistry that is contained within the ExtensibleConfigurableCacheFactory.Dependencies

  1. ResourceRegistry resourceRegistry = dependencies.getResourceRegistry();
  2. resourceRegistry.registerResource(InterceptorRegistry.class, new Registry());
  3. resourceRegistry.registerResource(EventDispatcherRegistry.class, new Registry());
  4. resourceRegistry.registerResource(XmlElement.class, "legacy-cache-config", new SimpleElement("cache-config"));
ResourceRegistry resourceRegistry = dependencies.getResourceRegistry();
resourceRegistry.registerResource(InterceptorRegistry.class, new Registry());
resourceRegistry.registerResource(EventDispatcherRegistry.class, new Registry());
resourceRegistry.registerResource(XmlElement.class, "legacy-cache-config", new SimpleElement("cache-config"));

…if you don’t do this it isn’t gonna work.

And finally we create the ExtensibleConfigurableCacheFactory

  1. ExtensibleConfigurableCacheFactory cacheFactory = new ExtensibleConfigurableCacheFactory(dependencies);
ExtensibleConfigurableCacheFactory cacheFactory = new ExtensibleConfigurableCacheFactory(dependencies);

Putting It all Together – Demo

So just to prove it works we will put it all together into a simple class with a main method

  1. package com.thegridman.coherence.noxml;
  2.  
  3. import com.tangosol.coherence.config.CacheConfig;
  4. import com.tangosol.coherence.config.CacheMapping;
  5. import com.tangosol.coherence.config.CacheMappingRegistry;
  6. import com.tangosol.coherence.config.ParameterList;
  7. import com.tangosol.coherence.config.ParameterMacroExpression;
  8. import com.tangosol.coherence.config.ResolvableParameterList;
  9. import com.tangosol.coherence.config.ServiceSchemeRegistry;
  10. import com.tangosol.coherence.config.builder.InstanceBuilder;
  11. import com.tangosol.coherence.config.scheme.BackingMapScheme;
  12. import com.tangosol.coherence.config.scheme.CacheStoreScheme;
  13. import com.tangosol.coherence.config.scheme.DistributedScheme;
  14. import com.tangosol.coherence.config.scheme.LocalScheme;
  15. import com.tangosol.coherence.config.scheme.ProxyScheme;
  16. import com.tangosol.coherence.config.scheme.ReadWriteBackingMapScheme;
  17. import com.tangosol.coherence.config.unit.Seconds;
  18. import com.tangosol.config.expression.Expression;
  19. import com.tangosol.config.expression.LiteralExpression;
  20. import com.tangosol.config.expression.Parameter;
  21. import com.tangosol.config.xml.DefaultProcessingContext;
  22. import com.tangosol.net.ExtensibleConfigurableCacheFactory;
  23. import com.tangosol.net.NamedCache;
  24. import com.tangosol.net.events.EventDispatcherRegistry;
  25. import com.tangosol.net.events.InterceptorRegistry;
  26. import com.tangosol.net.events.internal.Registry;
  27. import com.tangosol.run.xml.SimpleElement;
  28. import com.tangosol.run.xml.XmlElement;
  29. import com.tangosol.util.ResourceRegistry;
  30.  
  31. /**
  32.  * @author Jonathan Knight
  33.  */
  34. public class NoXmlDemo
  35. {
  36.     public static void main(String[] args) throws Exception
  37.     {
  38.         // Create a local-scheme
  39.         LocalScheme localScheme = new LocalScheme();
  40.         // Set expiry on our local scheme of 10 seconds
  41.         localScheme.setExpiryDelay(new LiteralExpression<>(new Seconds(10)));
  42.  
  43.         // Create a read-write-backing-map-scheme
  44.         ParameterList parameterList = new ResolvableParameterList();
  45.         Expression<?> expression = new ParameterMacroExpression<>("{cache-name}", String.class);
  46.         parameterList.add(new Parameter("cache-name", String.class, expression));
  47.  
  48.         CacheStoreScheme cacheStoreScheme = new CacheStoreScheme();
  49.         InstanceBuilder<Object> cacheStoreBuilder = new InstanceBuilder<>(MyCacheStore.class);
  50.         cacheStoreBuilder.setConstructorParameterList(parameterList);
  51.         cacheStoreScheme.setCustomBuilder(cacheStoreBuilder);
  52.  
  53.         ReadWriteBackingMapScheme readWriteBackingMapScheme = new ReadWriteBackingMapScheme();
  54.         readWriteBackingMapScheme.setCacheStoreScheme(cacheStoreScheme);
  55.         readWriteBackingMapScheme.setInternalScheme(localScheme);
  56.  
  57.         // Create a distributed-scheme
  58.         DistributedScheme distributedScheme = new DistributedScheme();
  59.         distributedScheme.setSchemeName("my-distributed-scheme");
  60.         distributedScheme.setServiceName("MyDistributedService");
  61.         distributedScheme.setXml(new SimpleElement("distributed-scheme"));
  62.         distributedScheme.setAutoStart(true);
  63.  
  64.         // Create the backing-map-scheme for the distributed-scheme
  65.         BackingMapScheme backingMapScheme = new BackingMapScheme();
  66.         backingMapScheme.setInnerScheme(readWriteBackingMapScheme);
  67.         distributedScheme.setBackingMapScheme(backingMapScheme);
  68.  
  69.         // Create the caching-schemes registry
  70.         ServiceSchemeRegistry serviceSchemeRegistry = new ServiceSchemeRegistry();
  71.         // Add the distributed scheme to the scheme registry
  72.         serviceSchemeRegistry.register(distributedScheme);
  73.  
  74.         // Create the cache-mapping
  75.         CacheMapping mapping = new CacheMapping("dist-*", "my-distributed-scheme");
  76.  
  77.         // Create the cache-scheme-mapping registry and add the mappings
  78.         CacheMappingRegistry mappingRegistry = new CacheMappingRegistry();
  79.         mappingRegistry.register(mapping);
  80.  
  81.         // Create the Cache Config and add the mappings and schemes
  82.         CacheConfig config = new CacheConfig(new DefaultProcessingContext().getDefaultParameterResolver());
  83.         config.setCacheMappingRegistry(mappingRegistry);
  84.         config.setServiceSchemeRegistry(serviceSchemeRegistry);
  85.  
  86.         // Create the ExtensibleConfigurableCacheFactory
  87.         ExtensibleConfigurableCacheFactory.Dependencies dependencies =
  88.                 new ExtensibleConfigurableCacheFactory.DefaultDependencies(config);
  89.  
  90.         ResourceRegistry resourceRegistry = dependencies.getResourceRegistry();
  91.         resourceRegistry.registerResource(InterceptorRegistry.class, new Registry());
  92.         resourceRegistry.registerResource(EventDispatcherRegistry.class, new Registry());
  93.         resourceRegistry.registerResource(XmlElement.class, "legacy-cache-config", new SimpleElement("cache-config"));
  94.  
  95.         // Create the actual Cache Factory
  96.         ExtensibleConfigurableCacheFactory cacheFactory = new ExtensibleConfigurableCacheFactory(dependencies);
  97.  
  98.         //Start the services;
  99.         cacheFactory.startServices();
  100.  
  101.         // Get a cache and put stuff in it
  102.         NamedCache cache = cacheFactory.ensureCache("dist-test", null);
  103.         cache.put("Key-1", "Value-1");
  104.         for (int i=0; i<11; i++)
  105.          {
  106.              System.err.println("Result: " + cache.get("Key-1"));
  107.              Thread.sleep(1000);
  108.          }
  109.      }
  110.  }
  111.  
package com.thegridman.coherence.noxml;

import com.tangosol.coherence.config.CacheConfig;
import com.tangosol.coherence.config.CacheMapping;
import com.tangosol.coherence.config.CacheMappingRegistry;
import com.tangosol.coherence.config.ParameterList;
import com.tangosol.coherence.config.ParameterMacroExpression;
import com.tangosol.coherence.config.ResolvableParameterList;
import com.tangosol.coherence.config.ServiceSchemeRegistry;
import com.tangosol.coherence.config.builder.InstanceBuilder;
import com.tangosol.coherence.config.scheme.BackingMapScheme;
import com.tangosol.coherence.config.scheme.CacheStoreScheme;
import com.tangosol.coherence.config.scheme.DistributedScheme;
import com.tangosol.coherence.config.scheme.LocalScheme;
import com.tangosol.coherence.config.scheme.ProxyScheme;
import com.tangosol.coherence.config.scheme.ReadWriteBackingMapScheme;
import com.tangosol.coherence.config.unit.Seconds;
import com.tangosol.config.expression.Expression;
import com.tangosol.config.expression.LiteralExpression;
import com.tangosol.config.expression.Parameter;
import com.tangosol.config.xml.DefaultProcessingContext;
import com.tangosol.net.ExtensibleConfigurableCacheFactory;
import com.tangosol.net.NamedCache;
import com.tangosol.net.events.EventDispatcherRegistry;
import com.tangosol.net.events.InterceptorRegistry;
import com.tangosol.net.events.internal.Registry;
import com.tangosol.run.xml.SimpleElement;
import com.tangosol.run.xml.XmlElement;
import com.tangosol.util.ResourceRegistry;

/**
 * @author Jonathan Knight
 */
public class NoXmlDemo
{
    public static void main(String[] args) throws Exception
    {
        // Create a local-scheme
        LocalScheme localScheme = new LocalScheme();
        // Set expiry on our local scheme of 10 seconds
        localScheme.setExpiryDelay(new LiteralExpression<>(new Seconds(10)));

        // Create a read-write-backing-map-scheme
        ParameterList parameterList = new ResolvableParameterList();
        Expression<?> expression = new ParameterMacroExpression<>("{cache-name}", String.class);
        parameterList.add(new Parameter("cache-name", String.class, expression));

        CacheStoreScheme cacheStoreScheme = new CacheStoreScheme();
        InstanceBuilder<Object> cacheStoreBuilder = new InstanceBuilder<>(MyCacheStore.class);
        cacheStoreBuilder.setConstructorParameterList(parameterList);
        cacheStoreScheme.setCustomBuilder(cacheStoreBuilder);

        ReadWriteBackingMapScheme readWriteBackingMapScheme = new ReadWriteBackingMapScheme();
        readWriteBackingMapScheme.setCacheStoreScheme(cacheStoreScheme);
        readWriteBackingMapScheme.setInternalScheme(localScheme);

        // Create a distributed-scheme
        DistributedScheme distributedScheme = new DistributedScheme();
        distributedScheme.setSchemeName("my-distributed-scheme");
        distributedScheme.setServiceName("MyDistributedService");
        distributedScheme.setXml(new SimpleElement("distributed-scheme"));
        distributedScheme.setAutoStart(true);

        // Create the backing-map-scheme for the distributed-scheme
        BackingMapScheme backingMapScheme = new BackingMapScheme();
        backingMapScheme.setInnerScheme(readWriteBackingMapScheme);
        distributedScheme.setBackingMapScheme(backingMapScheme);

        // Create the caching-schemes registry
        ServiceSchemeRegistry serviceSchemeRegistry = new ServiceSchemeRegistry();
        // Add the distributed scheme to the scheme registry
        serviceSchemeRegistry.register(distributedScheme);

        // Create the cache-mapping
        CacheMapping mapping = new CacheMapping("dist-*", "my-distributed-scheme");

        // Create the cache-scheme-mapping registry and add the mappings
        CacheMappingRegistry mappingRegistry = new CacheMappingRegistry();
        mappingRegistry.register(mapping);

        // Create the Cache Config and add the mappings and schemes
        CacheConfig config = new CacheConfig(new DefaultProcessingContext().getDefaultParameterResolver());
        config.setCacheMappingRegistry(mappingRegistry);
        config.setServiceSchemeRegistry(serviceSchemeRegistry);

        // Create the ExtensibleConfigurableCacheFactory
        ExtensibleConfigurableCacheFactory.Dependencies dependencies =
                new ExtensibleConfigurableCacheFactory.DefaultDependencies(config);

        ResourceRegistry resourceRegistry = dependencies.getResourceRegistry();
        resourceRegistry.registerResource(InterceptorRegistry.class, new Registry());
        resourceRegistry.registerResource(EventDispatcherRegistry.class, new Registry());
        resourceRegistry.registerResource(XmlElement.class, "legacy-cache-config", new SimpleElement("cache-config"));

        // Create the actual Cache Factory
        ExtensibleConfigurableCacheFactory cacheFactory = new ExtensibleConfigurableCacheFactory(dependencies);

        //Start the services;
        cacheFactory.startServices();

        // Get a cache and put stuff in it
        NamedCache cache = cacheFactory.ensureCache("dist-test", null);
        cache.put("Key-1", "Value-1");
        for (int i=0; i<11; i++)
         {
             System.err.println("Result: " + cache.get("Key-1"));
             Thread.sleep(1000);
         }
     }
 }
 

There is everything inside a single method. If you run the code it will create a ExtensibleConfigurableCacheFactory in the way described above.

Lines 99 - 108 are extra to prove it all works.
Line 99 tells the cache factory to start all of its autostart services.
Line 102 gets a NamedCache called dist-test from the cache factory.
Line 103 puts a key/value pair into the cache.
Line 104 - 108 just loop round getting the value from the cache every second for 11 seconds. This is to prove that the 10 seconds expiry we set way back when we created the local scheme is actually working.

If you run the class you will see something like this...

dist-test store called key=Key-1 value=Value-1
Result: Value-1
Result: Value-1
Result: Value-1
Result: Value-1
Result: Value-1
Result: Value-1
Result: Value-1
Result: Value-1
Result: Value-1
Result: Value-1
dist-test load called key=Key-1
Result: null

The first line is the store() method our cache store being called in response to the initial put on the cache. You can see that the cache name at the start of the line is correct, so out cache store macro parameter worked.
Then there are a number of lines where the result of the get() method is printed every second.
Eventually after 10 seconds the value will be expired from the cache so we then see the load() method being called on the cache store.
Finally we see the "Result: null" line as our cache store's load() method returned null.

So there we are, it all worked.

But We Missed Some Bits...

Yes, there are a couple of things missing from this very simple example.

The Serializer

The eagle eyed of you will have spotted that in the cache configuration XML at the start of the post there is a defaults section that specifies the default serializer to use in the rest of the configuration. We did not actually do anything with this above and if you run the code in a debugger and look at the serializer being used by the CacheService you will see it is not a PofSerializer.

There isn't actually anything in the classes we have mentioned above to handle the defaults (at least not anything I have found yet). We need to fall back to goo old XML, which I already mentioned when creating the distributed-scheme. You will remember that we had a line that injected an empty XmlElement into the DistributedScheme...

  1. distributedScheme.setXml(new SimpleElement("distributed-scheme"));
distributedScheme.setXml(new SimpleElement("distributed-scheme"));

Well, this is where we also need to inject the serializer, which we can do like this...

  1. SimpleElement xml = new SimpleElement("distributed-scheme");
  2. xml.addElement("serializer").setString("pof");
  3. distributedScheme.setXml(xml);
SimpleElement xml = new SimpleElement("distributed-scheme");
xml.addElement("serializer").setString("pof");
distributedScheme.setXml(xml);

So in effect we are injecting a snippet of XML that looks like this...

  1. <distributed-scheme>
  2.     <serializer>pof</serializer>
  3. </distributed-scheme>
<distributed-scheme>
    <serializer>pof</serializer>
</distributed-scheme>

...which is enough to make the service use the default POF serializer.

The proxy-service

Finally we come back to the proxy-service which I skipped previously. The reason for that is there is no proper scheme for a proxy service, or rather, there is a scheme but you need to configure it with XML - so the title of the post should be "You almost Don't Need That Cache Config XML File".

To configure the Proxy Scheme we need to do this...

  1. XmlElement proxyConfigElement = new SimpleElement("proxy-scheme");
  2. XmlElement acceptorElement = proxyConfigElement.addElement("acceptor-config");
  3. XmlElement tcpAcceptorElement = acceptorElement.addElement("tcp-acceptor");
  4. XmlElement localAddressElement = tcpAcceptorElement.addElement("local-address");
  5. XmlElement addressElement = localAddressElement.addElement("address");
  6. XmlElement portElement = localAddressElement.addElement("port");
  7. addressElement.setString("localhost");
  8. portElement.setInt(9099);
  9.  
  10. ProxyScheme proxyScheme = new ProxyScheme();
  11. proxyScheme.setSchemeName("my-proxy-scheme");
  12. proxyScheme.setServiceName("TcpProxyService");
  13. proxyScheme.setAutoStart(true);
  14. proxyScheme.setXml(proxyConfigElement);
XmlElement proxyConfigElement = new SimpleElement("proxy-scheme");
XmlElement acceptorElement = proxyConfigElement.addElement("acceptor-config");
XmlElement tcpAcceptorElement = acceptorElement.addElement("tcp-acceptor");
XmlElement localAddressElement = tcpAcceptorElement.addElement("local-address");
XmlElement addressElement = localAddressElement.addElement("address");
XmlElement portElement = localAddressElement.addElement("port");
addressElement.setString("localhost");
portElement.setInt(9099);

ProxyScheme proxyScheme = new ProxyScheme();
proxyScheme.setSchemeName("my-proxy-scheme");
proxyScheme.setServiceName("TcpProxyService");
proxyScheme.setAutoStart(true);
proxyScheme.setXml(proxyConfigElement);

...First we programatically create the XML to configure the proxy service. We could have done this various ways, but here we chose to use a few lines of code. We could have put the XML in a file and loaded it with the XmlHelper too if we wanted.
After we create the XML we create a ProxyScheme which following the same pattern as everything else, is the class that contains the configuration for the proxy service. We then set the scheme name and service name and set the configuration XML.

Now we have a proxy service we can add it to the service registry, where previously we just added the distributed scheme

  1. ServiceSchemeRegistry serviceSchemeRegistry = new ServiceSchemeRegistry();
  2. serviceSchemeRegistry.register(distributedScheme);
  3. serviceSchemeRegistry.register(proxyScheme);
ServiceSchemeRegistry serviceSchemeRegistry = new ServiceSchemeRegistry();
serviceSchemeRegistry.register(distributedScheme);
serviceSchemeRegistry.register(proxyScheme);

Now if we run the demo class above with these changes you will see in the logs that the proxy service is started and is listening on port 9099.

Running a DefaultCacheServer

I mentioned previously that we could use our programatically created cache factory inside a server node running a DefaultCacheServer. We can do this with a couple of lines of code like this...

  1. // Create the Cache Factory
  2. ExtensibleConfigurableCacheFactory cacheFactory = new ExtensibleConfigurableCacheFactory(dependencies);
  3. DefaultCacheServer dcs = new DefaultCacheServer(cacheFactory);
  4. dcs.startAndMonitor(5000L);
// Create the Cache Factory
ExtensibleConfigurableCacheFactory cacheFactory = new ExtensibleConfigurableCacheFactory(dependencies);
DefaultCacheServer dcs = new DefaultCacheServer(cacheFactory);
dcs.startAndMonitor(5000L);

Would I Use This in Production

Ha Ha Ha...

Whilst the code above was really to prove a point and show that you can now do a lot without an XML configuration file the work is not quite complete. There are still a few places where we need to fall back on XML. The classes used above are all internal to Coherence and not part of the documented API, so could and probably will change in forthcoming releases.

Right now, it has been fun to play with and occasionally useful when testing a bit of functionality as I can fire bits of Coherence up without a configuration file, so my test code is not full of XML files.

At least with an XML configuration file you have some idea of what your cluster should look like, although having said that, with custom namespaces and the ability to combine configuration files, this is becoming more complicated. I can imagine the poor guys in the Oracle Coherence Support Team not looking forward to the day when they ask for your XML configuration only to be sent some Java code instead!

Coherence 12.1.2 Custom Namespaces and Oracle NoSQL

NoSQL Expert

Oracle Coherence 12.1.2 Custom Namespaces is a new feature in Coherence 12.1.2 that allows you to add custom functionality to the Coherence cache configuration file. If anyone has used custom namespaces with something like the Spring Framework, then Coherence namespaces are very similar.

This blog post is going to work through an example of building a custom namespace. In this case we are going to use it to integrate Coherence with the Oracle NoSQL database. Now, those of you that have looked at Oracle NoSQL, especially the enterprise edition might be saying, “hang on, Oracle NoSQL already has Coherence integration” and you would be correct, but it does not work with Coherence version 12.1.2 so we are going to change that here. All the code from this blog is available here on my GitHub repository

First a little history lesson…

Custom namespace support has been in Coherence for quite some time now if you’ve been using the Coherence Incubator Commons library. It all started…
A long time ago in a galaxy far, far away….
OK, it started in Canary Wharf, London, in the heady days at the tail end of financial crash of 2008 something like this…
I remember seeing the first presentation of the Incubator Push Replication at the London Coherence SIG back in 2008, which at that time was all configured in code. I was flying from London to the US for Christmas just after the SIG, so during the flight I wrote some code to allow Push Replication to be configured inside the cache configuration file and e-mailed it to Brian Oliver at Oracle. On my return from London and on chatting to Brian, this had obviously triggered some thinking on his side and he had a lot of ideas about how custom namespaces could work (and typically for Brian, promised to write it over the weekend – LOL!). Well, it took a bit longer than a weekend to see the first namespaces support in the Incubator and finally now it is now fully supported in the core product. The new namespaces support in 12.1.2 is quite impressive and a lot of the classes that are used are actually there to allow the removal of the XML configuration file, but that is another blog post. You might think that was a long time to get something into the core product, and I’d agree, but this was quite a bit of work for the engineering team to refactor the whole of the cache configuration framework. Thinking back to those initial conversations, it is great to see something like that now in the core product – as they say “Mighty Oaks from little acorns grow”.

Coherence & Oracle NoSQL Integration

So, on with the meat of the post and back to our initial requirement – integrate Oracle NoSQL and Coherence. If you look at the Oracle NoSQL site you will see support for Coherence documented. In effect this support is currently (as of version 2.1.8) a couple of Coherence CacheStore classes that allow you to connect to NoSQL in the same way you would any other data source using a CacheStore. I’m not going to cover all about how this works; you are free to look at the Oracle NoSQL site to see the documentation. The problem with it though as I have said, is that it only works with versions of Coherence prior to 12.1.2. This is because the two CacheStore classes, oracle.kv.coherence.NoSQLBinaryStore and oracle.kv.coherence.NoSQLAvroCacheStore, and their base class oracle.kv.coherence.NoSQLStoreBase implement com.tangosol.run.xml.XmlConfigurable which means that any init-param parameters in cache configuration file where they are configured are passed to the setConfig of the class as a snippet of XML. The problem in 12.1.2 is that this is no longer supported – as I found out the hard way when porting a project from 3.7 to 12.1.2. Maybe it is documented but hey, how many developers read documentation ;-)

So, now we know Oracle NoSQL does not work with 12.1.2 how do we fix it? Well, as I’ve said, I had to do this on a project already, so I know one answer (and I think the elegant answer) is to use custom namespaces. In this way, Coherence does not do anything itself to build or configure the NoSQL CacheStore classes; instead it will delegate the job to our custom namespace classes. One other requirement of this is that whatever code we write, we will not be changing the actual NoSQL classes, so they will still work with the previous versions of Coherence.

This is how the NoSQL documentation says to configure a NoSQLBinaryStore cache store…

  1. <distributed-scheme>
  2.   <scheme-name>BinaryCacheScheme</scheme-name>
  3.   <service-name>BinaryCacheService</service-name>
  4.   <serializer>pof</serializer>
  5.   <backing-map-scheme>
  6.     <read-write-backing-map-scheme>
  7.       <internal-cache-scheme>
  8.         <local-scheme />
  9.       </internal-cache-scheme>
  10.       <cachestore-scheme>
  11.         <class-scheme>
  12.           <class-name>oracle.kv.coherence.NoSQLBinaryStore</class-name>
  13.           <init-params>
  14.             <init-param>
  15.               <param-name>storeName</param-name>
  16.               <param-value>kvstore</param-value>
  17.             </init-param>
  18.             <init-param>
  19.               <param-name>helperHosts</param-name>
  20.               <param-value>localhost:5000</param-value>
  21.             </init-param>
  22.           </init-params>
  23.         </class-scheme>
  24.       </cachestore-scheme>
  25.     </read-write-backing-map-scheme>
  26.   </backing-map-scheme>
  27.   <autostart>true</autostart>
  28. </distributed-scheme>
<distributed-scheme>
  <scheme-name>BinaryCacheScheme</scheme-name>
  <service-name>BinaryCacheService</service-name>
  <serializer>pof</serializer>
  <backing-map-scheme>
    <read-write-backing-map-scheme>
      <internal-cache-scheme>
        <local-scheme />
      </internal-cache-scheme>
      <cachestore-scheme>
        <class-scheme>
          <class-name>oracle.kv.coherence.NoSQLBinaryStore</class-name>
          <init-params>
            <init-param>
              <param-name>storeName</param-name>
              <param-value>kvstore</param-value>
            </init-param>
            <init-param>
              <param-name>helperHosts</param-name>
              <param-value>localhost:5000</param-value>
            </init-param>
          </init-params>
        </class-scheme>
      </cachestore-scheme>
    </read-write-backing-map-scheme>
  </backing-map-scheme>
  <autostart>true</autostart>
</distributed-scheme>

…That is taken directly from the NoSQL examples code. You can see that the cache store is defined using a normal class-scheme, and this would be fine if NoSQLBinaryStore had a constructor that took the two init-parm values as arguments, but it does not. In this case there are a lot of arguments that can be passed to the NoSQLBinaryStore and it would be impractical to have constructors that take all the different combinations, which is probably why it was written to use XMLConfigurable.

Using custom namespaces in 12.1.2 we will configure the same cache store like this…
Define the custom namespace in the cache-config element…

  1. <cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  2.               xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
  3.               xmlns:kv="class://oracle.kv.coherence.NoSQLNamespaceHandler"
  4.               xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd">
<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
               xmlns:kv="class://oracle.kv.coherence.NoSQLNamespaceHandler"
               xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd">

…you can see we have specified out namespace with xmlns:kv

Use our namespace inside the cache-store-scheme element…

  1. <distributed-scheme>
  2.   <scheme-name>BinaryCacheScheme</scheme-name>
  3.   <service-name>BinaryCacheService</service-name>
  4.   <serializer>pof</serializer>
  5.   <backing-map-scheme>
  6.     <read-write-backing-map-scheme>
  7.       <internal-cache-scheme>
  8.         <local-scheme />
  9.       </internal-cache-scheme>
  10.       <cachestore-scheme>
  11.         <kv:binary-cache-store>
  12.           <kv:store-name system-property="oracle.kv.storename">kvstore</kv:store-name>
  13.           <kv:helper-hosts system-property="oracle.kv.helperhosts"/> localhost:5000</ kv:helper-hosts>
  14.         </kv:binary-cache-store>
  15.       </cachestore-scheme>
  16.     </read-write-backing-map-scheme>
  17.   </backing-map-scheme>
  18.   <autostart>true</autostart>
  19. </distributed-scheme>
<distributed-scheme>
  <scheme-name>BinaryCacheScheme</scheme-name>
  <service-name>BinaryCacheService</service-name>
  <serializer>pof</serializer>
  <backing-map-scheme>
    <read-write-backing-map-scheme>
      <internal-cache-scheme>
        <local-scheme />
      </internal-cache-scheme>
      <cachestore-scheme>
        <kv:binary-cache-store>
          <kv:store-name system-property="oracle.kv.storename">kvstore</kv:store-name>
          <kv:helper-hosts system-property="oracle.kv.helperhosts"/> localhost:5000</ kv:helper-hosts>
        </kv:binary-cache-store>
      </cachestore-scheme>
    </read-write-backing-map-scheme>
  </backing-map-scheme>
  <autostart>true</autostart>
</distributed-scheme>

Now, that is very nice, what’s more, the custom namespace can have its own XSD file to allow the XML to be validated, which is a big plus for developers who can turn this validation on inside their IDEs (if you don’t do this you should be doing it).

So, when Coherence needs to instantiate a cache that maps to the BinaryCacheScheme scheme it will call our handlers to instantiate a CacheStore instance.

Custom Namespace Implementation

OK, that was the easy part, showing the finished configuration, but how do we make it work. We will start the explanation bottom up, starting with the class that builds the CacheStore and work our way back up to the namespace handler.

Schemes and Builders

As I said earlier there has been a lot of refactoring and new work done around how Coherence configuration works. A whole new set of classes called schemes and builders were written that basically configure and build the various parts of Coherence. Scheme appear to be the configuration and work either on their own or with builders to instantiate instances of various Coherence components. What this means in effect is that you do not actually need an XML configuration file to run Coherence. I have had a whole cluster running all configured in code with no XML – I’ll write that up in another post.

So, for our NoSQL integration we need a scheme that will hold the configuration for our CacheStore instances. There are two CacheStore classes in the NoSQL integration framework, but there is a lot of commonality, so we will write a scheme for each with a common base class.

The scheme class is what Coherence will call to inject in the various parameters from the XML configuration.

Here is the skeleton of our scheme for creating the NoSQL cache stores

  1. public abstract class NoSQLBaseCacheStoreScheme
  2.     extends AbstractScheme
  3.     implements ParameterizedBuilder<NoSQLStoreBase>, ParameterizedBuilder.ReflectionSupport {
  4.  
  5.     public NoSQLBaseCacheStoreScheme() {
  6.     }
  7.  
  8.     @Override
  9.     public NoSQLStoreBase realize(ParameterResolver resolver, ClassLoader classLoader, ParameterList parameterList)
  10.     {
  11.     return null;
  12.     }
  13.  
  14.     @Override
  15.     public boolean realizes(Class<?> paramClass, ParameterResolver paramParameterResolver, ClassLoader paramClassLoader)
  16.     {
  17.         return true;
  18.     }
  19. }
public abstract class NoSQLBaseCacheStoreScheme 
    extends AbstractScheme 
    implements ParameterizedBuilder<NoSQLStoreBase>, ParameterizedBuilder.ReflectionSupport {

    public NoSQLBaseCacheStoreScheme() {
    }

    @Override
    public NoSQLStoreBase realize(ParameterResolver resolver, ClassLoader classLoader, ParameterList parameterList)
    {
    return null;
    }

    @Override
    public boolean realizes(Class<?> paramClass, ParameterResolver paramParameterResolver, ClassLoader paramClassLoader) 
    {
        return true;
    }
}

This is the bear minimum for our scheme class. The realize method is the one that will actually be responsible for instantiating our cache store instance. The realizes method is used by Coherence to check whether this scheme realizes classes of the required type.

But where is the XML configuration set? For our scheme to be passed the values from the XML configuration we need to add set methods for each parameter, that is, for each XML tag we want a value for. These methods are then annotated to tell Coherence which XML element to inject.

For example, looking at the configuration above that used our new namespace we have this…

  1. <kv:binary-cache-store>
  2.   <kv:store-name system-property="oracle.kv.storename">kvstore</kv:store-name>
  3.   <kv:helper-hosts system-property="oracle.kv.helperhosts"/> localhost:5000</ kv:helper-hosts>
  4. </kv:binary-cache-store>
<kv:binary-cache-store>
  <kv:store-name system-property="oracle.kv.storename">kvstore</kv:store-name>
  <kv:helper-hosts system-property="oracle.kv.helperhosts"/> localhost:5000</ kv:helper-hosts>
</kv:binary-cache-store>

This XML contains two elements (the full scheme will contain more) that need injecting, these are kv:store-name and kv:helper-hosts so we need two setters.

  1. @Injectable("store-name")
  2. public void setStoreName(Expression<String> expression)
  3. {
  4.     this.storeNameExpression = expression;
  5. }
  6.  
  7. @Injectable("helper-hosts")
  8. public void setHelperHosts(Expression<String> expression)
  9. {
  10.      this.helperHostsExpression = expression;
  11. }
@Injectable("store-name")
public void setStoreName(Expression<String> expression)
{
    this.storeNameExpression = expression;
}

@Injectable("helper-hosts")
public void setHelperHosts(Expression<String> expression)
{
     this.helperHostsExpression = expression;
}

As you can see these are simple setter methods. Coherence passes in an Expression of the required type (in this case both attributes are Strings). The @Injectable annotation tells Coherence which XML element to inject, all very simple.

When we need to build an instance of our cache store we then evaluate the expressions passed in to give us the real values to use. We do this in the realize method.

  1. @Override
  2. public NoSQLStoreBase realize(ParameterResolver resolver, ClassLoader classLoader, ParameterList parameterList)
  3. {
  4.     String storeName = storeNameExpression.evaluate(resolver);
  5.     String helperHosts = storeNameExpression.evaluate(resolver);
  6.  
  7.     NoSQLStoreBase store = new
  8.  
  9.     Return store;
  10. }
@Override
public NoSQLStoreBase realize(ParameterResolver resolver, ClassLoader classLoader, ParameterList parameterList)
{
    String storeName = storeNameExpression.evaluate(resolver);
    String helperHosts = storeNameExpression.evaluate(resolver);

    NoSQLStoreBase store = new …

    Return store;
}

One of the requirements I mentioned earlier is that we are not going to change any of the existin NoSQL code as we ant it to remain compatible with earlier Coherence versions. This means that we need to instantiate our cache store implementations by calling new and then calling setConfig and pass in some XML. To do this we will just build up the relevant XML configuration in our realize method and pass it to the cache store; for example:

  1. @Override
  2. public NoSQLStoreBase realize(ParameterResolver resolver, ClassLoader classLoader, ParameterList parameterList)
  3. {
  4.     XmlElement configXml = new SimpleElement("config");
  5.     addElementIfNotNull(configXml, XmlConfigParser.OPT_STORENAME, getStoreName(resolver));
  6.     addElementIfNotNull(configXml, XmlConfigParser.OPT_HELPERHOSTS, getHelperHosts(resolver));
  7.  
  8.     NoSQLStoreBase store = instantiateStore();
  9.     store.setConfig(configXml);
  10.     return store;
  11. }
  12.  
  13. public String getStoreName(ParameterResolver resolver)
  14. {
  15.     return (storeNameExpression != null) ? storeNameExpression.evaluate(resolver) : null;
  16. }
  17.  
  18. public String getHelperHosts(ParameterResolver resolver)
  19. {
  20.     return (helperHostsExpression == null) ? null : helperHostsExpression.evaluate(resolver);
  21. }
  22.  
  23. private void addElementIfNotNull(XmlElement parent, String elementName, Object value)
  24. {
  25.     if (value == null)
  26.     {
  27.         return;
  28.     }
  29.  
  30.     parent.addElement(elementName).setString(String.valueOf(value));
  31. }
@Override
public NoSQLStoreBase realize(ParameterResolver resolver, ClassLoader classLoader, ParameterList parameterList)
{
    XmlElement configXml = new SimpleElement("config");
    addElementIfNotNull(configXml, XmlConfigParser.OPT_STORENAME, getStoreName(resolver));
    addElementIfNotNull(configXml, XmlConfigParser.OPT_HELPERHOSTS, getHelperHosts(resolver));

    NoSQLStoreBase store = instantiateStore();
    store.setConfig(configXml);
    return store;
}

public String getStoreName(ParameterResolver resolver)
{
    return (storeNameExpression != null) ? storeNameExpression.evaluate(resolver) : null;
}

public String getHelperHosts(ParameterResolver resolver)
{
    return (helperHostsExpression == null) ? null : helperHostsExpression.evaluate(resolver);
}

private void addElementIfNotNull(XmlElement parent, String elementName, Object value)
{
    if (value == null)
    {
        return;
    }

    parent.addElement(elementName).setString(String.valueOf(value));
}

The instantiateStore () method used above is not shown here as this code is all in our base scheme class. The instantiateStore() method will be in the sub-classes so that it can create the correct store implementation. For example, the sub-class looks like this

  1. public class NoSQLBinaryStoreScheme extends NoSQLBaseCacheStoreScheme {
  2.  
  3.     public NoSQLBinaryStoreScheme() {
  4.         super(EnumSet.complementOf(EnumSet.of(XmlConfigParser.ConfigOption.AVRO_FORMAT, XmlConfigParser.ConfigOption.SCHEMA_FILES)));
  5.     }
  6.  
  7.     @Override
  8.     protected NoSQLStoreBase instantiateStore() {
  9.         return new NoSQLBinaryStore();
  10.     }
  11.  
  12.     @Override
  13.     public boolean realizes(Class<?> aClass, ParameterResolver resolver, ClassLoader classLoader) {
  14.         return aClass.isAssignableFrom(NoSQLBinaryStore.class);
  15.     }
  16. }
public class NoSQLBinaryStoreScheme extends NoSQLBaseCacheStoreScheme {

    public NoSQLBinaryStoreScheme() {
        super(EnumSet.complementOf(EnumSet.of(XmlConfigParser.ConfigOption.AVRO_FORMAT, XmlConfigParser.ConfigOption.SCHEMA_FILES)));
    }

    @Override
    protected NoSQLStoreBase instantiateStore() {
        return new NoSQLBinaryStore();
    }

    @Override
    public boolean realizes(Class<?> aClass, ParameterResolver resolver, ClassLoader classLoader) {
        return aClass.isAssignableFrom(NoSQLBinaryStore.class);
    }
}

If we were only going to create Coherence cache factory implementations programmatically then a scheme would be enough for us to work with. For most people though we will now need to pug our scheme into the XML configuration processing.

ElementProcessor

For our custom namespace to work we need to register an instance of a com.tangosol.config.xml.ElementProcessor for each custom XML element we process in our namespace. In our case we are going to have two top level elements, one for each of the custom CacheStore implementations. In Oracle NoSQL there is the basic oracle.kv.coherence.NoSQLBinaryStore and the AVRO supporting oracle.kv.coherence.NoSQLAvroCacheStore classes. So we will have two XML elements called binary-cache-store and avro-cache-store so we end up with two ElementProcessor classes.

  1. public class NoSQLBinaryStoreElementProcessor implements ElementProcessor<NoSQLBinaryStoreScheme>
  2. {
  3.     @Override
  4.     public NoSQLBinaryStoreScheme process(ProcessingContext context, XmlElement element) throws ConfigurationException
  5.     {
  6.         return context.inject(new NoSQLBinaryStoreScheme(), element);
  7.     }
  8. }
public class NoSQLBinaryStoreElementProcessor implements ElementProcessor<NoSQLBinaryStoreScheme>
{
    @Override
    public NoSQLBinaryStoreScheme process(ProcessingContext context, XmlElement element) throws ConfigurationException
    {
        return context.inject(new NoSQLBinaryStoreScheme(), element);
    }
}
  1. public class NoSQLAvroCacheStoreElementProcessor implements ElementProcessor<NoSQLAvroCacheStoreScheme>
  2. {
  3.     @Override
  4.     public NoSQLAvroCacheStoreScheme process(ProcessingContext context, XmlElement element) throws ConfigurationException
  5.     {
  6.         return context.inject(new NoSQLAvroCacheStoreScheme(), element);
  7.     }
  8. }
public class NoSQLAvroCacheStoreElementProcessor implements ElementProcessor<NoSQLAvroCacheStoreScheme>
{
    @Override
    public NoSQLAvroCacheStoreScheme process(ProcessingContext context, XmlElement element) throws ConfigurationException
    {
        return context.inject(new NoSQLAvroCacheStoreScheme(), element);
    }
}

As you can see these are very simple classes. In our case the only thing they need to do is register the correct scheme with the ProcessingContext.

NamespaceHandler

Almost there now, the final thing we need is the actual com.tangosol.config.xml.NamespaceHandler implementation that Coherence will delegate to in order to handle any of our custom XML. In this case our implementation is very straight forward, all it needs to do is register the two ElementProcessor implementations with the relevant XML tags.

  1. public class NoSQLNamespaceHandler extends AbstractNamespaceHandler
  2. {
  3.  
  4.     public static final String XMLTAG_BINARY_CACHE_STORE = "binary-cache-store";
  5.     public static final String XMLTAG_AVRO_CACHE_STORE = "avro-cache-store";
  6.  
  7.     public NoSQLNamespaceHandler()
  8.     {
  9.         registerProcessor(XMLTAG_BINARY_CACHE_STORE, new NoSQLBinaryStoreElementProcessor());
  10.         registerProcessor(XMLTAG_AVRO_CACHE_STORE, new NoSQLAvroCacheStoreElementProcessor());
  11.     }
  12.  
  13. }
public class NoSQLNamespaceHandler extends AbstractNamespaceHandler
{

    public static final String XMLTAG_BINARY_CACHE_STORE = "binary-cache-store";
    public static final String XMLTAG_AVRO_CACHE_STORE = "avro-cache-store";

    public NoSQLNamespaceHandler()
    {
        registerProcessor(XMLTAG_BINARY_CACHE_STORE, new NoSQLBinaryStoreElementProcessor());
        registerProcessor(XMLTAG_AVRO_CACHE_STORE, new NoSQLAvroCacheStoreElementProcessor());
    }

}

If you go back to the cache configuration file you will see that this is the class that is declared in the namespace declaration in the root cache-config element of the configuration XML.

  1. <cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  2.               xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
  3.               xmlns:kv="class://oracle.kv.coherence.NoSQLNamespaceHandler"
  4.               xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd">
<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
               xmlns:kv="class://oracle.kv.coherence.NoSQLNamespaceHandler"
               xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd">

Custom XSD Validation

One final thing we can include, although this is optional, is an XSD file for our custom namespace. This has a couple of advantages, first Coherence will use the XSD to validate the XML and second, which I’ve mentioned already, we can use it in IDEs at development time to validate the XML.

To enable XSD validation we just add a schema location to the cache-config element at the top of the cache configuration file like this.

  1. <cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  2.               xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
  3.               xmlns:kv="class://oracle.kv.coherence.NoSQLNamespaceHandler"
  4.               xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd
  5.                                   class://oracle.kv.coherence.NoSQLNamespaceHandler coherence-nosql-config.xsd">
<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
               xmlns:kv="class://oracle.kv.coherence.NoSQLNamespaceHandler"
               xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd
                                   class://oracle.kv.coherence.NoSQLNamespaceHandler coherence-nosql-config.xsd">

You can see the location specified on line 5 that corresponds to the xmlns declared on line 3. Obviously our custom XSD needs to be called coherence-nosql-config.xsd and be on the classpath. I’m not going to post the XSD here but it is on GitHub.

In Summary

That’s about it and enough custom namespace work to allow Oracle NoSQL to work with Coherence 12.1.2 as a cache store DB. If you want to try it out go ahead and get the code from my GitHub repository. This is a Maven project and you will need Coherence 12.1.2 and Oracle NoSQL in your repository to use it.

There is a lot more that you can do with custom namespaces, this post has just scratched the surface. As you have seen we have used it here to inject our own implementation of a cache store. I have used it on other projects to inject various classes in places in the configuration we would previously have used a class-scheme. If you want to see more examples then a good place to look would be the release of the Coherence Incubator that corresponds to Coherence 12.1.2 (which at the time I am writing this hasn’t been publicly released yet).

Coherence .Net Testing with Oracle Tools

oracle coherence .net tools

This blog post is about making life easier when testing Oracle Coherence .Net client applications by using the Oracle Tools library on Git Hub. In my last blog about Oracle Tools I talked about how you can use Oracle Tools to easily test Oracle Coherence applications in Java and now it is the turn of the Oracle Coherence .Net developer to have life made easier. Previously I covered how you can use Oracle Tools to run an Oracle Coherence cluster in-process in the same JVM as the test code is running. Using Oracle Tools it is easy to configure and control the cluster members from within the test code and be safe in the knowledge that the cluster will die when the tests finish. Well, all this is now available in the .Net world too in my Oracle Tools .Net GitHub repository. If you are like other .Net developers I know, you resort to forking cluster processes when writing any sort of integration or acceptance test, this will be a thing of the past with Oracle Tools for Coherence .Net so test suites and CI builds can be more reliable andtest easier to run and configure from unit test code.

I have been wanting to put a .Net wrapper around this code for a while, way back when it was still part of the Oracle Coherence Incubator. The refactoring into Oracle Tools has actually made this job a bit easier as the code is much more modular and self contained and easier to build.

How It Works

Before we dive into some examples I will give a brief desription of how this works. I have not re-written all of Oracle Tools in .Net, that would have taken too long, and would not really be possible. Not everyone realizes this, but the Windows versions of Java come with a jvm.dll file that allows you to run a JVM inside a normal Windows process. You can communicate with the jvm.dll using the Java Native Interface. Once you have a JNI session instantiated and the JVM running it is possible to instantiate Java classes from your Windows code (in this case .Net) and communicate with those classes. The only down side is that JNI is a pig to work with so anything that makes JNI easier is very welcome.

There are a few commercial products that will do the job of making JNI easier, but Oracle Tools is open source so I did not want to go down that route. There are also a few open source JNI wrappers but they all seemd to have missing functionality until I found jni4net. The jni4net library allows you to generate .Net wrapper classes around your Java classes (and vice-versa). The classes and methods look identical in both languages, which is really cool, what’s more it works and is easy to set-up and integrate into a build.

So, the simple thing would be to say that I just ran jni4net over the Oracle Tools library and hey-presto we were away – but life is never that simple. There are a few little gotchas I needed to overcome. The Oracle Tools library is Java and some of it does not map well directly to the .Net world – I’m not the worlds best .Net developer by a long way, but even I could see where things didn’t look right to someone with a .Net brain. Finally jni4net does not wrap generics so even though generics are present on the Java classes they are lost in the generated code, so I needed to fix that too – by hand coding the stuff rather than fixing jni4net. All in all there was still a fair bit of .Net coding to be done – which believe me, for a hard core Java developer like me was a bit of a learning curve – all I can say is thank heavens for JetBrains Resharper. Being a massive IntelliJ fan getting Resharper saved my life as it makes Visual Studio look a lot like IntelliJ.

Oracle Tools .Net Examples

I will leave how to build Oracle Tools .Net until later in the post, first we will jump straight into some examples. If you have read the previous post they will look very familiar – which is the whole point. I will cover the examples in the same order as the last post, just to be consistent. The code for these examples are available int the code in the GitHub repository. All of these examples are written as NUnit test classes.

Initialising Oracle Tools .Net

As I have mentioned already, the way all of this code works is that it wraps classes running in an embedded JVM inside the .Net process. This JVM needs to be initialised, which jni4net provides code for, before it can be used. All of this initialisation code has been wrappered away inside a class called OracleTools in the Oracle.Tools namespace.
To make life a bit easier the Oracle Tools.Net dll comes with all of the relevant Java jar files embedded inside it so you do not need to worry about dependencies. When Oracle Tools is initialised and the JVM started all of these jars can be extracted onto the classpath of the embedded JVM. These dependencies are not the same as your applications dependencies, they are just for running Oracle Tools so you do not need to worry about trying to add your own jar files to this classpath.

The OracleTools class is a singleton, you cannot construct it, the only instance is available using the OracleTools.Wrapper member which you can then call methods on. The majority of the methods are a fluent style interface, that is they return the instance of OracleTools so you can chain calls together.

All of the examples we see below use the same, or similar, initialisation code

  1. public const string LibFolder = "lib";
public const string LibFolder = "lib";
  1. if (!OracleTools.Wrapper.Initialised)
  2. {
  3.     OracleTools.Wrapper
  4.         .AddEmbeddedJarsToClassPath(LibFolder)
  5.         .Initialise();
  6. }
if (!OracleTools.Wrapper.Initialised)
{
    OracleTools.Wrapper
        .AddEmbeddedJarsToClassPath(LibFolder)
        .Initialise();
}

The code above first checks to see whether Oracle Tools has already been initialise – you cannot start the JVM twice.
If OracleTools has not been initialised we call the AddEmbeddedJarsToClassPath then the Initialise method.
The AddEmbeddedJarsToClassPath tells OracleTools that we want it to add the jar files embedded in the OracleTools.dll to the JVM classpath. To do this they will be extracted to the directory named in the parameter to the AddEmbeddedJarsToClassPath method. In this case we extract them to a folder called “lib”, which is relative to the working directory, so we will create a folder called “lib” below the working directory and extract the jar files into it.
The Initialise method then starts the JVM with the class path we told it to use.

Class Path Options

Note The classpath of the embedded JVM is not the same as the classpath of the application you are going to test, they are completly independent. The embedded JVM only needs the jar file dependencies for oracle tools. In most cases you would not need to add any more jar files than those embedded in the DLL, but if you do you can call other methods on OracleTools.Wrapper

OracleTools.Wrapper.AddClassPath(path) This method adds the specified entry to the classpath. Just like Java, this can be a directory containing class files and resources or it can be the path to a jar file.

OracleTools.Wrapper.AddAllJarsToClassPath(folder) This method adds all of the jar files in the specified folder to the classpath. It is the same as using wild cards in a normal Java classpath.

Java Home – Which Version of Java is Used?

Obviously you already have one or more versions of Java JDK installed on your PC. By default Oracle Tools will look in the Windows Registry and find the default version of Java – which is usually the latest version you installed. If you have multiple versions and you want Oracle tools to use a specific one you can use OracleTools.Wrapper.SetJavaHome(home) where the parameter is the path to the location the JDK is installed.

JVM Memory

It is likely that you will want to set things like the various memory options for the embedded JVM, especially if you are going to run multiple container processes inside it, for example running multiple cluster members as part of a test. Oracle Tools will use the default JVM settings unless told otherwise, these settings vary with O/S type, 64 or 32 bit and Java version. You can change these with the AddJvmOption method. You call this method for each JVM option you want to set, the options are normal JVM options, for example

  1.     OracleTools.Wrapper
  2.         .AddJvmOption("-Xms1024m")
  3.         .AddJvmOption("-Xms1024m")
  4.         .AddJvmOption("-XX:MaxPermSize=300m")
    OracleTools.Wrapper
        .AddJvmOption("-Xms1024m")
        .AddJvmOption("-Xms1024m")
        .AddJvmOption("-XX:MaxPermSize=300m")

The above options, for the standard Oracle JVM, set the minimum (-Xms) and maximum (-Xmx) heap memory to 1024m (1GByte) and set the maximum Perm Space to 300m (300 MBytes). Apologies if this is a bit basic if you are a Java developer but I am assuming that some people reading this are mostly .Net developers who may not be too familiar with JVM settings.

Other JVM Settings

There may be occasions when for some reason you need to pass other arguments to the embedded JVM. You can use the same AddJvmOption to pass any valid option to the JVM, just put one option in each call to AddJvmOption. Remember though that this embedded JVM is not going to run your application, it is only going to run the Java Oracle Tools which in turn will run your application code. The settings and classpath you configure do not need to be the same as your application.

Simple External Process Example

The first example is running an external non-Java forked process, which in this case will be the equivalent of cmd.exe -C dir which as you all know is just going to run the dir command in the current directory.

  1. using NUnit.Framework;
  2. using Oracle.Tools.Runtime;
  3. using Oracle.Tools.Runtime.Console;
  4.  
  5. namespace Oracle.Tools.Examples
  6. {
  7.     [TestFixture]
  8.     public class RunSomethingNonJavaTest
  9.     {
  10.         public const string LibFolder = "lib";
  11.  
  12.         [TestFixtureSetUp]
  13.         public void ClassInitialize()
  14.         {
  15.             if (!OracleTools.Wrapper.Initialised)
  16.             {
  17.                 OracleTools.Wrapper
  18.                     .AddJvmOption("-Xmx1024m")
  19.                     .AddJvmOption("-XX:MaxPermSize=300m")
  20.                     .AddEmbeddedJarsToClassPath(LibFolder)
  21.                     .Initialise();
  22.             }
  23.         }
  24.  
  25.         [Test]
  26.         public void ShouldRunSomethingNonJava()
  27.         {
  28.             SimpleApplicationSchema schema = new SimpleApplicationSchema("cmd.exe")
  29.                 .AddArgument("/C")
  30.                 .AddArgument("dir");
  31.  
  32.             IApplicationBuilder<SimpleApplication,SimpleApplicationSchema> builder = new SimpleApplicationBuilder();
  33.  
  34.             var application = builder.Realize(schema, "CMD", new SystemApplicationConsole());
  35.  
  36.             application.WaitFor();
  37.             var exitCode = application.Destroy();
  38.             Assert.AreEqual(0, exitCode);
  39.         }
  40.     }
  41. }
using NUnit.Framework;
using Oracle.Tools.Runtime;
using Oracle.Tools.Runtime.Console;

namespace Oracle.Tools.Examples
{
    [TestFixture]
    public class RunSomethingNonJavaTest
    {
        public const string LibFolder = "lib";

        [TestFixtureSetUp]
        public void ClassInitialize()
        {
            if (!OracleTools.Wrapper.Initialised)
            {
                OracleTools.Wrapper
                    .AddJvmOption("-Xmx1024m")
                    .AddJvmOption("-XX:MaxPermSize=300m")
                    .AddEmbeddedJarsToClassPath(LibFolder)
                    .Initialise();
            }
        }

        [Test]
        public void ShouldRunSomethingNonJava()
        {
            SimpleApplicationSchema schema = new SimpleApplicationSchema("cmd.exe")
                .AddArgument("/C")
                .AddArgument("dir");

            IApplicationBuilder<SimpleApplication,SimpleApplicationSchema> builder = new SimpleApplicationBuilder();

            var application = builder.Realize(schema, "CMD", new SystemApplicationConsole());

            application.WaitFor();
            var exitCode = application.Destroy();
            Assert.AreEqual(0, exitCode);
        }
    }
}

The first thing you can see is the ClassInitialize method at the top of the class. This method is annotated with [TestFixtureSetup] which means that NUnit should run this method once before any tests in this class are run. This method runs the oracle Tools initialisation code I covered in the previous section.
Note: The initialisation code above sets the maximum heap to 1GB and the Perm Gen size to 300MB. I am running this code in a 64bit Windows 7 session on VMWare Fusion with lots of memory on a 16GB Mac Book Pro. If you are trying to run it on certain lower spec Windows configurations – like the 32bit XP at my current client – then you might not be able to create a JVM with that much heap so you may need to drop it down to say -Xmx900m.

The [Test] annotated method ShouldRunSomethingNonJava is the code that will run our cmd.exe /C dir process.

  • First we create the instance of SimpleApplicationSchema which is the schema class for non-Java applications. Its constructor argument is the executable to run, in this case cmd.exe
  • Then we call the AddArgument method a couple of times on the schema to set the command line arguments, in this case /C and dir
  • Next we create a builder that will realize our process; here we create a SimpleApplicationBuilder which is the builder for creating simple forked processes

There are various methods on SimpleApplicationSchema to set other properties of the application, for example you could add the following test method to the code above just to prove that you can configure environemtn variables for the process.

  1. [Test]
  2. public void ShouldRunSomethingNonJavaWithEnvironment()
  3. {
  4.     SimpleApplicationSchema schema = new SimpleApplicationSchema("cmd.exe")
  5.         .SetEnvironmentVariable("DIR_NAME", "C:\\")
  6.         .AddArgument("/C")
  7.         .AddArgument("dir")
  8.         .AddArgument("%DIR_NAME%");
  9.  
  10.     IApplicationBuilder<SimpleApplication, SimpleApplicationSchema> builder = new SimpleApplicationBuilder();
  11.  
  12.     var application = builder.Realize(schema, "CMD", new SystemApplicationConsole());
  13.  
  14.     application.WaitFor();
  15.     var exitCode = application.Destroy();
  16.     Assert.AreEqual(0, exitCode);
  17. }
[Test]
public void ShouldRunSomethingNonJavaWithEnvironment()
{
    SimpleApplicationSchema schema = new SimpleApplicationSchema("cmd.exe")
        .SetEnvironmentVariable("DIR_NAME", "C:\\")
        .AddArgument("/C")
        .AddArgument("dir")
        .AddArgument("%DIR_NAME%");

    IApplicationBuilder<SimpleApplication, SimpleApplicationSchema> builder = new SimpleApplicationBuilder();

    var application = builder.Realize(schema, "CMD", new SystemApplicationConsole());

    application.WaitFor();
    var exitCode = application.Destroy();
    Assert.AreEqual(0, exitCode);
}

In the above example we use a system property to specify the name of the directory we want a listing of, which in the above test is C:\

You can use SimpleApplicationSchema and SimpleApplicationBuilder to configure and run any process at all, although there are already ways to do this in .Net. The power of Oracle Tools comes with running Java processes, in particular processes running inside the embedded JVM.

If you run the above test you will see in the console output the captured stdout of the dir command and the directory listing.

External Non-Coherence Java Process

This example code is going to run a simple externally forked Java process.

  1. using NUnit.Framework;
  2. using Oracle.Tools.Runtime.Console;
  3. using Oracle.Tools.Runtime.Java;
  4.  
  5. namespace Oracle.Tools.Examples
  6. {
  7.     [TestFixture]
  8.     public class RunExternalJavaProcessTest
  9.     {
  10.         public const string LibFolder = "lib";
  11.  
  12.         [TestFixtureSetUp]
  13.         public void ClassInitialize()
  14.         {
  15.             if (!OracleTools.Wrapper.Initialised)
  16.             {
  17.                 OracleTools.Wrapper
  18.                     .AddJvmOption("-Xmx1024m")
  19.                     .AddJvmOption("-XX:MaxPermSize=300m")
  20.                     .AddEmbeddedJarsToClassPath(LibFolder)
  21.                     .Initialise();
  22.             }
  23.         }
  24.  
  25.         [Test]
  26.         public void ShouldRunExternalJavaApplication()
  27.         {
  28.             const string classpath = LibFolder + "/oracle-tools-net-java-tests.jar";
  29.             const string classToRun = "com.oracle.tools.test.MyTestClass";
  30.  
  31.             var schema = new SimpleJavaApplicationSchema(classToRun, classpath)
  32.                 .AddArgument("Arg0")
  33.                 .AddArgument("Arg1");
  34.  
  35.             var builder = new ExternalJavaApplicationBuilder<SimpleJavaApplication, SimpleJavaApplicationSchema>();
  36.  
  37.             var application = builder.Realize(schema, "JavaTest", new SystemApplicationConsole());
  38.  
  39.             var exitCode = application.WaitFor();
  40.             application.Destroy();
  41.             Assert.AreEqual(0, exitCode);
  42.         }
  43.     }
  44. }
using NUnit.Framework;
using Oracle.Tools.Runtime.Console;
using Oracle.Tools.Runtime.Java;

namespace Oracle.Tools.Examples
{
    [TestFixture]
    public class RunExternalJavaProcessTest
    {
        public const string LibFolder = "lib";

        [TestFixtureSetUp]
        public void ClassInitialize()
        {
            if (!OracleTools.Wrapper.Initialised)
            {
                OracleTools.Wrapper
                    .AddJvmOption("-Xmx1024m")
                    .AddJvmOption("-XX:MaxPermSize=300m")
                    .AddEmbeddedJarsToClassPath(LibFolder)
                    .Initialise();
            }
        }

        [Test]
        public void ShouldRunExternalJavaApplication()
        {
            const string classpath = LibFolder + "/oracle-tools-net-java-tests.jar";
            const string classToRun = "com.oracle.tools.test.MyTestClass";

            var schema = new SimpleJavaApplicationSchema(classToRun, classpath)
                .AddArgument("Arg0")
                .AddArgument("Arg1");

            var builder = new ExternalJavaApplicationBuilder<SimpleJavaApplication, SimpleJavaApplicationSchema>();

            var application = builder.Realize(schema, "JavaTest", new SystemApplicationConsole());

            var exitCode = application.WaitFor();
            application.Destroy();
            Assert.AreEqual(0, exitCode);
        }
    }
}

You will see there is the same initialisation code, the main bit of the example is in the ShouldRunExternalJavaApplication method. In this example we are going to run a simple Java test class that I have included. All this class does is dumps to stdout the System properties and program arguments. Its main purpose is to prove that what is configured in the schema is what is passed to the process.

  • The first thing we need to do is set up the class path of our application being tested. In this case we only need the jar file containing the test class.
  • We then create a SimpleJavaApplicationSchema instance specifying the name of our class to run com.oracle.tools.test.MyTestClass and the classpath.
  • Using the schema we specify that we want to pass two arguments to the process, “Arg0″ and “Arg1″, we should see these dumped out in the process output. There are a number of other methods on SimpleJavaApplicationSchema that allow you to configure various Java specific properties of the application.
  • We next create our builder that will realize the running process. In this case we use an ExternalJavaApplicationBuilder as we want the process to be forked
  • Now we have a builder we can call Realize to get the application instance. We pass the schema to the builder’s Realize method along with the application name (in this case we call it “JavaTest” and we use a SystemApplicationConsole to capture the application output.
  • We then call WaitFor on the application to wait for it to terminate and once it has finished we assert that the exit code was zero.

This was a very simple example but the SimpleJavaApplicationSchema allows you to configure any Java application.

Virtualized Non-Coherence Java Process Example

We can run the exact same Java class again but this time as a virtual process contained withing the same .Net process that is running the .Net code.

  1.     using NUnit.Framework;
  2.     using Oracle.Tools.Runtime.Console;
  3.     using Oracle.Tools.Runtime.Java;
  4.  
  5.     namespace Oracle.Tools.Examples
  6.     {
  7.         [TestFixture]
  8.         public class RunVirtualizedJavaProcessTest
  9.         {
  10.             public const string LibFolder = "lib";
  11.  
  12.             [TestFixtureSetUp]
  13.             public void ClassInitialize()
  14.             {
  15.                 if (!OracleTools.Wrapper.Initialised)
  16.                 {
  17.                     OracleTools.Wrapper
  18.                         .AddJvmOption("-Xmx1024m")
  19.                         .AddJvmOption("-XX:MaxPermSize=300m")
  20.                         .AddEmbeddedJarsToClassPath(LibFolder)
  21.                         .Initialise();
  22.                 }
  23.             }
  24.  
  25.             [Test]
  26.             public void ShouldRunVirtualizedJavaApplication()
  27.             {
  28.                 const string classpath = LibFolder + "/oracle-tools-net-java-tests.jar";
  29.                 const string classToRun = "com.oracle.tools.test.MyTestClass";
  30.  
  31.                 var schema = new SimpleJavaApplicationSchema(classToRun, classpath)
  32.                     .AddArgument("Arg0")
  33.                     .AddArgument("Arg1");
  34.  
  35.                 var builder = new VirtualizedJavaApplicationBuilder<SimpleJavaApplication, SimpleJavaApplicationSchema>();
  36.  
  37.                 var application = builder.Realize(schema, "JavaTest", new SystemApplicationConsole());
  38.  
  39.                 var exitCode = application.WaitFor();
  40.                 Assert.AreEqual(0, exitCode);
  41.             }
  42.         }
  43.     }
    using NUnit.Framework;
    using Oracle.Tools.Runtime.Console;
    using Oracle.Tools.Runtime.Java;

    namespace Oracle.Tools.Examples
    {
        [TestFixture]
        public class RunVirtualizedJavaProcessTest
        {
            public const string LibFolder = "lib";

            [TestFixtureSetUp]
            public void ClassInitialize()
            {
                if (!OracleTools.Wrapper.Initialised)
                {
                    OracleTools.Wrapper
                        .AddJvmOption("-Xmx1024m")
                        .AddJvmOption("-XX:MaxPermSize=300m")
                        .AddEmbeddedJarsToClassPath(LibFolder)
                        .Initialise();
                }
            }

            [Test]
            public void ShouldRunVirtualizedJavaApplication()
            {
                const string classpath = LibFolder + "/oracle-tools-net-java-tests.jar";
                const string classToRun = "com.oracle.tools.test.MyTestClass";

                var schema = new SimpleJavaApplicationSchema(classToRun, classpath)
                    .AddArgument("Arg0")
                    .AddArgument("Arg1");

                var builder = new VirtualizedJavaApplicationBuilder<SimpleJavaApplication, SimpleJavaApplicationSchema>();

                var application = builder.Realize(schema, "JavaTest", new SystemApplicationConsole());

                var exitCode = application.WaitFor();
                Assert.AreEqual(0, exitCode);
            }
        }
    }

You can see that this code is exactly the same as the previous example except for the builder used to realize the application. This time on line 34 we use a VirtualizedJavaApplicationBuilder so that the application runs within the sandboxed JVM inside the .Net process.

External Coherence Process Example

We can use SimpleJavaApplicationSchema to configure any Java application, including Oracle Coherence, but Oracle Tools makes life easier for us by providing a schema specifically for building Coherence processes. This schema is ClusterMemberSchema and includes a number of methods that allow us to configure the application without having to remember all of the correct Java system property names.

  1. using NUnit.Framework;
  2. using Oracle.Tools.Runtime.Coherence;
  3. using Oracle.Tools.Runtime.Console;
  4. using Oracle.Tools.Runtime.Java;
  5. using Assert = Oracle.Tools.Testing.Assert;
  6.  
  7. namespace Oracle.Tools.Examples
  8. {
  9.     [TestFixture]
  10.     public class RunExternalCoherenceNodeTest
  11.     {
  12.         public const string LibFolder = "lib";
  13.  
  14.         [TestFixtureSetUp]
  15.         public void ClassInitialize()
  16.         {
  17.             if (!OracleTools.Wrapper.Initialised)
  18.             {
  19.                 OracleTools.Wrapper
  20.                     .AddEmbeddedJarsToClassPath(LibFolder)
  21.                     .Initialise();
  22.             }
  23.         }
  24.  
  25.         [Test]
  26.         public void ShouldRunCoherenceNode()
  27.         {
  28.             const string classpath = LibFolder + "/coherence.jar";
  29.             var schema = new ClusterMemberSchema(ClusterMemberSchema.DefaultCacheServerClassname, classpath)
  30.                                 .SetClusterName("TestCluster")
  31.                                 .SetSingleServerMode()
  32.                                 .SetStorageEnabled(true)
  33.                                 .SetJmxManagementMode(JmxManagementMode.All);
  34.  
  35.  
  36.             var builder = new ExternalJavaApplicationBuilder<ClusterMember, ClusterMemberSchema>();
  37.  
  38.             var clusterMember = builder.Realize(schema, "Storage", new SystemApplicationConsole());
  39.  
  40.             Assert.Eventually(() => clusterMember.ClusterSize, Is.EqualTo(1));
  41.  
  42.             clusterMember.Destroy();
  43.         }
  44.     }
  45. }
using NUnit.Framework;
using Oracle.Tools.Runtime.Coherence;
using Oracle.Tools.Runtime.Console;
using Oracle.Tools.Runtime.Java;
using Assert = Oracle.Tools.Testing.Assert;

namespace Oracle.Tools.Examples
{
    [TestFixture]
    public class RunExternalCoherenceNodeTest
    {
        public const string LibFolder = "lib";

        [TestFixtureSetUp]
        public void ClassInitialize()
        {
            if (!OracleTools.Wrapper.Initialised)
            {
                OracleTools.Wrapper
                    .AddEmbeddedJarsToClassPath(LibFolder)
                    .Initialise();
            }
        }

        [Test]
        public void ShouldRunCoherenceNode()
        {
            const string classpath = LibFolder + "/coherence.jar";
            var schema = new ClusterMemberSchema(ClusterMemberSchema.DefaultCacheServerClassname, classpath)
                                .SetClusterName("TestCluster")
                                .SetSingleServerMode()
                                .SetStorageEnabled(true)
                                .SetJmxManagementMode(JmxManagementMode.All);


            var builder = new ExternalJavaApplicationBuilder<ClusterMember, ClusterMemberSchema>();

            var clusterMember = builder.Realize(schema, "Storage", new SystemApplicationConsole());

            Assert.Eventually(() => clusterMember.ClusterSize, Is.EqualTo(1));

            clusterMember.Destroy();
        }
    }
}

The example above is very similar to the previous examples with the same basic steps, initialise, set up classpath, create schema, create builder, realize application, stop application.

  • The initialisation code is again the same as previous – in reality this code would go into a common base class for all of your tests.
  • For this test we only require the Coherence jar on the class path
  • We now use a ClusterMemberSchema to configure our Coherence node. The ClusterMemberSchema contains various methods to set the different parameters of the node. In this case we are keeping it all very basic but you should be able to see by looking at the available methods on ClusterMemberSchema what other functionality is available
  • . There is also documentation in the Java version’s Java Docs.

  • We use ExternalJavaApplicationBuilder to realise the running node in the same way we did previously and capture the output with a SystemApplicationConsole
  • The next part is different – we want to wait until the node has formed a cluster. In this case it will cluster with itself so there will only be a single cluster member. We do the check using a deferred assertion to wait until the result of calling clusterMember.ClusterSize is equal to 1
  • Finally after the cluster is running we destroy the process and wait for it to stop

Virtualized Coherence Process Example

As with running a virtual basic Java application in the example above we can run a virtual Coherence node in the same process as the .Net code by just changing the builder.

  1. using NUnit.Framework;
  2. using Oracle.Tools.Runtime.Coherence;
  3. using Oracle.Tools.Runtime.Console;
  4. using Oracle.Tools.Runtime.Java;
  5. using Oracle.Tools.Runtime.Network;
  6. using com.oracle.tools.runtime.java.virtualization;
  7. using Assert = Oracle.Tools.Testing.Assert;
  8.  
  9. namespace Oracle.Tools.Examples
  10. {
  11.     [TestFixture]
  12.     public class RunVirtualizedCoherenceNodeTest
  13.     {
  14.         public const string LibFolder = "lib";
  15.  
  16.         [TestFixtureSetUp]
  17.         public void ClassInitialize()
  18.         {
  19.             if (!OracleTools.Wrapper.Initialised)
  20.             {
  21.                 OracleTools.Wrapper
  22.                     .AddJvmOption("-Xmx1024m")
  23.                     .AddJvmOption("-XX:MaxPermSize=300m")
  24.                     .AddEmbeddedJarsToClassPath(LibFolder)
  25.                     .Initialise();
  26.             }
  27.         }
  28.  
  29.         [Test]
  30.         public void ShouldRunCoherenceNode()
  31.         {
  32.             const string classpath = LibFolder + "/coherence.jar";
  33.             AvailablePortEnumerator ports = new AvailablePortEnumerator(40000);
  34.  
  35.             var schema = new ClusterMemberSchema(ClusterMemberSchema.DefaultCacheServerClassname, classpath)
  36.                                 .SetClusterName("TestCluster")
  37.                                 .SetSingleServerMode()
  38.                                 .SetStorageEnabled(true)
  39.                                 .SetJmxManagementMode(JmxManagementMode.All)
  40.                                 .SetJmxPort(ports)
  41.                                 .SetRoleName("RunVirtualizedCoherenceNodeTest");
  42.  
  43.             var builder = new VirtualizedJavaApplicationBuilder<ClusterMember, ClusterMemberSchema>();
  44.  
  45.             var clusterMember = builder.Realize(schema, "Storage", new SystemApplicationConsole());
  46.  
  47.             Assert.Eventually(() => clusterMember.ClusterSize, Is.EqualTo(1));
  48.  
  49.             clusterMember.Destroy();
  50.         }
  51.  
  52.     }
  53. }
using NUnit.Framework;
using Oracle.Tools.Runtime.Coherence;
using Oracle.Tools.Runtime.Console;
using Oracle.Tools.Runtime.Java;
using Oracle.Tools.Runtime.Network;
using com.oracle.tools.runtime.java.virtualization;
using Assert = Oracle.Tools.Testing.Assert;

namespace Oracle.Tools.Examples
{
    [TestFixture]
    public class RunVirtualizedCoherenceNodeTest
    {
        public const string LibFolder = "lib";

        [TestFixtureSetUp]
        public void ClassInitialize()
        {
            if (!OracleTools.Wrapper.Initialised)
            {
                OracleTools.Wrapper
                    .AddJvmOption("-Xmx1024m")
                    .AddJvmOption("-XX:MaxPermSize=300m")
                    .AddEmbeddedJarsToClassPath(LibFolder)
                    .Initialise();
            }
        }

        [Test]
        public void ShouldRunCoherenceNode()
        {
            const string classpath = LibFolder + "/coherence.jar";
            AvailablePortEnumerator ports = new AvailablePortEnumerator(40000);

            var schema = new ClusterMemberSchema(ClusterMemberSchema.DefaultCacheServerClassname, classpath)
                                .SetClusterName("TestCluster")
                                .SetSingleServerMode()
                                .SetStorageEnabled(true)
                                .SetJmxManagementMode(JmxManagementMode.All)
                                .SetJmxPort(ports)
                                .SetRoleName("RunVirtualizedCoherenceNodeTest");

            var builder = new VirtualizedJavaApplicationBuilder<ClusterMember, ClusterMemberSchema>();

            var clusterMember = builder.Realize(schema, "Storage", new SystemApplicationConsole());

            Assert.Eventually(() => clusterMember.ClusterSize, Is.EqualTo(1));

            clusterMember.Destroy();
        }

    }
}

You can see the only change is the use of a VirtualizedJavaApplicationBuilder on lines 43 and 45 to realise the cluster member.

Coherence Cluster Example

As with the Java version of Oracle Tools, the .Net version allows you to manage a whole cluster of processes, as that is how we normally work with Coherence applications and how you will most likely want to test your Coherence .Net application. As we have already seen the only difference between external Java applications and applications contained within the .Net process is which builder you use I am just going to use the internal version in this example to run a whole Oracle Coherence cluster inside the .Net process.

  1. using NUnit.Framework;
  2. using Oracle.Tools.Runtime.Coherence;
  3. using Oracle.Tools.Runtime.Console;
  4. using Oracle.Tools.Runtime.Java;
  5. using Oracle.Tools.Runtime.Network;
  6. using com.oracle.tools.runtime.java.virtualization;
  7. using Assert = Oracle.Tools.Testing.Assert;
  8.  
  9. namespace Oracle.Tools.Examples
  10. {
  11.     [TestFixture]
  12.     public class RunVirtualizedCoherenceClusterTest
  13.     {
  14.         public const string LibFolder = "lib";
  15.  
  16.         [TestFixtureSetUp]
  17.         public void ClassInitialize()
  18.         {
  19.             if (!OracleTools.Wrapper.Initialised)
  20.             {
  21.                 OracleTools.Wrapper
  22.                     .AddJvmOption("-XX:MaxPermSize=300m")
  23.                     .AddEmbeddedJarsToClassPath(LibFolder)
  24.                     .Initialise();
  25.             }
  26.         }
  27.  
  28.         [Test]
  29.         public void ShouldRunVirtualizedCoherenceCluster()
  30.         {
  31.             var memberBuilder = new VirtualizedJavaApplicationBuilder<ClusterMember, ClusterMemberSchema>();
  32.  
  33.             var clusterBuilder = new ClusterBuilder();
  34.             clusterBuilder.AddBuilder(memberBuilder, CreateStorageNodeSchema(), "Data", 2);
  35.             clusterBuilder.AddBuilder(memberBuilder, CreateExtendProxySchema(), "Proxy", 1);
  36.  
  37.             var cluster = clusterBuilder.Realize(new SystemApplicationConsole());
  38.             var memberCount = cluster.Count;
  39.  
  40.             var clusterMember = cluster["Data-0"];
  41.             Assert.Eventually(() => clusterMember.ClusterSize, Is.EqualTo(memberCount));
  42.  
  43.             var proxyMember = cluster["Proxy-0"];
  44.             Assert.Eventually(() => proxyMember.IsServiceRunning("TcpProxyService"), Is.EqualTo(true));
  45.  
  46.             cluster.Destroy();
  47.         }
  48.  
  49.         private ClusterMemberSchema CreateStorageNodeSchema()
  50.         {
  51.             return CreateCommonSchema()
  52.                     .SetStorageEnabled(true)
  53.                     .SetRoleName("VirtualizedCoherenceClusterData");
  54.         }
  55.  
  56.         private ClusterMemberSchema CreateExtendProxySchema()
  57.         {
  58.             return CreateCommonSchema()
  59.                     .SetStorageEnabled(false)
  60.                     .SetSystemProperty("tangosol.coherence.extend.enabled", "true")
  61.                     .SetSystemProperty("tangosol.coherence.extend.port", new AvailablePortEnumerator(40000))
  62.                     .SetRoleName("VirtualizedCoherenceClusterProxy");
  63.         }
  64.  
  65.         private ClusterMemberSchema CreateCommonSchema()
  66.         {
  67.             const string classpath = LibFolder + "/coherence.jar";
  68.             return new ClusterMemberSchema(ClusterMemberSchema.DefaultCacheServerClassname, classpath)
  69.                         .SetClusterName("TestCluster")
  70.                         .SetSingleServerMode()
  71.                         .SetJmxManagementMode(JmxManagementMode.LocalOnly)
  72.                         .SetJmxPort(new AvailablePortEnumerator(40000));
  73.         }
  74.     }
  75. }
using NUnit.Framework;
using Oracle.Tools.Runtime.Coherence;
using Oracle.Tools.Runtime.Console;
using Oracle.Tools.Runtime.Java;
using Oracle.Tools.Runtime.Network;
using com.oracle.tools.runtime.java.virtualization;
using Assert = Oracle.Tools.Testing.Assert;

namespace Oracle.Tools.Examples
{
    [TestFixture]
    public class RunVirtualizedCoherenceClusterTest
    {
        public const string LibFolder = "lib";

        [TestFixtureSetUp]
        public void ClassInitialize()
        {
            if (!OracleTools.Wrapper.Initialised)
            {
                OracleTools.Wrapper
                    .AddJvmOption("-XX:MaxPermSize=300m")
                    .AddEmbeddedJarsToClassPath(LibFolder)
                    .Initialise();
            }
        }

        [Test]
        public void ShouldRunVirtualizedCoherenceCluster()
        {
            var memberBuilder = new VirtualizedJavaApplicationBuilder<ClusterMember, ClusterMemberSchema>();

            var clusterBuilder = new ClusterBuilder();
            clusterBuilder.AddBuilder(memberBuilder, CreateStorageNodeSchema(), "Data", 2);
            clusterBuilder.AddBuilder(memberBuilder, CreateExtendProxySchema(), "Proxy", 1);

            var cluster = clusterBuilder.Realize(new SystemApplicationConsole());
            var memberCount = cluster.Count;

            var clusterMember = cluster["Data-0"];
            Assert.Eventually(() => clusterMember.ClusterSize, Is.EqualTo(memberCount));

            var proxyMember = cluster["Proxy-0"];
            Assert.Eventually(() => proxyMember.IsServiceRunning("TcpProxyService"), Is.EqualTo(true));

            cluster.Destroy();
        }

        private ClusterMemberSchema CreateStorageNodeSchema()
        {
            return CreateCommonSchema()
                    .SetStorageEnabled(true)
                    .SetRoleName("VirtualizedCoherenceClusterData");
        }

        private ClusterMemberSchema CreateExtendProxySchema()
        {
            return CreateCommonSchema()
                    .SetStorageEnabled(false)
                    .SetSystemProperty("tangosol.coherence.extend.enabled", "true")
                    .SetSystemProperty("tangosol.coherence.extend.port", new AvailablePortEnumerator(40000))
                    .SetRoleName("VirtualizedCoherenceClusterProxy");
        }

        private ClusterMemberSchema CreateCommonSchema()
        {
            const string classpath = LibFolder + "/coherence.jar";
            return new ClusterMemberSchema(ClusterMemberSchema.DefaultCacheServerClassname, classpath)
                        .SetClusterName("TestCluster")
                        .SetSingleServerMode()
                        .SetJmxManagementMode(JmxManagementMode.LocalOnly)
                        .SetJmxPort(new AvailablePortEnumerator(40000));
        }
    }
}

This example is a little more involved but the basics are the same as the others; the main work gets done in the ShouldRunVirtualizedCoherenceCluster test method.

  • First we create a builder that will be used to build the individual members of our cluster; we are using the VirtualizedJavaApplicationBuilder to run the cluster internally.
  • Next we create our ClusterBuilder which will build all of the cluster members in one go. The ClusterBuilder is really just a utility that builds a Cluster which is a just a fancy container around a number of processes
  • Once we have the ClusterBuilder we tell it what to build by adding schema to it. First we add a storage node schema which we create in the CreateStorageNodeSchema() method. We tell the cluster builder to use the VirtualizedJavaApplicationBuilder to build the storage nodes, we give the nodes the name prefix “Data” and we tell the ClusterBuilder we want two instances of the storage node creating
  • Next we add the extend proxy to the ClusterBuilder. We tell the ClusterBuilder to use the same VirtualizedJavaApplicationBuilder, the schema created by the CreateExtendProxySchema() method, give the node the name prefix “Proxy” and tell the ClusterBuilder we want one instance creating.
  • We then call Realize on the ClusterBuilder to create our cluster; in this case two storage nodes and a storage disabled proxy node.

Once the ClusterBuilder returns the Cluster we know the processes have started but the cluster is not actually ready to do anything useful as it takes Coherence some time to form the cluster and start all the services in each node; we need to wait for everything to be ready.

We can get individual applications from the cluster container by name. The Cluster is an implementation of a .Net Dictionary so you can access members by their name. The name of the member will be the name prefix we used when we added the schema to the ClusterBuilder followed by an instance number. So in the above example we have two members with the prefix “Data” and one with the prefix “Proxy”. The instance numbers start at zero, so our member names will be Data-0 Data-1 and Proxy-0

  • We get the first storage node from the cluster and then do a deferred assertion on it to wait until the cluster has the correct size – in this case three. Once this happens we know all of the nodes have joined the cluster
  • As this is supposed to be a Coherence .Net client test we would be connecting over Coherence*Extend so we need to make sure the proxy node has started its Proxy Service. We do this with another deferred assertion that waits for the service with a specified name to start. As we are using the default Coherence cache configuration file from the Coherence jar this proxy service is called TcpProxyService

End to End NUnit Coherence .Net Example

So now you have seen how to start a cluster the final example is a full blown end-to-end test that starts the cluster and does some cache operations, so it is more typical of a test you would have in a .Net client application.

  1. using NUnit.Framework;
  2. using Oracle.Tools.Runtime.Coherence;
  3. using Oracle.Tools.Runtime.Console;
  4. using Oracle.Tools.Runtime.Java;
  5. using Oracle.Tools.Runtime.Network;
  6. using Tangosol.Net;
  7. using Tangosol.Util.Aggregator;
  8. using Tangosol.Util.Filter;
  9. using Assert = Oracle.Tools.Testing.Assert;
  10.  
  11. namespace Oracle.Tools.Examples
  12. {
  13.     [TestFixture]
  14.     public class EndToEndTest
  15.     {
  16.         public const string LibFolder = "lib";
  17.         private Cluster _cluster;
  18.  
  19.         [TestFixtureSetUp]
  20.         public void StartCluster()
  21.         {
  22.             if (!OracleTools.Wrapper.Initialised)
  23.             {
  24.                 OracleTools.Wrapper
  25.                     .AddJvmOption("-Xmx1024m")
  26.                     .AddJvmOption("-XX:MaxPermSize=300m")
  27.                     .AddEmbeddedJarsToClassPath(LibFolder)
  28.                     .Initialise();
  29.             }
  30.  
  31.             var memberBuilder = new VirtualizedJavaApplicationBuilder<ClusterMember, ClusterMemberSchema>();
  32.  
  33.             var clusterBuilder = new ClusterBuilder();
  34.             clusterBuilder.AddBuilder(memberBuilder, CreateStorageNodeSchema(), "Data", 2);
  35.             clusterBuilder.AddBuilder(memberBuilder, CreateExtendProxySchema(), "Proxy", 1);
  36.  
  37.             _cluster = clusterBuilder.Realize(new SystemApplicationConsole());
  38.         }
  39.  
  40.         [SetUp]
  41.         public void AssertThatClusterIsReady()
  42.         {
  43.             var memberCount = _cluster.Count;
  44.             var clusterMember = _cluster["Data-0"];
  45.             Assert.That(clusterMember, Is.Not.Null);
  46.             Assert.Eventually(() => clusterMember.ClusterSize, Is.EqualTo(memberCount));
  47.             var proxyMember = _cluster["Proxy-0"];
  48.             Assert.That(proxyMember, Is.Not.Null);
  49.             Assert.Eventually(() => proxyMember.IsServiceRunning("TcpProxyService"), Is.EqualTo(true));
  50.         }
  51.  
  52.         [Test]
  53.         public void ShouldInteractWithCluster()
  54.         {
  55.             var cache = CacheFactory.GetCache("dist-test");
  56.             Assert.That(cache.Count, Is.EqualTo(0));
  57.  
  58.             cache["Key-1"] = "Value-1";
  59.             Assert.That(cache.Count, Is.EqualTo(1));
  60.             Assert.That((string)cache["Key-1"], Is.EqualTo("Value-1"));
  61.  
  62.             cache["Key-2"] = "Value-2";
  63.             cache["Key-3"] = "Value-3";
  64.  
  65.             var count = (int)cache.Aggregate(AlwaysFilter.Instance, new Count());
  66.             Assert.That(count, Is.EqualTo(3));
  67.         }
  68.  
  69.         [TestFixtureTearDown]
  70.         public void StopCluster()
  71.         {
  72.             if (_cluster != null)
  73.             {
  74.                 _cluster.Destroy();
  75.             }
  76.         }
  77.  
  78.         private static ClusterMemberSchema CreateStorageNodeSchema()
  79.         {
  80.             return CreateCommonSchema()
  81.                     .SetStorageEnabled(true);
  82.         }
  83.  
  84.         private static ClusterMemberSchema CreateExtendProxySchema()
  85.         {
  86.             return CreateCommonSchema()
  87.                     .SetStorageEnabled(false)
  88.                     .SetSystemProperty("tangosol.coherence.extend.enabled", "true")
  89.                     .SetSystemProperty("tangosol.coherence.extend.port", "9099");
  90.         }
  91.  
  92.         private static ClusterMemberSchema CreateCommonSchema()
  93.         {
  94.             const string classpath = LibFolder + "/coherence.jar";
  95.             return new ClusterMemberSchema(ClusterMemberSchema.DefaultCacheServerClassname, classpath)
  96.                         .SetClusterName("TestCluster")
  97.                         .SetSingleServerMode()
  98.                         .SetJmxManagementMode(JmxManagementMode.LocalOnly)
  99.                         .SetJmxPort(new AvailablePortEnumerator(40000))
  100.                         .SetPofEnabled(true);
  101.         }
  102.     }
  103. }
using NUnit.Framework;
using Oracle.Tools.Runtime.Coherence;
using Oracle.Tools.Runtime.Console;
using Oracle.Tools.Runtime.Java;
using Oracle.Tools.Runtime.Network;
using Tangosol.Net;
using Tangosol.Util.Aggregator;
using Tangosol.Util.Filter;
using Assert = Oracle.Tools.Testing.Assert;

namespace Oracle.Tools.Examples
{
    [TestFixture]
    public class EndToEndTest
    {
        public const string LibFolder = "lib";
        private Cluster _cluster;

        [TestFixtureSetUp]
        public void StartCluster()
        {
            if (!OracleTools.Wrapper.Initialised)
            {
                OracleTools.Wrapper
                    .AddJvmOption("-Xmx1024m")
                    .AddJvmOption("-XX:MaxPermSize=300m")
                    .AddEmbeddedJarsToClassPath(LibFolder)
                    .Initialise();
            }

            var memberBuilder = new VirtualizedJavaApplicationBuilder<ClusterMember, ClusterMemberSchema>();

            var clusterBuilder = new ClusterBuilder();
            clusterBuilder.AddBuilder(memberBuilder, CreateStorageNodeSchema(), "Data", 2);
            clusterBuilder.AddBuilder(memberBuilder, CreateExtendProxySchema(), "Proxy", 1);

            _cluster = clusterBuilder.Realize(new SystemApplicationConsole());
        }

        [SetUp]
        public void AssertThatClusterIsReady()
        {
            var memberCount = _cluster.Count;
            var clusterMember = _cluster["Data-0"];
            Assert.That(clusterMember, Is.Not.Null);
            Assert.Eventually(() => clusterMember.ClusterSize, Is.EqualTo(memberCount));
            var proxyMember = _cluster["Proxy-0"];
            Assert.That(proxyMember, Is.Not.Null);
            Assert.Eventually(() => proxyMember.IsServiceRunning("TcpProxyService"), Is.EqualTo(true));
        }

        [Test]
        public void ShouldInteractWithCluster()
        {
            var cache = CacheFactory.GetCache("dist-test");
            Assert.That(cache.Count, Is.EqualTo(0));

            cache["Key-1"] = "Value-1";
            Assert.That(cache.Count, Is.EqualTo(1));
            Assert.That((string)cache["Key-1"], Is.EqualTo("Value-1"));

            cache["Key-2"] = "Value-2";
            cache["Key-3"] = "Value-3";

            var count = (int)cache.Aggregate(AlwaysFilter.Instance, new Count());
            Assert.That(count, Is.EqualTo(3));
        }

        [TestFixtureTearDown]
        public void StopCluster()
        {
            if (_cluster != null)
            {
                _cluster.Destroy();
            }
        }

        private static ClusterMemberSchema CreateStorageNodeSchema()
        {
            return CreateCommonSchema()
                    .SetStorageEnabled(true);
        }

        private static ClusterMemberSchema CreateExtendProxySchema()
        {
            return CreateCommonSchema()
                    .SetStorageEnabled(false)
                    .SetSystemProperty("tangosol.coherence.extend.enabled", "true")
                    .SetSystemProperty("tangosol.coherence.extend.port", "9099");
        }

        private static ClusterMemberSchema CreateCommonSchema()
        {
            const string classpath = LibFolder + "/coherence.jar";
            return new ClusterMemberSchema(ClusterMemberSchema.DefaultCacheServerClassname, classpath)
                        .SetClusterName("TestCluster")
                        .SetSingleServerMode()
                        .SetJmxManagementMode(JmxManagementMode.LocalOnly)
                        .SetJmxPort(new AvailablePortEnumerator(40000))
                        .SetPofEnabled(true);
        }
    }
}

This example is a little more like a real test class would be, although in a whole suite of test classes we would probably extract the initialisation and cluster start into a method in a test base class that runs once when the suite starts and we would destory the cluster when the suite ends. This is much better than starting the cluster for every test as your test suite will run much faster if you only build your cluster once for the suite.

We have a StartCluster() method annotated with [TestFixtureSetUp] that NUnit will run once before the anything else in the class is run. In this method we start our cluster in the same way we did in the previous example.

As mentioned previously we cannot proceed with the tests unless we know the cluster is ready. I have put this code into the AssertThatClusterIsReady() method annotated with [Setup] which NUnit will run before every test method so we know before each method that the cluster is still running. This code is pretty much the same as the code we used in the previous example to verify the cluster size and make sure the proxy service is running.

There is only a single [Test] method ShouldInteractWithCluster() in this example. It basically gets a cache, puts data into it and does some assertions. Very simple, but you get the idea and should be able to see how you can pretty much put any client side code in here to test your application.

We have a single method annotated with [TestFixtureTearDown] called DestroyCluster() that NUnit will call after all the tests in the class have run and as its name suggests it just calls Destroy on the cluster to stop all of the nodes.

The other methods in the class are the schema creation methods which should be pretty self explanitory.

Easy Coherence .Net Testing

So that is all of the examples. They have been pretty basic in terms of the applications they have run but you should get the general idea of how Oracle Tools .Net works and how to use it. There are a lot of methods on the various schema classes I have not covered but what they do should be pretty obvious from their names and you can look at the documentation in the Java Docs for the Oracle Tool Java version.

You can see from the examples, especially the last one, that it is very easy to start an Oracle Coherence cluster inside an embedded JVM inside the same .Net process that your test code is running in. The cluster is easy to configure from your unit test code and eveything is all in one place, which is better that trying to spawn a forked cluster. Under the covers it uses the Oracle Tools Java code which is a project run and maintained by the guys at Oracle so should be pretty stable and funtional.

A good place to start is by looking at the OracleTools.Example Visual Studio project in the Oracle Tools .Net source on GitHub as this will show you the dependencies your test project will require. It also has all of the above examples.

Deferred Assertions

In a few of the examples I have used deferred assertions to verify a condition is true within a certain time, which is very useful for something mostly asynchronous like Coherence. The Java version of Oracle Tools has this functionality and it is also in Oracle Tools .Net but it works in a different way. The Java version relies on the Java dynaimic proxy functionality to work and although there is a way in .Net to have proxies it is more limited than the Java equivalent and would not have worked. In the end I wrote something similar using .Net delegates.

In the code you can do something like this

  1. using Assert = Oracle.Tools.Testing.Assert;
  2.     ...
  3. Assert.Eventually(() => clusterMember.ClusterSize, Is.EqualTo(3));
using Assert = Oracle.Tools.Testing.Assert;
    ...
Assert.Eventually(() => clusterMember.ClusterSize, Is.EqualTo(3));

This code will run the lamda () => clusterMember.ClusterSize and assert the return value is equal to 3. If the assertion fails it will be retried until the timeout has passed, then the assertion will fail. The default timeout is 30 seconds but if you want a different timeout value you can do this

  1. Assert.Within(TimeSpan.FromSeconds(10)).That(() => clusterMember.ClusterSize, Is.EqualTo(3));
Assert.Within(TimeSpan.FromSeconds(10)).That(() => clusterMember.ClusterSize, Is.EqualTo(3));

The Assert.Within method takes a normal C# TimeSpan object (in the above example 10 seconds) and the That method then specifies the assertion that should be true within that time.

Building Oracle Tools .Net

Currently Oracle Tools .Net is not available as a pre-built package. If you want to use it you are free to clone my GitHub repository and build it yourself. The build process is pretty straightforward. So, step 1, clone the repo, or download the zip of the repo from https://github.com/thegridman/oracle-tools-net.

Maven

The build process uses Maven, which is a popular Java build tool and is used by the Java versions of Oracle Tools and the Coherence Incubator. If you are a hard core .Net debveloper who does not know Maven I assume you have at least one friendly Java developer on your team if your are building Oracle Coherence applications so this should be easy to get set up. You can get Maven here http://maven.apache.org If you already know Maven then how to build the project should be plain enough.

Once you have Maven installed you need to make sure you have a version of the Oracle Coherence jar file in your Maven repository. I am not providing one due to licensing, but I assume you have one already. As I write this Oracle Tools .Net is dependant on Coherence 3.7.1.7 although you can change the dependency by changing the property in the top level Maven pom.xml file for the Project.

Build Settings

The build will compile all of the C# code and run the NUnit tests but to do this it needs to know the location of various things. These are configured in the pom.xml file in the root folder of the project. If you look in the properties section you will see these lines

  1.     <dotnet.path>C:\WINDOWS\Microsoft.NET\Framework\v4.0.30319</dotnet.path>
  2.     <dotnet.devenv.path>C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE</dotnet.devenv.path>
  3.     <nunit.path>C:\Program Files (x86)\NUnit 2.6.2</nunit.path>
    <dotnet.path>C:\WINDOWS\Microsoft.NET\Framework\v4.0.30319</dotnet.path>
    <dotnet.devenv.path>C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE</dotnet.devenv.path>
    <nunit.path>C:\Program Files (x86)\NUnit 2.6.2</nunit.path>

The above properties are form my system which is Windows 7 64 bit with .Net v4 and Nunit. You may need to change them to match your own system. I know there are ways to automate this using Maven profiles but I have not found the time to do it yet.

Run the Build

Now everything is ready, open a command prompt in the top level folder of the project and run
mvn clean install
This will run the full build of the project, along with all of the tests. At the end, there will be various build artefacts in the different target sub-directories. The Oracle Tools library will be in a file oracle-tools-net\target\oracle-tools-net-csharp-0.0.1-SNAPSHOT.zip. All of the artefacts will also be in your local Maven repository.

Visual Studio Projects

There are a few Visual Studio solution files for the project if you want to do anything with the C# code. These are in the oracle-tools-net-csharp folder and oracle-tools-net-examples folder. I have Visual Studio 2012, so there are solution files for that version. There are some solution files for Visual Studio 2010 too but I cannot guarantee they are up to date. I am sure if you are a competent .Net developer you will be able to work out how to create a solution around the various C# project files.

The main code base is in the oracle-tools-net-csharp\OracleTools.sln file. If you open this solution you can see all of the code that makes up the .Net version of Oracle tools along with the unit tests. There are two project files in this solutiion, oracle-tools-net-csharp\OracleTools\OracleTools.csproj and oracle-tools-net-csharp\OracleTools.Test\OracleTools.Test.csproj

The usage examples code is in the oracle-tools-net-examples\OracleTools.Examples.sln file. If you open this you can see the examples from this blog post written as NUnit tests. There is only a single project file in this solution, oracle-tools-net-examples\OracleTools.Examples\OracleTools.Examples.csproj

Hopefully that is enough information to allow you to take the code and get it working in your own applications. If it does not quite fit what you need it is straight forward enough and your are obviously free to fork my repository and do whatever you want with the code.

Oracle Coherence Testing with Oracle Tools

oracle coherence tools

This blog post is going to cover the new Oracle Tools project hosted on the Coherence Comunity GitHub site and how to use this to help test your applications, specifically Java and Oracle Coherence applications. For anyone who has been using the Oracle Coherence Incubator you will already be familiar with some of this post and I have blogged about the Incubator runtime package in the past here.

There have been a lot of changes around the Oracle Coherence Incubator over the last year thanks to a lot of work by the guys at Oracle. The biggest positive is that everything is now properly open sourced on GitHub. You can take the code and do whatever you like with it but more importantly it makes keeping up to date with changes much easier and it makes contributing changes and fixes back to the project a lot easier too. If you look at the Coherence Comunity on GitHub you will see it is made up of a number of different repositories. In this post I am going to talk about just one of those, the Oracle Tools project. If you do not want to get the source you can just pull the jar files from the central Maven repository as the releases of Oracle Tools are all published to Maven under com.oracle.tools.

After some discussions a while back it was clear that there was a lot of code in the Oracle Coherence Incubator Commons runtime package that would be useful for development other applications that may have nothing to do with Oracle Coherence. The runtime package allowed you to control any external process or virtualised Java process but being part of the Incubator dragged in the dependency on Coherence. A decision was made to refactor all of this code out of the Incubator and into its own project and so Oracle Tools was born.

If you look at the project you will see that it is pretty much all of the code from the old Incubator runtime package refactored into various modules. On the subject of modules, all of the projects on GitHub are now proper Maven projects so there should be no more headaches trying to figure out how to build them. I’m not getting into the pros and cons of build tools, Maven was chosen as the build tool and that is the way it is.

So, on with the meat of the blog post…

The Oracle Tools Modules

Oracle Tools is made up of the following modules:

  • oracle-tools-core
  • oracle-tools-runtime
  • oracle-tools-testing-support
  • oracle-tools-runtime-tests
  • oracle-tools-coherence

The functionality provided by Oracle Tools right now is pretty much all geared around running “processes” and supporting easy testing of your applications and as you would expect specifically testing clustered applications made up of a number of processes. This means that typically to use Oracle Tools only your tests need a dependency on the Oracle Tools artefacts, your application code does not. I think this is quite an important point, I know a number of teams that are somehow scared of using the Incubator code in their application even though it is open source and after all, who does not already use open source code in their applications. But for those of a nervous disposition, you can add Oracle Tools as a test dependancy without impacting your production code. Another point regarding dependencies is that although the Oracle Tools Coherence module is built against the current version of Coherence and has a dependdency on it as far as I know it will work fine with older versions so you should be fine using Oracle Tools even if your project is on an older version of Coherence (or your project is like mine where you have clients who may take your client API and run it with old versions).

For Maven users these are the dependencies you want.

  1. <dependency>
  2.     <groupId>com.oracle.tools</groupId>
  3.     <artifactId>oracle-tools-core</artifactId>
  4.     <version>${oracle.tools.revision}</version>
  5. </dependency>
  6. <dependency>
  7.     <groupId>com.oracle.tools</groupId>
  8.     <artifactId>oracle-tools-runtime</artifactId>
  9.     <version>${oracle.tools.revision}</version>
  10. </dependency>
  11. <dependency>
  12.     <groupId>com.oracle.tools</groupId>
  13.     <artifactId>oracle-tools-testing-support</artifactId>
  14.     <version>${oracle.tools.revision}</version>
  15. </dependency>
  16. <dependency>
  17.     <groupId>com.oracle.tools</groupId>
  18.     <artifactId>oracle-tools-coherence</artifactId>
  19.     <version>${oracle.tools.revision}</version>
  20. </dependency>
<dependency>
    <groupId>com.oracle.tools</groupId>
    <artifactId>oracle-tools-core</artifactId>
    <version>${oracle.tools.revision}</version>
</dependency>
<dependency>
    <groupId>com.oracle.tools</groupId>
    <artifactId>oracle-tools-runtime</artifactId>
    <version>${oracle.tools.revision}</version>
</dependency>
<dependency>
    <groupId>com.oracle.tools</groupId>
    <artifactId>oracle-tools-testing-support</artifactId>
    <version>${oracle.tools.revision}</version>
</dependency>
<dependency>
    <groupId>com.oracle.tools</groupId>
    <artifactId>oracle-tools-coherence</artifactId>
    <version>${oracle.tools.revision}</version>
</dependency>

Where ${oracle.tools.revision} is whatever the latest version is.

Deferred – Waiting for conditions to be met

The main functionality in the oracle-tools-core module is the deferred package. This code provides a way to reference objects that may or may not yet exist but probably will at some point in the future. Thinking about Coherence, this could typically be something like an Extend Proxy Service that may not yet exist but at some point will and at some other future point will be started and listening for connections. Used in combination with the oracle-tools-test-support module that provides deferred assertions it becomes very easy to write tests cases based on waiting for something to happen.

Still keeping with the Coherence example, as that is what I mostly work with, you can imagine that using the deferred functionality can be useful in tests to make sure that the pre-conditions of the tests have been met, for example, wait until the cluster has started and has the correct number of members then wait until the proxy is running, once those conditions are met, then the tests can start. Now you could write that code yourself using various combinations of waits or sleeps, catching exceptions when services are not present etc, but that is all now nicely wrapped away.

Typically, waiting for a cluster to be started and have a specific number of nodes is as simple as this

  1. assertThat(eventually(invoking(cluster).getClusterSize()), is(3));
assertThat(eventually(invoking(cluster).getClusterSize()), is(3));

The code above will wait for a default time of 30 seconds for the condition to be met, otherwise an exception is thrown and the test would fail. If you want to wait for a different amount of time then you can pass other parameters, for example to wait for five minutes…

  1. assertThat(eventually(invoking(cluster).getClusterSize(), 5, TimeUnit.MINUTES), is(3));
assertThat(eventually(invoking(cluster).getClusterSize(), 5, TimeUnit.MINUTES), is(3));

As you can see, this makes waiting for conditions to be met in your tests very easy and can be used in a number of other situations, for example waiting for events, messages etc…

The deferred package contains a number of different classes that allow you to work with various types of deferred values. From a Java and Coherence perspective, one of the most useful is the deferred JMX functionality that allows you to obtain values from JMX for a process but instead of you having to write code to wait for the process to start, wait for the JMX server to start and wait for the MBean to be available, that is all taken care of. For example, to wait for the proxy service to start on the extend proxy cluster member you can write this in your test…

  1. Int nodeId = proxyMember.getLocalMemberId();
  2. ObjectName objectName = new ObjectName("Coherence:type=Service,name=ExtendTcpProxyService,nodeId=" + nodeId));
  3. assertThat(eventually(invoking(proxyMember).getMBeanAttribute(objectName, "Running", Boolean.class)), is(true));
Int nodeId = proxyMember.getLocalMemberId();
ObjectName objectName = new ObjectName("Coherence:type=Service,name=ExtendTcpProxyService,nodeId=" + nodeId));
assertThat(eventually(invoking(proxyMember).getMBeanAttribute(objectName, "Running", Boolean.class)), is(true));

You will see these assertions used in the examples below.

Runtime – Process Control

As its name suggests the oracle-tools-runtime module contains the code from the old Coherence incubator’s runtime package, in this case the non-Coherence specific code. As I have mentioned in a previous post the Runtime functionality is all about making it easy to configure and control proceses that you may need as part of your application testing. These processes can be Java or non-Java and can be externally forked processes, or if they are Java, they can be isolated, virtualized processes inside the same JVM as your application tests. The runtime package is similar to what I previously blogged about with a few package and class name changes and new functionality.

The runtime package is based on the concept of Schema that define a process and builders that take a schema and realize a running process. Oracle Tools comes with schema for various types of processes, Java, non-Java and Coherence and builders for external processes and virtualized java processes.

I will run through a few simple examples to show how easy this is to use.

Simple External Processes

To run a simple external process you will need to have a dependency on the jar files from the oracle-tool-core and oracle-tools-runtime modules; so no Coherence dependencies yet.

This is a very basic example of running an external executable. For this example I wanted to do something pretty generic that would work anywhere. I work at home on a Mac Book Pro, in the office on Windows and I know some of you work on various flavours of Linux desktop so I chose to run the Java executable. It is a bit pointless in terms of doing anything useful, but it shows the functionality.

  1. package com.thegridman.oracle.tools.examples;
  2.  
  3. import com.oracle.tools.runtime.SimpleApplication;
  4. import com.oracle.tools.runtime.SimpleApplicationBuilder;
  5. import com.oracle.tools.runtime.SimpleApplicationSchema;
  6. import com.oracle.tools.runtime.console.SystemApplicationConsole;
  7.  
  8. public class RunSomethingForked
  9. {
  10.     public static void main(String[] args) throws Exception
  11.     {
  12.         SimpleApplicationSchema schema =
  13.                 new SimpleApplicationSchema("java")
  14.                     .setArgument("-version");
  15.  
  16.         SimpleApplicationBuilder builder
  17.                 = new SimpleApplicationBuilder();
  18.  
  19.         SimpleApplication application
  20.                 = builder.realize(schema, "Java", new SystemApplicationConsole());
  21.  
  22.         int exitCode = application.waitFor();
  23.     }
  24. }
package com.thegridman.oracle.tools.examples;

import com.oracle.tools.runtime.SimpleApplication;
import com.oracle.tools.runtime.SimpleApplicationBuilder;
import com.oracle.tools.runtime.SimpleApplicationSchema;
import com.oracle.tools.runtime.console.SystemApplicationConsole;

public class RunSomethingForked
{
    public static void main(String[] args) throws Exception
    {
        SimpleApplicationSchema schema =
                new SimpleApplicationSchema("java")
                    .setArgument("-version");

        SimpleApplicationBuilder builder
                = new SimpleApplicationBuilder();

        SimpleApplication application
                = builder.realize(schema, "Java", new SystemApplicationConsole());

        int exitCode = application.waitFor();
    }
}

The interesting bit is lines 12 – 22.

  • On line 12 we create an instance of a SimpleApplicationSchema which defines the process we want to run. This is the most simple type of process that requires as a minimum just the name of the executable to run; in this case “java”. We then call setArgument to pass in the command line argument -version that will cause the Java executable to print out the version information then exit.
  • On line 16 we create an instance of the SimpleApplicationBuilder that will realize a running process from a given schema. As the name suggests SimpleApplicationBuilder can only build processes based on a SimpleApplicationSchema.
  • On line 19 we call realize on the SimpleApplicationBuilder to build a running instance of a process based on our SimpleApplicationSchema. There are various overloaded versions of the realize method. At a minimum you need to pass a SimpleApplicationSchema. You also have the option to pass a process name and finally the version we have used where we pass in a name and an instance of an ApplicationConsole, in this case a SystemApplicationConsole.
    An ApplicationConsole is used to capture the output of the process and send it somewhere useful. In this case the SystemApplicationConsole captures the process output and sends it to the stdout of our parent process. There are a few built in implementations for sending the output to stdout or stderr or just ignoring it but it is easy enough to add your own if you want to do anything special.
  • Finally on line 22 we wait for the process to exit and capture it’s exit code.

If we run the code above we see something like this displayed on the console (this image is from running inside IntelliJ)

Run Something Forked

This is the captured output of our process that has been formatted by the ApplicationConsole. The format of a line starts with [application-name:output-type:pid] where application-name is the name of the application that we passed to the builder’s realize method. The output-type is either out for the process stdout stream or err for the process stderr stream. Finally pid is the O/S PID of the process. This is then followed by a line number and colon. So [Java:err:37401] 1: is line 1 of the stderr stream of our process. We gave the process the name “Java” and the O/S assigned it a PID of 37401.
As you can see the first two err lines are typically what you would expect from running java -version. The two lines containing (terminated) are from the Oracle Tools framework to signify the end of the process output for that stream, so there will be one for stderr and one for stdout.

So what we have ended up with is basically the same as running java -version from a command line in our current working directory.

That is the most basic example of running a process. Using SimplaApplicationSchema we can configure the process executable, its command line arguments, working directory and environment variables. This is pretty much all you would need to run any externally forked process, after all it is no less than you get with Java’s own ProcessBuilder. Oracle Tools though has some specialisations of the ApplicationSchema that make it easier to configure Java applications and Coherence applications.

Non-Coherence Java Processes

To run an external non-Coherence java process you will need to have a dependency on the jar files from the oracle-tool-core and oracle-tools-runtime modules; so still no Coherence dependencies yet.

When working with Java processes you now have two options on how to run them. You can run them externally, i.e. as a forked process as we did above; or you can run them in a virtualised sandbox inside the same process. Running internally has advantages, especially when testing, in that you are not left with various forked processes still running if your parent process dies.

To configure a Java process instead of SimpleApplicationSchema we can use SimpleJavaApplicationSchema which has a number of useful methods to configure settings specific to Java processes. At a minimum we need to create an instance of SimpleJavaApplicationSchema and specify a class name to run. We then have the option to specify a whole host of other options such as class path, system properties, JVM options and JMX configuration.

External Java Process

First we will run an example of a simple Java class. As with our first example above I wanted to run something that everyone will have on their system without me having to use a custom class. In this case given my site is pretty much all about Oracle Coherence I assume you have a Coherence jar file somewhere so I am going to run com.tangosol.net.DefaultCacheServer. I know this is a Coherence class and this section of the blog is supposed to be running a non-Coherence Java process, so we will have to pretend for a while.

  1. package com.thegridman.oracle.tools.examples;
  2.  
  3. import com.oracle.tools.runtime.console.SystemApplicationConsole;
  4. import com.oracle.tools.runtime.java.ExternalJavaApplicationBuilder;
  5. import com.oracle.tools.runtime.java.JavaApplication;
  6. import com.oracle.tools.runtime.java.SimpleJavaApplicationSchema;
  7.  
  8. import java.io.File;
  9.  
  10. public class RunJavaClassForked
  11. {
  12.     public static void main(String[] args) throws Exception
  13.     {
  14.         String classpath = "/m2/repository/com/oracle/coherence/coherence/3.7.1.7/coherence-3.7.1.7.jar";
  15.  
  16.         SimpleJavaApplicationSchema schema =
  17.                   new SimpleJavaApplicationSchema("com.tangosol.net.DefaultCacheServer", classpath)
  18.                     .setArgument("coherence-cache-config.xml")
  19.                     .setOption("-Xmx256m")
  20.                     .setSystemProperty("tangosol.coherence.clusterport", "12345")
  21.                     .setWorkingDirectory(new File("/Users/jonathanknight"));
  22.  
  23.         ExternalJavaApplicationBuilder builder =
  24.                 new ExternalJavaApplicationBuilder();
  25.  
  26.         JavaApplication application =
  27.                 builder.realize(schema, "DCS", new SystemApplicationConsole());
  28.  
  29.         Thread.sleep(10000);
  30.         application.destroy();
  31.     }
  32. }
package com.thegridman.oracle.tools.examples;

import com.oracle.tools.runtime.console.SystemApplicationConsole;
import com.oracle.tools.runtime.java.ExternalJavaApplicationBuilder;
import com.oracle.tools.runtime.java.JavaApplication;
import com.oracle.tools.runtime.java.SimpleJavaApplicationSchema;

import java.io.File;

public class RunJavaClassForked
{
    public static void main(String[] args) throws Exception
    {
        String classpath = "/m2/repository/com/oracle/coherence/coherence/3.7.1.7/coherence-3.7.1.7.jar";

        SimpleJavaApplicationSchema schema =
                  new SimpleJavaApplicationSchema("com.tangosol.net.DefaultCacheServer", classpath)
                    .setArgument("coherence-cache-config.xml")
                    .setOption("-Xmx256m")
                    .setSystemProperty("tangosol.coherence.clusterport", "12345")
                    .setWorkingDirectory(new File("/Users/jonathanknight"));

        ExternalJavaApplicationBuilder builder =
                new ExternalJavaApplicationBuilder();

        JavaApplication application =
                builder.realize(schema, "DCS", new SystemApplicationConsole());

        Thread.sleep(10000);
        application.destroy();
    }
}

As with the previous example the interesting bit is lines 14 – 30.

  • The first thing we need to do on lines 14 is work out what the class path will be for running our class. In this case we need a coherence.jar on the class path and on my Mac Book Pro this lives in a Maven repository.
  • On line 16 and 17 we create our SimpleJavaApplicationSchema that configures the java application, we specify the class to run, in this case com.tangosol.net.DefaultCacheServer and the class path
    At this point we have nothing else you need to configure to run DefaultCacheServer, as it will happily start up with no more configuration at all but that would not be much of an example so we will set some other options
  • On line 18 we set an argument that will be passed to the class’s main method – in this case coherence-cache-config.xml as DefaultCacheServer allows you to pass the name of the cache configuration file as the first argument..
    On line 19 we set a JVM option to set the maximum heap size to 256 MB.
    On line 20 we set a the multi-cast cluster port using the relevant system property
    On line 21 we set the working directory to my home directory.
  • On line 23 we create our application builder, in this case we want to run a forked external Java process so we create an instance of ExternalJavaApplicationBuilder
  • On line 26 we realize a running instance of our JConsole application, which we have named “DCS” and we are using a SystemApplicationConsole to capture the output
  • On line 29 we sleep for 10 seconds and on line 30 we destroy (kill) the process (there is no real need to sleep, but this example would run so fast otherwise the process would be killed before anything was output). This is different to the previous example where we waited for the application to finish by calling waitFor(). DefaultCacheServer is a server application that does not end until we somehow shut it down it. If we used waitFor() as in the previous example then this code would block until we shut down the DefaultCacheServer. The code we have here will force the DefaultCacheServer to close after 10 seconds.

So if you run the code above you will see in the console output DefaultCacheServer start then after 10 seconds die as the process is killed.
When running an external Java process like the example above the class being run must have a public static void main(String[] args) method as with any Java class being run from the command line. In effect what we have just run above would equate to a command line of…
java -Xmx256m -tangosol.coherence.clusterport=12345 -cp=/m2/repository/com/oracle/coherence/coherence/3.7.1.7/coherence-3.7.1.7.jar com.tangosol.net.DefaultCacheServer coherence-cache-config.xml

While DefaultCacheServer is running, if you look at the processes running on your machine you should see an new Java process that is DefaultCacheServer and the PID should match the PID in the console output.

You can see that when we relized the process we recieved an instance of JavaApplication instead of the SimpleApplication in the previous examples. The JavaApplication class has a number of Java specific methods on it.

  • You can obtain the System properties that were used to start the process. These are just the properties used at start up and do not reflect any changes that might be made to the process system properties while it is running.
  • You can also interact with the process JMX server and perform JMX queries. This can be useful for verifying application state if the process you are running supports JMX
  • Internal Java Process

    A feature of Oracle Tools is that it allows you to run Java processes in a virtualized sandbox inside the parent process. This sandbox isolates the applications class path, system properties and JMX server from the parent so for most purposes it can be treated like a separate application.

    There are a few differences to be aware of between external and virtualized processes.

    • You are not restricted to using the usual Java main method to start the virtualized process, you can use any method with the caveat that the method must not block.
    • It is not possible for the destroy method to kill the process in the same way that it does with an external process. To stop the virtualized process you can specify a stop method that will be called on the class to programatically shut it down.

    As an example we will run the same DefaultCacheServer again but this time as an internal virtualized process.

    1. package com.thegridman.oracle.tools.examples;
    2.  
    3. import com.oracle.tools.runtime.console.SystemApplicationConsole;
    4. import com.oracle.tools.runtime.java.JavaApplication;
    5. import com.oracle.tools.runtime.java.SimpleJavaApplicationSchema;
    6. import com.oracle.tools.runtime.java.VirtualizedJavaApplicationBuilder;
    7.  
    8. import java.io.File;
    9.  
    10. public class RunJavaClassInProcess
    11. {
    12.     public static void main(String[] args) throws Exception
    13.     {
    14.         String classpath = "/m2/repository/com/oracle/coherence/coherence/3.7.1.7/coherence-3.7.1.7.jar";
    15.  
    16.         SimpleJavaApplicationSchema schema =
    17.                   new SimpleJavaApplicationSchema("com.tangosol.net.DefaultCacheServer", classpath)
    18.                       .setStartMethodName("startDaemon")
    19.                       .setStopMethodName("shutdown")
    20.                       .setSystemProperty("tangosol.coherence.cacheconfig", "coherence-cache-config.xml")
    21.                       .setOption("-Xmx256m")
    22.                       .setSystemProperty("tangosol.coherence.clusterport", "12345")
    23.                       .setWorkingDirectory(new File("/Users/jonathanknight"));
    24.  
    25.         VirtualizedJavaApplicationBuilder builder =
    26.                 new VirtualizedJavaApplicationBuilder();
    27.  
    28.         JavaApplication application =
    29.                 builder.realize(schema, "DCS", new SystemApplicationConsole());
    30.  
    31.         Thread.sleep(10000);
    32.         application.destroy();
    33.     }
    34. }
    package com.thegridman.oracle.tools.examples;
    
    import com.oracle.tools.runtime.console.SystemApplicationConsole;
    import com.oracle.tools.runtime.java.JavaApplication;
    import com.oracle.tools.runtime.java.SimpleJavaApplicationSchema;
    import com.oracle.tools.runtime.java.VirtualizedJavaApplicationBuilder;
    
    import java.io.File;
    
    public class RunJavaClassInProcess
    {
        public static void main(String[] args) throws Exception
        {
            String classpath = "/m2/repository/com/oracle/coherence/coherence/3.7.1.7/coherence-3.7.1.7.jar";
    
            SimpleJavaApplicationSchema schema =
                      new SimpleJavaApplicationSchema("com.tangosol.net.DefaultCacheServer", classpath)
                          .setStartMethodName("startDaemon")
                          .setStopMethodName("shutdown")
                          .setSystemProperty("tangosol.coherence.cacheconfig", "coherence-cache-config.xml")
                          .setOption("-Xmx256m")
                          .setSystemProperty("tangosol.coherence.clusterport", "12345")
                          .setWorkingDirectory(new File("/Users/jonathanknight"));
    
            VirtualizedJavaApplicationBuilder builder =
                    new VirtualizedJavaApplicationBuilder();
    
            JavaApplication application =
                    builder.realize(schema, "DCS", new SystemApplicationConsole());
    
            Thread.sleep(10000);
            application.destroy();
        }
    }

    You can see that the code is almost identical

  • On line 18 we call setStartMethodName to specify the name of the method to call to start the process. In this case we do not call the DefaultCacheServer.main method as this is a blocking method that will not return, so instead we use DefaultCacheServer.startDaemon
  • On line 19 we call setStopMethodName to set the method to call to stop the process, in this case we call DefaultCacheServer.shutdown.
  • On line 20 we have change how we set the cache configuration file. The DefaultCacheServer.startDaemon does not take any arguments so we cannot use the setArgument method so instead we set the relevant system property.
  • Line 25 is where we now create a VirtualizedJavaApplicationBuilder to build our application as an internal virtualized process instead of external as in the previous example.
  • The rest of the code is identical to the previous example. If we run the code you can see exactly the same output in the console as previously with the exception that there is no PID shown at the start of each line. This is because there is no external process and if you look at the processes running on your machine while the code runs you will see not other processes are spawned.

    Coherence Processes

    As we saw above it is possible to start a Coherence cache server by treating it as just another Java class, which it obviously is, but there is an easier way and more functionaly rich way if we use the Coherence specific parts of Oracle Tools. To run a Coherence process you will need to have a dependency on the jar files from the oracle-tool-core, oracle-tools-runtime and oracle-tools-coherence modules as well as providing a coherence.jar file. As Coherence applications are Java processes you have the same options regarding whether you run externally or in a virtualized process. Oracle Tools also provides a cluster builder that allows you to control a whole cluster of processes at once.

    To create Coherence cluster member processes instead of using SimpleJavaApplicationSchema we use ClusterMemberSchema. This class has all of the same functionality as the previous two schema classes we have looked at, so you can set all of the normal process stuff, and Java specific properties, but it also has a host of Coherence specific settings too. For example, instead of setting the cache configuration using the setSystemProperty method as we did above, and having to remember to type the correct property name, we can just use ClusterMemberSchema.setCacheConfigURI instead.

    So, on with a few examples…

    External Coherence Process

    In this example we will run a single cache server process, the same as the previous example but using ClusterMemberSchema.

    1. package com.thegridman.oracle.tools.examples;
    2.  
    3. import com.oracle.tools.runtime.coherence.ClusterMember;
    4. import com.oracle.tools.runtime.coherence.ClusterMemberSchema;
    5. import com.oracle.tools.runtime.console.SystemApplicationConsole;
    6. import com.oracle.tools.runtime.java.ExternalJavaApplicationBuilder;
    7.  
    8. import java.io.File;
    9.  
    10. import static com.oracle.tools.deferred.DeferredAssert.assertThat;
    11. import static com.oracle.tools.deferred.DeferredHelper.eventually;
    12. import static com.oracle.tools.deferred.DeferredHelper.invoking;
    13. import static org.hamcrest.CoreMatchers.is;
    14.  
    15. public class RunCacheServerForked
    16. {
    17.     public static void main(String[] args) throws Exception
    18.     {
    19.         String classpath = "/m2/repository/com/oracle/coherence/coherence/3.7.1.7/coherence-3.7.1.7.jar";
    20.  
    21.         ClusterMemberSchema schema = new ClusterMemberSchema("com.tangosol.net.DefaultCacheServer", classpath)
    22.                 .setCacheConfigURI("coherence-cache-config.xml")
    23.                 .setPofEnabled(true)
    24.                 .setPofConfigURI("pof-config.xml")
    25.                 .setClusterPort(12345)
    26.                 .setStorageEnabled(true)
    27.                 .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
    28.                 .setOption("-Xmx256m")
    29.                 .setWorkingDirectory(new File("/Users/jonathanknight"));
    30.  
    31.         ExternalJavaApplicationBuilder<ClusterMember, ClusterMemberSchema> builder =
    32.                     new ExternalJavaApplicationBuilder<ClusterMember, ClusterMemberSchema>();
    33.  
    34.         ClusterMember member = builder.realize(schema, "Data", new SystemApplicationConsole());
    35.  
    36.         assertThat(eventually(invoking(member).getClusterSize()), is(1));
    37.  
    38.         Thread.sleep(10000);
    39.         member.destroy();
    40.     }
    41. }
    package com.thegridman.oracle.tools.examples;
    
    import com.oracle.tools.runtime.coherence.ClusterMember;
    import com.oracle.tools.runtime.coherence.ClusterMemberSchema;
    import com.oracle.tools.runtime.console.SystemApplicationConsole;
    import com.oracle.tools.runtime.java.ExternalJavaApplicationBuilder;
    
    import java.io.File;
    
    import static com.oracle.tools.deferred.DeferredAssert.assertThat;
    import static com.oracle.tools.deferred.DeferredHelper.eventually;
    import static com.oracle.tools.deferred.DeferredHelper.invoking;
    import static org.hamcrest.CoreMatchers.is;
    
    public class RunCacheServerForked
    {
        public static void main(String[] args) throws Exception
        {
            String classpath = "/m2/repository/com/oracle/coherence/coherence/3.7.1.7/coherence-3.7.1.7.jar";
    
            ClusterMemberSchema schema = new ClusterMemberSchema("com.tangosol.net.DefaultCacheServer", classpath)
                    .setCacheConfigURI("coherence-cache-config.xml")
                    .setPofEnabled(true)
                    .setPofConfigURI("pof-config.xml")
                    .setClusterPort(12345)
                    .setStorageEnabled(true)
                    .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
                    .setOption("-Xmx256m")
                    .setWorkingDirectory(new File("/Users/jonathanknight"));
    
            ExternalJavaApplicationBuilder<ClusterMember, ClusterMemberSchema> builder =
                        new ExternalJavaApplicationBuilder<ClusterMember, ClusterMemberSchema>();
    
            ClusterMember member = builder.realize(schema, "Data", new SystemApplicationConsole());
    
            assertThat(eventually(invoking(member).getClusterSize()), is(1));
    
            Thread.sleep(10000);
            member.destroy();
        }
    }

    So, very similar to our previous examples, we create a schema and use a builder to realize the running process. This time though things are a bit more Coherence specific.

  • In lines 21 – 29 we set up our ClusterMemberSchema with the configuration we require. You can see that we have used various custom methods to set things like the cache configuration, enable POF, set the POF configuration and set the JMX mode. This is much easier than using properties.
  • In line 31 we create the same ExternalJavaApplicationBuilder that we used previously to create an externally forked process.
  • In line 34 we realize our process using the builder, we name it “Data” and use a SystemApplicationConsole to capture the output. You will see that the process created is an instance of ClusterMember which has other useful methods on.
  • In line 36 we use one of these methods ClusterMember.getClusterSize() to tell us the size of the cluster (we expect it to eventually be one once the process starts up). We use a deferred assertion, which I covered above, to basically assert that the process starts properly within a reasonable time.
  • Finally on lines 38 and 39 we wait for 10 seconds then kill the process (again we don’t really need to sleep, and would in a normal test, but this allows the exmaple time to do something before it is killed).


  • When we realize the Coherence process the builder returns an instance of ClusterMember which is a Coherence specific version of the previous JavaApplication. We can still get all of the same functionality as the JavaProcess but we also get some extra Coherence methods.

  • We can get the cluster size, in our example this was one, but if running more nodes we would get a different value. The getClusterSize() method uses JMX to communicate the the JMX server of the process and interrogate the Coherence Cluster MBean. Obviously the MBean Server may take a while to start along with the process so that is why we checked the value using a deferred assertion.
  • You can obtain the local member ID of the process. This is the id that this node has been assigned in the cluster and again is obtained via a JMX call. This value is then useful when constructing ObjectName values to query other Coherence MBeans for this process.
  • Internal Coherence Process

    Running our Coherence process as a virtualized process is nothing more than change the builder used.

    1. package com.thegridman.oracle.tools.examples;
    2.  
    3. import com.oracle.tools.runtime.coherence.ClusterMember;
    4. import com.oracle.tools.runtime.coherence.ClusterMemberSchema;
    5. import com.oracle.tools.runtime.console.SystemApplicationConsole;
    6. import com.oracle.tools.runtime.java.VirtualizedJavaApplicationBuilder;
    7.  
    8. import java.io.File;
    9.  
    10. import static com.oracle.tools.deferred.DeferredAssert.assertThat;
    11. import static com.oracle.tools.deferred.DeferredHelper.eventually;
    12. import static com.oracle.tools.deferred.DeferredHelper.invoking;
    13. import static org.hamcrest.CoreMatchers.is;
    14.  
    15. public class RunCacheServerInProcess
    16. {
    17.     public static void main(String[] args) throws Exception
    18.     {
    19.         String classpath = "/m2/repository/com/oracle/coherence/coherence/3.7.1.7/coherence-3.7.1.7.jar";
    20.  
    21.         ClusterMemberSchema schema = new ClusterMemberSchema("com.tangosol.net.DefaultCacheServer", classpath)
    22.                 .setCacheConfigURI("coherence-cache-config.xml")
    23.                 .setPofEnabled(true)
    24.                 .setPofConfigURI("pof-config.xml")
    25.                 .setClusterPort(12345)
    26.                 .setStorageEnabled(true)
    27.                 .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
    28.                 .setOption("-Xmx256m")
    29.                 .setWorkingDirectory(new File("/Users/jonathanknight"));
    30.  
    31.         VirtualizedJavaApplicationBuilder<ClusterMember, ClusterMemberSchema> builder =
    32.                     new VirtualizedJavaApplicationBuilder<ClusterMember, ClusterMemberSchema>();
    33.  
    34.         ClusterMember member = builder.realize(schema, "Data", new SystemApplicationConsole());
    35.  
    36.         assertThat(eventually(invoking(member).getClusterSize()), is(1));
    37.  
    38.         Thread.sleep(10000);
    39.         member.destroy();
    40.     }
    41. }
    package com.thegridman.oracle.tools.examples;
    
    import com.oracle.tools.runtime.coherence.ClusterMember;
    import com.oracle.tools.runtime.coherence.ClusterMemberSchema;
    import com.oracle.tools.runtime.console.SystemApplicationConsole;
    import com.oracle.tools.runtime.java.VirtualizedJavaApplicationBuilder;
    
    import java.io.File;
    
    import static com.oracle.tools.deferred.DeferredAssert.assertThat;
    import static com.oracle.tools.deferred.DeferredHelper.eventually;
    import static com.oracle.tools.deferred.DeferredHelper.invoking;
    import static org.hamcrest.CoreMatchers.is;
    
    public class RunCacheServerInProcess
    {
        public static void main(String[] args) throws Exception
        {
            String classpath = "/m2/repository/com/oracle/coherence/coherence/3.7.1.7/coherence-3.7.1.7.jar";
    
            ClusterMemberSchema schema = new ClusterMemberSchema("com.tangosol.net.DefaultCacheServer", classpath)
                    .setCacheConfigURI("coherence-cache-config.xml")
                    .setPofEnabled(true)
                    .setPofConfigURI("pof-config.xml")
                    .setClusterPort(12345)
                    .setStorageEnabled(true)
                    .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
                    .setOption("-Xmx256m")
                    .setWorkingDirectory(new File("/Users/jonathanknight"));
    
            VirtualizedJavaApplicationBuilder<ClusterMember, ClusterMemberSchema> builder =
                        new VirtualizedJavaApplicationBuilder<ClusterMember, ClusterMemberSchema>();
    
            ClusterMember member = builder.realize(schema, "Data", new SystemApplicationConsole());
    
            assertThat(eventually(invoking(member).getClusterSize()), is(1));
    
            Thread.sleep(10000);
            member.destroy();
        }
    }

    You can see the code above is identical to the previous example except that it uses a VirtualizedJavaApplicationBuilder and runs the Coherence node as a virtualized process. We still have all the same functioanlity as the previous external process.

    External Coherence Cluster

    Typically when working with and testing Coherence application we are not working with a single process but a whole cluster. It is uaual in an accepance or integration test to run a couple of storage nodes and maybe, if you require it, an extend proxy node. This is where Oracle Tools makes it easier by allowing you to build and control a whole cluster together.

    1. package com.thegridman.oracle.tools.examples;
    2.  
    3. import com.oracle.tools.deferred.DeferredAssert;
    4. import com.oracle.tools.runtime.coherence.Cluster;
    5. import com.oracle.tools.runtime.coherence.ClusterBuilder;
    6. import com.oracle.tools.runtime.coherence.ClusterMember;
    7. import com.oracle.tools.runtime.coherence.ClusterMemberSchema;
    8. import com.oracle.tools.runtime.console.SystemApplicationConsole;
    9. import com.oracle.tools.runtime.java.ExternalJavaApplicationBuilder;
    10. import com.oracle.tools.runtime.network.AvailablePortIterator;
    11.  
    12. import java.io.File;
    13.  
    14. import static com.oracle.tools.deferred.DeferredHelper.eventually;
    15. import static com.oracle.tools.deferred.DeferredHelper.invoking;
    16. import static org.hamcrest.CoreMatchers.is;
    17.  
    18. public class RunClusterForked
    19. {
    20.     public static void main(String[] args) throws Exception
    21.     {
    22.         AvailablePortIterator ports = new AvailablePortIterator(40000);
    23.  
    24.         ClusterMemberSchema storage
    25.             = new ClusterMemberSchema()
    26.                     .setCacheConfigURI("coherence-cache-config.xml")
    27.                     .setPofEnabled(true)
    28.                     .setPofConfigURI("pof-config.xml")
    29.                     .setClusterPort(12345)
    30.                     .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
    31.                     .setJMXPort(ports)
    32.                     .setOption("-Xmx256m")
    33.                     .setWorkingDirectory(new File("/Users/jonathanknight"))
    34.                     .setStorageEnabled(true);
    35.  
    36.         ClusterMemberSchema extend
    37.             = new ClusterMemberSchema()
    38.                     .setCacheConfigURI("coherence-cache-config.xml")
    39.                     .setPofEnabled(true)
    40.                     .setPofConfigURI("pof-config.xml")
    41.                     .setClusterPort(12345)
    42.                     .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
    43.                     .setJMXPort(ports)
    44.                     .setOption("-Xmx256m")
    45.                     .setWorkingDirectory(new File("/Users/jonathanknight"))
    46.                     .setStorageEnabled(false)
    47.                     .setSystemProperty("tangosol.coherence.extend.enabled", true)
    48.                     .setSystemProperty("tangosol.coherence.extend.port", ports);
    49.  
    50.         ExternalJavaApplicationBuilder<ClusterMember,ClusterMemberSchema> builder
    51.                 = new ExternalJavaApplicationBuilder<ClusterMember,ClusterMemberSchema>();
    52.  
    53.         ClusterBuilder clusterBuilder = new ClusterBuilder();
    54.         clusterBuilder.addBuilder(builder, storage, "Data", 2);
    55.         clusterBuilder.addBuilder(builder, extend, "Proxy", 1);
    56.  
    57.         Cluster cluster = clusterBuilder.realize(new SystemApplicationConsole());
    58.  
    59.         DeferredAssert.assertThat(eventually(invoking(cluster).getClusterSize()), is(3));
    60.  
    61.         cluster.destroy();
    62.     }
    63. }
    package com.thegridman.oracle.tools.examples;
    
    import com.oracle.tools.deferred.DeferredAssert;
    import com.oracle.tools.runtime.coherence.Cluster;
    import com.oracle.tools.runtime.coherence.ClusterBuilder;
    import com.oracle.tools.runtime.coherence.ClusterMember;
    import com.oracle.tools.runtime.coherence.ClusterMemberSchema;
    import com.oracle.tools.runtime.console.SystemApplicationConsole;
    import com.oracle.tools.runtime.java.ExternalJavaApplicationBuilder;
    import com.oracle.tools.runtime.network.AvailablePortIterator;
    
    import java.io.File;
    
    import static com.oracle.tools.deferred.DeferredHelper.eventually;
    import static com.oracle.tools.deferred.DeferredHelper.invoking;
    import static org.hamcrest.CoreMatchers.is;
    
    public class RunClusterForked
    {
        public static void main(String[] args) throws Exception
        {
            AvailablePortIterator ports = new AvailablePortIterator(40000);
    
            ClusterMemberSchema storage
                = new ClusterMemberSchema()
                        .setCacheConfigURI("coherence-cache-config.xml")
                        .setPofEnabled(true)
                        .setPofConfigURI("pof-config.xml")
                        .setClusterPort(12345)
                        .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
                        .setJMXPort(ports)
                        .setOption("-Xmx256m")
                        .setWorkingDirectory(new File("/Users/jonathanknight"))
                        .setStorageEnabled(true);
    
            ClusterMemberSchema extend
                = new ClusterMemberSchema()
                        .setCacheConfigURI("coherence-cache-config.xml")
                        .setPofEnabled(true)
                        .setPofConfigURI("pof-config.xml")
                        .setClusterPort(12345)
                        .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
                        .setJMXPort(ports)
                        .setOption("-Xmx256m")
                        .setWorkingDirectory(new File("/Users/jonathanknight"))
                        .setStorageEnabled(false)
                        .setSystemProperty("tangosol.coherence.extend.enabled", true)
                        .setSystemProperty("tangosol.coherence.extend.port", ports);
    
            ExternalJavaApplicationBuilder<ClusterMember,ClusterMemberSchema> builder
                    = new ExternalJavaApplicationBuilder<ClusterMember,ClusterMemberSchema>();
    
            ClusterBuilder clusterBuilder = new ClusterBuilder();
            clusterBuilder.addBuilder(builder, storage, "Data", 2);
            clusterBuilder.addBuilder(builder, extend, "Proxy", 1);
    
            Cluster cluster = clusterBuilder.realize(new SystemApplicationConsole());
    
            DeferredAssert.assertThat(eventually(invoking(cluster).getClusterSize()), is(3));
    
            cluster.destroy();
        }
    }

    If you run the code above you will see in the console output that there are three Coherence nodes started and all three should form a cluster. As soon as the assertion has determined that all three nodes have joined the cluster, the cluster is destroyed. If you are quick enough you should see the three external processes fire up on your machine.

    Based on what we have already covered it should be easy to follow what the code is doing.

    • One new things is line 22, where we create an AvailablePortIterator. This allows us to obtain free port numbers without having to hard code them in our tests and risk clashing or in use port numbers. The ports will be dynamically assigned as required when applications are realized from schema that use the iterator. We have a few places in the code where we assign ports, lines 31 and 43 where we assign JMX ports and line 48 where we set the extend port.
    • Lines 24 – 34 create the schema that configures a storage node
    • Lines 36 – 48 create the schema that configures an extend proxy node
    • Lines 50 and 51 create the ExternalJavaApplicationBuilder as we want to fork the cluster as external processes
    • Line 53 is the new part where we create our ClusterBuilder that will realize and control the whole cluster of processes
    • Line 54 we tell the ClusterBuilder we want to realize two instances of processes from the storage node schema, the processes should be realized using the ExternalJavaApplicationBuilder and we will name the processes with the prefix “Data”. When the processes are realized they will actually be named Data-n where n is the instance number from zero to the number of required instance.
    • Line 55 we tell the ClusterBuilder we want to realize a single instance of a process using the proxy schema that we will name “Proxy” and realize with the ExternalJavaApplicationBuilder
    • Line 57 is where we actually realize the running cluster using the ClusterBuilder and obtain an instance of a Cluster. We use a SystemApplicationConsole to capture the output of all of the processes to stdout
    • Line 59 we wait for the result of cluster.getClusterSize() to be three, the expected cluster size
    • Line 61 we destroy the cluster, which will destroy all of the processes

    Being able to control and group all the various members of the cluster together makes the code a bit tider than having everything separate.

    Internal Coherence Cluster

    As with the previous internal example if we want to run all the members of the cluster as virtualized processes all we need to do is change a single line in the code. We would change line 50 to create a VirtualizedJavaApplicationBuilder and that would be it, the whole thing would run just the same. I will not bother repeating all the code again.

    Testing With Oracle Tools

    To finish off the examples here is an example test class that uses what I have covered above to start a Coherence cluster then run a test method that interacts with the cluster. This example uses JUnit but you should easily be able to convert it if you use another test framework.

    1. package com.thegridman.oracle.tools.examples;
    2.  
    3. import com.oracle.tools.deferred.DeferredAssert;
    4. import com.oracle.tools.runtime.coherence.Cluster;
    5. import com.oracle.tools.runtime.coherence.ClusterBuilder;
    6. import com.oracle.tools.runtime.coherence.ClusterMember;
    7. import com.oracle.tools.runtime.coherence.ClusterMemberSchema;
    8. import com.oracle.tools.runtime.console.SystemApplicationConsole;
    9. import com.oracle.tools.runtime.java.VirtualizedJavaApplicationBuilder;
    10. import com.oracle.tools.runtime.network.AvailablePortIterator;
    11. import com.tangosol.net.CacheFactory;
    12. import com.tangosol.net.CacheFactoryBuilder;
    13. import com.tangosol.net.ConfigurableCacheFactory;
    14. import com.tangosol.net.NamedCache;
    15. import com.tangosol.run.xml.XmlDocument;
    16. import com.tangosol.run.xml.XmlElement;
    17. import com.tangosol.run.xml.XmlHelper;
    18. import com.tangosol.run.xml.XmlValue;
    19. import org.junit.After;
    20. import org.junit.AfterClass;
    21. import org.junit.Assert;
    22. import org.junit.Before;
    23. import org.junit.BeforeClass;
    24. import org.junit.Test;
    25.  
    26. import javax.management.ObjectName;
    27. import java.io.File;
    28. import java.util.List;
    29. import java.util.Properties;
    30.  
    31. import static com.oracle.tools.deferred.DeferredHelper.eventually;
    32. import static com.oracle.tools.deferred.DeferredHelper.invoking;
    33. import static org.hamcrest.CoreMatchers.is;
    34. import static org.hamcrest.CoreMatchers.notNullValue;
    35.  
    36. public class ClusterTest
    37. {
    38.     private static Cluster cluster;
    39.  
    40.     private ConfigurableCacheFactory clientCacheFactory;
    41.  
    42.     @BeforeClass
    43.     public static void startCluster() throws Exception
    44.     {
    45.         AvailablePortIterator ports = new AvailablePortIterator(40000);
    46.  
    47.         ClusterMemberSchema storage = createStorageNodeSchema(ports);
    48.         ClusterMemberSchema extend =  createExtendProxySchema(ports);
    49.  
    50.         VirtualizedJavaApplicationBuilder<ClusterMember,ClusterMemberSchema> builder
    51.                 = new VirtualizedJavaApplicationBuilder<ClusterMember,ClusterMemberSchema>();
    52.  
    53.         ClusterBuilder clusterBuilder = new ClusterBuilder();
    54.         clusterBuilder.addBuilder(builder, storage, "Data", 2);
    55.         clusterBuilder.addBuilder(builder, extend, "Proxy", 1);
    56.  
    57.         cluster = clusterBuilder.realize(new SystemApplicationConsole());
    58.     }
    59.  
    60.     @Before
    61.     public void setupTest() throws Exception
    62.     {
    63.         // Assert the cluster is ready
    64.         Assert.assertThat(cluster, is(notNullValue()));
    65.         DeferredAssert.assertThat(eventually(invoking(cluster).getClusterSize()), is(3));
    66.  
    67.         // Assert the Proxy Service is running
    68.         assertServiceIsRunning("Proxy-0", "TcpProxyService");
    69.  
    70.         // Get the extend port the proxy is using
    71.         ClusterMember proxyNode = cluster.getApplication("Proxy-0");
    72.         String extendPort = proxyNode.getSystemProperty("tangosol.coherence.extend.port");
    73.  
    74.         // Create the client properties
    75.         Properties properties = new Properties(System.getProperties());
    76.         properties.setProperty("tangosol.coherence.extend.port", extendPort);
    77.  
    78.         // Create the client Cache Factory
    79.         clientCacheFactory = createClientCacheFactory("client-cache-config.xml", properties);
    80.     }
    81.  
    82.     @Test
    83.     public void shouldPutDataIntoCache() throws Exception
    84.     {
    85.         NamedCache cache = clientCacheFactory.ensureCache("dist-test", null);
    86.         cache.put("Key-1", "Value-1");
    87.         Assert.assertThat((String)cache.get("Key-1"), is("Value-1"));
    88.     }
    89.  
    90.     @After
    91.     public void shutdownClient()
    92.     {
    93.         if (clientCacheFactory != null)
    94.         {
    95.             CacheFactory.getCacheFactoryBuilder().release(clientCacheFactory);
    96.             clientCacheFactory = null;
    97.         }
    98.     }
    99.  
    100.     @AfterClass
    101.     public static void stopCluster()
    102.     {
    103.         if (cluster != null)
    104.         {
    105.             cluster.destroy();
    106.             cluster = null;
    107.         }
    108.     }
    109.  
    110.     public static ClusterMemberSchema createStorageNodeSchema(AvailablePortIterator ports) {
    111.         return createCommonSchema(ports)
    112.                     .setStorageEnabled(true);
    113.     }
    114.  
    115.     public static ClusterMemberSchema createExtendProxySchema(AvailablePortIterator ports) {
    116.         return createCommonSchema(ports)
    117.                     .setStorageEnabled(false)
    118.                     .setSystemProperty("tangosol.coherence.extend.enabled", true)
    119.                     .setSystemProperty("tangosol.coherence.extend.port", ports);
    120.     }
    121.  
    122.     public static ClusterMemberSchema createCommonSchema(AvailablePortIterator ports) {
    123.         return new ClusterMemberSchema()
    124.                 .setCacheConfigURI("coherence-cache-config.xml")
    125.                 .setPofEnabled(true)
    126.                 .setPofConfigURI("pof-config.xml")
    127.                 .setClusterPort(12345)
    128.                 .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
    129.                 .setJMXPort(ports)
    130.                 .setOption("-Xmx256m")
    131.                 .setWorkingDirectory(new File("/Users/jonathanknight"));
    132.     }
    133.  
    134.     private void assertServiceIsRunning(String memberName, String serviceName) throws Exception
    135.     {
    136.         Assert.assertThat(cluster, is(notNullValue()));
    137.         ClusterMember member = cluster.getApplication(memberName);
    138.         int nodeId = member.getLocalMemberId();
    139.         ObjectName objectName = new ObjectName(String.format("Coherence:type=Service,name=%s,nodeId=%d", serviceName, nodeId));
    140.         DeferredAssert.assertThat(eventually(invoking(member).getMBeanAttribute(objectName, "Running", Boolean.class)), is(true));
    141.     }
    142.  
    143.     private ConfigurableCacheFactory createClientCacheFactory(String cacheConfigurationURI, Properties properties)
    144.     {
    145.         XmlDocument clientConfigXml = XmlHelper.loadFileOrResource(cacheConfigurationURI, "Client");
    146.         replacePropertiesInXml(clientConfigXml, "system-property", properties);
    147.  
    148.         CacheFactoryBuilder cacheFactoryBuilder = CacheFactory.getCacheFactoryBuilder();
    149.         cacheFactoryBuilder.setCacheConfiguration(cacheConfigurationURI, null, clientConfigXml);
    150.         return cacheFactoryBuilder.getConfigurableCacheFactory(cacheConfigurationURI, null);
    151.     }
    152.  
    153.     @SuppressWarnings("unchecked")
    154.     private void replacePropertiesInXml(XmlElement xml, String propertyAttributeName, Properties properties)
    155.     {
    156.         XmlValue attribute = xml.getAttribute(propertyAttributeName);
    157.         if (attribute != null)
    158.         {
    159.             xml.setAttribute(propertyAttributeName, null);
    160.             try
    161.             {
    162.                 String propertyValue = properties.getProperty(attribute.getString());
    163.                 if (propertyValue != null)
    164.                 {
    165.                     xml.setString(propertyValue);
    166.                 }
    167.             }
    168.             catch (Exception _ignored)
    169.             {
    170.                 // ignored on purpose
    171.             }
    172.         }
    173.  
    174.         for (XmlElement child : (List<XmlElement>) xml.getElementList())
    175.         {
    176.             replacePropertiesInXml(child, propertyAttributeName, properties);
    177.         }
    178.     }
    179. }
    package com.thegridman.oracle.tools.examples;
    
    import com.oracle.tools.deferred.DeferredAssert;
    import com.oracle.tools.runtime.coherence.Cluster;
    import com.oracle.tools.runtime.coherence.ClusterBuilder;
    import com.oracle.tools.runtime.coherence.ClusterMember;
    import com.oracle.tools.runtime.coherence.ClusterMemberSchema;
    import com.oracle.tools.runtime.console.SystemApplicationConsole;
    import com.oracle.tools.runtime.java.VirtualizedJavaApplicationBuilder;
    import com.oracle.tools.runtime.network.AvailablePortIterator;
    import com.tangosol.net.CacheFactory;
    import com.tangosol.net.CacheFactoryBuilder;
    import com.tangosol.net.ConfigurableCacheFactory;
    import com.tangosol.net.NamedCache;
    import com.tangosol.run.xml.XmlDocument;
    import com.tangosol.run.xml.XmlElement;
    import com.tangosol.run.xml.XmlHelper;
    import com.tangosol.run.xml.XmlValue;
    import org.junit.After;
    import org.junit.AfterClass;
    import org.junit.Assert;
    import org.junit.Before;
    import org.junit.BeforeClass;
    import org.junit.Test;
    
    import javax.management.ObjectName;
    import java.io.File;
    import java.util.List;
    import java.util.Properties;
    
    import static com.oracle.tools.deferred.DeferredHelper.eventually;
    import static com.oracle.tools.deferred.DeferredHelper.invoking;
    import static org.hamcrest.CoreMatchers.is;
    import static org.hamcrest.CoreMatchers.notNullValue;
    
    public class ClusterTest
    {
        private static Cluster cluster;
    
        private ConfigurableCacheFactory clientCacheFactory;
    
        @BeforeClass
        public static void startCluster() throws Exception
        {
            AvailablePortIterator ports = new AvailablePortIterator(40000);
    
            ClusterMemberSchema storage = createStorageNodeSchema(ports);
            ClusterMemberSchema extend =  createExtendProxySchema(ports);
    
            VirtualizedJavaApplicationBuilder<ClusterMember,ClusterMemberSchema> builder
                    = new VirtualizedJavaApplicationBuilder<ClusterMember,ClusterMemberSchema>();
    
            ClusterBuilder clusterBuilder = new ClusterBuilder();
            clusterBuilder.addBuilder(builder, storage, "Data", 2);
            clusterBuilder.addBuilder(builder, extend, "Proxy", 1);
    
            cluster = clusterBuilder.realize(new SystemApplicationConsole());
        }
    
        @Before
        public void setupTest() throws Exception
        {
            // Assert the cluster is ready
            Assert.assertThat(cluster, is(notNullValue()));
            DeferredAssert.assertThat(eventually(invoking(cluster).getClusterSize()), is(3));
    
            // Assert the Proxy Service is running
            assertServiceIsRunning("Proxy-0", "TcpProxyService");
    
            // Get the extend port the proxy is using
            ClusterMember proxyNode = cluster.getApplication("Proxy-0");
            String extendPort = proxyNode.getSystemProperty("tangosol.coherence.extend.port");
    
            // Create the client properties
            Properties properties = new Properties(System.getProperties());
            properties.setProperty("tangosol.coherence.extend.port", extendPort);
    
            // Create the client Cache Factory
            clientCacheFactory = createClientCacheFactory("client-cache-config.xml", properties);
        }
    
        @Test
        public void shouldPutDataIntoCache() throws Exception
        {
            NamedCache cache = clientCacheFactory.ensureCache("dist-test", null);
            cache.put("Key-1", "Value-1");
            Assert.assertThat((String)cache.get("Key-1"), is("Value-1"));
        }
    
        @After
        public void shutdownClient()
        {
            if (clientCacheFactory != null)
            {
                CacheFactory.getCacheFactoryBuilder().release(clientCacheFactory);
                clientCacheFactory = null;
            }
        }
    
        @AfterClass
        public static void stopCluster()
        {
            if (cluster != null)
            {
                cluster.destroy();
                cluster = null;
            }
        }
    
        public static ClusterMemberSchema createStorageNodeSchema(AvailablePortIterator ports) {
            return createCommonSchema(ports)
                        .setStorageEnabled(true);
        }
    
        public static ClusterMemberSchema createExtendProxySchema(AvailablePortIterator ports) {
            return createCommonSchema(ports)
                        .setStorageEnabled(false)
                        .setSystemProperty("tangosol.coherence.extend.enabled", true)
                        .setSystemProperty("tangosol.coherence.extend.port", ports);
        }
    
        public static ClusterMemberSchema createCommonSchema(AvailablePortIterator ports) {
            return new ClusterMemberSchema()
                    .setCacheConfigURI("coherence-cache-config.xml")
                    .setPofEnabled(true)
                    .setPofConfigURI("pof-config.xml")
                    .setClusterPort(12345)
                    .setJMXManagementMode(ClusterMemberSchema.JMXManagementMode.LOCAL_ONLY)
                    .setJMXPort(ports)
                    .setOption("-Xmx256m")
                    .setWorkingDirectory(new File("/Users/jonathanknight"));
        }
    
        private void assertServiceIsRunning(String memberName, String serviceName) throws Exception
        {
            Assert.assertThat(cluster, is(notNullValue()));
            ClusterMember member = cluster.getApplication(memberName);
            int nodeId = member.getLocalMemberId();
            ObjectName objectName = new ObjectName(String.format("Coherence:type=Service,name=%s,nodeId=%d", serviceName, nodeId));
            DeferredAssert.assertThat(eventually(invoking(member).getMBeanAttribute(objectName, "Running", Boolean.class)), is(true));
        }
    
        private ConfigurableCacheFactory createClientCacheFactory(String cacheConfigurationURI, Properties properties)
        {
            XmlDocument clientConfigXml = XmlHelper.loadFileOrResource(cacheConfigurationURI, "Client");
            replacePropertiesInXml(clientConfigXml, "system-property", properties);
    
            CacheFactoryBuilder cacheFactoryBuilder = CacheFactory.getCacheFactoryBuilder();
            cacheFactoryBuilder.setCacheConfiguration(cacheConfigurationURI, null, clientConfigXml);
            return cacheFactoryBuilder.getConfigurableCacheFactory(cacheConfigurationURI, null);
        }
    
        @SuppressWarnings("unchecked")
        private void replacePropertiesInXml(XmlElement xml, String propertyAttributeName, Properties properties)
        {
            XmlValue attribute = xml.getAttribute(propertyAttributeName);
            if (attribute != null)
            {
                xml.setAttribute(propertyAttributeName, null);
                try
                {
                    String propertyValue = properties.getProperty(attribute.getString());
                    if (propertyValue != null)
                    {
                        xml.setString(propertyValue);
                    }
                }
                catch (Exception _ignored)
                {
                    // ignored on purpose
                }
            }
    
            for (XmlElement child : (List<XmlElement>) xml.getElementList())
            {
                replacePropertiesInXml(child, propertyAttributeName, properties);
            }
        }
    }

    We will go through the class bit by bit

    @BeforeClass Method – Starting the Cluster

    The obvious place to start the cluster for a test class is before anything else runs, so in this case we use a method annotated with @BeforeClass I have called obviously enough startCluster().

    1. @BeforeClass
    2. public static void startCluster() throws Exception
    3. {
    4.     AvailablePortIterator ports = new AvailablePortIterator(40000);
    5.  
    6.     ClusterMemberSchema storage = createStorageNodeSchema(ports);
    7.     ClusterMemberSchema extend =  createExtendProxySchema(ports);
    8.  
    9.     VirtualizedJavaApplicationBuilder<ClusterMember,ClusterMemberSchema> builder
    10.             = new VirtualizedJavaApplicationBuilder<ClusterMember,ClusterMemberSchema>();
    11.  
    12.     ClusterBuilder clusterBuilder = new ClusterBuilder();
    13.     clusterBuilder.addBuilder(builder, storage, "Data", 2);
    14.     clusterBuilder.addBuilder(builder, extend, "Proxy", 1);
    15.  
    16.     cluster = clusterBuilder.realize(new SystemApplicationConsole());
    17. }
    @BeforeClass
    public static void startCluster() throws Exception
    {
        AvailablePortIterator ports = new AvailablePortIterator(40000);
    
        ClusterMemberSchema storage = createStorageNodeSchema(ports);
        ClusterMemberSchema extend =  createExtendProxySchema(ports);
    
        VirtualizedJavaApplicationBuilder<ClusterMember,ClusterMemberSchema> builder
                = new VirtualizedJavaApplicationBuilder<ClusterMember,ClusterMemberSchema>();
    
        ClusterBuilder clusterBuilder = new ClusterBuilder();
        clusterBuilder.addBuilder(builder, storage, "Data", 2);
        clusterBuilder.addBuilder(builder, extend, "Proxy", 1);
    
        cluster = clusterBuilder.realize(new SystemApplicationConsole());
    }

    The startCluster() method should be easy to understand as it is pretty identical to the last example. We have extracted out the schema creation into two other methods createStorageNodeSchema and createExtendProxySchema. If you look at these methods you will see we have a further optimisation where we extract creation of all the common schema settings into another method createCommonSchema. This meas there is less code and is obviously less error prone.

    @Before – Verify The Cluster is Ready & Setup the Test

    Before we run any test methods we want to make sure that the cluster is available, and also in our case we are going to connect over Extend, so we want to make sure the proxy is running. On top of that we will also set up the client Cache Factory that the tests will use to connect to the cluster. It makes sense to do this in one common method rather than every test method (even though in this example we only have one test method). All of the code I have put in the @Before method could just have easily been put in the @BeforeClass method if you only want to do it once for the whole test class (or even somewhere once for a suite).

    1. @Before
    2. public void setupTest() throws Exception
    3. {
    4.     // Assert the cluster is ready
    5.     Assert.assertThat(cluster, is(notNullValue()));
    6.     DeferredAssert.assertThat(eventually(invoking(cluster).getClusterSize()), is(3));
    7.  
    8.     // Assert the Proxy Service is running
    9.     assertServiceIsRunning("Proxy-0", "TcpProxyService");
    10.  
    11.     // Get the extend port the proxy is using
    12.     ClusterMember proxyNode = cluster.getApplication("Proxy-0");
    13.     String extendPort = proxyNode.getSystemProperty("tangosol.coherence.extend.port");
    14.  
    15.     // Create the client properties
    16.     Properties properties = new Properties(System.getProperties());
    17.     properties.setProperty("tangosol.coherence.extend.port", extendPort);
    18.  
    19.     // Create the client Cache Factory
    20.     clientCacheFactory = createClientCacheFactory("client-cache-config.xml", properties);
    21. }
    @Before
    public void setupTest() throws Exception
    {
        // Assert the cluster is ready
        Assert.assertThat(cluster, is(notNullValue()));
        DeferredAssert.assertThat(eventually(invoking(cluster).getClusterSize()), is(3));
    
        // Assert the Proxy Service is running
        assertServiceIsRunning("Proxy-0", "TcpProxyService");
    
        // Get the extend port the proxy is using
        ClusterMember proxyNode = cluster.getApplication("Proxy-0");
        String extendPort = proxyNode.getSystemProperty("tangosol.coherence.extend.port");
    
        // Create the client properties
        Properties properties = new Properties(System.getProperties());
        properties.setProperty("tangosol.coherence.extend.port", extendPort);
    
        // Create the client Cache Factory
        clientCacheFactory = createClientCacheFactory("client-cache-config.xml", properties);
    }

    So, in our @Before method the first thing we do is assert we actually have a cluster.
    Next we assert that the cluster size is three, using a deferred assertion the same as we have done in other examples.
    Next we assert that the proxy service is running. For this we use a utility method that checks via JMX that the service is running.

    1. private void assertServiceIsRunning(String memberName, String serviceName) throws Exception
    2. {
    3.     Assert.assertThat(cluster, is(notNullValue()));
    4.     ClusterMember member = cluster.getApplication(memberName);
    5.     int nodeId = member.getLocalMemberId();
    6.     ObjectName objectName = new ObjectName(String.format("Coherence:type=Service,name=%s,nodeId=%d", serviceName, nodeId));
    7.     DeferredAssert.assertThat(eventually(invoking(member).getMBeanAttribute(objectName, "Running", Boolean.class)), is(true));
    8. }
    private void assertServiceIsRunning(String memberName, String serviceName) throws Exception
    {
        Assert.assertThat(cluster, is(notNullValue()));
        ClusterMember member = cluster.getApplication(memberName);
        int nodeId = member.getLocalMemberId();
        ObjectName objectName = new ObjectName(String.format("Coherence:type=Service,name=%s,nodeId=%d", serviceName, nodeId));
        DeferredAssert.assertThat(eventually(invoking(member).getMBeanAttribute(objectName, "Running", Boolean.class)), is(true));
    }

    This is quite a useful method and I may at some point feed it back into the Oracle Tool project.
    The method takes two parameters, the name of the member the service is on and the name of the service. If you remember our proxy member will be called Proxy-0 as we told the ClusterBuilder to use the prefix “Proxy” for the extend proxy member and there is only a single instance which will be instance zero. The second parameter is the name of the service; this name comes from the cache configuration file. We are using the coherence-cache-config.xml file from inside the coherence jar file and the name of the proxy service in this file is TcpProxyService.

    • The first thing we do is assert we have a cluster, this is better than just throwing a NullPointerException later.
    • Next we get the proxy member from the cluster using the specified member name.
    • Now we have the member we need to get its cluster node ID, which Oracle Tools does via JMX.
    • Once we have the node ID we can create the JMX ObjectName for the service. All service names in Coherence follow the same naming pattern so this is easy to do. There is a method on the member called getMBeanAttribute that will retunr the value of a given MBean attribute. We use this to get the value of the “Running” attribute of the service. We wrap this in a deferred assertion to give the service a reasonable time to start up.

    Now we know the proxy is started we can configure our Cache Factory that the tests will use. If you look at the code of the test method you will see that the NamedCache is not obtained using the static CacheFactory.getCache method but from a real instance of a ConfigurableCacheFactory. Personally I would avoid using the static methods in Coherence to do things as much as you can. Using statics makes tests hard to control and makes things hard to mock. I know there are mock frameworks that let you mock static calls but doing this is like worshiping the devil, so better not to use statics in the first place. So, how do we create our client cache factory and tell it the ports for the proxy…

  • First we need to know the extend port. If you remember we have not hard coded this, but instead we used an AvailablePortIterator to pick a port when the process is realized. You will rember I covered earlier that the Java process and hence ClusterMember instance has a method on it to obtain the values of System properties used to realize the process and this is how we get the port used.
    1. ClusterMember proxyNode = cluster.getApplication("Proxy-0");
    2. String extendPort = proxyNode.getSystemProperty("tangosol.coherence.extend.port");
    ClusterMember proxyNode = cluster.getApplication("Proxy-0");
    String extendPort = proxyNode.getSystemProperty("tangosol.coherence.extend.port");
  • Now we have the port we can use another useful utility method createClientCacheFactory to create the client Cache Factory. This method takes the name of the cache configuration file to use and a set of properties to use to replace any properties in the XML from the configuration file – in our case the extend port.
    1. // Create the client properties
    2. Properties properties = new Properties(System.getProperties());
    3. properties.setProperty("tangosol.coherence.extend.port", extendPort);
    4.  
    5. // Create the client Cache Factory
    6. clientCacheFactory = createClientCacheFactory("client-cache-config.xml", properties);
    // Create the client properties
    Properties properties = new Properties(System.getProperties());
    properties.setProperty("tangosol.coherence.extend.port", extendPort);
    
    // Create the client Cache Factory
    clientCacheFactory = createClientCacheFactory("client-cache-config.xml", properties);

    So in our setup method we create the Properties adding the extend port property, then call the createClientCacheFactory method with the client configuration name and the properties.

    A very brief description of the createClientCacheFactory method…

    1. private ConfigurableCacheFactory createClientCacheFactory(String cacheConfigurationURI, Properties properties)
    2. {
    3.     XmlDocument clientConfigXml = XmlHelper.loadFileOrResource(cacheConfigurationURI, "Client");
    4.     replacePropertiesInXml(clientConfigXml, "system-property", properties);
    5.  
    6.     CacheFactoryBuilder cacheFactoryBuilder = CacheFactory.getCacheFactoryBuilder();
    7.     cacheFactoryBuilder.setCacheConfiguration(cacheConfigurationURI, null, clientConfigXml);
    8.     return cacheFactoryBuilder.getConfigurableCacheFactory(cacheConfigurationURI, null);
    9. }
    private ConfigurableCacheFactory createClientCacheFactory(String cacheConfigurationURI, Properties properties)
    {
        XmlDocument clientConfigXml = XmlHelper.loadFileOrResource(cacheConfigurationURI, "Client");
        replacePropertiesInXml(clientConfigXml, "system-property", properties);
    
        CacheFactoryBuilder cacheFactoryBuilder = CacheFactory.getCacheFactoryBuilder();
        cacheFactoryBuilder.setCacheConfiguration(cacheConfigurationURI, null, clientConfigXml);
        return cacheFactoryBuilder.getConfigurableCacheFactory(cacheConfigurationURI, null);
    }

    First we load the XML for the specified configuration URI. When we have the XML we replace any system properties in it using another utility method. That method should be easy enough for you to work out what it does. We then get the CacheFactoryBuilder using one of the few static Coherence methods used. If we wanted to we could have just created a new instance of the relevant concrete ConfigurableCacheFactory but I am going to use the builder. Once we have the builder we register the configuration XML we created then obtain a ConfigurableCacheFactory from that.

    So, now we have the client Cache factory we are ready to run the test.

    @Test – The Actual Test Method

    In this class we only have one test method that does something very simple.

    1. @Test
    2. public void shouldPutDataIntoCache() throws Exception
    3. {
    4.     NamedCache cache = clientCacheFactory.ensureCache("dist-test", null);
    5.     cache.put("Key-1", "Value-1");
    6.     Assert.assertThat((String)cache.get("Key-1"), is("Value-1"));
    7. }
    @Test
    public void shouldPutDataIntoCache() throws Exception
    {
        NamedCache cache = clientCacheFactory.ensureCache("dist-test", null);
        cache.put("Key-1", "Value-1");
        Assert.assertThat((String)cache.get("Key-1"), is("Value-1"));
    }

    As I’ve already mentioned, we have a concrete ConfigurableCacheFactory so we use the ConfigurableCacheFactory.ensureCache method to get our cache instances. All the test does is put a value into the cache and then asserts the same value comes back with a get call.

    @After – Cleaning Up the Test

    After each test has run I have decided I want to clean up the client before the @Before method runs again.

    1. @After
    2. public void shutdownClient()
    3. {
    4.     if (clientCacheFactory != null)
    5.     {
    6.         CacheFactory.getCacheFactoryBuilder().release(clientCacheFactory);
    7.         clientCacheFactory = null;
    8.     }
    9. }
    @After
    public void shutdownClient()
    {
        if (clientCacheFactory != null)
        {
            CacheFactory.getCacheFactoryBuilder().release(clientCacheFactory);
            clientCacheFactory = null;
        }
    }

    All we are doing in our clean up is deregistering the Cache Factory and configuration from the static CacheFactoryBuilder.

    @AfterClass – Destroy The Cluster

    Finally after all of our tests have run we want to stop the cluster so we have an @AfterClass annotated method that does just that.

    1. @AfterClass
    2. public static void stopCluster()
    3. {
    4.     if (cluster != null)
    5.     {
    6.         cluster.destroy();
    7.         cluster = null;
    8.     }
    9. }
    @AfterClass
    public static void stopCluster()
    {
        if (cluster != null)
        {
            cluster.destroy();
            cluster = null;
        }
    }

    And that is it, a full test class example.

    Testing Tips

    So, you have seen how easy it is to define different types of processes and control them from within code. This makes it simple to add to your automated functional test suite as part of a CI build. The project I work on for my current client uses this functionality and it works very well. It is possible to run a wide mixture of different processes, Java and non-Java, internal and external and mix and match if you really wanted to. There are a few tips and things to be aware of when using the above techniques to test Coherence applications so I’ve noted a few of them below.

    Start Your Cluster Once

    Typically a project will only have a single Coherence cluster so it is a very good idea to start and configure you test cluster once at the beginning of the functional test suite. Starting and stopping a cluster for every test class can add significant amounts of time to your test run. How you do this depends on your test framework, for example JUnit provides Rules that can be used to make sure the cluster has started. TestNG provides a @BeforeSuite method that you can use in a super-class of all your functional tests.

    Wait for the cluster to be ready before starting the tests – this is important for a stable test suite. If you just called realize on all your builders to fire up a cluster then immediately carried on into your tests methods the cluster may or may not actually be ready for the tests meaning tests may intermittently fail – the worst kind of test failure. With the deferred assertions mentioned previously it is easy to wait for all the cluster members to have joined the cluster, wait for the extend proxy to start etc. Again using my current client’s project as an example this application even has an internal MBean that monitors the application status and sets an MBean attribute when the application has been fully initialised and is ready to start accepting client requests. We then do a deferred assertion on this MBean attribute in the test suite to wait until the application is ready before continuing on to the tests.

    The start it once rule really applies to starting up any part of your test suite that takes some time. In my current project we start a Coherence cluster, embedded database and JMS server at the start of our test suites and tear it all down at the end and run over 25,000 tests in a shade over 25 minutes. The test run would take forever if we performed the initialisation too often.

    Memory Usage

    When using virtualized processes in your tests – which is the most sensible way to make sure everything is killed off when the tests finish – you need to make sure you have a big enough JVM heap to run everything so make sure your test harness is configured with a big heap. My current project is quite complex and runs a lot of things in a single JVM and we use a 1024MB heap. More important though is Perm Gen size as each virtualized process is isolated by ClassLoader so will increase Perm Gen usage as classes will be loaded multiple times. You can set Perm Space with a JVM argument like this

    -XX:MaxPermSize=256M

    The exact size will depend on your use case but we typically run with 256M or 300M.

    You might also want to try lowering the thread stack size for tests as this will reduce heap usage as long as you do not have any methods with very deep recursive calls. Running a number of Coherence cluster members in a single JVM can use a lot of threads so having a lower stack size can be a big help. Stack size is set using a JVM parameter like this

    -Xss64k

    Again, you might need a different value but we typically drop the stack down to 64k.

    So that’s it, not too complicated and very useful functionality. It is well worth looking at Oracle Tools to see how it can help simplify testing of applications, especially clustered applications. It is all open source so if it does not quite do what you want it is easy to change and feedback on new requirements is always welcome.

    Enjoy…

  • Oracle Coherence Top n Query

    excel-pivot-one-row-two-values

    For this blog I am going to talk about how to do an Oracle Coherence Top n query – that is a query such as find me the top n things in a cache – where the top is based on an attribute of the cache entry, for example the top 5 orders based on order value. Obviously we could also find the bottom n or other such queries. The blog has a bit of preamble to set the scene and cover my example scenario, then we will look at the code to actually perform an Oracle Coherence Top n query and finally look at a few more examples using different aggregators (and we will learn something not everyone realises about ValueExtractors too). Apologies to any .Net or C++ developers but the code here is all Java. If you want to use this in either C# of C++ then you only need to create a stub of the LimitAggregator class I describe and then deploy the Java version to your server.
    So, with a little help from Shakespeare, we will go and count some words…

    Whilst playing around with code from my previous blog on Oracle Coherence Pivot Table Queries I had a scenario where I wanted to perform a Top n query; I was trying to replicate something I had seen elsewhere as an example and realised that out of the box Coherence does not have anything built in that allows you to do this type of query.

    — First Witch:
    “In the poysond Entrailes throw
    Toad, that vnder cold stone,
    Dayes and Nights, ha’s thirty one:
    Sweltred Venom sleeping got,
    Boyle thou first i’th’ charmed pot”
    Macbeth (IV, i)

    The exact scenario was a trivial example where I have a cache of “words” and the word entry has a count of how many times that word occurred in a book, or in our case a play. I then want to query for something like the top n most used words. I could just as easily have wanted the least used words or some other similar query.

    Caching Words

    So, lets start with the scenario. I need a cache of words so I need a simple Word class, the attributes will be the word itself, the usage count and the length of the word. I know I could get the length from the String without needing a field but it makes it easier to extract using a PofExtractor later.

    1. public class Word implements PortableObject
    2. {
    3.     private String word;
    4.  
    5.     private long count = 0;
    6.  
    7.     private int length;
    8.  
    9.     public Word()
    10.     {
    11.     }
    12.  
    13.     public Word(String word)
    14.     {
    15.         this.word = word;
    16.         this.length = word.length();
    17.     }
    18.  
    19.     public String getWord()
    20.     {
    21.         return word;
    22.     }
    23.  
    24.     public int getLength()
    25.     {
    26.         return length;
    27.     }
    28.  
    29.     public long getCount()
    30.     {
    31.         return count;
    32.     }
    33.  
    34.     public void setCount(long count)
    35.     {
    36.         this.count = count;
    37.     }
    38.  
    39.     public void incrementCount()
    40.     {
    41.         count++;
    42.     }
    43.  
    44.     @Override
    45.     public void readExternal(PofReader pofReader) throws IOException
    46.     {
    47.         word = pofReader.readString(0);
    48.         count = pofReader.readLong(1);
    49.         length = pofReader.readInt(2);
    50.     }
    51.  
    52.     @Override
    53.     public void writeExternal(PofWriter pofWriter) throws IOException
    54.     {
    55.         pofWriter.writeString(0, word);
    56.         pofWriter.writeLong(1, count);
    57.         pofWriter.writeInt(2, length);
    58.     }
    59.  
    60.     @Override
    61.     public String toString()
    62.     {
    63.         return "Word{" +
    64.                "count=" + count +
    65.                ", word='" + word + '\'' +
    66.                ", length=" + length +
    67.                '}';
    68.     }
    69. }
    public class Word implements PortableObject
    {
        private String word;
    
        private long count = 0;
    
        private int length;
    
        public Word()
        {
        }
    
        public Word(String word)
        {
            this.word = word;
            this.length = word.length();
        }
    
        public String getWord()
        {
            return word;
        }
    
        public int getLength()
        {
            return length;
        }
    
        public long getCount()
        {
            return count;
        }
    
        public void setCount(long count)
        {
            this.count = count;
        }
    
        public void incrementCount()
        {
            count++;
        }
    
        @Override
        public void readExternal(PofReader pofReader) throws IOException
        {
            word = pofReader.readString(0);
            count = pofReader.readLong(1);
            length = pofReader.readInt(2);
        }
    
        @Override
        public void writeExternal(PofWriter pofWriter) throws IOException
        {
            pofWriter.writeString(0, word);
            pofWriter.writeLong(1, count);
            pofWriter.writeInt(2, length);
        }
    
        @Override
        public String toString()
        {
            return "Word{" +
                   "count=" + count +
                   ", word='" + word + '\'' +
                   ", length=" + length +
                   '}';
        }
    }

    It does not get much simpler than that. The Word class implements PortableObject so it goes over the wire and has a couple of methods for getting and setting the usage count and other fields.

    — Macbeth:
    “Is this a Dagger, which I see before me,
    The Handle toward my Hand? Come, let me clutch thee:
    I haue thee not, and yet I see thee still.
    Art thou not fatall Vision, sensible
    To feeling, as to sight? or art thou but
    A Dagger of the Minde, a false Creation,
    Proceeding from the heat-oppressed Braine?”
    Macbeth (II, ii)

    My Word cache will be a simple distributed cache, for this example any distributed cache configuration will do as all I want to do is run an aggregator. I will not bother showing a cache configuration file as there is nothing special in it, even the default file in the Coherence jar would do for this example.

    Loading the Word Cache

    Now I have a class to represent a word I need to load some words into the cache. Well, Shakespeare knew a lot of words so what better than one of his plays, so we will go to Project Gutenberg where you can download some Shakespeare – and it should be obvious from the quotes that we are going to use Macbeth for this example.

    Now we have a file we can write a trivial bit of code to load the words.

    1. private static void loadMacbeth(int count) throws Exception
    2. {
    3.     NamedCache wordsCache = CacheFactory.getCache("words");
    4.  
    5.     WordUpdater updater = new WordUpdater();
    6.  
    7.     BufferedReader reader = new BufferedReader(new FileReader("/Users/jonathanknight/Documents/macbeth.txt"));
    8.     String line = reader.readLine();
    9.  
    10.     // skip 382 lines to miss the Project Gutenberg introduction
    11.     for (int i=0; i<382; i++)
    12.      {
    13.          line = reader.readLine();
    14.      }
    15.  
    16.      int wordCount = 0;
    17.      while(line != null && wordCount < count)
    18.      {
    19.          System.out.println(line);
    20.          String[] words = line.split("\\s|((?!['-])\\p{Punct})");
    21.  
    22.          for (String word : words)
    23.          {
    24.              if (word.length() > 0)
    25.             {
    26.                 word = word.toLowerCase();
    27.                 wordsCache.invoke(word, updater);
    28.                 wordCount++;
    29.             }
    30.         }
    31.         line = reader.readLine();
    32.     }
    33. }
    private static void loadMacbeth(int count) throws Exception
    {
        NamedCache wordsCache = CacheFactory.getCache("words");
    
        WordUpdater updater = new WordUpdater();
    
        BufferedReader reader = new BufferedReader(new FileReader("/Users/jonathanknight/Documents/macbeth.txt"));
        String line = reader.readLine();
    
        // skip 382 lines to miss the Project Gutenberg introduction
        for (int i=0; i<382; i++)
         {
             line = reader.readLine();
         }
     
         int wordCount = 0;
         while(line != null && wordCount < count)
         {
             System.out.println(line);
             String[] words = line.split("\\s|((?!['-])\\p{Punct})");
     
             for (String word : words)
             {
                 if (word.length() > 0)
                {
                    word = word.toLowerCase();
                    wordsCache.invoke(word, updater);
                    wordCount++;
                }
            }
            line = reader.readLine();
        }
    }

    And the WordUpdater entry processor used to update the counts in the cache…

    1. public class WordUpdater extends AbstractProcessor implements PortableObject
    2. {
    3.     @Override
    4.     public Object process(InvocableMap.Entry entry)
    5.     {
    6.         Word word;
    7.         if (entry.isPresent())
    8.         {
    9.             word = (Word) entry.getValue();
    10.         } else {
    11.             word = new Word((String) entry.getKey());
    12.         }
    13.         word.incrementCount();
    14.         entry.setValue(word);
    15.         return null;
    16.     }
    17.  
    18.     @Override
    19.     public void readExternal(PofReader pofReader) throws IOException
    20.     {
    21.     }
    22.  
    23.     @Override
    24.     public void writeExternal(PofWriter pofWriter) throws IOException
    25.     {
    26.     }
    27. }
    public class WordUpdater extends AbstractProcessor implements PortableObject
    {
        @Override
        public Object process(InvocableMap.Entry entry)
        {
            Word word;
            if (entry.isPresent())
            {
                word = (Word) entry.getValue();
            } else {
                word = new Word((String) entry.getKey());
            }
            word.incrementCount();
            entry.setValue(word);
            return null;
        }
    
        @Override
        public void readExternal(PofReader pofReader) throws IOException
        {
        }
    
        @Override
        public void writeExternal(PofWriter pofWriter) throws IOException
        {
        }
    }

    All the code does is read lines from the .txt file (skipping the first 382 as this is an introduction by Project Gutenberg) and for each line it uses the String.split() method to split the string. The regex expression used to split the line into words “\\s|((?![‘])\\p{Punct})” splits on punctuation as well as on spaces. For those not up to speed on regex I shall explain the parts

    • The first part \\s matches whitespace
    • Then we have a | which is OR
    • Then we have ((?![‘-])\\p{Punct}) which is all punctuation except for single quote
    • The \\p{Punct} matches all punctuation
    • The (?![‘-]) is a negative look ahead – i.e. not whatever is inside the square brackets – in this case single quote and hyphen

    This is because I want words that end with say a comma or full stop or come straight after say a double quote to be treated like a normal word. For example, taking the opening part of the play…

    Thunder and Lightning. Enter three Witches.

    1. When shall we three meet againe?
    In Thunder, Lightning, or in Raine?
    2. When the Hurley-burley’s done,
    When the Battaile’s lost, and wonne

    …I want the first “Thunder” in “Thunder and lightning” to be treated as the same word as “Thunder,” in “In Thunder, Lightning, or in Raine?” so I need to strip the comma from the second thunder; the same would apply to stripping the question mark from the end of “Raine?”. Single quotes and dashes are excluded from splitting words they are is used as hyphens to join words and apostrophes in words like “Hurley-burley’s” and we do not want this to appear as three words “Hurley”, “burley” and “s”.

    The reason I use the WordUpdater EntryProcessor to do the update of the count is so that it is atomic. The scenario I was trying to work on originally involved multiple simultaneous books/plays being loaded together so word count needs to be updated atomically as two or more threads could try to update the same word at the same time.

    Counting Words

    To start with we will do a simple query to bring back the word count so we can see what we have in the words cache. Rather than just bring back all the entries, that would a long list, I will do a query where I return a count and a list of all the words that have that usage count; oh… and one more thing, we will only look at words with five or more letters so we do not just end up with a list of words like “a”, “to”, “and” etc…

    I can do this query with standard Coherence Aggregators, in this case a combination of the DistinctValues aggregator and a GroupAggregator. To do the word length filtering I just use a PofExtractor to extract word length. So here is the code to count everything…

    1. NamedCache wordsCache = CacheFactory.getCache("words");
    2.  
    3. ValueExtractor wordExtractor = new PofExtractor(String.class, 0);
    4. ValueExtractor countExtractor = new PofExtractor(Long.class, 1);
    5.  
    6. GroupAggregator groupAggregator =
    7.         GroupAggregator.createInstance(countExtractor, new DistinctValues(wordExtractor));
    8.  
    9. Filter wordSizeFilter = new GreaterFilter(new PofExtractor(Integer.class, 4), 4);
    10.  
    11. Map<Long,Collection> result = (Map<Long, Collection>) wordsCache.aggregate(wordSizeFilter, groupAggregator);
    12.  
    13. System.out.println("Result: shouldCountWords");
    14. for (Map.Entry<Long,Collection> entry : result.entrySet())
    15. {
    16.     System.out.println(entry.getKey() + "\t" + entry.getValue());
    17. }
    NamedCache wordsCache = CacheFactory.getCache("words");
    
    ValueExtractor wordExtractor = new PofExtractor(String.class, 0);
    ValueExtractor countExtractor = new PofExtractor(Long.class, 1);
    
    GroupAggregator groupAggregator =
            GroupAggregator.createInstance(countExtractor, new DistinctValues(wordExtractor));
    
    Filter wordSizeFilter = new GreaterFilter(new PofExtractor(Integer.class, 4), 4);
    
    Map<Long,Collection> result = (Map<Long, Collection>) wordsCache.aggregate(wordSizeFilter, groupAggregator);
    
    System.out.println("Result: shouldCountWords");
    for (Map.Entry<Long,Collection> entry : result.entrySet())
    {
        System.out.println(entry.getKey() + "\t" + entry.getValue());
    }

    If we load the cache with the first 10,000 words from the play and then run the code above we get the following output…

    1 [manhood, blowes, drowsie, inuest, scoena, bright, folly, inuite, particular, sleeue, augment, surmise, colour …
    2 [robes, hearts, neyther, wearie, hauing, sences, spoke, ready, began, banquet, filthie, partner, instant …
    3 [gentlemen, sorrow, halfe, greater, honors, treason, shine, ghost, winde, thanks, enemie, secunda, throat …
    4 [forth, darke, faces, second, attend, reason, innocent, ‘gainst, become, chance, bring, together, takes …
    5 [might, state, house, hearke, through, fortune, though, let’s, father, while, giuen, whence, almost …
    6 [would’st, words, liues, honor, feares, shake, leaue, lesse, earth, fleans, comes, chamber]
    7 [hence, donalbaine, being, houre, about, royall, cannot, knocking, friends, who’s, malcolme, peace …
    8 [businesse, thinke, murther, lords, kings, morrow, further, keepe, sight, present, stand, three …
    9 [o’th’, highnesse, euery, could, heare, against, murth, thought, duncan, onely]
    10 [haile, heart, other, whose, before, seruant, heere, downe, bloody]
    11 [againe, still, without, within, i’th’]
    12 [these, there’s]
    13 [worthy, death]
    14 [those, feare, scena]
    15 [strange, where, nature]
    17 [exeunt, blood, looke]
    16 [knock, things]
    19 [selfe, lenox, sleepe, should]
    21 [great, cawdor]
    20 [speake, there]
    23 [thane, would]
    25 [night]
    26 [rosse]
    30 [banquo]
    34 [macbeth]
    32 [their]
    36 [shall]
    48 [enter]
    57 [which]
    And yes, for those of you that are not native English speakers, they really are English words, or rather it is quite old English, or I should say Olde English as Shakespeare wrote Macbeth in around 1603. The document I used is scanned from an original first edition and has some archaic spellings and printing idiosyncrasies that are expliained very well in the introduction of the document. If you want a more readable version there is another one here More readable Macbeth

    I have truncated some of the longer lists of words otherwise the results would have filled the page. At first it appears that the list (or Map rather) is in some sort of order, but it is not really and you can see the later entries are not in usage count order.





    An Oracle Coherence Top n Query – Find the Top 5 Word Usages

    So back to our original requirement of trying to find the most used words. For my query requirement I want the top five lists of most used words – and I want them in order. From the list above this would be…

    57 [which]
    48 [enter]
    36 [shall]
    34 [macbeth]
    32 [their]

    …we would expect to see Macbeth make it into the top list with 34 occurrences but poor old Banquo just missed out on 30 occurrences – although that is the least of Banquo’s problems as Macbeth has him murdered.

    Now, to obtain the top five I could do the same query I just did above then when I have the results back on the client I could sort them and pull off the top five entries, but that is cheating a bit and although in this case the data set is small that may not always be the case. Say I had a cache of many GBytes of data, which is not unusual on the size of cluster I work with, I could not bring all of that back to a client to sort and cut down, far better, and far faster, to get each cluster member to do a bit of the work.

    — Macbeth:
    Auant, & quit my sight, let the earth hide thee:
    Thy bones are marrowlesse, thy blood is cold:
    Thou hast no speculation in those eyes
    Which thou dost glare with.
    Macbeth (III, iv)

    Coherence has two ways to limit the results returned from a query, either a Filter or an Aggregator, or combination of both. In this case a Filter on its own will not do the job as this just restricts the entries returned to matching a criteria. An aggregator on the other hand does have the ability to do what we want. In my example I am using a GroupAggregator but I obviously I do not want to write a specialised case of GroupAggregator that limits its result set as this would not work if I wanted to limit other types of aggregator. For example, say I didn’t want the GroupBy/DistinctValues combination but just wanted the top ten longest words – I would just use a ReducerAggregator. What I need to write is an aggregator that wraps another aggregator and can limit (optionally after ordering) the results and without further ado (or keeping the Shakespear theme “without much ado“) here is the code…

    1. /*
    2.  * File: LimitAggregator.java
    3.  *
    4.  * Copyright (c) 2012. All Rights Reserved. Jonathan Knight.
    5.  *
    6.  * Jonathan Knight makes no representations or warranties about the
    7.  * suitability of the software, either express or implied, including but not
    8.  * limited to the implied warranties of merchantability, fitness for a
    9.  * particular purpose, or non-infringement. Jonathan Knight shall not be
    10.  * liable for any damages suffered by licensee as a result of using, modifying
    11.  * or distributing this software or its derivatives.
    12.  *
    13.  * This notice may not be removed or altered.
    14.  */
    15. package com.thegridman.coherence.aggregators;
    16.  
    17. import com.tangosol.io.pof.PofReader;
    18. import com.tangosol.io.pof.PofWriter;
    19. import com.tangosol.io.pof.PortableObject;
    20. import com.tangosol.util.Filter;
    21. import com.tangosol.util.InvocableMap;
    22. import com.tangosol.util.SimpleMapEntry;
    23. import com.tangosol.util.comparator.EntryComparator;
    24.  
    25. import java.io.IOException;
    26. import java.util.*;
    27.  
    28. /**
    29.  * This class is an Aggregator that wraps another aggregator to limit the amount of data returned.
    30.  * It would typically be used with aggregators that return Collections or Maps to perform the
    31.  * equivalent of Top <i>n</i> queries, for example, find the to 5 valuations by valuation amount.
    32.  *
    33.  * @author Jonathan Knight
    34.  */
    35. public class LimitAggregator implements InvocableMap.ParallelAwareAggregator, PortableObject
    36. {
    37.     /** The underlying aggregator that will obtain the un-ordered, unlimited results */
    38.     private InvocableMap.EntryAggregator aggregator;
    39.  
    40.     /** The maximum number of results to return */
    41.     private long limit;
    42.  
    43.     /** A flag that indicates whether the results should be ordered prior to limiting */
    44.     private boolean ordered;
    45.  
    46.     /** A comparator to use to order the results if the ordered flag is true.
    47.      * If null and ordered is true the results should be Comparable */
    48.     private Comparator comparator;
    49.  
    50.     private int mapComparatorTarget = EntryComparator.CMP_KEY;
    51.  
    52.     /** Empty constructor for POF */
    53.     public LimitAggregator()
    54.     {
    55.     }
    56.  
    57.     /**
    58.      * Create an instance of a LimitAggregator that will imit the results of the specified
    59.      * aggregator to a maximum number of optionally sorted results.
    60.      *
    61.      * @param aggregator - the aggregator to get the data
    62.      * @param limit      - the maximum number of results to return
    63.      * @param ordered    - should the results be sorted prior to limiting
    64.      * @param comparator - optional Comparator to use to sort results
    65.      */
    66.     public LimitAggregator(InvocableMap.EntryAggregator aggregator, long limit, boolean ordered, Comparator comparator)
    67.     {
    68.         this(aggregator, limit, ordered, comparator, EntryComparator.CMP_KEY);
    69.     }
    70.  
    71.     /**
    72.      * Create an instance of a LimitAggregator that will imit the results of the specified
    73.      * aggregator to a maximum number of optionally sorted results.
    74.      *
    75.      * @param aggregator - the aggregator to get the data
    76.      * @param limit      - the maximum number of results to return
    77.      * @param ordered    - should the results be sorted prior to limiting
    78.      * @param comparator - optional Comparator to use to sort results
    79.      * @param mapComparatorTarget - specified how to sort a Map result set prior to truncation. See {@link EntryComparator}
    80.      */
    81.     public LimitAggregator(InvocableMap.EntryAggregator aggregator, long limit, boolean ordered, Comparator comparator, int mapComparatorTarget)
    82.     {
    83.         this.aggregator = aggregator;
    84.         this.limit = limit;
    85.         this.ordered = ordered;
    86.         this.comparator = comparator;
    87.         this.mapComparatorTarget = mapComparatorTarget;
    88.     }
    89.  
    90.     /**
    91.      * Implementation of {@link InvocableMap.ParallelAwareAggregator} getParallelAggregator method
    92.      *
    93.      * @return this aggregator
    94.      */
    95.     @Override
    96.     public InvocableMap.EntryAggregator getParallelAggregator()
    97.     {
    98.         return this;
    99.     }
    100.  
    101.     /**
    102.      * Implementation of {@link InvocableMap.ParallelAwareAggregator} aggregate method
    103.      *
    104.      * @return the result of this aggregator
    105.      */
    106.     @Override
    107.     public Object aggregate(Set entries)
    108.     {
    109.         return aggregator.aggregate(entries);
    110.     }
    111.  
    112.     /**
    113.      * Implementation of {@link InvocableMap.ParallelAwareAggregator} method to aggregate the
    114.      * Collection of results.
    115.      *
    116.      * @param results - the results to be aggregated.
    117.      * @return the aggregated results
    118.      */
    119.     @SuppressWarnings({"unchecked"})
    120.     @Override
    121.     public Object aggregateResults(Collection results)
    122.     {
    123.         Object limited;
    124.         if (aggregator instanceof InvocableMap.ParallelAwareAggregator)
    125.         {
    126.             Object result = ((InvocableMap.ParallelAwareAggregator)aggregator).aggregateResults(results);
    127.             if (result instanceof Collection)
    128.             {
    129.                 limited = truncateCollection((Collection) result);
    130.             }
    131.             else if (result instanceof Map)
    132.             {
    133.                 limited = truncateMap((Map) result);
    134.             }
    135.             else
    136.             {
    137.                 limited = Collections.singleton(result);
    138.             }
    139.         }
    140.         else
    141.         {
    142.             limited = truncateCollection(results);
    143.         }
    144.         return limited;
    145.     }
    146.  
    147.     /**
    148.      * Method called by clients to execute the aggregator.
    149.      *
    150.      * This is required to make sure that any results are correctly sorted
    151.      * as when a SortedMap is serialized on a server and deserialized on the
    152.      * client it is no longer sorted.
    153.      *
    154.      * @param map    - the Map (cache) to aggregate
    155.      * @param filter - the Filter to apply to the aggregate call
    156.      * @return the results of the limited aggregator call.
    157.      */
    158.     @SuppressWarnings({"unchecked"})
    159.     public Object aggregate(InvocableMap map, Filter filter)
    160.     {
    161.         Object results = map.aggregate(filter, this);
    162.         if (results instanceof Collection)
    163.         {
    164.             results = copyCollection((Collection) results);
    165.         }
    166.         else if (results instanceof Map)
    167.         {
    168.             results = copyMap((Map) results);
    169.         }
    170.         return results;
    171.     }
    172.  
    173.     /**
    174.      * Method called by clients to execute the aggregator.
    175.      *
    176.      * This is required to make sure that any results are correctly sorted
    177.      * as when a SortedMap is serialized on a server and deserialized on the
    178.      * client it is no longer sorted.
    179.      *
    180.      * @param map    - the Map (cache) to aggregate
    181.      * @param keys   - the set of keys to aggregate
    182.      * @return the results of the limited aggregator call.
    183.      */
    184.     @SuppressWarnings({"unchecked"})
    185.     public Object aggregate(InvocableMap map, Collection keys)
    186.     {
    187.         Object results = map.aggregate(keys, this);
    188.         if (results instanceof Collection)
    189.         {
    190.             results = copyCollection((Collection) results);
    191.         }
    192.         else if (results instanceof Map)
    193.         {
    194.             results = copyMap((Map) results);
    195.         }
    196.         return results;
    197.     }
    198.     /**
    199.      * Truncate a Map down to the required size.
    200.      *
    201.      * @param mapToTruncate - the map to be truncated
    202.      * @return a copy of the specified Map truncated to the required number of entries.
    203.      */
    204.     public <K,V> Map<K,V> truncateMap(Map<K,V> mapToTruncate)
    205.     {
    206.         Map<K,V> map = copyMap(mapToTruncate);
    207.         truncate(map.keySet());
    208.         return map;
    209.     }
    210.  
    211.     /**
    212.      * Truncate a Collection down to the required size.
    213.      * The Collection will be sorted prior to truncation if required.
    214.      *
    215.      * @param collectionToTruncate - the Collection to be truncated
    216.      * @return a copy of the Collection truncated to the required size.
    217.      */
    218.     public <T> Collection<T> truncateCollection(Collection<T> collectionToTruncate)
    219.     {
    220.         Collection<T> collection = copyCollection(collectionToTruncate);
    221.         truncate(collection);
    222.         return collection;
    223.     }
    224.  
    225.     /**
    226.      * Truncate a Collection.
    227.      *
    228.      * @param collection - the collection to truncate
    229.      */
    230.     private void truncate(Collection collection)
    231.     {
    232.         Iterator it = collection.iterator();
    233.         int count = 0;
    234.         while(it.hasNext())
    235.         {
    236.             it.next();
    237.             if (count < limit)
    238.              {
    239.                  count++;
    240.                  continue;
    241.              }
    242.              it.remove();
    243.          }
    244.      }
    245.  
    246.      /**
    247.       * Create a copy of the specified Collection.
    248.       * If the ordered field is true then the resulting Collection
    249.       * will be ordered.
    250.       *
    251.       * @param collection - the collection to copy and optionally sort.
    252.       * @return a copy, optionally sorted, of the specified Collection.
    253.       */
    254.      @SuppressWarnings({"unchecked"})
    255.      public <T> Collection<T> copyCollection(Collection<T> collection)
    256.     {
    257.         List list = new ArrayList(collection);
    258.         if (ordered)
    259.         {
    260.             Collections.sort(list, comparator);
    261.         }
    262.         return list;
    263.     }
    264.  
    265.     /**
    266.      * Create a copy of the specified Map.
    267.      * If the ordered field is true then the resulting Map
    268.      * will be ordered.
    269.      *
    270.      * @param map - the Map to copy and optionally sort.
    271.      * @return a copy, optionally sorted, of the specified Map.
    272.      */
    273.     @SuppressWarnings({"unchecked"})
    274.     public <K,V> Map<K,V> copyMap(Map<K, V> map)
    275.     {
    276.         Map<K,V> newMap;
    277.         if (ordered)
    278.         {
    279.             newMap = new LinkedHashMap<K, V>();
    280.             List<Map.Entry<K,V>> entries = new ArrayList<Map.Entry<K,V>>();
    281.             for (Map.Entry entry : map.entrySet())
    282.             {
    283.                 entries.add(new SimpleMapEntry(entry.getKey(), entry.getValue()));
    284.             }
    285.             EntryComparator entryComparator = new EntryComparator(comparator, mapComparatorTarget);
    286.             Collections.sort(entries, entryComparator);
    287.             for (Map.Entry<K,V> entry : entries)
    288.             {
    289.                 newMap.put(entry.getKey(), entry.getValue());
    290.             }
    291.         }
    292.         else
    293.         {
    294.             newMap = new HashMap<K,V>();
    295.             newMap.putAll(map);
    296.         }
    297.         return newMap;
    298.     }
    299.  
    300.     @Override
    301.     public void readExternal(PofReader pofReader) throws IOException
    302.     {
    303.         aggregator = (InvocableMap.EntryAggregator) pofReader.readObject(0);
    304.         limit = pofReader.readLong(1);
    305.         comparator = (Comparator) pofReader.readObject(2);
    306.         ordered = pofReader.readBoolean(3);
    307.         mapComparatorTarget = pofReader.readInt(4);
    308.     }
    309.  
    310.     @Override
    311.     public void writeExternal(PofWriter pofWriter) throws IOException
    312.     {
    313.         pofWriter.writeObject(0, aggregator);
    314.         pofWriter.writeLong(1, limit);
    315.         pofWriter.writeObject(2, comparator);
    316.         pofWriter.writeBoolean(3, ordered);
    317.         pofWriter.writeInt(4, mapComparatorTarget);
    318.     }
    319.  
    320.     public InvocableMap.EntryAggregator getAggregator()
    321.     {
    322.         return aggregator;
    323.     }
    324.  
    325.     public long getLimit()
    326.     {
    327.         return limit;
    328.     }
    329.  
    330.     public boolean isOrdered()
    331.     {
    332.         return ordered;
    333.     }
    334.  
    335.     public Comparator getComparator()
    336.     {
    337.         return comparator;
    338.     }
    339.  
    340. }
    /*
     * File: LimitAggregator.java
     *
     * Copyright (c) 2012. All Rights Reserved. Jonathan Knight.
     *
     * Jonathan Knight makes no representations or warranties about the
     * suitability of the software, either express or implied, including but not
     * limited to the implied warranties of merchantability, fitness for a
     * particular purpose, or non-infringement. Jonathan Knight shall not be
     * liable for any damages suffered by licensee as a result of using, modifying
     * or distributing this software or its derivatives.
     *
     * This notice may not be removed or altered.
     */
    package com.thegridman.coherence.aggregators;
    
    import com.tangosol.io.pof.PofReader;
    import com.tangosol.io.pof.PofWriter;
    import com.tangosol.io.pof.PortableObject;
    import com.tangosol.util.Filter;
    import com.tangosol.util.InvocableMap;
    import com.tangosol.util.SimpleMapEntry;
    import com.tangosol.util.comparator.EntryComparator;
    
    import java.io.IOException;
    import java.util.*;
    
    /**
     * This class is an Aggregator that wraps another aggregator to limit the amount of data returned.
     * It would typically be used with aggregators that return Collections or Maps to perform the
     * equivalent of Top <i>n</i> queries, for example, find the to 5 valuations by valuation amount.
     *
     * @author Jonathan Knight
     */
    public class LimitAggregator implements InvocableMap.ParallelAwareAggregator, PortableObject
    {
        /** The underlying aggregator that will obtain the un-ordered, unlimited results */
        private InvocableMap.EntryAggregator aggregator;
    
        /** The maximum number of results to return */
        private long limit;
    
        /** A flag that indicates whether the results should be ordered prior to limiting */
        private boolean ordered;
    
        /** A comparator to use to order the results if the ordered flag is true.
         * If null and ordered is true the results should be Comparable */
        private Comparator comparator;
    
        private int mapComparatorTarget = EntryComparator.CMP_KEY;
    
        /** Empty constructor for POF */
        public LimitAggregator()
        {
        }
    
        /**
         * Create an instance of a LimitAggregator that will imit the results of the specified
         * aggregator to a maximum number of optionally sorted results.
         *
         * @param aggregator - the aggregator to get the data
         * @param limit      - the maximum number of results to return
         * @param ordered    - should the results be sorted prior to limiting
         * @param comparator - optional Comparator to use to sort results
         */
        public LimitAggregator(InvocableMap.EntryAggregator aggregator, long limit, boolean ordered, Comparator comparator)
        {
            this(aggregator, limit, ordered, comparator, EntryComparator.CMP_KEY);
        }
    
        /**
         * Create an instance of a LimitAggregator that will imit the results of the specified
         * aggregator to a maximum number of optionally sorted results.
         *
         * @param aggregator - the aggregator to get the data
         * @param limit      - the maximum number of results to return
         * @param ordered    - should the results be sorted prior to limiting
         * @param comparator - optional Comparator to use to sort results
         * @param mapComparatorTarget - specified how to sort a Map result set prior to truncation. See {@link EntryComparator}
         */
        public LimitAggregator(InvocableMap.EntryAggregator aggregator, long limit, boolean ordered, Comparator comparator, int mapComparatorTarget)
        {
            this.aggregator = aggregator;
            this.limit = limit;
            this.ordered = ordered;
            this.comparator = comparator;
            this.mapComparatorTarget = mapComparatorTarget;
        }
    
        /**
         * Implementation of {@link InvocableMap.ParallelAwareAggregator} getParallelAggregator method
         *
         * @return this aggregator
         */
        @Override
        public InvocableMap.EntryAggregator getParallelAggregator()
        {
            return this;
        }
    
        /**
         * Implementation of {@link InvocableMap.ParallelAwareAggregator} aggregate method
         *
         * @return the result of this aggregator
         */
        @Override
        public Object aggregate(Set entries)
        {
            return aggregator.aggregate(entries);
        }
    
        /**
         * Implementation of {@link InvocableMap.ParallelAwareAggregator} method to aggregate the
         * Collection of results.
         *
         * @param results - the results to be aggregated.
         * @return the aggregated results
         */
        @SuppressWarnings({"unchecked"})
        @Override
        public Object aggregateResults(Collection results)
        {
            Object limited;
            if (aggregator instanceof InvocableMap.ParallelAwareAggregator)
            {
                Object result = ((InvocableMap.ParallelAwareAggregator)aggregator).aggregateResults(results);
                if (result instanceof Collection)
                {
                    limited = truncateCollection((Collection) result);
                }
                else if (result instanceof Map)
                {
                    limited = truncateMap((Map) result);
                }
                else
                {
                    limited = Collections.singleton(result);
                }
            }
            else
            {
                limited = truncateCollection(results);
            }
            return limited;
        }
    
        /**
         * Method called by clients to execute the aggregator.
         *
         * This is required to make sure that any results are correctly sorted
         * as when a SortedMap is serialized on a server and deserialized on the
         * client it is no longer sorted.
         *
         * @param map    - the Map (cache) to aggregate
         * @param filter - the Filter to apply to the aggregate call
         * @return the results of the limited aggregator call.
         */
        @SuppressWarnings({"unchecked"})
        public Object aggregate(InvocableMap map, Filter filter)
        {
            Object results = map.aggregate(filter, this);
            if (results instanceof Collection)
            {
                results = copyCollection((Collection) results);
            }
            else if (results instanceof Map)
            {
                results = copyMap((Map) results);
            }
            return results;
        }
    
        /**
         * Method called by clients to execute the aggregator.
         *
         * This is required to make sure that any results are correctly sorted
         * as when a SortedMap is serialized on a server and deserialized on the
         * client it is no longer sorted.
         *
         * @param map    - the Map (cache) to aggregate
         * @param keys   - the set of keys to aggregate
         * @return the results of the limited aggregator call.
         */
        @SuppressWarnings({"unchecked"})
        public Object aggregate(InvocableMap map, Collection keys)
        {
            Object results = map.aggregate(keys, this);
            if (results instanceof Collection)
            {
                results = copyCollection((Collection) results);
            }
            else if (results instanceof Map)
            {
                results = copyMap((Map) results);
            }
            return results;
        }
        /**
         * Truncate a Map down to the required size.
         *
         * @param mapToTruncate - the map to be truncated
         * @return a copy of the specified Map truncated to the required number of entries.
         */
        public <K,V> Map<K,V> truncateMap(Map<K,V> mapToTruncate)
        {
            Map<K,V> map = copyMap(mapToTruncate);
            truncate(map.keySet());
            return map;
        }
    
        /**
         * Truncate a Collection down to the required size.
         * The Collection will be sorted prior to truncation if required.
         *
         * @param collectionToTruncate - the Collection to be truncated
         * @return a copy of the Collection truncated to the required size.
         */
        public <T> Collection<T> truncateCollection(Collection<T> collectionToTruncate)
        {
            Collection<T> collection = copyCollection(collectionToTruncate);
            truncate(collection);
            return collection;
        }
    
        /**
         * Truncate a Collection.
         *
         * @param collection - the collection to truncate
         */
        private void truncate(Collection collection)
        {
            Iterator it = collection.iterator();
            int count = 0;
            while(it.hasNext())
            {
                it.next();
                if (count < limit)
                 {
                     count++;
                     continue;
                 }
                 it.remove();
             }
         }
     
         /**
          * Create a copy of the specified Collection.
          * If the ordered field is true then the resulting Collection
          * will be ordered.
          *
          * @param collection - the collection to copy and optionally sort.
          * @return a copy, optionally sorted, of the specified Collection.
          */
         @SuppressWarnings({"unchecked"})
         public <T> Collection<T> copyCollection(Collection<T> collection)
        {
            List list = new ArrayList(collection);
            if (ordered)
            {
                Collections.sort(list, comparator);
            }
            return list;
        }
    
        /**
         * Create a copy of the specified Map.
         * If the ordered field is true then the resulting Map
         * will be ordered.
         *
         * @param map - the Map to copy and optionally sort.
         * @return a copy, optionally sorted, of the specified Map.
         */
        @SuppressWarnings({"unchecked"})
        public <K,V> Map<K,V> copyMap(Map<K, V> map)
        {
            Map<K,V> newMap;
            if (ordered)
            {
                newMap = new LinkedHashMap<K, V>();
                List<Map.Entry<K,V>> entries = new ArrayList<Map.Entry<K,V>>();
                for (Map.Entry entry : map.entrySet())
                {
                    entries.add(new SimpleMapEntry(entry.getKey(), entry.getValue()));
                }
                EntryComparator entryComparator = new EntryComparator(comparator, mapComparatorTarget);
                Collections.sort(entries, entryComparator);
                for (Map.Entry<K,V> entry : entries)
                {
                    newMap.put(entry.getKey(), entry.getValue());
                }
            }
            else
            {
                newMap = new HashMap<K,V>();
                newMap.putAll(map);
            }
            return newMap;
        }
    
        @Override
        public void readExternal(PofReader pofReader) throws IOException
        {
            aggregator = (InvocableMap.EntryAggregator) pofReader.readObject(0);
            limit = pofReader.readLong(1);
            comparator = (Comparator) pofReader.readObject(2);
            ordered = pofReader.readBoolean(3);
            mapComparatorTarget = pofReader.readInt(4);
        }
    
        @Override
        public void writeExternal(PofWriter pofWriter) throws IOException
        {
            pofWriter.writeObject(0, aggregator);
            pofWriter.writeLong(1, limit);
            pofWriter.writeObject(2, comparator);
            pofWriter.writeBoolean(3, ordered);
            pofWriter.writeInt(4, mapComparatorTarget);
        }
    
        public InvocableMap.EntryAggregator getAggregator()
        {
            return aggregator;
        }
    
        public long getLimit()
        {
            return limit;
        }
    
        public boolean isOrdered()
        {
            return ordered;
        }
    
        public Comparator getComparator()
        {
            return comparator;
        }
    
    }

    …and for those that want it (and have JUnit, Mockito and Hamcrest) here is the corresponding unit test class…

    1. /*
    2.  * File: LimitAggregatorTest.java
    3.  *
    4.  * Copyright (c) 2012. All Rights Reserved. Jonathan Knight.
    5.  *
    6.  * Jonathan Knight makes no representations or warranties about the
    7.  * suitability of the software, either express or implied, including but not
    8.  * limited to the implied warranties of merchantability, fitness for a
    9.  * particular purpose, or non-infringement. Jonathan Knight shall not be
    10.  * liable for any damages suffered by licensee as a result of using, modifying
    11.  * or distributing this software or its derivatives.
    12.  *
    13.  * This notice may not be removed or altered.
    14.  */
    15. package com.thegridman.coherence.aggregators;
    16.  
    17. import com.tangosol.io.pof.ConfigurablePofContext;
    18. import com.tangosol.util.Binary;
    19. import com.tangosol.util.ExternalizableHelper;
    20. import com.tangosol.util.Filter;
    21. import com.tangosol.util.InvocableMap;
    22. import com.tangosol.util.aggregator.DistinctValues;
    23. import com.tangosol.util.comparator.EntryComparator;
    24. import com.tangosol.util.comparator.InverseComparator;
    25. import com.tangosol.util.filter.EqualsFilter;
    26. import org.junit.Test;
    27.  
    28. import java.util.Arrays;
    29. import java.util.Collection;
    30. import java.util.HashMap;
    31. import java.util.HashSet;
    32. import java.util.Map;
    33. import java.util.Set;
    34.  
    35. import static org.hamcrest.CoreMatchers.instanceOf;
    36. import static org.hamcrest.CoreMatchers.is;
    37. import static org.hamcrest.CoreMatchers.not;
    38. import static org.hamcrest.CoreMatchers.sameInstance;
    39. import static org.hamcrest.collection.IsIterableContainingInOrder.contains;
    40. import static org.hamcrest.collection.IsMapContaining.hasEntry;
    41. import static org.junit.Assert.assertThat;
    42. import static org.mockito.Matchers.any;
    43. import static org.mockito.Matchers.anyCollection;
    44. import static org.mockito.Matchers.anySet;
    45. import static org.mockito.Matchers.same;
    46. import static org.mockito.Mockito.mock;
    47. import static org.mockito.Mockito.verify;
    48. import static org.mockito.Mockito.when;
    49.  
    50. /**
    51.  * @author Jonathan Knight
    52.  */
    53. public class LimitAggregatorTest
    54. {
    55.     @Test
    56.     public void shouldSerializeAndDeserialize() throws Exception
    57.     {
    58.         ConfigurablePofContext pofContext = new ConfigurablePofContext("gridman-pof-config.xml");
    59.  
    60.         LimitAggregator aggregator = new LimitAggregator(new DistinctValues(), 100L, true, new EntryComparator());
    61.         Binary binary = ExternalizableHelper.toBinary(aggregator, pofContext);
    62.         LimitAggregator result = (LimitAggregator) ExternalizableHelper.fromBinary(binary, pofContext);
    63.  
    64.         assertThat(result.getAggregator(), is(instanceOf(DistinctValues.class)));
    65.         assertThat(result.getLimit(), is(100L));
    66.         assertThat(result.isOrdered(), is(true));
    67.         assertThat(result.getComparator(), is(instanceOf(EntryComparator.class)));
    68.     }
    69.  
    70.     @Test
    71.     public void shouldReturnSelfForParallelAggregator() throws Exception
    72.     {
    73.         LimitAggregator aggregator = new LimitAggregator(new DistinctValues(), 100L, true, new EntryComparator());
    74.         assertThat((LimitAggregator) aggregator.getParallelAggregator(), is(sameInstance(aggregator)));
    75.     }
    76.  
    77.     @Test
    78.     public void shouldCreateCopyOfCollectionInSameOrder() throws Exception
    79.     {
    80.         LimitAggregator aggregator = new LimitAggregator(new DistinctValues(), 100L, false, null);
    81.  
    82.         Collection<String> collection = Arrays.asList("1", "3", "4", "2");
    83.  
    84.         Collection<String> result = aggregator.copyCollection(collection);
    85.         assertThat(result, is(not(sameInstance(collection))));
    86.         assertThat(result, contains("1", "3", "4", "2"));
    87.     }
    88.  
    89.     @Test
    90.     public void shouldCreateOrderedCopyOfCollection() throws Exception
    91.     {
    92.         LimitAggregator aggregator = new LimitAggregator(new DistinctValues(), 100L, true, null);
    93.  
    94.         Collection<String> collection = Arrays.asList("1", "4", "3", "2");
    95.  
    96.         Collection<String> result = aggregator.copyCollection(collection);
    97.         assertThat(result, is(not(sameInstance(collection))));
    98.         assertThat(result, contains("1", "2", "3", "4"));
    99.     }
    100.  
    101.     @Test
    102.     public void shouldCreateOrderedCopyOfCollectionUsingComparator() throws Exception
    103.     {
    104.         LimitAggregator aggregator = new LimitAggregator(new DistinctValues(), 100L, true, new InverseComparator());
    105.  
    106.         Collection<String> collection = Arrays.asList("1", "4", "3", "2");
    107.  
    108.         Collection<String> result = aggregator.copyCollection(collection);
    109.         assertThat(result, is(not(sameInstance(collection))));
    110.         assertThat(result, contains("4", "3", "2", "1"));
    111.     }
    112.  
    113.     @Test
    114.     public void shouldCreateTruncatedCopyOfCollectionInSameOrder() throws Exception
    115.     {
    116.         LimitAggregator aggregator = new LimitAggregator(new DistinctValues(), 3, false, null);
    117.  
    118.         Collection<String> collection = Arrays.asList("1", "3", "4", "2");
    119.  
    120.         Collection<String> result = aggregator.truncateCollection(collection);
    121.         assertThat(result, is(not(sameInstance(collection))));
    122.         assertThat(result, contains("1", "3", "4"));
    123.     }
    124.  
    125.     @Test
    126.     public void shouldCreateTruncatedOrderedCopyOfCollection() throws Exception
    127.     {
    128.         LimitAggregator aggregator = new LimitAggregator(new DistinctValues(), 2, true, null);
    129.  
    130.         Collection<String> collection = Arrays.asList("1", "4", "3", "2");
    131.  
    132.         Collection<String> result = aggregator.truncateCollection(collection);
    133.         assertThat(result, is(not(sameInstance(collection))));
    134.         assertThat(result, contains("1", "2"));
    135.     }
    136.  
    137.     @Test
    138.     public void shouldCreateCopyOfMap() throws Exception
    139.     {
    140.         LimitAggregator aggregator = new LimitAggregator(new DistinctValues(), 100L, false, null);
    141.  
    142.         Map<String,String> map = new HashMap<String,String>();
    143.         map.put("1", "value-1");
    144.         map.put("4", "value-4");
    145.         map.put("3", "value-3");
    146.         map.put("2", "value-2");
    147.  
    148.         Map<String,String> result = aggregator.copyMap(map);
    149.         assertThat(result, is(not(sameInstance(map))));
    150.         assertThat(result, is(map));
    151.     }
    152.  
    153.     @Test
    154.     public void shouldCreateOrderedCopyOfMap() throws Exception
    155.     {
    156.         LimitAggregator aggregator = new LimitAggregator(new DistinctValues(), 100L, true, null);
    157.  
    158.         Map<String,String> map = new HashMap<String,String>();
    159.         map.put("1", "value-1");
    160.         map.put("4", "value-4");
    161.         map.put("3", "value-3");
    162.         map.put("2", "value-2");
    163.  
    164.         Map<String,String> result = aggregator.copyMap(map);
    165.         assertThat(result, is(not(sameInstance(map))));
    166.         assertThat(result, hasEntry("1", "value-1"));
    167.         assertThat(result, hasEntry("4", "value-4"));
    168.         assertThat(result, hasEntry("3", "value-3"));
    169.         assertThat(result, hasEntry("2", "value-2"));
    170.         assertThat(result.keySet(), contains("1", "2", "3", "4"));
    171.     }
    172.  
    173.     @Test
    174.     public void shouldCreateOrderedCopyOfMapUsingComparator() throws Exception
    175.     {
    176.         LimitAggregator aggregator = new LimitAggregator(new DistinctValues(), 100L, true, new InverseComparator());
    177.  
    178.         Map<String,String> map = new HashMap<String,String>();
    179.         map.put("1", "value-1");
    180.         map.put("4", "value-4");
    181.         map.put("3", "value-3");
    182.         map.put("2", "value-2");
    183.  
    184.         Map<String,String> result = aggregator.copyMap(map);
    185.         assertThat(result, is(not(sameInstance(map))));
    186.         assertThat(result, hasEntry("1", "value-1"));
    187.         assertThat(result, hasEntry("4", "value-4"));
    188.         assertThat(result, hasEntry("3", "value-3"));
    189.         assertThat(result, hasEntry("2", "value-2"));
    190.         assertThat(result.keySet(), contains("4", "3", "2", "1"));
    191.     }
    192.  
    193.     @Test
    194.     public void shouldCreateO