Coherence 12.1.2 Custom Namespaces and Oracle NoSQL

Oracle Coherence 12.1.2 Custom Namespaces is a new feature in Coherence 12.1.2 that allows you to add custom functionality to the Coherence cache configuration file. If anyone has used custom namespaces with something like the Spring Framework, then Coherence namespaces are very similar.

This blog post is going to work through an example of building a custom namespace. In this case we are going to use it to integrate Coherence with the Oracle NoSQL database. Now, those of you that have looked at Oracle NoSQL, especially the enterprise edition might be saying, “hang on, Oracle NoSQL already has Coherence integration” and you would be correct, but it does not work with Coherence version 12.1.2 so we are going to change that here. All the code from this blog is available here on my GitHub repository

First a little history lesson…

Custom namespace support has been in Coherence for quite some time now if you’ve been using the Coherence Incubator Commons library. It all started…
A long time ago in a galaxy far, far away….
OK, it started in Canary Wharf, London, in the heady days at the tail end of financial crash of 2008 something like this…
I remember seeing the first presentation of the Incubator Push Replication at the London Coherence SIG back in 2008, which at that time was all configured in code. I was flying from London to the US for Christmas just after the SIG, so during the flight I wrote some code to allow Push Replication to be configured inside the cache configuration file and e-mailed it to Brian Oliver at Oracle. On my return from London and on chatting to Brian, this had obviously triggered some thinking on his side and he had a lot of ideas about how custom namespaces could work (and typically for Brian, promised to write it over the weekend – LOL!). Well, it took a bit longer than a weekend to see the first namespaces support in the Incubator and finally now it is now fully supported in the core product. The new namespaces support in 12.1.2 is quite impressive and a lot of the classes that are used are actually there to allow the removal of the XML configuration file, but that is another blog post. You might think that was a long time to get something into the core product, and I’d agree, but this was quite a bit of work for the engineering team to refactor the whole of the cache configuration framework. Thinking back to those initial conversations, it is great to see something like that now in the core product – as they say “Mighty Oaks from little acorns grow”.

Coherence & Oracle NoSQL Integration

So, on with the meat of the post and back to our initial requirement – integrate Oracle NoSQL and Coherence. If you look at the Oracle NoSQL site you will see support for Coherence documented. In effect this support is currently (as of version 2.1.8) a couple of Coherence CacheStore classes that allow you to connect to NoSQL in the same way you would any other data source using a CacheStore. I’m not going to cover all about how this works; you are free to look at the Oracle NoSQL site to see the documentation. The problem with it though as I have said, is that it only works with versions of Coherence prior to 12.1.2. This is because the two CacheStore classes, oracle.kv.coherence.NoSQLBinaryStore and oracle.kv.coherence.NoSQLAvroCacheStore, and their base class oracle.kv.coherence.NoSQLStoreBase implement com.tangosol.run.xml.XmlConfigurable which means that any init-param parameters in cache configuration file where they are configured are passed to the setConfig of the class as a snippet of XML. The problem in 12.1.2 is that this is no longer supported – as I found out the hard way when porting a project from 3.7 to 12.1.2. Maybe it is documented but hey, how many developers read documentation ;-)

So, now we know Oracle NoSQL does not work with 12.1.2 how do we fix it? Well, as I’ve said, I had to do this on a project already, so I know one answer (and I think the elegant answer) is to use custom namespaces. In this way, Coherence does not do anything itself to build or configure the NoSQL CacheStore classes; instead it will delegate the job to our custom namespace classes. One other requirement of this is that whatever code we write, we will not be changing the actual NoSQL classes, so they will still work with the previous versions of Coherence.

This is how the NoSQL documentation says to configure a NoSQLBinaryStore cache store…

<distributed-scheme>
  <scheme-name>BinaryCacheScheme</scheme-name>
  <service-name>BinaryCacheService</service-name>
  <serializer>pof</serializer>
  <backing-map-scheme>
    <read-write-backing-map-scheme>
      <internal-cache-scheme>
        <local-scheme />
      </internal-cache-scheme>
      <cachestore-scheme>
        <class-scheme>
          <class-name>oracle.kv.coherence.NoSQLBinaryStore</class-name>
          <init-params>
            <init-param>
              <param-name>storeName</param-name>
              <param-value>kvstore</param-value>
            </init-param>
            <init-param>
              <param-name>helperHosts</param-name>
              <param-value>localhost:5000</param-value>
            </init-param>
          </init-params>
        </class-scheme>
      </cachestore-scheme>
    </read-write-backing-map-scheme>
  </backing-map-scheme>
  <autostart>true</autostart>
</distributed-scheme>

…That is taken directly from the NoSQL examples code. You can see that the cache store is defined using a normal class-scheme, and this would be fine if NoSQLBinaryStore had a constructor that took the two init-parm values as arguments, but it does not. In this case there are a lot of arguments that can be passed to the NoSQLBinaryStore and it would be impractical to have constructors that take all the different combinations, which is probably why it was written to use XMLConfigurable.

Using custom namespaces in 12.1.2 we will configure the same cache store like this…
Define the custom namespace in the cache-config element…

<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
              xmlns:kv="class://oracle.kv.coherence.NoSQLNamespaceHandler"
              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd">

…you can see we have specified out namespace with xmlns:kv

Use our namespace inside the cache-store-scheme element…

<distributed-scheme>
  <scheme-name>BinaryCacheScheme</scheme-name>
  <service-name>BinaryCacheService</service-name>
  <serializer>pof</serializer>
  <backing-map-scheme>
    <read-write-backing-map-scheme>
      <internal-cache-scheme>
        <local-scheme />
      </internal-cache-scheme>
      <cachestore-scheme>
        <kv:binary-cache-store>
          <kv:store-name system-property="oracle.kv.storename">kvstore</kv:store-name>
          <kv:helper-hosts system-property="oracle.kv.helperhosts"/> localhost:5000</ kv:helper-hosts>
        </kv:binary-cache-store>
      </cachestore-scheme>
    </read-write-backing-map-scheme>
  </backing-map-scheme>
  <autostart>true</autostart>
</distributed-scheme>

Now, that is very nice, what’s more, the custom namespace can have its own XSD file to allow the XML to be validated, which is a big plus for developers who can turn this validation on inside their IDEs (if you don’t do this you should be doing it).

So, when Coherence needs to instantiate a cache that maps to the BinaryCacheScheme scheme it will call our handlers to instantiate a CacheStore instance.

Custom Namespace Implementation

OK, that was the easy part, showing the finished configuration, but how do we make it work. We will start the explanation bottom up, starting with the class that builds the CacheStore and work our way back up to the namespace handler.

Schemes and Builders

As I said earlier there has been a lot of refactoring and new work done around how Coherence configuration works. A whole new set of classes called schemes and builders were written that basically configure and build the various parts of Coherence. Scheme appear to be the configuration and work either on their own or with builders to instantiate instances of various Coherence components. What this means in effect is that you do not actually need an XML configuration file to run Coherence. I have had a whole cluster running all configured in code with no XML – I’ll write that up in another post.

So, for our NoSQL integration we need a scheme that will hold the configuration for our CacheStore instances. There are two CacheStore classes in the NoSQL integration framework, but there is a lot of commonality, so we will write a scheme for each with a common base class.

The scheme class is what Coherence will call to inject in the various parameters from the XML configuration.

Here is the skeleton of our scheme for creating the NoSQL cache stores

public abstract class NoSQLBaseCacheStoreScheme 
    extends AbstractScheme 
    implements ParameterizedBuilder<NoSQLStoreBase>, ParameterizedBuilder.ReflectionSupport {

    public NoSQLBaseCacheStoreScheme() {
    }

    @Override
    public NoSQLStoreBase realize(ParameterResolver resolver, ClassLoader classLoader, ParameterList parameterList)
    {
    return null;
    }

    @Override
    public boolean realizes(Class<?> paramClass, ParameterResolver paramParameterResolver, ClassLoader paramClassLoader) 
    {
        return true;
    }
}

This is the bear minimum for our scheme class. The realize method is the one that will actually be responsible for instantiating our cache store instance. The realizes method is used by Coherence to check whether this scheme realizes classes of the required type.

But where is the XML configuration set? For our scheme to be passed the values from the XML configuration we need to add set methods for each parameter, that is, for each XML tag we want a value for. These methods are then annotated to tell Coherence which XML element to inject.

For example, looking at the configuration above that used our new namespace we have this…

<kv:binary-cache-store>
  <kv:store-name system-property="oracle.kv.storename">kvstore</kv:store-name>
  <kv:helper-hosts system-property="oracle.kv.helperhosts"/> localhost:5000</ kv:helper-hosts>
</kv:binary-cache-store>

This XML contains two elements (the full scheme will contain more) that need injecting, these are kv:store-name and kv:helper-hosts so we need two setters.

@Injectable("store-name")
public void setStoreName(Expression<String> expression)
{
    this.storeNameExpression = expression;
}

@Injectable("helper-hosts")
public void setHelperHosts(Expression<String> expression)
{
     this.helperHostsExpression = expression;
}

As you can see these are simple setter methods. Coherence passes in an Expression of the required type (in this case both attributes are Strings). The @Injectable annotation tells Coherence which XML element to inject, all very simple.

When we need to build an instance of our cache store we then evaluate the expressions passed in to give us the real values to use. We do this in the realize method.

@Override
public NoSQLStoreBase realize(ParameterResolver resolver, ClassLoader classLoader, ParameterList parameterList)
{
    String storeName = storeNameExpression.evaluate(resolver);
    String helperHosts = storeNameExpression.evaluate(resolver);

    NoSQLStoreBase store = new …

    Return store;
}

One of the requirements I mentioned earlier is that we are not going to change any of the existin NoSQL code as we ant it to remain compatible with earlier Coherence versions. This means that we need to instantiate our cache store implementations by calling new and then calling setConfig and pass in some XML. To do this we will just build up the relevant XML configuration in our realize method and pass it to the cache store; for example:

@Override
public NoSQLStoreBase realize(ParameterResolver resolver, ClassLoader classLoader, ParameterList parameterList)
{
    XmlElement configXml = new SimpleElement("config");
    addElementIfNotNull(configXml, XmlConfigParser.OPT_STORENAME, getStoreName(resolver));
    addElementIfNotNull(configXml, XmlConfigParser.OPT_HELPERHOSTS, getHelperHosts(resolver));

    NoSQLStoreBase store = instantiateStore();
    store.setConfig(configXml);
    return store;
}

public String getStoreName(ParameterResolver resolver)
{
    return (storeNameExpression != null) ? storeNameExpression.evaluate(resolver) : null;
}

public String getHelperHosts(ParameterResolver resolver)
{
    return (helperHostsExpression == null) ? null : helperHostsExpression.evaluate(resolver);
}

private void addElementIfNotNull(XmlElement parent, String elementName, Object value)
{
    if (value == null)
    {
        return;
    }

    parent.addElement(elementName).setString(String.valueOf(value));
}

The instantiateStore () method used above is not shown here as this code is all in our base scheme class. The instantiateStore() method will be in the sub-classes so that it can create the correct store implementation. For example, the sub-class looks like this

public class NoSQLBinaryStoreScheme extends NoSQLBaseCacheStoreScheme {

    public NoSQLBinaryStoreScheme() {
        super(EnumSet.complementOf(EnumSet.of(XmlConfigParser.ConfigOption.AVRO_FORMAT, XmlConfigParser.ConfigOption.SCHEMA_FILES)));
    }

    @Override
    protected NoSQLStoreBase instantiateStore() {
        return new NoSQLBinaryStore();
    }

    @Override
    public boolean realizes(Class<?> aClass, ParameterResolver resolver, ClassLoader classLoader) {
        return aClass.isAssignableFrom(NoSQLBinaryStore.class);
    }
}

If we were only going to create Coherence cache factory implementations programmatically then a scheme would be enough for us to work with. For most people though we will now need to pug our scheme into the XML configuration processing.

ElementProcessor

For our custom namespace to work we need to register an instance of a com.tangosol.config.xml.ElementProcessor for each custom XML element we process in our namespace. In our case we are going to have two top level elements, one for each of the custom CacheStore implementations. In Oracle NoSQL there is the basic oracle.kv.coherence.NoSQLBinaryStore and the AVRO supporting oracle.kv.coherence.NoSQLAvroCacheStore classes. So we will have two XML elements called binary-cache-store and avro-cache-store so we end up with two ElementProcessor classes.

public class NoSQLBinaryStoreElementProcessor implements ElementProcessor<NoSQLBinaryStoreScheme>
{
    @Override
    public NoSQLBinaryStoreScheme process(ProcessingContext context, XmlElement element) throws ConfigurationException
    {
        return context.inject(new NoSQLBinaryStoreScheme(), element);
    }
}

public class NoSQLAvroCacheStoreElementProcessor implements ElementProcessor<NoSQLAvroCacheStoreScheme>
{
    @Override
    public NoSQLAvroCacheStoreScheme process(ProcessingContext context, XmlElement element) throws ConfigurationException
    {
        return context.inject(new NoSQLAvroCacheStoreScheme(), element);
    }
}

As you can see these are very simple classes. In our case the only thing they need to do is register the correct scheme with the ProcessingContext.

NamespaceHandler

Almost there now, the final thing we need is the actual com.tangosol.config.xml.NamespaceHandler implementation that Coherence will delegate to in order to handle any of our custom XML. In this case our implementation is very straight forward, all it needs to do is register the two ElementProcessor implementations with the relevant XML tags.

public class NoSQLNamespaceHandler extends AbstractNamespaceHandler
{

    public static final String XMLTAG_BINARY_CACHE_STORE = "binary-cache-store";
    public static final String XMLTAG_AVRO_CACHE_STORE = "avro-cache-store";

    public NoSQLNamespaceHandler()
    {
        registerProcessor(XMLTAG_BINARY_CACHE_STORE, new NoSQLBinaryStoreElementProcessor());
        registerProcessor(XMLTAG_AVRO_CACHE_STORE, new NoSQLAvroCacheStoreElementProcessor());
    }

}

If you go back to the cache configuration file you will see that this is the class that is declared in the namespace declaration in the root cache-config element of the configuration XML.

<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
              xmlns:kv="class://oracle.kv.coherence.NoSQLNamespaceHandler"
              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd">

Custom XSD Validation

One final thing we can include, although this is optional, is an XSD file for our custom namespace. This has a couple of advantages, first Coherence will use the XSD to validate the XML and second, which I’ve mentioned already, we can use it in IDEs at development time to validate the XML.

To enable XSD validation we just add a schema location to the cache-config element at the top of the cache configuration file like this.

<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
              xmlns:kv="class://oracle.kv.coherence.NoSQLNamespaceHandler"
              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd
                                  class://oracle.kv.coherence.NoSQLNamespaceHandler coherence-nosql-config.xsd">

You can see the location specified on line 5 that corresponds to the xmlns declared on line 3. Obviously our custom XSD needs to be called coherence-nosql-config.xsd and be on the classpath. I’m not going to post the XSD here but it is on GitHub.

In Summary

That’s about it and enough custom namespace work to allow Oracle NoSQL to work with Coherence 12.1.2 as a cache store DB. If you want to try it out go ahead and get the code from my GitHub repository. This is a Maven project and you will need Coherence 12.1.2 and Oracle NoSQL in your repository to use it.

There is a lot more that you can do with custom namespaces, this post has just scratched the surface. As you have seen we have used it here to inject our own implementation of a cache store. I have used it on other projects to inject various classes in places in the configuration we would previously have used a class-scheme. If you want to see more examples then a good place to look would be the release of the Coherence Incubator that corresponds to Coherence 12.1.2 (which at the time I am writing this hasn’t been publicly released yet).