5 minutes with – JGroup

In a cluster environment could happen that you need to replicate the cached information over all the nodes that are included in cluster.

For this purpose it’s very useful to use some replication system as Jgroups. In this articles we’ll see how configure and use that with easy configuration.

Jgroups is a distributed replication system that allow to replicate the cached information in all the enviroment which are listening for receiving these information.

In this article we’ll talk about jgroups integrated with ehcache system cache and we’ll see how it works with two different protocols type (Tcp and UDP). I think they work very well together.

Briefly there are three sections in our configuration file:

cacheManagerPeerProviderFactory

It’s used for saying “Hello, I’m here!” and allows to discover the other CacheManager in the cluster and let be discovered from other nodes.

It accepts only two argument (class and properties) and we’ll see later the details.

cacheEventListenerFactory

It’s used for receiving notification about cache update by other nodes in cache cluster.

bootstrapCacheLoaderFactory

It’s used for starting cache system and synchronize the cached elements in the cluster

I’ve been very brief to describe these points because I think you can easily find all the configuration information at official web site here http://ehcache.org/documentation/replication/jgroups-replicated-caching. My purpose is to show you how can you configure it in very few time and run it.

Tcp and Udp

The most used protocol in this configuration is User Datagram Protocol (Udp). The key point of this protocol is the node flexible to subscribe and unsubscribe at the replication service at runtime. Udp is not reliable protocol but Jgroups helps us to fix this feature. We’ll see that in more detail on the code.

On the other hand Tcp needs to know all the nodes in the cluster before starting the system. It’s full reliable without any implementation from Jgroups.

Which is the best option? I don’t think there’s the best or worst option. I used both in different enviroments (for example I used tcp to skip proxy issue) without any problem.

Synchronous and Asynchronous

Ok, this is the common typical issue on the replication system. Take a breath and think it as database replication enviroment.

Synchronous means high data refresh with high I/O network traffic over the net.

Asynchronous means working with old data but you save the network traffic.

Which is the best choice? It depends on your application and, specially, about the data type cached.

Now it’s time to see the code. First, the ehcache configuration file:

<?xml version="1.0" encoding="UTF-8"?>
<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd"
	updateCheck="false" name="JGroupsCache">

	<cacheManagerPeerProviderFactory
		class="net.sf.ehcache.distribution.jgroups.JGroupsCacheManagerPeerProviderFactory"
		properties="file=udp.xml" />

	<defaultCache maxElementsInMemory="50" eternal="false"
		overflowToDisk="false" memoryStoreEvictionPolicy="LFU">
		<cacheEventListenerFactory
			class="net.sf.ehcache.distribution.jgroups.JGroupsCacheReplicatorFactory"
			properties="replicateAsynchronously=true, replicatePuts=true,
			replicateUpdates=true, replicateUpdatesViaCopy=false,
			replicateRemovals=true" />

		<bootstrapCacheLoaderFactory
			class="net.sf.ehcache.distribution.jgroups.JGroupsBootstrapCacheLoaderFactory"
			properties="bootstrapAsynchronously=false" />
	</defaultCache>

	<cache name="wordCache" eternal="false" maxElementsInMemory="100"
		overflowToDisk="false" diskPersistent="false" timeToIdleSeconds="0"
		timeToLiveSeconds="60" memoryStoreEvictionPolicy="LRU">
		<cacheEventListenerFactory
			class="net.sf.ehcache.distribution.jgroups.JGroupsCacheReplicatorFactory"
			properties="replicateAsynchronously=true, replicatePuts=true,
			replicateUpdates=true, replicateUpdatesViaCopy=false,
			replicateRemovals=true" />

		<bootstrapCacheLoaderFactory
			class="net.sf.ehcache.distribution.jgroups.JGroupsBootstrapCacheLoaderFactory"
			properties="bootstrapAsynchronously=false" />
	</cache>
</ehcache>

As you can see, in this first code I used a udp protocol. The content of the file is the follow.

<config>
  <UDP mcast_addr="228.10.10.10" mcast_port="45588"/>
  <PING timeout="2000"/>
  <MERGE2/>
  <FD_SOCK/>
  <VERIFY_SUSPECT timeout="1500"  />
  <pbcast.NAKACK retransmit_timeout="2400,4800"/>
  <UNICAST/>
  <pbcast.STABLE/>
  <pbcast.GMS/>
</config>

You have to put your multicast address and port into the firt node.

Take a look at pbcast.NAKACK node. As said at the official web site: “NAKACK provides reliable delivery and FIFO (= First In First Out) properties for messages sent to all nodes in a cluster”.

If you would like to use tcp protocol you’ll have to configure into the ehcache file canching the properties file name file=tcp.xml which contains the follow configuration.

<?xml version="1.0" encoding="UTF-8"?>
<config>
<TCP bind_addr="localhost" bind_port="7831" />
<TCPPING timeout="3000"
    initial_hosts="localhost[7831],localhost[7832]"
    port_range="1"
    num_initial_members="2"/>
<VERIFY_SUSPECT timeout="1500"  />
<pbcast.NAKACK use_mcast_xmit="false" gc_lag="100"
    retransmit_timeout="300,600,1200,2400,4800"
    discard_delivered_msgs="true"/>
<pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000" max_bytes="400000"/>
<pbcast.GMS print_local_addr="true" join_timeout="5000" shun="false" view_bundling="true"/>
</config>

As I said before,  you must put all the nodes in the cluster. You can see these at initial_hosts properties (comma separated).

 The above file contains a very common configuration that it’s fine at 99% cases.

Before running the example you need the rest of the code. That’s very easy and is not the focus of the article to explain it. You can find better details on article “5 minutes with – Spring Cache” on this blog.

 /WEB-INF/spring-servlet.xml

<?xml version="1.0" encoding="UTF-8"?>

	<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
	xmlns:context="http://www.springframework.org/schema/context"
	xmlns:cache="http://www.springframework.org/schema/cache"
	xsi:schemaLocation="http://www.springframework.org/schema/beans
		http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
		http://www.springframework.org/schema/context
		http://www.springframework.org/schema/context/spring-context-3.0.xsd
		http://www.springframework.org/schema/cache
        http://www.springframework.org/schema/cache/spring-cache.xsd">

	<context:component-scan base-package="it.springwebjgroups.controller" />

	<!-- Process cache annotations -->
	<cache:annotation-driven />

	<!-- Configuration for using Ehcache as the cache manager-->
	<bean id="cacheManager" class="org.springframework.cache.ehcache.EhCacheCacheManager"
		p:cache-manager-ref="ehcache" />
	<bean id="ehcache"
		class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean"
		p:config-location="classpath:ehcache.xml" />

 	<bean id="Speaker" class="it.springwebjgroups.bean.Speaker" />

	<bean id="viewResolver"
		class="org.springframework.web.servlet.view.UrlBasedViewResolver">
		<property name="viewClass"
			value="org.springframework.web.servlet.view.JstlView" />
		<property name="prefix" value="/WEB-INF/jsp/" />
		<property name="suffix" value=".jsp" />
	</bean>

</beans>

/WEB-INF/web.xml

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns="http://java.sun.com/xml/ns/javaee"
	xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
	xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
	id="WebApp_ID" version="2.5">
	<display-name>Spring3MVC</display-name>
	<welcome-file-list>
		<welcome-file>index.jsp</welcome-file>
	</welcome-file-list>

	<servlet>
		<servlet-name>spring</servlet-name>
		<servlet-class>
			org.springframework.web.servlet.DispatcherServlet
		</servlet-class>
		<load-on-startup>1</load-on-startup>
	</servlet>
	<servlet-mapping>
		<servlet-name>spring</servlet-name>
		<url-pattern>*.html</url-pattern>
	</servlet-mapping>
</web-app>

Speaker.java

package it.springwebjgroups.bean;

import java.util.Date;
import org.springframework.cache.annotation.CacheEvict;
import org.springframework.cache.annotation.Cacheable;

public class Speaker {
	@Cacheable(value = "wordCache", key = "#word")
	public String SayHello(String word) {
		System.out.println("Key: " + word + " not cached, it's now in cache");
		return word + "#" + new Date().toString();
	}

	@CacheEvict(value = "wordCache", key = "#word")
	public String SayHelloClear(String word) {
		System.out.println("Clear all keys");
		return "Ok";
	}
}

HelloWorldController.java

package it.springwebjgroups.controller;

import it.springwebjgroups.bean.Speaker;
import net.sf.ehcache.Cache;
import net.sf.ehcache.CacheManager;
import net.sf.ehcache.Element;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.servlet.ModelAndView;

@Controller
public class HelloWorldController {

	@Autowired
	Speaker speaker;

	@RequestMapping("/hello")
	public ModelAndView helloWorld(
			@RequestParam(required = true) String message) {

		message = speaker.SayHello(message);
		return new ModelAndView("hello", "message", message);
	}

	@RequestMapping("/helloclear")
	public ModelAndView helloWorldClear(
			@RequestParam(required = true) String message) {

		message = speaker.SayHelloClear(message);
		return new ModelAndView("hello", "message", message);
	}

	@RequestMapping("/list")
	public ModelAndView listCachedWords() {
		CacheManager manager = CacheManager.ALL_CACHE_MANAGERS.get(0);

		Cache cache = manager.getCache("wordCache");

		StringBuilder cachedValue = new StringBuilder();
		for (Object key : cache.getKeys()) {
			Element element = cache.get(key);
			if (element != null) {
				cachedValue.append(element.getObjectValue().toString());
				cachedValue.append("
");
			}
		}
		return new ModelAndView("hello", "message", cachedValue.toString());
	}
}

jsp/hello.jsp


${message}

jsp/list.jsp

</pre>
<table>
<tbody>
<tr>
<td>Key</td>
<td>Value</td>
</tr>
<tr>
<td> </td>
<td>${message}</td>
</tr>
</tbody>
</table>
<pre>

And last the maven file


<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>it.springwebjgroups</groupId>
	<artifactId>SpringWebJGroups</artifactId>
	<packaging>war</packaging>
	<version>1.0.0</version>
	<name>SpringWebJGroups Maven Webapp</name>
	<url>http://maven.apache.org</url>
	<dependencies>
		<dependency>
			<groupId>junit</groupId>
			<artifactId>junit</artifactId>
			<version>3.8.1</version>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-core</artifactId>
			<version>3.1.0.RELEASE</version>
		</dependency>
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-context</artifactId>
			<version>3.1.0.RELEASE</version>
		</dependency>
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-web</artifactId>
			<version>3.1.0.RELEASE</version>
		</dependency>
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-webmvc</artifactId>
			<version>3.1.0.RELEASE</version>
		</dependency>
		<dependency>
			<groupId>net.sf.ehcache</groupId>
			<artifactId>ehcache-core</artifactId>
			<version>2.6.2</version>
		</dependency>
		<dependency>
			<groupId>net.sf.ehcache</groupId>
			<artifactId>ehcache-jgroupsreplication</artifactId>
			<version>1.6</version>
		</dependency>
		<dependency>
			<groupId>org.jgroups</groupId>
			<artifactId>jgroups</artifactId>
			<version>2.10.0.GA</version>
		</dependency>
		<dependency>
			<groupId>cglib</groupId>
			<artifactId>cglib</artifactId>
			<version>2.2</version>
		</dependency>
		<dependency>
			<groupId>jstl</groupId>
			<artifactId>jstl</artifactId>
			<version>1.2</version>
		</dependency>
		<dependency>
			<groupId>javax.ws.rs</groupId>
			<artifactId>jsr311-api</artifactId>
			<version>1.1</version>
		</dependency>

	</dependencies>
	<build>
		<finalName>SpringWebJGroups</finalName>
	</build>
</project>

Run the code at the url

<host1>/SpringWebJGroups/hello.html?message=Hello
<host1>/SpringWebJGroups/hello.html?message=Goodbye

<host2>/SpringWebJGroups/list.html
<host2>/SpringWebJGroups/list.html

The result should be the follow:

springwebjgroups1

Finally the resource I think you can find very useful.

Balamaci’s Blog: http://balamaci.wordpress.com/category/java/caching/
EhCache Jgroups: http://ehcache.org/documentation/replication/jgroups-replicated-caching
Protocol configuration: http://www.jgroups.org/manual/html_single/index.html

Advertisements

2 thoughts on “5 minutes with – JGroup

  1. Hi my name is Giuseppe and I’m having some problem with cache replication. I’ve two ehcache files (one for each webapp). These cacheManagers have different caches but when a cache’s timeToLive expires in the webapp1 the new object is also sent to the webapp2. This cause a java.lang.IllegalArgumentException: java.lang.ClassNotFoundException (couldn
    ‘t deliver message [dst: ..). The object cannot be received from webapp2 because it is part of another domain. Any idea about this strange behaviour?

    • Hi Giuseppe.
      I don’t think the problem is due to different webapps domains.

      The ClassNotFoundException is some classes that are missing in the webapp2.

      Are you sure that the webapp2 has that class?
      I also suggest checking the 2 domains have no communication problems at the JGroup port.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s