Ehcache 2.5 goes beta! (explanation included)

The new release of Ehcache has gone beta less than 2 weeks ago, and there are many cool things packed in there, I wanted to talk a bit about it. If you’re using Ehcache as a memory cache or as a cache for Hibernate, you’ll find some useful information here.

Ehcache as Hibernate cache

Many of you have probably used Ehcache as a second level cache for Hibernate. Usually, Ehcache works well enough and you don’t even need to configure it.
Still, if you want to take full advantage of it, and improve the performances of your application, you will need to tune it.

Everything started with a default cache

<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd">
  <defaultCache
     eternal="false"
     overflowToDisk="true"
     maxElementsInMemory="10000"/>
</ehcache>

This simple ehcache.xml configuration tells Ehcache to define all caches following a default configuration. Here we have caches that will hold at most 10000 elements.

Let’s apply this cache to Hibernate

So far, you’re probably familiar with this part, we define ehcache as a cache for hibernate just by adding the following setting in the hibernate config:
hibernate.cache.region.factory_class=net.sf.ehcache.hibernate.EhCacheRegionFactory

Let’s imagine two example entities:  Country and City
There are around 200 countries in the world, so it is pretty easy to optimize a cache related to Country.

<cache name="com.jsoft.tutorial.domain.Country" 
       eternal="false" 
       overflowToDisk="true" 
       maxElementsInMemory="200"/>

But what about the City entity? Obviously, there can be so many cities, that we will keep in the cache just a subset of all existing entities.
We can rely on the default cache and keep 10000 City entities in the cache at most, or tune the cache and set a different number.

<cache name="com.jsoft.tutorial.domain.City"
       eternal="false" 
       overflowToDisk="true" 
       maxElementsInMemory="50000"/>

We see that choosing that maximum number of elements in the cache in memory becomes more difficult.
And there’s another interrogation, how much memory can we allocate to the cache to be efficient?
Let’s take the hypothesis where 1 City element takes about 1Kb of memory. Holding 50000 elements would take around 50Mb of memory.

But using a lot of memory can become a problem in Java. When using more than 4Gb of heap memory, the garbage collector can induce long pauses and freeze your application. So in order to have an efficient application, we should try not to use more than 4Gb of memory. (We can also use BigMemory with Ehcache, but that’s another story).

In our hypothesis, if our application uses 2Gb of memory, we could safely allocate at least 1Gb to our set of caches, and 50Mb for our City cache would be ok, there will remain enough memory for other caches.

We still meet 2 difficulties:
– we might want to add more elements in the cache so our application can scale better, but we need to tune ourselves the limit.
– if our City entity would change and get more attributes, its size in memory would change as well, and our cache would then take more memory. This is something we would need to monitor and update.

Tuning is a lot of work, so let’s not do it (ourselves).

Many times I have seen enterprise applications using ehcache and having NO tuning at all. It was still working fine because ehcache is a stable and mature product, but those applications could have been better optimized with a bit of tuning (not only on Ehcache).

Indeed, for any part of an application, there’s no such thing as one-time tuning. That is a job in itself which requires constant work (like you would do on a car, you can tune your engine but you’ll have to check it regularly). It’s no surprise then that often, tuning is simply forgotten (until the application is under high load and scaling is a necessity).

This is where Ehcache 2.5 comes to the rescue!

Instead of defining the maximum number of elements a cache can hold in memory, you can now define the memory a cache can use.
Ehcache will take care of managing the number of elements so your cache is optimally used.

The configuration is easy, the minimum to do is to define the total memory you will allocate for the whole set of caches, this is done in the <ehcache> tag

<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd"
        maxBytesLocalHeap="300m">
...
</ehcache>

With this, Ehcache will manage itself the memory and your caches will dynamically get what’s available.

Let’s apply this on our previous configuration

<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd"
        maxBytesLocalHeap="300m">

   <defaultCache
    eternal="true"
    overflowToDisk="true"/>

  <cache eternal="false" name="com.jsoft.tutorial.Country" overflowToDisk="true"/>

  <cache eternal="false" name="com.jsoft.tutorial.City" overflowToDisk="true"/>

</ehcache>

Here, the caches will be filled until the 300m are totally used. And what’s best is that caches sizes can vary dynamically. For instance, if the Country cache were using 200m, the City cache could use 100m. But let’s imagine that 30 minutes later, some of the elements in cache Country have been evicted and the cache now is using only 150m, the newly freed 50m could be used by the City cache to hold more elements.

Self-tuning is great, but I want SOME control!

Let’s see how we can refine Ehcache’s self tuning behavior.
By default, a cache will use what’s available in the total memory

But it’s also possible to allocate a fixed memory size to a cache, by using the maxLocalBytesOnheap attribute on the <cache> tag

e.g.

<cache eternal="false" name="com.jsoft.tutorial.Country" overflowToDisk="true" maxLocalBytesOnheap="10m"/>

We can even give a % of the total available memory:
e.g.

<cache eternal="false" name="com.jsoft.tutorial.Country" overflowToDisk="true" maxLocalBytesOnheap="20%"/>

This is very simple and very powerful, you just decide how much memory you’re willing to give to your caches, or you can let Ehcache take care of it, and you will be sure that your cache usage is optimal.

There is more than memory…

I’ve talked about memory tuning, but Ehcache is not limited to use the heap memory, it has a multi-tier architecture.
Ehcache follows a 3 tiers architecture to store elements.
In front, you’ll have heap memory, the fastest, but subject to the Garbage Collector pauses issue.
Then offheap memory, which is still memory, but not in heap. It is slower, but it is not garbage-collected, so no problem, you can use hundreds of gigabytes of memory.
The last tier is on disk, it’s the slowest, but still faster than accessing the database, and it can be huge, as it is limited only by the size of your disk.

Using those tiers, you can cache hundreds of gigabytes of data, and scale you application without changing its architecture. It’s up to you to decide how much data you want to cache, thus deciding which tiers you want to use (Heap is always in front, offheap and disk are optional).

Ehcache’s self tuning feature is available for each tier, with the following parameters : maxBytesLocalHeap, maxBytesLocalOffHeap and maxBytesLocalDisk.

Just remember that in the current beta release, if you want to use offheap, you’ll have to add overflowToOffHeap=”true”, and if you want to use the disk, you’ll have to add overflowToDisk=”true”.

A full config would look like:

<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd"
        maxBytesLocalHeap="60m"
        maxBytesLocalOffHeap="300m"
        maxBytesLocalDisk="500m">

  <defaultCache
    eternal="true"
    overflowToDisk="true"/>

  <cache eternal="false" name="com.jsoft.tutorial.Country" overflowToDisk="true"/>

  <cache eternal="false" name="com.jsoft.tutorial.City"
       maxBytesLocalOffHeap="10m"
       overflowToOffHeap="true"
       maxBytesLocalHeap="15%"
       overflowToDisk="true"
       maxBytesLocalDisk="300m"/>

</ehcache>

It doesn’t stop there

Ehcache 2.5 has many more improvements. One is pinning.
For instance, in my example, it would be nice to keep some entities in memory no matter what.
Let’s have a look at the Country entity, we know that it is a limited set, and we can assume that those entities are regularly used, so it would be good to force them to remain in the cache. This is where we can take advantage of cache pinning.

We would configure it like this:

<cache eternal="false" name="com.jsoft.tutorial.Country" overflowToDisk="true">
  <pinning store="localHeap"/>
</cache>

If we had a bigger set, we could set the cache to pin elements either in heap or offheap.
If we had a huge set, we could even pin elements amongst all tiers (heap, offheap and disk).
One caution though… Because pinning forces elements to remain in the cache, if there are too many, the cache could grow beyond available heap and cause an OutOfMemoryException.

Time to take a pause

I will stop now but there are more new features in Ehcache 2.5, like cache warming, copyOnRead, copyOnWrite… There’s a lot to discover in the official documentation.

Also note that I talked here about Ehcache when used as a 2nd level cache for Hibernate, but those new features apply as well to Ehcache as a standalone cache solution for your application.

Thanks for reading!