There always comes a time in your professional career when you get stuck, and for days at a time! Every developer worth his salt has definitely gone through this phase. Another such moment occurred in my career too recently.
We are building a product which uses FIFO queues. Simple right. Each request that we get needs to be queued with an always-incrementing-by-1 sequence counter. No problem here. A single application server can have AtomicLong
‘s incrementAndGet()
and you can be rest assured that the counter values are always incrementing. Piece of cake. What if you have 2 application servers? Oh oh! Your immediate answer would be – use MySQL’s AUTO_INCREMENT
. What if MySQL is not the right tool? In our product, we had to go ahead with DynamoDB. And it does not support auto-increment. What now?
There are some elegant solutions such as Twitter’s Snowflake. It always guarantees unique, roughly sortable, increasing counters in a distributed environment. But we needed a solution that guarantees increment by 1.
As mentioned earlier, the “days at a time” started. 🙂 Thank God it wasn’t “weeks at a time”, though it seemed that way, as we soon came across Hazelcast! Hazelcast is an in-memory Open Source data grid based on Java. And its “Cross-JVM communication / shared storage” using Distributed Primitives was the perfect fit for our problem, i.e. a cluster-wide distributed sequence counter.
Let get our hands dirty, shall we? I will only touch upon IAtomicLong
in this blog though we have extensively used ILock
as well.
import com.hazelcast.client.HazelcastClient; import com.hazelcast.core.HazelcastInstance; import com.hazelcast.core.IAtomicLong; String hazelcastCounter = "<name of your counter>"; // eg.: web_requests_counter HazelcastInstance client = HazelcastClient.newHazelcastClient(); IAtomicLong counter = client.getAtomicLong(hazelcastCounter); Long incrementedValue = counter.incrementAndGet();
It is simple, isn’t it! As a developer, it seems like all the operations are carried out locally because of the familiarity with functions such as incrementAndGet()
. But under the hood lies the complexity of data synchronization across the cluster. And that, my friends, is the beauty of Hazelcast.
How do you form a cluster of AWS EC2 instances using Hazelcast? I will be back after a short commercial break. 🙂
GridGain also has atomic long support. On top of it, it also provide AtomicSequence which is useful for performant primary key generation in memory: http://atlassian.gridgain.com/wiki/display/GG60/Distributed+Data+Structures
Hi John,
Thanks for pointing out an alternative. I had a look at the link and it looks impressive!