There always comes a time in your professional career when you get stuck, and for days at a time! Every developer worth his salt has definitely gone through this phase. Another such moment occurred in my career too recently.
We are building a product which uses FIFO queues. Simple right. Each request that we get needs to be queued with an always-incrementing-by-1 sequence counter. No problem here. A single application server can have
incrementAndGet() and you can be rest assured that the counter values are always incrementing. Piece of cake. What if you have 2 application servers? Oh oh! Your immediate answer would be – use MySQL’s
AUTO_INCREMENT. What if MySQL is not the right tool? In our product, we had to go ahead with DynamoDB. And it does not support auto-increment. What now?
There are some elegant solutions such as Twitter’s Snowflake. It always guarantees unique, roughly sortable, increasing counters in a distributed environment. But we needed a solution that guarantees increment by 1.
As mentioned earlier, the “days at a time” started. 🙂 Thank God it wasn’t “weeks at a time”, though it seemed that way, as we soon came across Hazelcast! Hazelcast is an in-memory Open Source data grid based on Java. And its “Cross-JVM communication / shared storage” using Distributed Primitives was the perfect fit for our problem, i.e. a cluster-wide distributed sequence counter.
Let get our hands dirty, shall we? I will only touch upon
IAtomicLong in this blog though we have extensively used
ILock as well.
import com.hazelcast.client.HazelcastClient; import com.hazelcast.core.HazelcastInstance; import com.hazelcast.core.IAtomicLong; String hazelcastCounter = "<name of your counter>"; // eg.: web_requests_counter HazelcastInstance client = HazelcastClient.newHazelcastClient(); IAtomicLong counter = client.getAtomicLong(hazelcastCounter); Long incrementedValue = counter.incrementAndGet();
It is simple, isn’t it! As a developer, it seems like all the operations are carried out locally because of the familiarity with functions such as
incrementAndGet(). But under the hood lies the complexity of data synchronization across the cluster. And that, my friends, is the beauty of Hazelcast.
How do you form a cluster of AWS EC2 instances using Hazelcast? I will be back after a short commercial break. 🙂
GridGain also has atomic long support. On top of it, it also provide AtomicSequence which is useful for performant primary key generation in memory: http://atlassian.gridgain.com/wiki/display/GG60/Distributed+Data+Structures
Thanks for pointing out an alternative. I had a look at the link and it looks impressive!