Redis on an elastic
I hope everyone is having an amazing day today. After all it is Friday!!
I’m here to make that Friday even more awesome by writing a bit about how cool Redis is on AWS Elasticache service(Maybe I should revisit things I call cool, specially on a Friday)
So Redis has been around for a long time now. If anyone knows the pain of setting up a Redis cluster (Be it sentinel or cluster) and managing it, please raise your hand;
Its nice to see people agreeing with you, specially on a Friday. Ok hands down. Lets move on.
Now we come to the alternate universe(yes I am a big Barry Allen fan) where Redis is married to AWS Elasticache. This is definitely not a monogamous relationship since the hand of Memcached was also given to AWS Elasticache.
Before we proceed further, I would like to mind map the things we would be looking as part of this write-up.
We will only be focusing on a subset of features on this post since Redis by itself is a broader area which can not be cramped up to one single post. AWS Elasticache(That is a lot of words, lets call it AWS-E) setup of a Redis cluster is literally as the old adage goes, a walk in the park(only if the park has WiFi of course). Let’s see a minimum set of things you need to bring up a Redis with cluster mode enabled;
- Setup a subnet group to deploy the cluster
- Setup a parameter group or use the default one provided
- Setup a security group for the cluster
- Setup the backup policy
- Create the cluster….
That was my first reaction too. Astonished. Simplicity is the new sexy I say. Don’t get me wrong, there is no better sense of accomplishment when you setup a Redis cluster by yourself. The blood, sweat and (I want to say tears but thats being a bit melodramatic) time spent on setting up a cluster is so fulfilling. But its not only about running a cluster successfully, the actual tears come when it comes to managing your cluster day in and day out. Now that is painful and I would not want to spend my time dealing with those operational atrocities.
Lets look at some features of Redis on AWS-E(I should hashtag this so that it catches on) that makes it appealing.
Performance
Redis is a high performant in-memory datastore as we know. AWS has taken it a bit further by adding a few tweaks to make it run even smoother with AWS-E. As you know, the writes to a Redis cluster always happens to the master nodes. The read replicas are then updated with the writes asynchronously. At peak write load times, the master nodes can receive many writes which would put a lot of pressure on the master nodes. AWS-E has introduced a mechanism by which they start to throttle the writes to the master nodes when they see an increase in load.
The other improvement they have done with respect to performance is the more efficient usage of the swap memory. During high throughput operations such as replica syncs and snapshots, memory might be swapped in and out as required. AWS-E has improved this whole process minimising the impact of these operations on the cluster.
Availability
Multi-AZ gives you the ability to span your Redis cluster within a region. You are allowed to select the zone(s) you want it to be deployed and can even control the slots and keyspace distribution if needed.
Auto-resharding for me is one of the key appealing features of AWS-E. The fact that I can add or remove nodes to the cluster gives me more control on how I would like to manage my infrastructure. This means that I can monitor through CloudWatch and then trigger events to scale up or down my Redis nodes. AWS-E handles re-sharding with minimum impact to the cluster and re-distributes the keys according to the changes.
The Redis cluster mode on AWS-E has a faster failover as well(15–30sec). As you are allowed a maximum 15 shards with 5 read replicas per shard, the impact of failover is further minimised. As part of the failover, if majority of the nodes fail in a shard, electing a master would be problematic since Redis works in a quorum based election system. AWS-E has made some improvements here by making more intelligent master election.
Security
With AWS-E, you are given encryption at rest and in-transit. For me personally, I do not see myself using the in-transit encryption for Redis. Only because I would usually always expose the Redis cluster only within my private subnet(s). If you do expose the Redis cluster externally the it would make sense. Note that also, if you enable in-transit encryption, the data moving between nodes in the Redis cluster will also be encrypted. Something to think about since this could possibly have a performance impact.
Encryption at-rest makes perfect sense and should be used where possible. All the disk writes done during sync and backup will be encrypted when this is enabled.
AWS-E is also HIPAA eligible which means you can store protected health information(PHI) if needed.
Redis AUTH can also be enabled if required whereby you provide a token to get access to the cluster. This token is created as part of setting up the Redis cluster.
Moving on from security, coming from the open source Redis, I wanted to see how we could enable the AOF(Append only file) on the cluster in AWS-E. Digging into the default parameter group created I can see that this option(mentioned as appendonly) is set to no by default and you cannot modify it. As mentioned in the AWS documentation, this feature is disabled on newer versions. A snippet from the documentation explains more eloquently as to why this is disabled;
That’s about it for today. Thank you for reading and if anyone has any interesting experience to share on using Redis on AWS Elasticache, please do leave a comment.
