In-Memory Caches, Grids, and Other Mystical Creatures: Navigating the In-Memory data store world.
The moment you decide your relational database can’t keep up with your ambitions: wait no more and look into the in-memory data stores world.
This one has creatures of all shapes and behaviours.
Some in-memory data stores are simple, elegant beings—fast key-value stores that exist purely to make your reads and writes sub-second. Others are powerful grids that not only store your data but also run computations on it, whispering SQL and filtering logic across a distributed cluster.
When considering a caching layer on top of your relational database, keep in mind that it's best suited for static content or short-lived write-once content and less suitable for data that is changing frequently. Moreover, data freshness might be compromised in a caching layer, so your application should be ok with some stale data.
In-memory stores usually trade durability for performance: they would not persist data on disk in order to speed up read/write operations, although they still have different durability options if needed.
There is a huge elephant in the corner. How do you figure out when data has changed in the relational database and replace it in the caching layer?
There are two major types of in-memory data stores
- Simple in-memory data store: stores data in RAM so read/writes are extremely fast, data structures are usually simple, mainly used for data caching and session stores ( Redis, Valkey, Memcached, DragonFly, KeyDB, Garnet)
- In-memory data grid: clustered, distributed in-memory platform designed not only to store data but also to compute on it, supports Sql, predicates and filters ( Hazelcast, Apache Ignite, Oracle Coherence, GigaSpaces)
Redis was once a standard for simple in-memory data stores, providing low latency access with optional persistence and high availability. Redis 8 version is moving up by introducing querying/filtering on values as well as vector search in real time. This still does not move them towards an in-memory data grid, as in Data grid system needs distributed compute ability, transactional semantics, structured schema and much more.
Valkey is the open-source community fork of Redis, started from Redis 7.2.4 codebase after Redis Labs changed Redis licensing from BSD-3 to Redis Source Available License (RSAL). Cloud providers (AWS, Google, Azure, etc.) can no longer offer Redis as a service commercially under this new license.
Lets look at what the biggest cloud providers offer:
Azure Managed Redis:
— Redis version 7.4.x
— Not using Valkey, uses last Redis open source version
— Native support for vector search
— Conditional update support to the SET command
— Supports for active-active geo-replication
— Supports a single logical database per instance
— you can follow on what new in Azure managed Redis here: https://learn.microsoft.com/en-us/azure/redis/whats-new
GCP Memorystore:
— supports Valkey version 8.0
— Availability: 99.9% (Memorystore for Redis) 99.99% (Memorystore for Redis Cluster)
— No query/filter data ( value) in real time
— No support for active-active geo-replication, supports primary in one region, only read replicas in other regions.
— A single store can contain up to 16 databases, a cluster can only contain 1 database.
— more information here: https://cloud.google.com/memorystore
Amazon ElastiCache for Redis:
— supports Valkey version 8.2
— Availability: 99.99%
— No query/filter data ( value) in real time
— Native support for vector search
— Conditional update support to the SET command
— A single store can contain up to 16 databases, a cluster can only contain 1 database.
— No support for active-active geo-replication, supports primary in one region, only read replicas in other regions.
— more information here : https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/Self-Service-Updates.html
When trying to understand differences between Redis alternatives, I have searched but could not find a fresh Gartner quadrant for in-memory data stores, unfortunately. However, here is this interesting recent study (22 Oct 2025) of next generation cloud-native in-memory stores https://arxiv.org/html/2510.19805v1 and mainly dedicated to Redis alternatives: Valkey, KeyDB, and Garnet, which are community-driven forks of Redis.
This study compares Redis v7.2.7, Valkey v8.0.2, KeyDB v6.3.4, and Garnet v1.0.62
There is another interesting study that I found https://seifrajhi.github.io/blog/in-memory-databases-comparison/ (note that it was published a year ago) on Redis, DragonFly, Skytable and KeyDB. This study compares in-memory stores in terms of architecture, feature set and performance.
One more good article is Redis vs Valkey, a complete guide to the future of in-memory databases https://www.jusdb.com/blog/redis-vs-valkey-a-complete-guide-to-the-future-of-in-memory-databases-jusdb
Valkey releases can be found in their GitHub repo: https://github.com/valkey-io/valkey/releases
I’d love to hear which in-memory system you are using and what real-world data challenge it helped you conquer? Please share your story in the comments!
Comments
No comments yet.
Leave a comment