I've seen this a lot in code reviews for engineers moving from monolithic apps to distributed systems: the belief that a distributed cache is basically just a Map that happens to live on a different server. They think, "I'll just wrap my ConcurrentHashMap in an API call, and boom—I've scaled my state."

Thinking it's just a Map over TCP

The problem with that mental model is that it ignores the "distributed" part of the equation. If you just have one giant remote Map, you've created a massive single point of failure and a performance bottleneck. But if you try to solve that by just copying the Map to five different servers, you've now entered a nightmare of cache coherence. Imagine updating a user's session token on Node A, but Node B still has the old one. Suddenly, your users are being randomly logged out because their requests are hitting different servers.

// The "Naïve" approach that fails at scale
case class CacheNode(id: String, data: mutable.Map[String, String])

class DistributedCache(nodes: List[CacheNode]) {
  def put(key: String, value: String): Unit = {
    // WRONG: Updating every node to keep them "in sync"
    // This kills write performance and doesn't scale.
    nodes.foreach(_.data.put(key, value))
  }
}


Partitioning the Truth with Consistent Hashing
To actually build a distributed cache, you have to stop thinking about "syncing" and start thinking about "ownership." Each piece of data should be owned by exactly one node at any given time. We do this using consistent hashing.

Instead of a simple key % nodeCount (which breaks the moment you add or remove a server), consistent hashing places both your servers and your keys on a logical ring. A key is assigned to the first server encountered while moving clockwise around that ring. This way, when you add a new node, you only have to move a small fraction of the keys, rather than reshuffling the entire dataset.

Building the Coordinator Pattern
In Scala, we can model this by separating the CacheNode (which holds the actual data) from the CacheCoordinator (which knows where the data lives). I usually recommend using a Case Class for the ring mapping to keep it immutable and easy to reason about.

You'll want your coordinator to handle the logic of:

• Hashing the key to a point on the ring.
• Finding the nearest node.
• Forwarding the request.

I've found that using Long or Int for the hash space is plenty for most exercises, but in production, you'd use something like MurmurHash3 to ensure the distribution is actually uniform. If your hash function is poor, you'll end up with "hot spots" where one server does 90% of the work while the others sit idle.

One last tip: don't forget about Time-to-Live (TTL). A cache that never expires is just a memory leak with a fancy name. Use a PriorityQueue or a scheduled executor to prune old entries, otherwise, your distributed system will eventually just be a distributed crash.

---

📋 Practical Task

Exercise: Implementing a Consistent Hashing Ring for a UserSession Cache

Your task is to build the core routing logic for a distributed session cache. You aren't building the network layer (we'll simulate that with method calls), but you must implement the distribution logic correctly.

Requirements:

• Create a CacheNode class that stores a simple Map[String, String].
• Implement a ConsistentHashRing class. It should take a list of node IDs and map them to a 360-degree circle (or a large integer range).
• Implement a CacheCoordinator that uses the ring to decide which node should handle a get or put request.
• The Twist: Your ring must support "Virtual Nodes." Instead of placing each physical node on the ring once, place it 3 times (using different hash seeds) to ensure a more even distribution of keys.

Validation Criteria: If you add three nodes (NodeA, NodeB, NodeC) and put 100 random keys into the coordinator, no single node should hold more than 45% of the keys. If you remove one node, only the keys belonging to that specific node should be redistributed.