A few years ago, I interviewed a candidate—let's call him Mark—who was arguably one of the best Kotlin coders I'd ever seen. He could write a custom DSL in his sleep and handled complex coroutine scopes with ease. But when we hit the system design portion of the interview, and I asked him how he'd scale a notification service to handle 10 million pushes per hour, he stumbled. He started describing a very sophisticated internal queue using Channel and Flow, which was technically impressive, but he forgot one critical detail: his logic lived entirely in a single JVM process. When I asked what happened if that server crashed or if we needed to add five more servers to handle the load, he realized he'd built a beautiful engine that could only ever fit in one car.

That's the trap many of us fall into when transitioning from "Kotlin developer" to "Backend Engineer." We get so focused on the elegance of the language—the null safety, the conciseness, the asynchronous primitives—that we forget the code is just a small part of a larger, often messy, distributed system. In a system design interview, the interviewer isn't testing your knowledge of sealed classes; they are testing your ability to manage state, latency, and failure across a network.

Scaling State Beyond the JVM

In a local project, a ConcurrentHashMap is your best friend for caching or tracking state. In a production backend, it's often a liability. The moment you introduce a load balancer and multiple instances of your Kotlin service, your local map becomes a "silo." User A might hit Server 1 and be cached, but their next request hits Server 2, and suddenly their session is gone. I always tell my mentees: if the data needs to survive a restart or be visible to another instance, it cannot live in the JVM heap.

This is where you start discussing distributed caches like Redis or Memcached. When you're explaining this in an interview, don't just say "I'll use Redis." Explain why. Mention the trade-off between the millisecond latency of local memory versus the network hop required for a remote cache. In a Kotlin context, you'll likely be using a client like Lettuce or Jedis, and you should be prepared to discuss how you'll handle the serialization of your data classes—perhaps using kotlinx.serialization—to move them efficiently across the wire.

The Coroutine Connection Pool Paradox

One of the biggest pitfalls for Kotlin developers in system design is overestimating the power of coroutines. It's intoxicating to know you can launch 100,000 coroutines without crashing your memory. However, your database doesn't care about your coroutines. If you have a PostgreSQL connection pool limited to 20 connections, launching 100,000 concurrent requests to the DB won't make them faster; it will just lead to a massive queue of suspended coroutines waiting for a connection, eventually timing out and cascading into a system-wide failure.

When designing your backend, you have to discuss "backpressure." You need to explain how you'll limit the concurrency of your outgoing requests to downstream services or databases. Whether you use a Semaphore to limit concurrent access or a message queue like RabbitMQ or Kafka to buffer the load, the goal is to ensure that your high-efficiency Kotlin code doesn't accidentally act as a Distributed Denial of Service (DDoS) attack against your own infrastructure.

Choosing the Right Persistence Layer

You'll inevitably be asked to choose a database. The "correct" answer is always "it depends," but you'll lose points if you can't explain what it depends on. I've seen candidates default to MongoDB because it's flexible, only to realize halfway through the design that they need complex ACID transactions for a financial ledger—something a relational database like PostgreSQL handles natively.

Think about the access patterns. If you're building a social media feed where you need to query "friends of friends," a Graph database like Neo4j makes more sense. If you're storing massive amounts of time-series telemetry data from IoT devices, look toward Cassandra or InfluxDB. In your interview, tie this back to your Kotlin models. Explain how a heavily normalized SQL schema maps to your domain entities, or how a document store allows your Kotlin data classes to evolve without needing a migration script for every single field change.

📋 Practical Task

Design a Distributed Rate Limiter for a Public API

You are tasked with designing a rate-limiting system for a high-traffic API. The requirement is that each API key is limited to 1,000 requests per minute across the entire global infrastructure (not just per server).

Instead of writing implementation code, provide a System Design Specification that addresses the following:

• Storage Choice: Where will you store the request counts? Justify why you chose a specific technology (e.g., Redis, Cassandra, etc.) over a local Kotlin Map.
• The Race Condition: Explain how you will handle the "read-modify-write" problem when two different servers try to increment the counter for the same API key at the exact same millisecond. Mention a specific strategy (e.g., atomic increments, Lua scripts, or distributed locks).
• Failure Mode: What happens to the API if the rate-limiter storage goes down? Should the API fail-closed (block all requests) or fail-open (allow all requests)? Justify your decision based on user experience versus system safety.
• Kotlin Integration: Briefly describe how you would implement this as a Ktor interceptor or a Spring Filter to ensure the business logic remains decoupled from the rate-limiting logic.