When you hit a whiteboard interview and they ask you to implement a rate limiter, they aren't just checking if you know the algorithm. They're looking at how you handle state over time and how you deal with edge cases—like "burstiness."

I've always preferred the Token Bucket algorithm for these discussions. It's intuitive: imagine a bucket that holds a maximum number of tokens. Every request costs one token. If the bucket is empty, the request is rejected. Meanwhile, tokens drip back into the bucket at a steady rate. It allows for a bit of "burst" traffic while keeping the average rate in check.

Defining the Bucket's State

First, I need to decide what information I actually need to track. To make this work, I don't want a background timer constantly adding tokens (that would be a nightmare for performance). Instead, I'll calculate how many tokens should have been added since the last time a request came in.

class RateLimiter
  def initialize(max_tokens, refill_rate)
    @max_tokens = max_tokens
    @refill_rate = refill_rate # tokens per second
    @tokens = max_tokens
    @last_refill_time = Time.now
  end
end

I'm setting the initial state to be "full." In a real-world scenario, you'd probably associate this instance with a specific User ID or IP address in a Redis store, but for the whiteboard, a simple class will do.

Calculating the Refill Logic

Now, I need a method that checks if a request can pass. This is where the "lazy refill" happens. I'll calculate the time elapsed, multiply it by the rate, and add that to the current token count.

  def allow_request?
    now = Time.now
    elapsed = now - @last_refill_time
    
    # Refill the bucket based on time passed
    @tokens += elapsed * @refill_rate
    @last_refill_time = now

    if @tokens >= 1
      @tokens -= 1
      return true
    end

    false
  end

Catching the "Infinite Token" Bug

Wait. If I leave the code like this, I've made a classic mistake. If a user doesn't make a request for an hour, elapsed * @refill_rate will be a huge number. My @tokens count will grow to thousands, effectively allowing the user to blast the API with thousands of requests instantly, completely bypassing the "rate" part of the rate limiter.

I need to cap the tokens at @max_tokens. This ensures that while we allow a small burst (the size of the bucket), we don't allow an infinite accumulation of credit.

  def allow_request?
    now = Time.now
    elapsed = now - @last_refill_time
    
    # Fix: Cap the tokens so they don't grow indefinitely
    @tokens = [@max_tokens, @tokens + (elapsed * @refill_rate)].min
    @last_refill_time = now

    if @tokens >= 1
      @tokens -= 1
      return true
    end

    false
  end

Putting it Together for a Test Drive

To make sure this actually behaves as expected, I'll simulate a few requests. I'll set a limit of 5 tokens, refilling at 1 token per second. This means I can burst 5 requests immediately, but then I'm limited to one per second.

limiter = RateLimiter.new(5, 1)

# Burst 6 requests
6.times do |i|
  puts "Request #{i+1}: #{limiter.allow_request? ? 'Allowed' : 'Rejected'}"
end

# Wait 2 seconds and try again
sleep 2
puts "Request 7 (after 2s): #{limiter.allow_request? ? 'Allowed' : 'Rejected'}"

If you run this, you'll see the first 5 pass, the 6th fail, and the 7th pass. It's a simple pattern, but the "cap" logic is usually what separates a junior implementation from a senior one in an interview setting.

📋 Practical Task

Exercise: Implement a Tiered Rate Limiter

In the real world, different users have different limits (e.g., "Free" vs "Premium" tiers). Your task is to modify the RateLimiter logic to support tiered configurations.

Build a class called TieredRateLimiter that takes a user_tier symbol (:free or :premium) during initialization. Use the following configurations:

• Free: Max 3 tokens, refills at 0.5 tokens per second.
• Premium: Max 10 tokens, refills at 2 tokens per second.

Your implementation should:

1. Assign the correct max_tokens and refill_rate based on the tier.
2. Maintain the "cap" logic so tokens do not exceed the tier's maximum.
3. Implement the allow_request? method to return true or false.