Imagine we've got a public API that serves expensive data—let's say it's a high-resolution weather forecast that hits a paid third-party provider. If a single aggressive client decides to script a loop and hit our /forecast endpoint 500 times a second, we're going to get a massive bill, or worse, the provider will ban our API key. We need a way to tell the client, "Slow down," before the request even hits our business logic.

The "Quick and Dirty" Map

My first instinct when I tackle this is usually the simplest one: just keep a count of requests per IP address. I'll throw together a map where the key is the IP and the value is an integer. If the integer hits a limit, I'll return a 429 Too Many Requests.

type limiter struct {
    counts map[string]int
}

func (l *limiter) Limit(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        ip := r.RemoteAddr
        l.counts[ip]++
        if l.counts[ip] > 10 {
            http.Error(w, "Too many requests", http.StatusTooManyRequests)
            return
        }
        next.ServeHTTP(w, r)
    })
}

I run this, and for the first few requests, it feels great. But then I remember that Go maps are not thread-safe. As soon as I hit this with a load testing tool like hey or ab, the server crashes instantly with a fatal error: concurrent map writes. This is a classic "gotcha" in Go middleware. Since every request runs in its own goroutine, we're hammering that map from dozens of directions at once.

The Panic and the Mutex

Alright, so I need to protect that map. I'll wrap it in a sync.Mutex. While I'm at it, I realize the current logic is broken: once a user hits 10 requests, they are blocked forever because I'm never resetting the counter. I'll add a crude "reset" by just wiping the map every minute using a goroutine.

type limiter struct {
    mu     sync.Mutex
    counts map[string]int
}

func NewLimiter() *limiter {
    l := &limiter{counts: make(map[string]int)}
    go func() {
        for range time.Tick(time.Minute) {
            l.mu.Lock()
            l.counts = make(map[string]int)
            l.mu.Unlock()
        }
    }()
    return l
}

This stops the crashing, but it's still a bit clunky. We've created a "fixed window" limiter. If a user sends 10 requests at 10:00:59 and another 10 at 10:01:01, they've essentially sent 20 requests in two seconds, which defeats the purpose of a strict rate limit. Plus, locking the entire map for every single request creates a bottleneck. In a high-traffic app, my middleware becomes the slowest part of the system.

Leveraging the Token Bucket

Instead of fighting with mutexes and timers, I'm going to use the golang.org/x/time/rate package. This implements a "Token Bucket" algorithm. Think of it like a bucket that fills with tokens at a steady rate. Every request takes a token. If the bucket is empty, the request is rejected. This allows for "bursts" (if the bucket is full, you can send a few requests quickly) while maintaining a strict average rate over time.

The tricky part here is that we need a separate bucket for every single IP address, otherwise, one user could exhaust the tokens for everyone on the internet.

import (
    "net/http"
    "sync"
    "golang.org/x/time/rate"
)

type visitor struct {
    limiter  *rate.Limiter
    lastSeen time.Time
}

var visitors = make(map[string]*visitor)
var mu sync.Mutex

func limitMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        ip := r.RemoteAddr // In production, check X-Forwarded-For

        mu.Lock()
        v, exists := visitors[ip]
        if !exists {
            // Allow 5 requests per second, with a burst capacity of 10
            v = &visitor{limiter: rate.NewLimiter(5, 10)}
            visitors[ip] = v
        }
        v.lastSeen = time.Now()
        mu.Unlock()

        if !v.limiter.Allow() {
            http.Error(w, "Too Many Requests", http.StatusTooManyRequests)
            return
        }

        next.ServeHTTP(w, r)
    })
}

The Final Leak

Wait, I've just introduced a memory leak. I'm adding a visitor struct to the visitors map for every single unique IP that hits my server. If I get a million unique visitors, that map grows indefinitely. I need a way to clean up old visitors who haven't been seen in a while.

I'll add a background cleanup loop that iterates through the map and deletes entries where lastSeen is older than, say, 3 minutes. This is the "professional" way to handle stateful middleware: create the limit, handle the concurrency, and always, always plan for the cleanup of the state you've created.

📋 Practical Task

Implement a Memory-Cleaning Loop for the Rate Limiter

In the lesson, I mentioned that the visitors map will grow indefinitely, leading to a memory leak. Your task is to complete the implementation by adding a cleanup function.

• Create a function cleanupVisitors() that runs in its own goroutine.
• The function should wake up every 1 minute.
• It must lock the mu mutex, iterate through the visitors map, and delete any visitor who hasn't made a request in the last 5 minutes.
• Integrate this function into the main() function or the middleware initialization so it starts when the server begins.