A few years ago, I was on call for a fintech project when our alerts started screaming at 3:00 AM. A partner had integrated our new API, but they had a bug in their retry logic. Instead of exponential backoff, their script was hitting our /transaction-status endpoint in a tight while loop. Within minutes, a single client was hammering us with 5,000 requests per second. Our database CPU spiked to 100%, and we didn't just lose that one partner's connection—we took down the entire platform for every other user. That's when I realized that "trusting your clients" is a great way to get woken up in the middle of the night.

To prevent this, we need a rate-limited API gateway. In Go, this usually means implementing a middleware layer that intercepts requests and decides—based on a set of rules—whether to let the request through or return a 429 Too Many Requests response. We aren't just talking about a global limit; we need a way to track limits per client (usually by IP address or API key) so that one bad actor doesn't starve everyone else of resources.

Selecting a Limiting Strategy

You'll see a lot of debate between "fixed window" and "token bucket" algorithms. Fixed windows are easy to implement—you just count requests in a 60-second bucket—but they suffer from "bursting" at the edges of the window. I almost always recommend the token bucket approach for Go services. Think of it as a bucket that refills with "tokens" at a steady rate. Every request consumes a token. If the bucket is empty, the request is rejected. This allows for a bit of burstiness (the size of the bucket) while maintaining a strict long-term average rate.

Go's golang.org/x/time/rate package is the gold standard here. It implements the token bucket algorithm beautifully. You don't need to write your own ticker loops and channel-based counters unless you're doing something highly specialized or are restricted from using external x-packages.

Handling State Across Concurrent Requests

Here is where most developers trip up: managing the limiters. You can't just have one global limiter; you need a map where the key is the client's IP and the value is their specific rate.Limiter instance. Because Go maps are not thread-safe, and your API gateway will be handling dozens of concurrent goroutines, you'll need a sync.RWMutex or a sync.Map to protect that registry.

I usually prefer a standard map with a sync.RWMutex because it gives me more control over how I clean up old entries. If you just keep adding IPs to a map forever, you've accidentally built a memory leak. In a production system, you'd use a TTL (Time To Live) cache or Redis, but for a standalone gateway, a simple cleanup goroutine that purges inactive limiters every few minutes is usually enough.

Integrating the Limiter into the Middleware Chain

The beauty of the http.Handler interface is how easily it wraps. Your rate limiter should be a function that takes an http.Handler and returns a new one. Inside that wrapper, you extract the IP, look up (or create) the limiter for that IP, and call Allow(). If it returns false, you write the 429 header and stop the chain right there. If it returns true, you call next.ServeHTTP(w, r) and let the request proceed to your actual business logic.

// A quick glimpse at the logic flow
func limitMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        ip := r.RemoteAddr // Simplification; usually you check X-Forwarded-For
        limiter := getLimiterForIP(ip)
        
        if !limiter.Allow() {
            http.Error(w, "Too Many Requests", http.StatusTooManyRequests)
            return
        }
        next.ServeHTTP(w, r)
    })
}

📋 Practical Task

Exercise: Implementing a Per-IP Rate Limiter for a Mock Payment Proxy

Your goal is to build a middleware-protected API gateway that proxies requests to a mock payment service. You must ensure that no single IP address can exceed a specific request threshold, preventing a single client from overwhelming the system.

Requirements:

• Create a LimiterManager struct that holds a map of *rate.Limiter objects, keyed by IP address, and a sync.RWMutex to handle concurrent access.
• Implement a method GetLimiter(ip string) *rate.Limiter that returns an existing limiter or creates a new one if the IP is seen for the first time.
• The limit should be set to 5 requests per second with a burst capacity of 10 requests.
• Write a middleware function that:
  • Extracts the client's IP from the request.
  • Uses the LimiterManager to check if the request is allowed.
  • Returns a 429 Too Many Requests status code if the limit is exceeded.
  • Calls the next handler if the request is permitted.
• Set up a simple http.HandleFunc at /pay that returns a "Payment Processed" message, and wrap it with your rate-limiting middleware.

Testing your work: Use a tool like curl in a loop or a benchmarking tool like hey or ab to verify that the first 10 requests go through quickly, but subsequent requests are throttled to 5 per second.