A few years ago, I was helping a teammate integrate a legacy financial API that had a very strict limit: exactly five requests per second. He had written a clean, concurrent worker pool to sync thousands of accounts. It worked perfectly in our staging environment because the staging API had no limits. But the moment he flipped the switch to production, the API started throwing 429 Too Many Requests errors like confetti. He tried to fix it by sprinkling time.Sleep(200 * time.Millisecond) throughout the code, but because of the way the workers were scheduled, they still overlapped and triggered the rate limiter. He spent an entire weekend chasing "ghost" bugs that were actually just the API shutting him out.

The lesson here is that rate limiting shouldn't be an afterthought or a series of sleeps scattered in your business logic. It needs to be a first-class citizen of your API client. In Go, the most elegant way to handle this is by using a channel as a "metronome" to coordinate your requests.

Using Tickers as a Request Metronome

If you need a steady, predictable cadence, time.Ticker is your best friend. Instead of telling a goroutine to sleep, you tell it to wait for a signal from the ticker. This ensures that no matter how many workers you have competing for the resource, they only proceed when the ticker "ticks."

type RateLimitedClient struct {
    client  *http.Client
    limiter <-chan time.Time
}

func NewRateLimitedClient(requestsPerSecond int) *RateLimitedClient {
    // We create a ticker that sends a signal every 1/N seconds
    ticker := time.NewTicker(time.Second / time.Duration(requestsPerSecond))
    return &RateLimitedClient{
        client:  &http.Client{Timeout: 10 * time.Second},
        limiter: ticker.C,
    }
}

func (c *RateLimitedClient) Do(req *http.Request) (*http.Response, error) {
    // This is the magic line. The goroutine blocks here until 
    // the ticker sends a value over the channel.
    <-c.limiter
    return c.client.Do(req)
}

I prefer this approach because it decouples the "when" from the "what." Your business logic just calls Do(), and the client handles the timing. If you increase your worker count from 10 to 100, the rate of requests remains exactly the same; the workers just queue up more densely behind the channel read.

Handling Bursts with Buffered Channels

The ticker approach is great for a strict "metronome" feel, but many modern APIs allow for "bursting." For example, an API might let you make 10 requests instantly, as long as you average 2 requests per second over a minute. A strict ticker would waste that capacity.

To handle this, we can use a buffered channel as a "token bucket." You fill the buffer with tokens, and each request consumes one. A separate background goroutine then refills those tokens at a steady rate. This allows your client to handle a sudden spike in traffic without immediately hitting the 429 limit, while still maintaining a long-term average.

type TokenBucketClient struct {
    client *http.Client
    tokens chan struct{}
}

func NewTokenBucketClient(maxBurst, refillRate int) *TokenBucketClient {
    c := &TokenBucketClient{
        client: &http.Client{},
        tokens: make(chan struct{}, maxBurst),
    }

    // Fill the bucket initially
    for i := 0; i < maxBurst; i++ {
        c.tokens <- struct{}{}
    }

    // Refill the bucket at the specified rate in the background
    go func() {
        ticker := time.NewTicker(time.Second / time.Duration(refillRate))
        for range ticker.C {
            select {
            case c.tokens <- struct{}{}:
            default: 
                // Bucket is full, drop the token
            }
        }
    }()

    return c
}

func (c *TokenBucketClient) Do(req *http.Request) (*http.Response, error) {
    <-c.tokens // Block until a token is available
    return c.client.Do(req)
}

One detail I should mention: notice the select block in the refill goroutine. Without the default case, the background goroutine would block forever once the bucket reached maxBurst, which would leak a goroutine or stop your ticker. Always ensure your refill logic is non-blocking.

📋 Practical Task

Exercise: The Shopify-Style Order Synchronizer

You are building a tool that syncs orders from a third-party store. The API allows a maximum of 3 requests per second, but it allows a burst of up to 5 requests if the client has been idle.

Your Task:

• Implement a SyncClient using the Token Bucket pattern described in the lesson.
• The client must support a burst of 5 and a refill rate of 3 per second.
• Write a main function that launches 20 concurrent goroutines, each attempting to make a "request" (for the sake of the exercise, a "request" can simply be printing "Request sent at %v" with the current timestamp).
• Verify that the first 5 requests happen almost instantaneously, and the subsequent requests are spaced out by approximately 333ms.