Skip to Content
Course content

135: http.Client Configuration

Click on the "Edit" button in the top corner of the screen to edit your slide content.

I can almost guarantee that at some point, you've written http.Get("https://api.example.com"). It's the fastest way to get a request out the door, and for a quick script or a local prototype, it's perfectly fine. But if you're writing code that's going to live in a production environment, using the default client is a ticking time bomb.

The "Default Client is Fine" Myth

The biggest misconception I see is that http.DefaultClient is a sensible, pre-configured client provided by the Go team. It's not. In reality, the default client has no timeout. None. Zero.

Think about what that means in a real-world scenario. Imagine you're building a service that fetches exchange rates from a third-party API. If that API's server hangs—meaning it accepts the TCP connection but just never sends the response headers—your goroutine will sit there forever. It will wait for a response that is never coming. If your service gets 100 requests per second, you'll leak 100 goroutines per second. Eventually, your memory spikes, your scheduler chokes, and your entire application crashes because of a dependency you don't even control.

// DON'T DO THIS IN PRODUCTION
resp, err := http.Get("https://slow-api.com/data") 
// If slow-api.com hangs, this line blocks indefinitely.

Taking Control with Explicit Timeouts and Transports

The correct approach is to instantiate your own http.Client. This gives you a handle on the Timeout field, which covers the entire exchange: from the initial Dial to reading the response body. I usually set this to a conservative value based on the SLA of the service I'm calling.

var apiClient = &http.Client{
    Timeout: 10 * time.Second,
}

func fetchData() {
    // Now, if the server hangs, the request will fail after 10s
    resp, err := apiClient.Get("https://api.example.com/metrics")
    if err != nil {
        log.Printf("Request failed: %v", err)
        return
    }
    defer resp.Body.Close()
}

But wait—there's more. Sometimes a global timeout is too blunt an instrument. If you need to tune how the underlying TCP connections are handled, you have to look at the Transport. The http.Client is actually just a high-level wrapper around an http.RoundTripper (usually an *http.Transport).

If you're making thousands of requests to the same host, you'll want to tweak the connection pooling. For instance, MaxIdleConnsPerHost defaults to 2. In a high-throughput system, that's way too low. You'll end up creating new TCP connections for every single request because the old ones are closed too quickly, leading to socket exhaustion (TIME_WAIT). I've spent an entire afternoon debugging "connection refused" errors only to realize I was just killing my own sockets by using the default transport settings.

customTransport := &http.Transport{
    MaxIdleConns:        100,
    MaxIdleConnsPerHost: 20, 
    IdleConnTimeout:     90 * time.Second,
}

var hardenedClient = &http.Client{
    Transport: customTransport,
    Timeout:   15 * time.Second,
}

By separating the Client (which handles higher-level logic like timeouts and redirects) from the Transport (which handles the "plumbing" of the network), you can ensure your application remains resilient even when the internet is acting up.




📋 Practical Task

Building a Resilient External API Consumer

You are tasked with writing a small utility that fetches data from a simulated "flaky" API. Your goal is to ensure the application never hangs indefinitely and manages its connections efficiently.

  • Create a package-level http.Client.
  • Configure the client with a total request timeout of 5 seconds.
  • Configure a custom http.Transport that allows up to 10 idle connections per host (instead of the default 2).
  • Write a function FetchData(url string) ([]byte, error) that uses this client to perform a GET request.
  • Ensure that the response body is closed correctly to avoid leaking connections.

Test your implementation by attempting to call a non-responsive IP (like http://10.255.255.1) and verify that your code returns a timeout error after 5 seconds rather than hanging forever.

Rating
0 0

There are no comments for now.

to be the first to leave a comment.