In a perfect world, every API call you make returns a 200 OK in under 100 milliseconds. But we don't live in that world. We live in a world of "transient faults"—those annoying, brief hiccups where a service is rebooting, a network switch is sneezing, or a database is momentarily overloaded. If you just let your application crash the first time a request fails, your users are going to hate you.

The manual loop that works until it doesn't

When I first started writing C#, my instinct was to just wrap the failing call in a for loop. It seems logical: try it three times, and if it still fails, then give up. It looks something like this:

public async Task<PaymentResponse> ProcessPayment(PaymentRequest request)
{
    int retryCount = 3;
    for (int i = 0; i < retryCount; i++)
    {
        try
        {
            return await _paymentGateway.ChargeAsync(request);
        }
        catch (HttpRequestException) when (i < retryCount - 1)
        {
            // Wait a bit before trying again
            await Task.Delay(1000);
        }
    }
    throw new Exception("Payment gateway is down after multiple attempts.");
}

On the surface, this solves the immediate problem. But here's where it falls apart in a production environment. First, it's rigid. If you want to change the retry logic for ten different services, you're copying and pasting this loop everywhere. Second, it uses a "fixed interval." If the payment gateway is struggling under a massive load, and 1,000 of your application instances all retry every exactly one second, you've just accidentally launched a Distributed Denial of Service (DDoS) attack against your own provider. I call this the "thundering herd" problem, and it's a great way to get your API key revoked.

Offloading the chaos to Polly

This is why we use Polly. It's the industry-standard resilience library for .NET. Instead of writing loops, you define a policy—a set of rules for how the system should behave when things go wrong—and then you wrap your call inside that policy. In the modern version of Polly (v8+), we use the ResiliencePipeline.

using Polly;
using Polly.Retry;

// Define the pipeline
var pipeline = new ResiliencePipelineBuilder()
    .AddRetry(new RetryStrategyOptions
    {
        ShouldHandle = new PredicateBuilder().Handle<HttpRequestException>(),
        BackoffType = DelayBackoffType.Exponential,
        UseJitter = true,
        MaxRetryAttempts = 3,
        Delay = TimeSpan.FromSeconds(1)
    })
    .Build();

// Use the pipeline to execute your call
var response = await pipeline.ExecuteAsync(async token => 
    await _paymentGateway.ChargeAsync(request), cancellationToken);

Notice the shift in responsibility. The ProcessPayment method no longer cares how the retry happens; it just knows that it's executing within a resilience pipeline. We've separated the "what" (charging a card) from the "how" (retrying when the network flickers).

Why waiting exactly one second is a bad idea

You'll notice I used DelayBackoffType.Exponential and UseJitter = true in the Polly example. This is the "secret sauce" of professional software engineering. Exponential backoff means the wait time grows (1s, 2s, 4s...), giving the failing service more breathing room to recover.

Jitter adds a random amount of time to that delay. Instead of everyone retrying at exactly 2.0 seconds, some will retry at 1.8s, others at 2.2s. This spreads the load across the timeline, preventing those spikes of traffic that crash recovered servers. It's a small configuration change, but it's the difference between a system that recovers gracefully and one that enters a death spiral.

Now, a word of caution: retries are only for transient faults. If the API returns a 401 Unauthorized or a 400 Bad Request, retrying a thousand times won't help; the request is fundamentally broken. Always make sure your ShouldHandle predicate is targeting the right exceptions or status codes. Don't retry a password failure; you'll just get your user's account locked.

📋 Practical Task

Exercise: Implementing a Resilient Weather API Client

You are building a weather dashboard that consumes a third-party API. The API is notoriously flaky and frequently throws HttpRequestException during peak hours. Your goal is to implement a resilient client using Polly.

Requirements:

• Create a WeatherService class with a method GetTemperatureAsync(string city).
• Configure a Polly ResiliencePipeline that:
  • Handles only HttpRequestException.
  • Retries up to 5 times.
  • Uses Exponential Backoff.
  • Enables Jitter to avoid the thundering herd problem.
  • Starts with an initial delay of 2 seconds.
• Wrap the API call (you can simulate the API call with a method that randomly throws an exception) within this pipeline.

Success Criteria: Your code should demonstrate that the application doesn't crash on the first failure and that the time between retries increases progressively rather than remaining constant.