Imagine you're trying to call a friend who isn't picking up. If you call them every five seconds for ten minutes straight, you're not being helpful—you're just spamming them. If their phone is dead or they're in a meeting, your constant calling doesn't fix the problem; it just ensures that the moment they do look at their phone, they'll be annoyed (or their battery will die even faster).

A better approach is to call once. If they don't answer, you wait five minutes. Still nothing? You wait fifteen. Then an hour. You're giving them space to resolve whatever is keeping them away while still making sure you eventually get through. That, in a nutshell, is exponential backoff.

Mapping the Phone Call to the Code

In the world of distributed systems, your "friend" is usually a flaky microservice or a third-party API. When that API returns a 503 Service Unavailable, the worst thing you can do is immediately hammer it with 1,000 more requests. You'll effectively perform a DDoS attack on your own infrastructure. Instead, we map our phone call logic like this:

• The Call: The execution of a block of code (like an HTTP request).
• The Silence: A caught exception or a "Failure" response.
• The Increasing Wait: A delay that grows exponentially (e.g., $2^{attempt} \times 100ms$).
• Giving Up: A maximum retry limit so we don't loop forever.

Calculating the Wait Period

I've seen a lot of developers just hardcode a few Thread.sleep(1000) calls, but that's brittle. The clean way to handle this in Scala is to calculate the delay based on the current attempt number. A common formula is baseDelay * Math.pow(2, attempt).

One pro tip: in a real production environment, you should add "jitter." Jitter is just a bit of random noise added to the delay. If 100 instances of your service all fail at the same time and all use the exact same exponential backoff, they'll all retry at the exact same millisecond, creating "thundering herd" spikes. Adding a random 0-100ms here and there smears those requests out.

Structuring the Retry Loop

Since we're in Scala, we want to avoid mutable while loops if we can. A recursive function is much more natural here. You pass the current attempt count as an argument, and if the operation fails and you haven't hit your limit, you call the function again with attempt + 1.

def retryWithBackoff[T](attempt: Int, maxRetries: Int, block: => T): T = {
  try {
    block
  } catch {
    case e: Exception if attempt < maxRetries =>
      val waitTime = Math.pow(2, attempt).toInt * 100
      println(s"Attempt $attempt failed. Retrying in ${waitTime}ms...")
      Thread.sleep(waitTime)
      retryWithBackoff(attempt + 1, maxRetries, block)
    case e: Exception =>
      println("Max retries reached. Giving up.")
      throw e
  }
}

Notice the block: => T. By using a call-by-name parameter, we ensure the code inside the block isn't executed until we actually call it inside the try block. If we used a regular parameter, the work would be done before the function was even called, which would defeat the whole purpose of retrying.

📋 Practical Task

Exercise: Implementing a Resilient Weather API Client with Backoff

You are building a weather dashboard that fetches data from a notoriously unstable external API. Your goal is to wrap the API call in a retry mechanism so that temporary network glitches don't crash the entire dashboard.

Requirements:

• Create a class WeatherServiceClient with a method fetchTemperature(city: String): Int.
• Inside fetchTemperature, simulate an API call. For the sake of this exercise, make the call fail (throw a RuntimeException) 70% of the time and succeed (return a random temperature) 30% of the time.
• Implement a retry mechanism using exponential backoff.
• The base delay should be 200ms.
• The maximum number of retries should be 4.
• If the call fails after all 4 retries, the method should finally throw the exception to the caller.

Expected Output: When you run the client, you should see log messages indicating the attempt number and the increasing wait time before the final success or failure.