In a perfect world, every API call succeeds and every database query is instant. In the real world, the third-party CRM you're syncing data to will go down for five minutes on a Tuesday, or it'll rate-limit you because you pushed too many updates at once. If your background job just fails and dies, you lose data. If it retries immediately and constantly, you're basically DDoS-ing a service that's already struggling.

That's where retry and backoff strategies come in. The goal isn't just to "try again," but to try again smartly.

Dealing with a Flaky CRM API

Let's build a job that pushes a user's updated profile to an external CRM. I'll assume we're using a standard queue worker setup where the job can be released back into the queue if it fails.

class SyncUserToCrmJob
{
    public function handle($userData)
    {
        $response = CRM::updateUser($userData);

        if ($response->failed()) {
            throw new \Exception("CRM API unavailable");
        }
    }
}


Now, if this throws an exception, most queue runners will just retry it. But the default behavior is often "retry immediately." If the CRM is returning a 429 (Too Many Requests), retrying a millisecond later is just going to get you another 429. You're wasting CPU cycles and making the API provider hate you.

The "Immediate Retry" Trap
When I first started writing these, I thought the best way to handle failures was a simple for loop inside the handle method. I figured, "I'll just try it three times right here."

// DON'T DO THIS
public function handle($userData)
{
    for ($i = 0; $i < 3; $i++) {
        try {
            return CRM::updateUser($userData);
        } catch (\Exception $e) {
            // Just loop and try again immediately
        }
    }
    throw new \Exception("Failed after 3 attempts");
}


I learned the hard way that this is a terrible idea. Why? Because it blocks the worker. If the API is timing out (taking 30 seconds per request), this one job now ties up your worker for 90 seconds. If you have 100 failed jobs, your entire queue grinds to a halt. Never block your worker with sleep() or tight loops. Instead, we want to "release" the job back to the queue with a delay.

Implementing Exponential Backoff
The professional way to do this is exponential backoff. Instead of waiting a fixed amount of time, we increase the delay with every failure. This gives the external service breathing room to recover.

I like to use a formula where the delay is (attempts ^ 4) + 15 (or something similar). This means the first retry is fast, but the fifth retry happens much later.

public function handle($userData)
{
    try {
        CRM::updateUser($userData);
    } catch (\Exception $e) {
        $attempts = $this->getJobAttempts();

        if ($attempts >= 5) {
            $this->fail($e);
            return;
        }

        // Exponential backoff: 15s, 31s, 99s, 271s...
        $delay = pow($attempts, 4) + 15;
        
        $this->release($delay);
    }
}


By using release($delay), the worker puts the job back in the queue and immediately moves on to the next task. The job stays invisible until the delay expires, which is exactly what we want.

Adding Jitter to Prevent Thundering Herds
There's one more detail that separates the juniors from the seniors: jitter. Imagine you have a network glitch and 1,000 jobs all fail at the exact same second. If they all use the exact same exponential backoff formula, they will all wake up and hit the API again at the exact same second. This is called the "thundering herd" problem.

To fix this, we add a bit of randomness—a "jitter"—to the delay.

$delay = pow($attempts, 4) + 15;
$jitter = rand(0, 10); // Add a few random seconds
$this->release($delay + $jitter);


It seems trivial, but in a high-scale system, that small random offset spreads the load and prevents your own retry logic from becoming a self-inflicted denial-of-service attack.

📋 Practical Task

Exercise: Implementing Jittered Backoff for a Weather Data Sync Job

You are maintaining a job that fetches current weather data from a public API and saves it to your database. The API is notoriously unstable and frequently returns 503 errors.

Your task: Modify the provided WeatherSyncJob class to implement a retry strategy with the following requirements:

• The job should not retry more than 5 times.
• Implement exponential backoff using the formula: (attempts * 10) * attempts (e.g., Attempt 1 = 10s, Attempt 2 = 40s, Attempt 3 = 90s).
• Add a random "jitter" between 1 and 5 seconds to the final delay.
• Ensure the job is released back to the queue rather than using sleep().

class WeatherSyncJob
{
    public function handle($city)
    {
        try {
            // This method is provided by the WeatherSDK
            WeatherSDK::fetchAndSave($city);
        } catch (\Exception $e) {
            // TODO: Implement your retry and backoff logic here
        }
    }

    // Mock method to simulate getting current attempt count
    protected function getJobAttempts() {
        return 1; // In a real app, this comes from the queue worker
    }

    // Mock method to simulate releasing the job back to the queue
    protected function release($seconds) {
        echo "Job released for {$seconds} seconds.";
    }

    // Mock method to mark the job as permanently failed
    protected function fail($exception) {
        echo "Job failed permanently: " . $exception->getMessage();
    }
}