Imagine we're building a system that generates a massive daily PDF report for our clients. We have five different worker pods running in Kubernetes to handle the load. If two pods start generating the same report for the same client at the same time, we're wasting resources and potentially sending the client two identical emails. We need a distributed lock—something that says, "I've got this client covered, everyone else back off."

The "Check-then-Set" Trap

When I first started playing with Redis for locking, I did what most people do. I wrote a simple check: ask Redis if a key exists; if it doesn't, create it. In Go, it looks something like this:

// DON'T DO THIS
exists, _ := rdb.Exists(ctx, "lock:client_123").Result()
if exists == 0 {
    rdb.Set(ctx, "lock:client_123", "locked", 0)
    // Do the work...
}

On a local machine with one worker, this works perfectly. But in production, this is a disaster waiting to happen. This is a classic "check-then-act" race condition. If two pods check for the key at the exact same millisecond, both will see that it doesn't exist, and both will proceed to set the lock and run the report. You haven't built a lock; you've built a suggestion.

Then there's the "zombie lock" problem. In the code above, I set the value with no expiration. If the pod crashes halfway through generating that PDF, the lock stays in Redis forever. No other pod will ever touch that client's report again until a human manually deletes the key from the CLI. That's a support ticket I don't want to deal with at 3 AM.

Atomic Set and the TTL Safety Net

To fix this, we have to stop thinking in terms of multiple commands and start thinking in terms of atomic operations. Redis provides a way to "Set if Not Exists" (NX) and set a "Time To Live" (PX) in a single atomic call. This eliminates the race condition and ensures the lock eventually expires even if the worker vanishes from the face of the earth.

// A much better approach
uniqueID := uuid.New().String()
success, err := rdb.SetNX(ctx, "lock:client_123", uniqueID, 30*time.Second).Result()
if err != nil || !success {
    return fmt.Errorf("could not acquire lock")
}
// Do the work...

By using `SetNX`, Redis handles the check and the assignment in one heartbeat. If the key is already there, it returns false. The 30-second TTL is our insurance policy. Now, if the pod dies, the lock naturally evaporates after 30 seconds, allowing another worker to pick up the slack.

Preventing Accidental Lock Theft

You might think we're done, but there's one more edge case that bites experienced devs: the "stale unlock." Imagine Pod A acquires the lock, but then hits a massive Garbage Collection pause or a slow network call that lasts 31 seconds. The lock expires. Pod B comes along and acquires the lock for the same client.

Suddenly, Pod A wakes up. It finishes its work and calls `rdb.Del(ctx, "lock:client_123")`. But Pod A isn't deleting its own lock anymore—it's deleting Pod B's lock! Now Pod C can jump in while Pod B is still working. We've lost our mutual exclusion.

The fix is to ensure that a worker can only delete a lock if the value inside the lock is the same unique ID they generated when they first acquired it. Since checking the value and deleting the key must happen together, we use a Lua script. Redis executes Lua scripts atomically, meaning no other command can run in the middle of our check-and-delete logic.

var unlockScript = redis.NewScript(`
    if redis.call("get", KEYS[1]) == ARGV[1] then
        return redis.call("del", KEYS[1])
    else
        return 0
    end
`)

// Use it like this:
err := unlockScript.Run(ctx, rdb, []string{"lock:client_123"}, uniqueID).Err()

This is the "professional" way to handle a simple distributed lock. It's not as bulletproof as Redlock (which handles Redis cluster failures), but for 99% of use cases, an atomic SetNX paired with a Lua-based release is exactly what you need.

📋 Practical Task

Exercise: Implementing a Safe Distributed Lock for a Daily Report Generator

You are tasked with implementing a locking mechanism for a report generator. Using the github.com/redis/go-redis/v9 library, write a Go function PerformLockedWork(ctx context.Context, rdb *redis.Client, clientID string, work func()) error that fulfills the following requirements:

• Acquisition: Attempt to acquire a lock using a key formatted as lock:report:{clientID}. The lock must be atomic and have a TTL of 15 seconds.
• Unique Ownership: Use a unique identifier (like a UUID or a random string) as the value of the lock to ensure only the owner can release it.
• Execution: If the lock is acquired, execute the provided work() function.
• Safe Release: After the work is complete (or if the work panics), release the lock using a Lua script to ensure you only delete the lock if you still own it.
• Failure Handling: If the lock cannot be acquired, return a specific error indicating the resource is currently locked.