We've all been there: your user clicks "Sign Up," and the browser just hangs. Why? Because your code is trying to connect to an external SMTP server to send a welcome email, and that server is taking three seconds to respond. The user thinks your site is broken, but in reality, you're just making them wait for a task that doesn't actually need to happen in real-time.

The solution is a job queue. While tools like RabbitMQ or Redis are the industry standard for high-scale apps, you don't always need that complexity. For a lot of projects, a simple database table is more than enough to get the job done.

Designing the holding pen

First, we need a place to store these pending tasks. I like to keep it lean. We need to know what the job is, when it was created, and whether it's already being handled. I'm going to use a payload column to store a JSON string; this gives us the flexibility to pass different types of data (like a user ID and an email template name) without changing the schema every time we add a new job type.

CREATE TABLE job_queue (
    id INT AUTO_INCREMENT PRIMARY KEY,
    payload TEXT NOT NULL,
    status ENUM('pending', 'processing', 'completed', 'failed') DEFAULT 'pending',
    attempts INT DEFAULT 0,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);

Offloading the work

Now, instead of calling mail() or a library like PHPMailer directly in our registration logic, we just "dispatch" the job. I've written this as a simple helper function. It just dumps the data into the table and lets the user move on with their life immediately.

function queueJob($jobType, $data) {
    $db = getDatabaseConnection();
    $payload = json_encode(['type' => $jobType, 'data' => $data]);
    
    $stmt = $db->prepare("INSERT INTO job_queue (payload) VALUES (?)");
    $stmt->execute([$payload]);
}

// Usage during registration:
queueJob('send_welcome_email', ['user_id' => 42, 'email' => 'user@example.com']);

The worker script and a critical race condition

Now for the worker. This is a PHP script that runs in the background (usually via a cron job or a supervisor process). It looks for 'pending' jobs and executes them. Initially, I wrote it like this:

// DON'T DO THIS - My first mistake
$jobs = $db->query("SELECT * FROM job_queue WHERE status = 'pending' LIMIT 5");
foreach ($jobs as $job) {
    processJob($job); 
    $db->prepare("UPDATE job_queue SET status = 'completed' WHERE id = ?")->execute([$job['id']]);
}

I realized quickly that this is a disaster waiting to happen. If this script takes 30 seconds to run and your cron job triggers every 10 seconds, you'll have multiple workers picking up the same 'pending' jobs. Your users will get five welcome emails, and you'll look like an amateur.

To fix this, we need to "claim" the job atomically. I'll update the status to 'processing' before I even touch the payload. By using a WHERE status = 'pending' in the update statement, the database ensures only one worker wins the race for that specific ID.

while (true) {
    $db = getDatabaseConnection();
    
    // Find one pending job
    $stmt = $db->query("SELECT id FROM job_queue WHERE status = 'pending' LIMIT 1");
    $job = $stmt->fetch();

    if (!$job) {
        sleep(5); // Nothing to do, take a nap
        continue;
    }

    // Atomically claim the job
    $update = $db->prepare("UPDATE job_queue SET status = 'processing' WHERE id = ? AND status = 'pending'");
    $update->execute([$job['id']]);

    if ($update->rowCount() === 0) {
        continue; // Someone else grabbed it first!
    }

    // Now we actually do the work
    $jobDetails = $db->prepare("SELECT payload FROM job_queue WHERE id = ?");
    $jobDetails->execute([$job['id']]);
    $data = json_decode($jobDetails->fetchColumn(), true);

    try {
        if ($data['type'] === 'send_welcome_email') {
            // Simulate sending email
            echo "Sending email to " . $data['data']['email'] . "\n";
        }
        $db->prepare("UPDATE job_queue SET status = 'completed' WHERE id = ?")->execute([$job['id']]);
    } catch (Exception $e) {
        $db->prepare("UPDATE job_queue SET status = 'failed', attempts = attempts + 1 WHERE id = ?")->execute([$job['id']]);
    }
}

Keeping the table clean

One last thing: if you leave this running for a year, your job_queue table will grow to millions of rows of 'completed' jobs. It'll eventually slow down your queries. In a real production environment, I'd set up a separate cleanup task to delete completed jobs older than 7 days. A simple DELETE FROM job_queue WHERE status = 'completed' AND updated_at < NOW() - INTERVAL 7 DAY does the trick.

📋 Practical Task

Build a Password Reset Token Queue

Using the logic from this lesson, implement a job queue system for password resets. Instead of sending the email immediately when a user requests a password reset, you must:

• Create a new entry in the job_queue table with a type of 'send_password_reset' and a payload containing the user's email and a unique reset token.
• Modify the worker script to handle the 'send_password_reset' job type.
• Ensure the worker correctly handles the "claiming" process to prevent the same reset email from being sent multiple times if multiple worker processes are running.
• Add a check in the worker: if a job has reached 3 attempts without succeeding, mark its status as 'failed' permanently rather than trying again.