Imagine you're running a busy coffee shop. You have a line of customers placing orders, but you don't want the person taking the order to also be the person steaming the milk, pulling the espresso shots, and handing over the cup. That would be a bottleneck. Instead, you have a "ticket rail"—a physical line where orders are clipped. Your baristas (the workers) just keep an eye on that rail. As soon as a ticket appears, the first available barista grabs it, finishes the drink, and then goes back to the rail to see if there's more work. If the rail is empty, they just hang out and wait.

In C++, a multithreaded task queue works exactly like that ticket rail. We have "producer" threads (the order takers) pushing tasks into a queue, and "worker" threads (the baristas) pulling those tasks out and executing them. The magic happens in how we manage the waiting; we don't want our worker threads spinning in a while(true) loop eating 100% of your CPU just to check if a task exists. We use a std::condition_variable to let the threads "sleep" until a new ticket is clipped to the rail.

The Ticket Rail: Guarding the Queue

Since multiple threads will be touching our queue at once—some adding tasks, some removing them—we can't just use a raw std::queue. We need a std::mutex to ensure that only one thread is modifying the queue at a time. I've seen plenty of developers try to "optimize" this by using lock-free queues early on, but unless you're writing a high-frequency trading engine, a mutex is your best friend. It's safe, readable, and usually fast enough.

#include <iostream>
#include <vector>
#include <queue>
#include <thread>
#include <mutex>
#include <condition_variable>
#include <functional>

class TaskQueue {
    // We use std::function<void()> so we can pass lambdas or bound functions
    std::queue<std::function<void()>> tasks;
    std::mutex queueMutex;
    std::condition_variable condition;
    std::vector<std::thread> workers;
    bool stop = false;

public:
    TaskQueue(size_t threads);
    ~TaskQueue();
    void enqueue(std::function<void()> task);
};

Wiring up the Worker Loop

When we initialize the TaskQueue, we spawn a set number of worker threads. Each thread runs a loop that basically says: "Lock the mutex, check if there's a task. If not, sleep until the condition variable wakes me up."

Notice the use of std::unique_lock here. Unlike std::lock_guard, a unique_lock can be unlocked and relocked, which is exactly what the condition_variable needs to do behind the scenes when it puts the thread to sleep.

TaskQueue::TaskQueue(size_t threads) {
    for(size_t i = 0; i < threads; ++i) {
        workers.emplace_back([this] {
            while(true) {
                std::function<void()> task;
                {
                    std::unique_lock<std::mutex> lock(this -> queueMutex);
                    
                    // Wait until there is a task or the queue is stopping
                    this -> condition.wait(lock, [this] { 
                        return this -> stop || !this -> tasks.empty(); 
                    });

                    if(this -> stop && this -> tasks.empty()) {
                        return; // Exit thread
                    }

                    task = std::move(this -> tasks.front());
                    this -> tasks.pop();
                }
                // Execute the task outside the lock! 
                // If you run the task inside the lock, you've just made your app single-threaded.
                task();
            }
        });
    }
}

Pushing Tasks and Waking Workers

Adding a task is the easy part. We lock the mutex, push the function into the queue, and then call notify_one(). This tells the OS to wake up exactly one of the sleeping worker threads to handle the new ticket.

void TaskQueue::enqueue(std::function<void()> task) {
    {
        std::lock_guard<std::mutex> lock(queueMutex);
        tasks.push(std::move(task));
    }
    condition.notify_one();
}

Shutting Down the Shop

You can't just let the TaskQueue object go out of scope; if you do, the worker threads will be destroyed while they're still running, and your program will crash with a nasty std::terminate. You need to signal the threads to stop and then join() them.

TaskQueue::~TaskQueue() {
    {
        std::lock_guard<std::mutex> lock(queueMutex);
        stop = true;
    }
    condition.notify_all(); // Wake everyone up to tell them to leave
    for(std::thread & worker : workers) {
        worker.join();
    }
}

I'll leave you with a tip: the most common bug here is forgetting to execute the task outside the scope of the unique_lock. If the task itself takes 2 seconds to run and you hold the lock, every other worker thread will be blocked from even checking the queue, defeating the entire purpose of having multiple threads.

📋 Practical Task

Exercise: Building a Multithreaded Image Processor Simulator

You are tasked with simulating a system that processes high-resolution images. Each "image process" is just a function that prints the image ID and sleeps for a random amount of time to simulate heavy computation.

Using the TaskQueue logic from this lesson, implement a program that:

• Initializes a TaskQueue with 4 worker threads.
• Enqueues 10 different tasks. Each task should be a lambda that captures an integer imageID, prints "Processing image [ID] on thread [Thread ID]...", and then calls std::this_thread::sleep_for for a random duration between 100ms and 500ms.
• Ensures that the main thread waits for all tasks to be processed before the program exits.

Requirement: Your output should demonstrate that images are being processed out of order (e.g., Image 3 might finish before Image 1), proving that your workers are operating in parallel.