I've seen this mistake dozens of times, usually in code written by developers who are just starting to move from sequential to concurrent programming. It looks innocent enough: you have a batch of work to do, and you want it to happen as fast as possible, so you wrap each single task in its own thread. It feels intuitive, but in a production environment, it's a ticking time bomb.

The "One Thread Per Task" Disaster

// Imagine we're processing 5,000 log files from a server
List<File> logFiles = getLogFiles();

for (File file : logFiles) {
    new Thread(() -> {
        processLogFile(file);
    }).start();
}

At first glance, this looks like it's maximizing parallelism. But here is what's actually happening under the hood: you're asking the Operating System to allocate a significant amount of memory for 5,000 different thread stacks. You're also forcing the CPU to perform "context switching" thousands of times per second—basically spending more time deciding which thread to run than actually running your code.

If you run this on a machine with limited resources, you'll eventually hit an OutOfMemoryError: unable to create new native thread. Even if it doesn't crash, your application will likely crawl to a halt because the overhead of managing those threads outweighs the benefit of the concurrency.

Stopping the Bleeding with ExecutorService

Instead of managing threads manually, we use a Thread Pool. The idea is simple: create a small, fixed number of worker threads and keep them alive. When a task comes in, it sits in a queue until one of those workers is free to pick it up. This keeps your memory usage predictable and your CPU efficient.

Here is how we rewrite that log processor using the ExecutorService:

// We create a pool with a fixed number of threads. 
// A good rule of thumb is based on your CPU cores.
int cores = Runtime.getRuntime().availableProcessors();
ExecutorService executor = Executors.newFixedThreadPool(cores);

for (File file : logFiles) {
    executor.submit(() -> {
        processLogFile(file);
    });
}

// This is the part people always forget: you must shut down the executor!
executor.shutdown();

Now, instead of 5,000 threads fighting for attention, we have (for example) 8 threads working through the queue. The throughput is usually much higher because the CPU stays in the "zone" longer, and the memory footprint remains flat regardless of whether you have 5,000 or 5 million files to process.

Picking the Right Pool Strategy

Not every problem fits a FixedThreadPool. Depending on what your tasks are doing, you might want a different strategy. I usually categorize tasks into two buckets: CPU-bound and IO-bound.

• CPU-bound: Tasks that do heavy math or data processing. Use a fixed pool size roughly equal to your number of cores. Adding more threads won't help because you physically don't have more cores to do the work.
• IO-bound: Tasks that spend most of their time waiting (database queries, API calls, reading files). You can get away with a much larger pool here—sometimes 10x or 20x the number of cores—because the threads spend most of their time sleeping, leaving the CPU free to manage others.

If you're dealing with a massive, unpredictable burst of short-lived tasks, Executors.newCachedThreadPool() is an option. It creates threads as needed and re-uses them, but be careful: if your tasks start piling up and blocking, a cached pool can grow indefinitely and bring you right back to that OutOfMemoryError we started with.

The Importance of a Graceful Shutdown

An ExecutorService doesn't just vanish when the method ends. It contains worker threads that are, by default, "non-daemon" threads. This means your JVM will actually keep running even after your main method finishes, simply because the thread pool is still idling and waiting for work.

Always call executor.shutdown(). If you need to ensure the tasks finish within a certain timeframe before the app closes, use the combination of shutdown() followed by awaitTermination(). It's the professional way to handle the lifecycle of your background workers.

📋 Practical Task

Exercise: Concurrent Website Availability Checker

You need to build a tool that checks the availability of a list of websites. Because network requests are IO-bound (meaning the thread spends most of its time waiting for the server to respond), you should use a thread pool to check multiple sites simultaneously.

Requirements:

• Create a list of at least 10 URLs (you can use sites like google.com, github.com, etc.).
• Implement an ExecutorService using a fixed thread pool of 4 threads.
• For each URL, submit a task that attempts to open a connection to the site and prints whether the site is "UP" or "DOWN".
• Ensure the program shuts down the executor service correctly so the JVM can exit.
• Bonus: Use Future<String> to collect the results of each check and print a final summary report once all tasks are complete.