Skip to Content
Course content

318: Combining Multiple Context Managers

Click on the "Edit" button in the top corner of the screen to edit your slide content.

At some point, you're going to run into a situation where you need to manage more than one resource at a time. Maybe you're reading from a configuration file and writing to a log, or perhaps you're transferring data between two different database connections. The instinct for most of us starting out is to just stack them.

The Nesting Trap

When I first started writing Python, I did exactly what you're probably thinking: I nested my with statements. It looks like this:

with open('source.txt', 'r') as src:
    with open('destination.txt', 'w') as dst:
        for line in src:
            if "ERROR" in line:
                dst.write(line)

Now, there is nothing technically "wrong" with this. It works, it's safe, and it ensures both files are closed properly. But as your logic grows, you end up with what I call the "Pyramid of Doom." If you need four or five resources, your actual logic is suddenly indented halfway across the screen. It's visually noisy and makes the code harder to scan. It's a cognitive tax you don't need to pay.

Flattening the Resource Stack

Python gives us a much cleaner way to handle this. You can comma-separate multiple context managers within a single with statement. It looks like this:

with open('source.txt', 'r') as src, open('destination.txt', 'w') as dst:
    for line in src:
        if "ERROR" in line:
            dst.write(line)

Under the hood, this is functionally identical to the nested version. Python enters the managers from left to right and exits them in reverse order. If the second open() fails, the first one is still closed correctly. It's cleaner, it keeps your indentation levels sane, and it tells whoever is reading your code, "These two resources are linked for this specific block of logic."

Handling an Unknown Number of Resources

Here is where things get tricky. The comma-separated approach is great when you know exactly how many files you're opening. But what if you're passed a list of ten filenames and told to merge them? You can't write a with statement with a dynamic number of arguments.

You might be tempted to use a loop to open them, but if you do with open(f) as file: inside a loop, the file closes the moment that iteration ends. You'll end up with a bunch of closed handles and no way to work with them simultaneously.

This is where contextlib.ExitStack comes in. I use this whenever I'm dealing with a dynamic set of resources. It essentially acts as a container for context managers that you can add to on the fly:

from contextlib import ExitStack

filenames = ['log1.txt', 'log2.txt', 'log3.txt']
with ExitStack() as stack:
    files = [stack.enter_context(open(fname, 'r')) for fname in filenames]
    # Now 'files' is a list of open file objects
    # All of them will be closed when the ExitStack block exits
    for f in files:
        print(f.read())

I consider ExitStack a "power user" tool. It prevents you from having to write complex try...finally blocks manually to ensure every single file in a list gets closed if the fifth one happens to throw an exception. It's the professional way to handle variable-width resource management.




📋 Practical Task

Build a Multi-File Log Aggregator

Write a function called aggregate_logs(source_files, output_filename). The function should take a list of strings (filenames) and a single string for the destination file.

  • Use contextlib.ExitStack to open all the source files simultaneously.
  • Use a second context manager (combined in the same block or nested) to open the output file for writing.
  • Read the contents of every source file and write them into the output file, adding a header line before each file's content (e.g., "--- Contents of log1.txt ---").
  • Ensure that all files are closed automatically, even if one of the source files is missing or corrupted during the process.
Rating
0 0

There are no comments for now.

to be the first to leave a comment.