I've been working on a tool to monitor a remote server's logs in real-time. The problem is that the connection involves a network handshake and the data arrives in chunks—it's all inherently asynchronous. I tried to wrap it in a standard context manager to make sure the connection closes properly, but I hit a wall immediately.

The problem with standard 'with' blocks

Here is my first attempt. I wanted a simple class that opens a connection and closes it when I'm done. Since the connection is an awaitable call, I tried putting it inside a regular __enter__ method:

class LogStreamer:
    def __init__(self, host):
        self.host = host

    def __enter__(self):
        # This is where I hit the snag
        return await connect_to_server(self.host) 

    def __exit__(self, exc_type, exc_val, exc_tb):
        # Need to close the connection here
        pass

async def main():
    with LogStreamer("logs.example.com") as stream:
        print("Connected!")

If you try to run this, Python will scream at you. You can't use await inside a synchronous function like __enter__. It's a fundamental rule: sync methods can't await async operations. I can't just make __enter__ an async def because the with statement expects a synchronous return.

Making the setup asynchronous

To fix this, we have to switch to the asynchronous version of the context manager: async with. For this to work, I need to rename my magic methods to __aenter__ and __aexit__.

class LogStreamer:
    def __init__(self, host):
        self.host = host
        self.connection = None

    async def __aenter__(self):
        print("Opening connection...")
        self.connection = await connect_to_server(self.host)
        return self

    async def __aexit__(self, exc_type, exc_val, exc_tb):
        print("Closing connection...")
        await self.connection.close()

async def main():
    async with LogStreamer("logs.example.com") as streamer:
        print("We are in!")

Now the async with keyword tells Python: "Wait for __aenter__ to finish before entering the block, and wait for __aexit__ to finish before leaving it." It's a clean way to handle resources that require network I/O to set up or tear down. I've used this pattern a lot with database pools and websocket connections.

When a regular loop isn't fast enough

Now that I'm connected, I want to iterate over the incoming logs. My instinct is to use a for loop, but the logs don't arrive all at once. Each log entry is pushed from the server, and I have to await the arrival of the next one.

If I try to implement a standard __next__ method, I run into the same problem as before: __next__ is synchronous, but fetching the next log is asynchronous. This is where async for comes in. To make a class "async iterable," I need __aiter__ and __anext__.

class LogStreamer:
    # ... (keeping the context manager code from above) ...

    def __aiter__(self):
        return self

    async def __anext__(self):
        log = await self.connection.receive_next_log()
        if log is None:
            # This is how we signal the end of an async iterator
            raise StopAsyncIteration
        return log

Notice the difference here. __aiter__ is actually a regular function (it just returns the iterator object, usually self), but __anext__ is an async def. This allows the loop to pause while the server is thinking, letting other tasks in your event loop run in the meantime.

Tying the stream together

When we combine both patterns, we get a very powerful, Pythonic way to handle streaming data. We use async with to manage the lifecycle of the connection and async for to process the data as it arrives. It reads almost like synchronous code, but it's non-blocking.

async def monitor_logs():
    try:
        async with LogStreamer("logs.example.com") as streamer:
            async for log in streamer:
                if "ERROR" in log:
                    print(f"Alert: {log}")
                else:
                    print(f"Log: {log}")
    except ConnectionError:
        print("Could not connect to the log server.")

# Now we run the loop
import asyncio
asyncio.run(monitor_logs())

I personally prefer this approach over manually calling connect() and close(). Why? Because if an exception happens inside that async for loop, __aexit__ is still guaranteed to run, ensuring we don't leave dangling connections on the server. It's essentially "async safety" for your resources.

📋 Practical Task

Build an Async API Pagination Fetcher

You are tasked with creating a class called AsyncPageFetcher that simulates fetching paginated data from an API. Since network calls are slow, this must be implemented using asynchronous context managers and iterators.

Requirements:

• Implement __aenter__ to simulate an API authentication process (use await asyncio.sleep(0.1)).
• Implement __aexit__ to simulate logging out or closing the session.
• Implement __aiter__ and __anext__ to yield "pages" of data.
• The iterator should yield strings like "Page 1", "Page 2", up to "Page 5".
• After "Page 5", it must raise StopAsyncIteration.
• Each page fetch in __anext__ should simulate a delay using await asyncio.sleep(0.1).

Verification: Your final code should be executed within an async def main() function using async with and async for to print every page fetched from the fetcher.