Think about a conveyor-belt sushi restaurant. If you ordered a full platter of 20 different rolls, the chef would have to prepare every single one, arrange them on a giant tray, and then bring the whole thing to your table. You're sitting there hungry, and the waiter is struggling to carry a massive tray. That's how Task<List<T>> works; you wait for the entire collection to be ready before you can even taste the first piece.

Now, consider the actual conveyor belt. The chef makes one piece, puts it on the belt, and it slides toward you. You eat it immediately. While you're chewing, the chef is already making the next one. You aren't waiting for the whole menu to be completed; you're consuming a stream of data as it becomes available. That is exactly what IAsyncEnumerable<T> is doing for your code.

Stop Waiting for the Giant Tray

I've seen too many developers default to returning a List because it's familiar. But when you're pulling 10,000 rows from a database or streaming logs from a cloud provider, buffering that entire list into memory is a recipe for a OutOfMemoryException—or at the very least, a very sluggish UI. By using IAsyncEnumerable, you're telling the caller: "I'll give you items as I find them. Don't wait for me to finish the whole search."

public async IAsyncEnumerable<string> StreamLogFilesAsync(string directoryPath)
{
    var files = Directory.GetFiles(directoryPath, "*.log");

    foreach (var file in files)
    {
        // Imagine this is a heavy network call or a slow disk read
        string content = await File.ReadAllTextAsync(file);
        
        // We 'yield' the result immediately to the consumer
        yield return content; 
    }
}

Notice the yield return. This is the magic. It doesn't return the whole sequence; it returns a "promise" of the next item. The method execution actually pauses here and hands control back to the person calling the method.

Consuming the Stream without Blocking

You can't use a standard foreach loop here because the items aren't ready yet. You need the await foreach. I like to think of this as the "patient loop." It knows that the next item might take a few milliseconds (or seconds) to arrive, and it's perfectly fine with that.

public async Task ProcessLogsAsync()
{
    // We aren't loading all logs into memory first.
    // We are processing them one by one as they are yielded.
    await foreach (var logEntry in StreamLogFilesAsync(@"C:\Logs"))
    {
        Console.WriteLine($"Processing: {logEntry.Substring(0, 20)}...");
        // Do some work with the log entry here
    }
}

Preventing Infinite Waits with Cancellation

Here is where things get professional. In a real production environment, you can't just let a stream run forever. What if the user closes the app or the network hangs? Since IAsyncEnumerable is asynchronous, it should almost always support a CancellationToken.

The syntax is a bit quirky here because the token is passed via the [EnumeratorCancellation] attribute. It looks strange the first time you see it, but it's how C# bridges the gap between the method call and the actual iterator.

public async IAsyncEnumerable<int> GetNumbersAsync(
    [EnumeratorCancellation] CancellationToken ct = default)
{
    for (int i = 0; i < 100; i++)
    {
        // Check if the caller wants to stop
        ct.ThrowIfCancellationRequested();
        
        await Task.Delay(100, ct); 
        yield return i;
    }
}

If you omit that attribute, the token you pass into the WithCancellation() extension method during the await foreach won't actually reach your logic inside the stream. I've spent a few hours debugging that exact mistake in the past—don't be that person.

When to Stick with a Simple List

I don't want you to use this everywhere. If you're returning five items from a configuration file, IAsyncEnumerable is overkill. It adds a small amount of overhead for the state machine. Use it when the data set is large, when the time between items is unpredictable, or when you want the user to see progress (like a loading list) rather than a blank screen for ten seconds followed by a sudden burst of data.

📋 Practical Task

Build a Paged API Result Streamer

You are tasked with creating a service that fetches "User Profiles" from a paginated API. Instead of fetching all 5 pages of users and returning a massive list, you must implement a streamer that fetches one page at a time and yields users individually.

Requirements:

• Create a method public async IAsyncEnumerable<User> StreamUsersAsync([EnumeratorCancellation] CancellationToken ct).
• Simulate a paginated API: Use a loop to iterate from page 1 to 5.
• Inside the loop, use await Task.Delay(500, ct) to simulate the network latency of fetching a page.
• For each page, yield 3 simulated User objects (you can just create a simple User class with a Name property).
• In your Main method, consume this stream using await foreach and ensure you pass a CancellationTokenSource` with a timeout of 2 seconds to see the cancellation in action.