Skip to Content
Course content

92: Channels for Producer-Consumer Patterns

Click on the "Edit" button in the top corner of the screen to edit your slide content.

Imagine you're running a busy gourmet burger joint. You've got one person taking orders at the counter (the Producer) and one person in the back flipping burgers (the Consumer). If the cashier just yelled every order across the kitchen, the cook would get overwhelmed and forget things. If the cashier tried to physically hand the order to the cook while the cook was mid-flip, they'd just get in each other's way.

Instead, you use a ticket rail. The cashier slides a slip of paper onto the rail and moves back. The cook, when they have a free hand, grabs the next ticket from the rail. The rail acts as a buffer. If the cashier is faster than the cook, the rail fills up. If the rail is full, the cashier has to pause for a second before taking the next order so the kitchen doesn't descend into total chaos. This is exactly what System.Threading.Channels does for your code.

The Conveyor Belt Logic

In C#, a Channel<T> is essentially a thread-safe queue designed specifically for this "handoff" pattern. You have a Writer and a Reader. The beauty here is that they are decoupled; the producer doesn't need to know who is consuming the data or how they're doing it—they just need to know the channel is open.

Let's look at a scenario where we're processing a batch of high-resolution images. We have one task scanning a directory for files and another task performing the actual resizing, which is CPU-intensive.

using System.Threading.Channels;

public class ImageProcessor
{
    public async Task ProcessImagesAsync()
    {
        // We create a bounded channel. If there are 10 images waiting, 
        // the producer will asynchronously wait before adding more.
        var channel = Channel.CreateBounded<string>(new BoundedChannelOptions(10) 
        { 
            FullMode = BoundedChannelFullMode.Wait 
        });

        // Start the consumer (the "cook")
        var consumerTask = ConsumeImagesAsync(channel.Reader);

        // Start the producer (the "cashier")
        var producerTask = ProduceImagesAsync(channel.Writer);

        await Task.WhenAll(producerTask, consumerTask);
    }

    private async Task ProduceImagesAsync(ChannelWriter<string> writer)
    {
        string[] files = { "img1.jpg", "img2.jpg", "img3.jpg", "img4.jpg" };
        foreach (var file in files)
        {
            Console.WriteLine($"[Producer] Found {file}, adding to queue...");
            await writer.WriteAsync(file);
        }
        
        // Crucial: tell the reader we're done, otherwise the consumer waits forever
        writer.Complete();
    }

    private async Task ConsumeImagesAsync(ChannelReader<string> reader)
    {
        // ReadAllAsync is a magic little helper that lets us use await foreach
        await foreach (var image in reader.ReadAllAsync())
        {
            Console.WriteLine($"[Consumer] Resizing {image}...");
            await Task.Delay(500); // Simulate heavy work
        }
        Console.WriteLine("All images processed.");
    }
}

Handling the Backpressure

You'll notice I used Channel.CreateBounded. In the software world, we call the "full rail" problem backpressure. If your producer can generate 1,000 items per second but your consumer can only process 10, an unbounded channel will just grow in memory until your app crashes with an OutOfMemoryException. I've been there; it's a bad day.

By bounding the channel to 10, await writer.WriteAsync(file) will actually pause the producer's execution once the limit is hit. It only resumes once the consumer has pulled an item out, freeing up a slot. This naturally throttles your system to the speed of your slowest component, which is exactly what you want for stability.

Cleaning Up the Queue

One detail that often trips people up is the writer.Complete() call. Think of this as the cashier putting a "Closed" sign on the register. The consumer doesn't stop immediately—they still finish processing whatever tickets are already on the rail—but once the rail is empty, the ReadAllAsync() loop terminates gracefully.

If you forget to call Complete(), your consumer task will hang indefinitely, waiting for a message that will never come. If you're using a CancellationToken, you can also use that to shut things down, but Complete() is the standard way to signal a natural end to the stream.




📋 Practical Task

Build a High-Throughput Log Filtering System

Your goal is to create a system that mimics a log aggregator. You need to implement a producer-consumer pattern using System.Threading.Channels to filter "Error" logs from a stream of data without blocking the main ingestion thread.

Requirements:

  • Create a bounded channel that holds a maximum of 5 strings.
  • The Producer: Create a method that generates 20 random log messages. Each message should be either "INFO: System OK" or "ERROR: Connection Failed". The producer should write these to the channel.
  • The Consumer: Create a method that reads from the channel. It should only print the message to the console if the string contains the word "ERROR".
  • Ensure the producer calls Complete() when finished and that the main program waits for both the producer and consumer to finish execution.
  • Test the backpressure by adding a small Task.Delay in the consumer to see how the producer is throttled.
Rating
0 0

There are no comments for now.

to be the first to leave a comment.