Skip to Content
Course content

92: Building Custom Readers and Writers

Click on the "Edit" button in the top corner of the screen to edit your slide content.

Up until now, we've mostly used the readers and writers provided by the standard library—things like os.File or bytes.Buffer. But the real power of Go's I/O system isn't in the concrete types; it's in the io.Reader and io.Writer interfaces. Because they are so simple, you can wrap them. You can create a "middleware" for your data stream that transforms, filters, or logs data as it passes through.

To see this in action, let's build a Rot13Reader. It's a simple cipher that replaces a letter with the 13th letter after it in the alphabet. Instead of reading a whole file into memory, transforming it, and writing it back, we'll build a reader that transforms the bytes on the fly as they are being read.

Designing the Rot13 Wrapper

The key here is composition. We don't want to implement the low-level logic of reading from a disk or a network socket. Instead, our Rot13Reader will take an existing io.Reader and "wrap" it. Whatever the inner reader produces, we'll modify before handing it to the caller.

type Rot13Reader struct {
    r io.Reader
}

func NewRot13Reader(r io.Reader) io.Reader {
    return &Rot13Reader{r: r}
}


Now we just need to satisfy the io.Reader interface by implementing the Read method. This method's job is to fill the provided slice p with data and return how many bytes were read.

Implementing the Transformation Logic

The logic for Rot13 is straightforward: if it's an uppercase letter, shift it within A-Z; if it's lowercase, shift it within a-z. Everything else stays the same.

func (r *Rot13Reader) Read(p []byte) (n int, err error) {
    n, err = r.r.Read(p)
    for i := 0; i < n; i++ {
        b := p[i]
        if (b >= 'A' && b <= 'Z') {
            p[i] = (b-'A'+13)%26 + 'A'
        } else if (b >= 'a' && b <= 'z') {
            p[i] = (b-'a'+13)%26 + 'a'
        }
    }
    return n, err
}


Where I messed up the buffer

When I first wrote a similar wrapper, I made a classic mistake. I tried to be "efficient" by creating a temporary buffer inside the Read method to do the transformation, and then I tried to copy that temporary buffer into p.

I wrote something like this (don't do this):

// WRONG WAY
func (r *Rot13Reader) Read(p []byte) (n int, err error) {
    temp := make([]byte, len(p)) // Creating a new slice every single call!
    n, err = r.r.Read(temp)
    // ... transform temp ...
    copy(p, temp)
    return n, err
}


I quickly realized this was a performance nightmare. If you're reading a 1GB file in 4KB chunks, you're allocating a new 4KB slice thousands of times. The garbage collector would be working overtime. The beauty of the io.Reader interface is that the caller provides the buffer p. The correct approach is to modify p in place, which is exactly what I did in the first implementation above. Always respect the buffer provided to you.

Wiring it up to a String Reader

To test this, we can wrap a strings.NewReader. This allows us to treat a simple string as if it were a file or a network stream, making it perfect for unit tests.

func main() {
    input := "Hello, Go World! Let's build some readers."
    reader := strings.NewReader(input)
    rotReader := NewRot13Reader(reader)

    // We use io.ReadAll to consume the custom reader until EOF
    result, err := io.ReadAll(rotReader)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println(string(result))
    // Output: Uryyb, Tb Jbeyq! Yrgf ohyna fbev erntebf.
}


By wrapping the reader, the io.ReadAll function has no idea it's talking to a Rot13 transformer; it just knows it's calling Read() on something that satisfies the interface. This is the "Decorator" pattern in action, and it's how Go handles everything from Gzip compression to TLS encryption.




📋 Practical Task

Exercise: Build a Sensitive Data Masking Writer

Your task is to create a custom io.Writer that masks sensitive information. Instead of a Reader, you will build a MaskingWriter that wraps another io.Writer (like os.Stdout or a bytes.Buffer).

Requirements:

  • Define a struct MaskingWriter that holds an underlying io.Writer.
  • Implement the Write(p []byte) (n int, err error) method.
  • In the Write method, look for the byte sequence "SECRET" (case-sensitive).
  • Replace every occurrence of "SECRET" with "******" before passing the data to the underlying writer.
  • Ensure that the number of bytes returned (n) matches the number of bytes processed from the original slice p, even if you modified the content.

Test your implementation: Wrap a bytes.Buffer with your MaskingWriter, write the string "My SECRET code is 12345" to it, and verify that the buffer contains "My ****** code is 12345".

Rating
0 0

There are no comments for now.

to be the first to leave a comment.