Why should I bother wrapping my files in BufReader?

You might think that calling read on a std::fs::File is as efficient as it gets, but here is the catch: every single time you make a system call to the OS to read a chunk of bytes, there's a performance penalty. If you're reading a large file one byte at a time, or even in small 10-byte chunks, your program will spend more time talking to the kernel than actually processing data.

I've seen this bite people when they start writing custom parsers. BufReader solves this by reading a large chunk of the file (usually 8KB) into an internal memory buffer in one go. When you ask for the next byte, Rust just hands it to you from that memory buffer. It's orders of magnitude faster because it minimizes those expensive trips to the hard drive.

How do I actually use this to read a file line-by-line?

This is probably the most common use case for BufReader. To get the lines() method, you need to import the BufRead trait. If you forget that import, you'll spend ten minutes wondering why the method doesn't exist on your reader.

use std::fs::File;
use std::io::{self, BufRead, BufReader};

fn main() -> io::Result<()> {
    let file = File::open("server_logs.txt")?;
    // Wrap the file in a BufReader
    let reader = BufReader::new(file);

    for line in reader.lines() {
        let line = line?; 
        if line.contains("ERROR") {
            println!("Found an error: {}", line);
        }
    }
    Ok(())
}

Notice how we wrap the File object. The BufReader essentially "owns" the file now and manages the buffering logic for you.

What is the deal with BufWriter and flushing?

BufWriter is the mirror image of BufReader. Instead of reading a big chunk from the disk, it collects your small writes in memory and only sends them to the OS when the buffer is full. It's a lifesaver when you're writing thousands of small strings to a file.

But there's a trap: because the data lives in a buffer, the last bit of your data might still be sitting in memory when your program finishes. Usually, when the BufWriter is dropped, it attempts to flush that data to disk. However, if a panic occurs or you need to ensure the data is written immediately before the writer goes out of scope, you must call .flush() manually.

use std::fs::File;
use std::io::{self, BufWriter, Write};

fn save_data() -> io::Result<()> {
    let file = File::create("output.dat")?;
    let mut writer = BufWriter::new(file);

    writer.write_all(b"Some important data\n")?;
    writer.write_all(b"More important data\n")?;

    // I always recommend an explicit flush if the writer
    // lives for a long time or handles critical data.
    writer.flush()?; 
    
    Ok(())
}

If you're just writing a small CLI tool, you can often get away without the explicit flush, but in production systems, it's a habit that will save you from "missing data" bugs that are a nightmare to debug.

📋 Practical Task

Exercise: Buffered Log Filtering Tool

Your task is to create a program that reads a large log file (input.log) and writes only the lines containing the keyword "CRITICAL" into a new file (critical_errors.log). To make this performant, you must use both BufReader and BufWriter.

• Create a dummy input.log file with several lines of text, some containing "INFO", "WARN", and "CRITICAL".
• Use BufReader::new() to read the input file line-by-line.
• Use BufWriter::new() to write the filtered lines to the output file.
• Ensure you call .flush() on your writer before the function returns to ensure all critical errors are safely saved to disk.