Skip to Content
Course content

209: The io/fs Abstraction

Click on the "Edit" button in the top corner of the screen to edit your slide content.

You've probably spent most of your time in Go using the os package to handle files. It's the instinctive choice. But as your projects grow, you'll notice a recurring pain point: whenever you write a function that reads a file, that function becomes tethered to the actual disk. This makes your tests slow, flaky, and dependent on the environment they're running in.

The io/fs package, introduced in Go 1.16, solves this by providing a set of interfaces. Instead of saying "I need a path on the hard drive," your code says "I need something that behaves like a filesystem."

The trap of hardcoding the OS

Let's say we're building a tool that loads configuration files from a directory. I'll start the way I used to—by just using os.ReadDir. I want to find every .json file in a folder and process it.

type ConfigLoader struct {
    Data map[string]string
}

func (cl *ConfigLoader) LoadFromDir(path string) error {
    files, err := os.ReadDir(path)
    if err != nil {
        return err
    }

    for _, file := range files {
        if !file.IsDir() && strings.HasSuffix(file.Name(), ".json") {
            // Read file logic here...
            fmt.Printf("Loading %s\n", file.Name())
        }
    }
    return nil
}

At first glance, this is fine. But here's where I hit the wall: testing. To test this function, I now have to actually create a temporary directory on my laptop, write a few dummy JSON files to it, run the test, and then remember to clean up the directory afterward. It's a lot of boilerplate for a simple logic check. I'm managing the disk just to test a loop.

Decoupling with the fs.FS interface

To fix this, I need to stop asking for a string path and start asking for an fs.FS. The fs.FS interface is incredibly simple—it only requires one method: Open(name string) (fs.File, error). Most of the functionality we actually use (like reading directories) is provided by helper functions in the io/fs package that take an fs.FS as an argument.

Here is how I'd refactor that loader. Notice that I'm no longer importing os inside the logic function:

import (
    "fmt"
    "io/fs"
    "strings"
)

type ConfigLoader struct {
    Data map[string]string
}

// Now we accept any filesystem implementation
func (cl *ConfigLoader) LoadFromFS(sys fs.FS) error {
    // fs.ReadDir is a helper that works on any fs.FS
    files, err := fs.ReadDir(sys, ".") 
    if err != nil {
        return err
    }

    for _, file := range files {
        if !file.IsDir() && strings.HasSuffix(file.Name(), ".json") {
            // fs.ReadFile is another helper we can use
            content, err := fs.ReadFile(sys, file.Name())
            if err != nil {
                return err
            }
            fmt.Printf("Loaded %s: %s\n", file.Name(), string(content))
        }
    }
    return nil
}

By shifting to fs.FS, the ConfigLoader doesn't care if the files are on a Linux server, inside a ZIP archive, or embedded directly into the binary. It just knows how to "ask" for a file.

Mocking the filesystem for instant tests

This is where the magic happens. Go provides a package called testing/fstest specifically for this. It gives us MapFS, which is essentially an in-memory map of filenames to content. No more os.MkdirTemp or cleaning up files after a test run.

I can now write a test that runs in microseconds and requires zero disk I/O:

import (
    "testing"
    "testing/fstest"
)

func TestLoadFromFS(t *testing.T) {
    // I define my "filesystem" as a simple map
    mockFS := fstest.MapFS{
        "config1.json": {Data: []byte(`{"key": "val1"}`)},
        "config2.json": {Data: []byte(`{"key": "val2"}`)},
        "readme.txt":   {Data: []byte(`Ignore me`)},
    }

    loader := &ConfigLoader{Data: make(map[string]string)}
    err := loader.LoadFromFS(mockFS)
    if err != nil {
        t.Fatalf("Expected no error, got %v", err)
    }
    
    // The logic is tested, but the disk was never touched.
}

If I actually want to use this in production with the real disk, I just pass os.DirFS("/etc/myapp/configs") into the function. os.DirFS returns an fs.FS that wraps a specific local directory. It's a clean, elegant hand-off.




📋 Practical Task

Build a Read-Only Asset Validator

Your goal is to create a utility that ensures a set of required assets (like images or templates) exist within a filesystem, regardless of whether that filesystem is on disk or embedded in the binary.

  • Create a function ValidateAssets(sys fs.FS, requiredFiles []string) ([]string, error).
  • The function should check if every file in the requiredFiles slice exists in the provided fs.FS.
  • If a file is missing, add its name to a "missing" slice.
  • The function should return the list of missing files and any unexpected errors encountered.
  • Write a test for this function using fstest.MapFS to simulate a filesystem where some required files are present and some are missing.
Rating
0 0

There are no comments for now.

to be the first to leave a comment.