When I first started building persistence layers in Go, I thought I was being clever by offloading the "slow" disk I/O to a background goroutine. I figured why make the user wait for a file write when the data is already updated in memory? It seemed like a free performance win.

Here is a snippet of the kind of code I wrote back then. It looks clean, but it's a landmine:

type Store struct {
    data map[string]string
}

func (s *Store) Set(key, value string) {
    s.data[key] = value
    // "Optimize" by saving in the background
    go s.saveToFile() 
}

func (s *Store) saveToFile() {
    bytes, _ := json.Marshal(s.data)
    os.WriteFile("store.json", bytes, 0644)
}

The Dreaded Concurrent Map Panic

If you run this in a high-traffic environment, it won't just be slow—it will crash. You'll see this terrifying message: fatal error: concurrent map read and map write.

Here is what's happening: Go's maps are not thread-safe. While your Set method is updating the map on the main thread, your saveToFile goroutine is reading that same map to marshal it into JSON. If those two actions happen at the exact same microsecond, the Go runtime panics and kills the process. I learned the hard way that "backgrounding" a task doesn't magically make the data access safe; it actually creates the race condition in the first place.

Guarding the Store with a Mutex

To fix this, we need to ensure that only one goroutine is touching the map at any given time. A sync.RWMutex is the right tool here because it allows multiple readers (which is fine) but only one writer.

type Store struct {
    mu   sync.RWMutex
    data map[string]string
}

func (s *Store) Set(key, value string) {
    s.mu.Lock()
    s.data[key] = value
    s.mu.Unlock()
    
    // We still want persistence, but we can't just pass the map
    s.saveToFile() 
}

func (s *Store) saveToFile() {
    s.mu.RLock()
    // We create a snapshot copy so we can unlock the map quickly
    // and do the slow disk I/O without blocking other requests.
    snapshot := make(map[string]string, len(s.data))
    for k, v := range s.data {
        snapshot[k] = v
    }
    s.mu.RUnlock()

    bytes, _ := json.Marshal(snapshot)
    os.WriteFile("store.json", bytes, 0644)
}

Notice how I used RLock() in the save method. This tells Go, "I'm just reading, so other readers can join in, but no one can write until I'm done." More importantly, I copied the map into a snapshot. If I held the lock during the os.WriteFile call, my entire application would freeze every time it saved to disk. Never hold a lock during I/O if you can avoid it.

Handling State Recovery on Startup

Persistence is useless if you can't get the data back. The standard pattern is to attempt a load during the initialization of your store. If the file doesn't exist (like the very first time the app runs), we should treat that as a normal state, not a fatal error.

func NewStore(filename string) (*Store, error) {
    s := &Store{
        data: make(map[string]string),
    }

    file, err := os.ReadFile(filename)
    if err != nil {
        if os.IsNotExist(err) {
            return s, nil // Fine, first run
        }
        return nil, err // Actual disk error
    }

    if err := json.Unmarshal(file, &s.data); err != nil {
        return nil, err
    }

    return s, nil
}

I prefer this approach because it separates the "empty state" from the "broken state." If the file is corrupted JSON, the app should fail to start; if the file simply isn't there, the app should start with a fresh map.

Preventing Data Corruption with Atomic Renames

There is one last "pro" tip I want to give you. os.WriteFile truncates the file and writes the new data. If your computer crashes or the power goes out exactly halfway through that write, your store.json is now half-written garbage. You've lost everything.

To make this "atomic," write the data to a temporary file first, then rename that file to the final destination. On most operating systems, os.Rename is an atomic operation. Either the new file completely replaces the old one, or the old one stays intact. No in-between.

func (s *Store) saveToFile() {
    s.mu.RLock()
    snapshot := make(map[string]string, len(s.data))
    for k, v := range s.data {
        snapshot[k] = v
    }
    s.mu.RUnlock()

    bytes, _ := json.Marshal(snapshot)
    
    tmpFile := "store.json.tmp"
    if err := os.WriteFile(tmpFile, bytes, 0644); err != nil {
        log.Printf("failed to write tmp file: %v", err)
        return
    }
    
    if err := os.Rename(tmpFile, "store.json"); err != nil {
        log.Printf("failed to rename file: %v", err)
    }
}

📋 Practical Task

Exercise: Implementing a TTL (Time-to-Live) for the Key-Value Store

In a real key-value store, you rarely want data to stay forever. Your task is to modify the Store implementation to support expiring keys.

• Modify the internal map to store a struct containing both the value and an expiry time (using time.Time).
• Update the Set method to accept a ttl time.Duration. If the duration is 0, the key should never expire.
• Update the Get method so that if a key is accessed after its expiry time, it returns a "not found" error (and ideally deletes the expired key from the map).
• Ensure your saveToFile and NewStore functions are updated to handle the new struct instead of a simple string.
• Bonus: Implement a "Cleanup" method that runs in a background goroutine every 60 seconds to purge all expired keys from the map to prevent memory leaks.