When you're building system tools or deployment scripts in Go, you'll eventually run into the need to group a bunch of files together. While zip is common, tar (Tape Archive) is the gold standard in Unix-like environments because it preserves file permissions and directory structures more faithfully.

Today, we're going to build a small utility that takes a folder of log files and bundles them into a single .tar archive. It's a simple concept, but the archive/tar package requires a specific sequence of operations that can trip you up if you're not careful.

Setting up our target files

Before we write the archiving logic, we need some data to work with. I'll assume we have a directory called logs/ with a few text files in it. In a real app, these would be your application logs, but for this example, we'll just imagine they exist.

package main

import (
	"archive/tar"
	"fmt"
	"io"
	"os"
	"path/filepath"
)

func main() {
	// We'll create a file named "backup.tar"
	tarFile, err := os.Create("backup.tar")
	if err != nil {
		panic(err)
	}
	defer tarFile.Close()

	// Initialize the tar writer
	tw := tar.NewWriter(tarFile)
	defer tw.Close()

	files := []string{"logs/app.log", "logs/error.log", "logs/system.log"}
	
	for _, file := range files {
		if err := addToTar(tw, file); err != nil {
			fmt.Printf("Error adding %s: %v\n", file, err)
		}
	}
}

The logic of the Tar Header

Here is the part that differs from writing a standard file. In a tar archive, every file is preceded by a "Header." This header tells the extractor the file's name, size, mode, and modification time. You can't just stream bytes into a tar writer; you have to tell the writer what's coming first.

I'll write a helper function to handle the heavy lifting of opening the file and creating that header.

func addToTar(tw *tar.Writer, filename string) error {
	file, err := os.Open(filename)
	if err != nil {
		return err
	}
	defer file.Close()

	stat, err := file.Stat()
	if err != nil {
		return err
	}

	// Create a header based on the file's stats
	header, err := tar.FileInfoHeader(stat, "")
	if err != nil {
		return err
	}

	// Ensure the header name matches the actual path we want in the archive
	header.Name = filename

	if err := tw.WriteHeader(header); err != nil {
		return err
	}

	_, err = io.Copy(tw, file)
	return err
}

The "Missing Data" Trap

I'll be honest: the first time I wrote a tar utility, I tried to be "clever" by manually creating the tar.Header struct instead of using tar.FileInfoHeader. I set the filename, but I forgot to set the Size field.

The result? The archive was created without errors, but when I tried to extract it, the files were all 0 bytes. The tar.Writer uses the size specified in the header to know exactly how many bytes to read from the subsequent stream. If you tell it the size is 0, it ignores the io.Copy data you send immediately after. Always use FileInfoHeader or explicitly set header.Size = stat.Size(). Don't trust your memory on this one.

Putting it all together

The final piece is ensuring the tar.Writer is closed properly. Because tar.Writer writes a footer (two empty 512-byte blocks) to signal the end of the archive, failing to call Close() will result in a corrupted archive that some extraction tools will complain about.

I used defer tw.Close() in the main function, which is usually fine, but in a production tool, you'd want to check the error returned by Close() to ensure the footer was actually written to disk.

Now, when you run this, Go iterates through your log list, reads the metadata, writes the header, streams the file content, and finally seals the archive. It's efficient because we're using io.Copy, meaning we aren't loading the entire log file into memory—we're just piping it from the disk to the archive.

📋 Practical Task

Exercise: Build a Recursive Directory Archiver

The example in the lesson only archived a hard-coded list of files. Your task is to expand this into a tool that can archive an entire directory recursively.

• Modify the program to accept a source directory path as a command-line argument.
• Use filepath.Walk (or filepath.WalkDir) to find all files within that directory and its subdirectories.
• Ensure that the header.Name preserves the relative path of the files so that the directory structure is maintained when the archive is extracted.
• Add a check to skip the .tar file itself if it happens to be created inside the source directory, to avoid an infinite loop of the archive archiving itself.