A few years ago, I was reviewing a PR from a developer who was building a log analysis tool. They had a simple pipeline: read a massive text file, filter for "ERROR" tags, map those lines to a specific Error object, and then take the first 10 to show in a dashboard. It looked clean, but the moment we ran it against a production-sized log file (about 4GB), the JVM hit an OutOfMemoryError and crashed. The problem was that they were using a standard Scala List. Even though they only wanted 10 results, Scala was eagerly filtering and mapping every single line in that 4GB file, creating massive intermediate collections in memory before ever hitting that .take(10).

Why Eager Collections Kill Your Heap

Most of the collections we've used so far in this course are "eager." When you call .map or .filter on a List, Scala does the work immediately and produces a brand new collection. In the log example I mentioned, if you have a million lines, .filter creates a new list of maybe 100,000 errors, and then .map creates another list of 100,000 objects. That is a lot of wasted allocation if you only ever intended to look at the first few items.

This is where LazyList (which replaced the older Stream in Scala 2.13) comes in. A LazyList doesn't compute its elements until they are actually accessed. Instead of processing the whole collection, it essentially stores a recipe for how to find the next element. When you chain .filter and .map on a LazyList, Scala doesn't actually perform those operations right away; it just wraps the original list in a series of "lazy" transformations. The actual work only happens when you call a method like .head, .toList, or .foreach.

val logs = LazyList.from(1 to 1000000).map(i => s"Log entry $i")
val errors = logs.filter(line => line.contains("ERROR")).take(5)
// Nothing has actually been filtered or mapped yet!

println(errors.toList) // NOW the work happens, and it stops after finding 5 matches.

Infinite Sequences and Memoization

Because LazyList only evaluates what it needs, it allows us to represent things that would be impossible with a standard List: infinite sequences. I've often used this pattern to generate a stream of unique IDs or to implement mathematical sequences where you don't know the upper bound in advance.

One thing you need to be careful about is "memoization." LazyList remembers the values it has already computed. If you traverse a LazyList and keep a reference to the head of that list, the JVM cannot garbage collect the elements you've already passed. If you're processing a truly massive stream, be mindful of whether you're holding onto the start of the list or just consuming it as you go.

Here is a classic example of an infinite sequence—the Fibonacci numbers—implemented using a LazyList. Notice how the list is defined in terms of itself:

val fibs: LazyList[BigInt] = 
  BigInt(0) #:: BigInt(1) #:: fibs.zip(fibs.tail).map { case (a, b) => a + b }

// I can take as many as I want without ever causing a StackOverflow or OOM
val firstTen = fibs.take(10).toList 

The #:: operator is the "cons" operator for LazyList. It prepends an element to the list lazily. I personally find this pattern incredibly powerful for decoupling the generation of data from the consumption of data. You define the infinite logic once, and the caller decides how much of it they actually need.

📋 Practical Task

Building a Lazy Prime Number Sieve

Your task is to implement a lazy prime number generator using a Sieve of Eratosthenes approach. Instead of calculating primes up to a fixed limit, you will create a LazyList that can provide an infinite sequence of primes.

Requirements:

• Create a LazyList starting from 2.
• Implement a recursive function that filters out multiples of the first element (the prime) from the rest of the stream.
• The final implementation should allow the user to call .take(n) to get the first n prime numbers without calculating more than necessary.
• Test your implementation by extracting the 100th prime number using .drop(99).head.

Starter Code:

def sieve(s: LazyList[Int]): LazyList[Int] = {
  // Your implementation here
}

val primes = sieve(LazyList.from(2))
println(primes.drop(99).head) // Should print 541