Skip to Content
Course content

39: Processing Data with Scala Collections

Click on the "Edit" button in the top corner of the screen to edit your slide content.

When you first start working with Scala, it's incredibly tempting to bring your Java or Python habits with you. I see it all the time: a developer who is brilliant at logic but treats a Scala List like an array from a C-style language. They create a mutable variable, loop through the collection, and manually update that variable based on some conditions. It works, sure, but it's fighting the very nature of the language.

Let's look at a real scenario. Imagine we're processing a list of Order objects. We need to calculate the total revenue from all "Completed" orders that were over $100, but only for a specific region. Here is how a lot of people write this when they're just trying to get the ticket closed:

case class Order(id: Int, status: String, amount: Double, region: String)

val orders = List(
  Order(1, "Completed", 150.0, "North"),
  Order(2, "Pending", 200.0, "North"),
  Order(3, "Completed", 50.0, "North"),
  Order(4, "Completed", 300.0, "South"),
  Order(5, "Completed", 120.0, "North")
)

var totalRevenue = 0.0
for (order <- orders) {
  if (order.status == "Completed" && order.amount > 100 && order.region == "North") {
    totalRevenue += order.amount
  }
}

The cost of mutable state

Now, looking at that code, your brain probably says "that's fine." But as a maintainer, this makes me nervous. The var totalRevenue is a piece of mutable state. In a small script, it's trivial. In a complex production system with concurrency, mutable state is where the bugs hide. If this logic were to move into a parallel processing block, you'd suddenly be dealing with race conditions and needing AtomicDouble or synchronized blocks.

Beyond the technical risk, this style is "imperative." You're telling the computer how to do the work (initialize a variable, iterate, check a condition, increment). In Scala, we want to describe what we want the data to become. The imperative approach forces the reader to simulate the loop in their head to understand the final result.

Thinking in transformations

The "Scala way" is to treat your data as a stream that you refine through a pipeline. Instead of a loop and a mutable accumulator, we use higher-order functions. I prefer to chain filter and map, or use collect if I'm doing both at once. Here's how I'd rewrite that same logic:

val totalRevenue = orders
  .filter(o => o.status == "Completed")
  .filter(o => o.region == "North")
  .filter(o => o.amount > 100)
  .map(_.amount)
  .sum

Notice how this reads like a sentence: "Take the orders, filter by status, then region, then amount, extract the amounts, and sum them up." There are no variables to track, no counters to increment, and no risk of off-by-one errors. It's declarative. If you need to change the criteria—say, including "Shipped" orders as well—you just tweak one line in the pipeline without worrying about how it affects the accumulator logic.

Dealing with the intermediate overhead

Now, a colleague might point out that every time we call .filter or .map, Scala creates a new intermediate collection. If you have ten thousand orders, creating three temporary lists just to sum them up feels wasteful. I'll admit, it is. But for most business logic, the overhead is negligible compared to the gain in maintainability.

If you actually hit a performance bottleneck, you don't go back to the var and while loop. Instead, you use a view. By adding .view to the start of the chain, Scala switches to lazy evaluation. It doesn't create those intermediate lists; it just builds a "recipe" for the transformation and only executes it when you call the final terminal operation, like .sum.

val totalRevenue = orders.view
  .filter(o => o.status == "Completed")
  .filter(o => o.region == "North")
  .filter(o => o.amount > 100)
  .map(_.amount)
  .sum

This gives you the best of both worlds: the clean, declarative syntax of functional collections and the performance of a single imperative loop. This is the level of abstraction where you should be spending your time in Scala.




📋 Practical Task

Exercise: High-Value Customer Loyalty Auditor

You are building a tool to audit customer loyalty points. You have a list of Transaction objects. Your goal is to calculate the total loyalty points for a specific customer, but with a few catches:

  • Only include transactions that are NOT marked as "Refunded".
  • Points are only earned on transactions over $20.00.
  • The point calculation is: $1.00 spent = 1 point.
  • You must use a functional pipeline (filter, map, sum) rather than a mutable var and a loop.

Starter Code:

case class Transaction(id: String, customerId: String, amount: Double, status: String)

val transactions = List(
  Transaction("T1", "CUST_01", 50.0, "Completed"),
  Transaction("T2", "CUST_01", 10.0, "Completed"),
  Transaction("T3", "CUST_01", 100.0, "Refunded"),
  Transaction("T4", "CUST_02", 200.0, "Completed"),
  Transaction("T5", "CUST_01", 75.0, "Completed")
)

val targetCustomer = "CUST_01"
// Your code here: Calculate total points for targetCustomer
Rating
0 0

There are no comments for now.

to be the first to leave a comment.