Skip to Content
Course content

106: Mutation Testing with Mutant

Click on the "Edit" button in the top corner of the screen to edit your slide content.

Wait, I already have 100% test coverage—why do I need this?

This is the first thing almost every developer asks me when I bring up mutation testing. Here is the hard truth: code coverage is a vanity metric. It tells you that a line of code was executed during your tests, but it says absolutely nothing about whether your tests actually verified the result of that execution.

Imagine we have a simple PricingEngine that applies a discount if a customer spends over $100:

class PricingEngine
  def calculate_total(amount)
    return amount * 0.9 if amount > 100
    amount
  end
end

Now, imagine your test looks like this: expect(engine.calculate_total(150)).to be < 150. That test passes, and your coverage tool shows that every line of the method was hit. You're at 100%. But did you actually test the logic? Not really. If I accidentally changed the code to amount * 0.5, your test would still pass. Your coverage is high, but your confidence is low.

Mutation testing solves this by playing the villain. It intentionally injects bugs (mutations) into your source code—like changing a > to a >= or a + to a -—and then runs your tests. If your tests still pass despite the bug, the mutation "survived." A survived mutation is a giant red flag that your tests are missing a critical edge case.

How do I actually get Mutant running on my project?

First, add gem 'mutant-rspec' to your Gemfile. I'm assuming you're using RSpec, as that's the most common pairing, but Mutant supports other frameworks too. One thing to keep in mind: Mutant is computationally expensive. It doesn't just run your tests once; it runs them hundreds of times against different versions of your code. Don't run it on your entire monolith during every save, or you'll be waiting forever.

To target a specific class, run it from your terminal like this:

bundle exec mutant coverage PricingEngine

Mutant will then analyze the PricingEngine class, find every possible place it can break something, and report back on how many of those breaks your test suite caught (killed) and how many it missed (survived).

What does it actually mean when a mutation "survives," and how do I fix it?

When Mutant tells you a mutation survived, it's basically saying, "I changed this line of code, and your tests didn't notice." Let's go back to our PricingEngine. Mutant might decide to mutate amount > 100 into amount >= 100.

If your only test case is calculate_total(150), that mutation will survive. Why? Because 150 is greater than 100 AND greater than or equal to 100. The result is the same for both.

To "kill" this mutation, you have to write a test for the exact boundary where the mutation creates a different behavior. In this case, you need a test for exactly 100:

it "does not apply a discount when the amount is exactly 100" do
  expect(engine.calculate_total(100)).to eq(100)
end

Now, if Mutant changes > to >=, the code will return 90 instead of 100, your test will fail, and the mutation is officially "killed." I find this process incredibly satisfying; it turns testing into a game of cat-and-mouse where the goal is to make your code impossible to break without a test noticing.




📋 Practical Task

Killing Survivors in the LoyaltyPointCalculator

You have been handed a LoyaltyPointCalculator class that determines how many points a user earns based on their spend. While the current RSpec suite shows 100% line coverage, a Mutant run has revealed several surviving mutations.

The Code:

class LoyaltyPointCalculator
  def points_for(spend)
    return 0 if spend < 0
    return spend * 2 if spend > 500
    spend
  end
end

The Current (Weak) Tests:

RSpec.describe LoyaltyPointCalculator do
  let(:calc) { LoyaltyPointCalculator.new }

  it "calculates points for high spend" do
    expect(calc.points_for(600)).to eq(1200)
  end

  it "calculates points for low spend" do
    expect(calc.points_for(100)).to eq(100)
  end

  it "handles negative spend" do
    expect(calc.points_for(-10)).to eq(0)
  end
end

Your Task:

  1. Identify the two boundary mutations that would survive this test suite (Hint: look at the comparison operators).
  2. Write two additional test cases that specifically target those boundaries to ensure any change to the logic (like changing > to >= or < to <=) would cause the tests to fail.
  3. Ensure your final test suite leaves no survivors for the points_for method.
Rating
0 0

There are no comments for now.

to be the first to leave a comment.