Skip to Content
Course content

290: Mutation Testing Concepts

Click on the "Edit" button in the top corner of the screen to edit your slide content.

I've seen this happen in almost every mid-level engineering team I've joined: a developer proudly presents a Pull Request with 100% line coverage. They've hit every single line of code with a unit test, and they feel safe. They think, "If the code changes and breaks, my tests will catch it."

The Mirage of 100% Code Coverage

Here is the hard truth: code coverage tells you which lines were executed, not which lines were verified. You can have 100% coverage and still have practically zero confidence in your logic.

Let's look at a real example. Imagine you're writing a simple loyalty point calculator for an e-commerce app:

public int calculatePoints(double spend) {
    if (spend > 100.0) {
        return 10;
    }
    return 0;
}

Now, look at these tests. They provide 100% line coverage:

@Test
void testHighSpend() {
    assertEquals(10, calculator.calculatePoints(150.0));
}

@Test
void testLowSpend() {
    assertEquals(0, calculator.calculatePoints(50.0));
}

Everything looks great, right? But what happens if a junior dev (or a tired version of you) accidentally changes the source code to spend >= 100.0? All your tests still pass. The coverage is still 100%, but your business logic just changed—you're now giving away points to people who spent exactly 100.0, which might be against company policy.

Killing the Mutant to Find the Gap

This is where mutation testing comes in. Instead of measuring which lines are hit, mutation testing asks: "If I intentionally break a piece of the logic, will any of my tests notice?"

In mutation testing terminology, that change (changing > to >=) is called a Mutant. If your tests still pass despite the change, the mutant has Survived. If at least one test fails, you've Killed the mutant.

To kill the mutant in our example, you don't change the code; you add a test case for the boundary condition:

@Test
void testExactThreshold() {
    assertEquals(0, calculator.calculatePoints(100.0));
}

Now, if someone changes > to >=, this specific test will fail. The mutant is dead, and your test suite is actually robust.

Thinking Beyond the Boundary

While boundary changes are the most common mutations, mutation testing engines (like PITest for Java) do much more. They perform various "mutations" to stress-test your assertions:

  • Conditionals Boundary: Changing < to <=.
  • Math Mutators: Changing a + to a -.
  • Void Method Call Removal: Deleting a call to a method that returns void (to see if that side-effect was actually being tested).
  • Return Value Mutators: Changing a method that returns true to return false.

I'll be honest: mutation testing is computationally expensive. Running it on a massive monolithic project can take hours because the tool has to re-run your test suite for every single mutation it creates. Because of this, I don't recommend running it on every single commit. Instead, use it as a "health check" for your critical business logic—the stuff where a single off-by-one error could cost the company money.




📋 Practical Task

Hardening the Order Threshold Validator

You are tasked with auditing a critical piece of logic in a shipping system. The following ShippingCalculator class has 100% line coverage, but it is vulnerable to mutations. Your goal is to identify the "surviving mutant" and write the missing test case to kill it.

public class ShippingCalculator {
    public double calculateShipping(double weight) {
        // Free shipping for packages under 5kg
        if (weight < 5.0) {
            return 0.0;
        }
        return 10.0;
    }
}

The current test suite is as follows:

@Test
void testLightPackage() {
    assertEquals(0.0, calc.calculateShipping(2.0));
}

@Test
void testHeavyPackage() {
    assertEquals(10.0, calc.calculateShipping(10.0));
}

Your Task:

  1. Identify which specific character change in the calculateShipping method would create a "surviving mutant" (a change that doesn't break the existing tests).
  2. Write the additional JUnit test method required to ensure that this specific mutation would be caught (killed).
Rating
0 0

There are no comments for now.

to be the first to leave a comment.