I've been there. You've spent the afternoon writing tests, you run your suite, and you see a wall of green dots. Every single test passed. You feel confident, you merge the PR, and then—about ten minutes after it hits production—you get a Slack notification that the app is crashing for a specific subset of users.

The problem isn't that your tests failed; it's that your tests didn't actually execute the code that was breaking. This is the gap between "passing tests" and "coverage."

The Illusion of a Green Test Suite

# payment_processor.py
def calculate_fee(amount, currency):
    rates = {"USD": 0.02, "EUR": 0.025, "GBP": 0.03}
    
    if currency not in rates:
        # I added this logic to handle unsupported currencies, 
        # but I forgot to test it.
        raise ValueError(f"Unsupported currency: {currency}")
        
    return amount * rates[currency]

# test_payment.py
def test_usd_fee():
    assert calculate_fee(100, "USD") == 2.0

def test_eur_fee():
    assert calculate_fee(100, "EUR") == 2.5

If you run these tests with pytest, they pass perfectly. You might think, "Great, the fee calculation works." But look closely at the calculate_fee function. I wrote a guard clause to handle unsupported currencies, but my tests only cover the "happy path" (USD and EUR). If a user tries to pay with JPY, the ValueError will trigger, and unless I've tested how the rest of the app handles that error, I've just shipped a potential crash.

Exposing the Blind Spots

This is where coverage.py comes in. It doesn't tell you if your code is *correct* (that's what the tests are for), but it tells you which lines of code were actually touched during the test run. It's a map of your ignorance.

First, you'll need to install it: pip install coverage. Instead of running your tests directly through pytest, you wrap the execution with the coverage tool:

coverage run -m pytest
coverage report

When you run this against the code above, you'll see a table showing that payment_processor.py has something like 75% coverage. The coverage report command lists the missing lines—specifically, the raise ValueError line. Now I know exactly where my testing is lacking.

Turning Data into Action

The terminal report is okay for small files, but once you're working on a real project, it's a nightmare to track line numbers manually. I always generate the HTML report. It's a game-changer because it lets you browse your source code in a browser with the executed lines highlighted in green and the missed lines in red.

coverage html

Now, I can open htmlcov/index.html, see that red line in the currency check, and realize I need to add a specific test case for the failure scenario:

def test_unsupported_currency():
    import pytest
    with pytest.raises(ValueError):
        calculate_fee(100, "JPY")

Run the coverage command again, and that line turns green. I haven't necessarily proven the code is "perfect," but I've eliminated the blind spot. I'm no longer guessing if my tests are comprehensive; I have the data to prove it.

📋 Practical Task

Closing the Gaps in the Password Validator

You've been handed a PasswordValidator class that is supposed to enforce several security rules. The previous developer wrote a few tests, but they claim the "test coverage is high." Your job is to prove them wrong using coverage.py.

The Code:

# validator.py
class PasswordValidator:
    def validate(self, password):
        if len(password) < 8:
            return False, "Too short"
        
        if not any(char.isdigit() for char in password):
            return False, "Must contain a number"
        
        if not any(char.isupper() for char in password):
            return False, "Must contain an uppercase letter"
            
        if " " in password:
            return False, "No spaces allowed"
            
        return True, "Valid"

# test_validator.py
from validator import PasswordValidator

def test_short_password():
    v = PasswordValidator()
    assert v.validate("Ab1") == (False, "Too short")

def test_valid_password():
    v = PasswordValidator()
    assert v.validate("SecurePass123") == (True, "Valid")

Your Goal:

1. Run coverage run -m pytest on this code.
2. Run coverage report to identify which validation rules (the if statements) are not being executed by the current tests.
3. Write additional test cases in test_validator.py to cover the missing scenarios (missing numbers, missing uppercase, and spaces).
4. Run coverage again and verify that validator.py reaches 100% coverage.