I've seen this happen a dozen times with junior analysts: they spend three hours writing a series of filter() and na.omit() calls to "clean" a dataset, and then they hand over the final table. When the stakeholder asks, "How much data did we actually lose?" or "Which columns were the biggest offenders?", the analyst has to go back and manually re-calculate those numbers. They've confused cleaning the data with reporting on its quality.

Cleaning is not the same as auditing

The misconception is that the act of removing bad data satisfies the requirement for data quality. It doesn't. Cleaning is a destructive process—you are removing information. Quality reporting is an audit; it's the process of quantifying the "dirtiness" of your raw input before you touch it. If you just clean the data without a report, you're flying blind. You might accidentally delete 40% of your rows because of a systematic error in the source system, and you'd never know because the final table just looks "clean."

In a professional pipeline, the report happens first. You generate a summary that tells you exactly what's wrong, you decide on a cleaning strategy based on those metrics, and then you apply the fix. I always tell my team: if you can't quantify the error, you can't justify the fix.

Building an automated audit pipeline

To do this right in R, you don't want to write a separate sum(is.na()) call for every single column. That's tedious and prone to copy-paste errors. Instead, we use a functional approach. We want a generator—a function that takes any dataframe and returns a summary table of quality metrics.

Think of it as creating a "meta-dataset" where the rows are your original columns and the columns are your quality metrics. I usually lean on purrr::map_df for this because it allows us to iterate over the columns and bind the results into a tidy dataframe immediately.

library(dplyr)
library(purrr)
library(tidyr)

# This is a skeletal version of a quality generator
generate_quality_report <- function(data) {
  data %>%
    map_df(~ {
      # We calculate metrics for each column 'x'
      tibble(
        missing_count = sum(is.na(.x)),
        missing_pct = mean(is.na(.x)) * 100,
        unique_count = n_distinct(.x),
        # Use a tryCatch or check if numeric to avoid errors on strings
        min_val = if(is.numeric(.x)) min(.x, na.rm = TRUE) else NA,
        max_val = if(is.numeric(.x)) max(.x, na.rm = TRUE) else NA
      )
    }, .id = "column_name")
}

# Example usage with a messy dataset
raw_data <- tibble(
  patient_id = c(1, 2, 3, 4, 5),
  age = c(25, NA, 30, 150, 22), # 150 is a likely outlier
  blood_pressure = c(120, 110, NA, 130, 120)
)

report <- generate_quality_report(raw_data)
print(report)

I added a small check for is.numeric in the code above. This is a crucial detail. If you try to find the min() of a character column, R will throw an error or give you an alphabetical minimum, which is useless for a quality report. When you're building these generators, always assume the input data is more chaotic than the documentation claims it is.

By structuring your report this way, you can now easily filter the report itself. For example, report %>% filter(missing_pct > 10) instantly tells you which columns are too empty to be useful for your analysis. That's the power of treating your quality report as data rather than just a printed list of notes.

📋 Practical Task

Exercise: Clinical Trial Patient Data Quality Auditor

You have been handed a raw dataset from a clinical trial. Before you can run any statistical tests, you need to build a robust quality report generator that identifies specific "red flags" in the data.

The Dataset: Create a tibble named trial_data with the following columns:

• patient_id: 100 rows (some duplicates).
• age: 100 values (include some NAs and some impossible values like -5 or 200).
• dose_mg: 100 values (include some NAs).
• treatment_group: 100 values (mostly "A" and "B", but include some typos like "a" or "B ").

Your Task: Write a function called audit_trial_data() that returns a summary dataframe. Your report must include the following metrics for every column:

1. The percentage of missing values.
2. The number of unique values.
3. A logical flag has_outliers: For the age column, this should be TRUE if any value is < 0 or > 120. For other columns, it can simply be FALSE.
4. A logical flag has_duplicates: TRUE if the column contains any duplicate values.

Finally, use your function to generate the report and print only the columns that have more than 5% missing data OR have outliers.