I've spent a lot of time reviewing R code from analysts who are brilliant at statistics but haven't quite grasped the ergonomics of the language. One of the most common things I see is what I call "The Onion." It's that habit of nesting functions so deeply that you have to read the code from the inside out to understand what's actually happening. When you're composing a series of transformations—cleaning data, transforming it, and then aggregating it—this approach quickly becomes a liability.

The Inside-Out Struggle

Imagine we're building a pipeline to process raw sensor data. We need to remove the NAs, normalize the scale, and then calculate a rolling average. The naive way to write this looks something like this:

# The "Onion" approach
final_data <- rolling_avg(normalize_scale(remove_outliers(raw_sensor_data)))

At first glance, it's compact. But as soon as you add a fourth or fifth step, it becomes a nightmare. If you want to check the output of remove_outliers to make sure it's working, you have to break the whole line apart and create a temporary variable. You're essentially reading the logic in reverse order of execution. It's mentally taxing, and it's where most bugs hide because it's so easy to misplace a closing parenthesis when you're five levels deep.

Now, some people try to fix this by creating a trail of temporary variables. They'll do data1 <- remove_outliers(raw_sensor_data), then data2 <- normalize_scale(data1), and so on. I've seen this in a lot of legacy scripts. While it's easier to debug, it litters your global environment with "junk" variables that serve no purpose other than to pass data to the next line. It's cluttered and makes the code feel like a grocery list rather than a logical process.

Thinking in Pipelines

The better way—and the way I want you to lean into—is using the native pipe operator |>. It allows you to write the code in the exact order the data moves through the system. It transforms the "Onion" into a conveyor belt.

# The pipeline approach
final_data <- raw_sensor_data |>
  remove_outliers() |>
  normalize_scale() |>
  rolling_avg()

The trade-off here is purely about readability and maintainability. Notice how the logic now flows top-to-bottom, left-to-right. If I decide that normalize_scale should actually happen before remove_outliers, I don't have to carefully surgically extract a nested function call; I just swap two lines of code. If I need to debug a specific step, I can just comment out the rest of the pipeline and see exactly what that one function is emitting.

One thing to keep in mind: this only works if your functions are designed for it. For a pipeline to be clean, the first argument of each function should be the data it operates on. If you're using a function that expects the data as the second or third argument, you'll find yourself fighting the pipe. That's usually a signal that the function needs a wrapper or a rewrite to be "pipeline-friendly."

📋 Practical Task

Exercise: Building a Clean Stock Price Volatility Pipeline

You are tasked with creating a data processing pipeline for a set of stock price observations. You have three existing functions: clean_ticks() (removes zeros), calc_returns() (calculates percentage change), and filter_volatility() (keeps only returns above a certain threshold).

Below is a naive, nested implementation. Your goal is to refactor this into a clean, readable pipeline using the |> operator. Ensure that the data flows logically from the raw input stock_data to the final high_volatility_events variable.

# Existing functions (do not change these)
clean_ticks <- function(df) { df[df$price != 0, ] }
calc_returns <- function(df) { df$returns <- c(NA, diff(df$price)/df$price[-1]); df }
filter_volatility <- function(df, threshold = 0.02) { df[abs(df$returns) > threshold, ] }

# Raw data
stock_data <- data.frame(price = c(100, 101, 0, 102, 105, 104, 108))

# REFACTOR THIS LINE:
high_volatility_events <- filter_volatility(calc_returns(clean_ticks(stock_data)))

Refactor the final line into a pipeline. Then, add a fourth step to the pipeline using head(n = 5) to ensure you only ever retrieve the first five high-volatility events.