I've seen this happen to a lot of developers moving into sports analytics. You spend hours scraping player stats, you build a model that looks great on your training data, and then the moment you try to run it against a new set of players—say, a fresh draft class—the whole thing crashes with a cryptic error about "factor levels."

# The "Broken" Setup
player_data <- data.frame(
  position = c("Guard", "Forward", "Center", "Guard"),
  minutes = c(30, 32, 28, 35),
  points = c(22, 18, 12, 25)
)

# We build a simple model to predict points
perf_model <- lm(points ~ position + minutes, data = player_data)

# Now we try to predict for a new player who is a "Wing" (a position not in training)
new_player <- data.frame(position = "Wing", minutes = 30)
predict(perf_model, new_player)
# Error in predict.lm(perf_model, new_player) : 
# factor position has new levels Wing

The Factor Level Mismatch

The problem here is that R treats categorical variables (factors) as a fixed set of levels. When you trained perf_model, R looked at the position column and decided the world only consists of Guards, Forwards, and Centers. When you handed it a "Wing," the model didn't know how to map that to a coefficient. It doesn't just assume it's a zero; it panics and stops.

In a real-world player performance model, this happens constantly. You might have a player with a rare position or a team that just entered the league. If your pipeline isn't robust, your production code will blow up the second an outlier appears.

Handling Unseen Categories with Level Standardization

The fix isn't just to add "Wing" to your training data—you can't always predict the future. Instead, you need to define your factor levels explicitly or handle "Other" categories. The most professional way to do this is to define a master list of allowable positions and coerce any unknown values into an "Other" category before the data ever hits the model.

# Define the "Source of Truth" for positions
valid_positions <- c("Guard", "Forward", "Center", "Other")

# A helper function to clean the data
clean_position <- function(pos_vec) {
  # If the position isn't in our valid list, call it "Other"
  pos_vec[!(pos_vec %in% valid_positions)] <- "Other"
  return(factor(pos_vec, levels = valid_positions))
}

# Apply the fix to training data
player_data$position <- clean_position(player_data$position)
perf_model <- lm(points ~ position + minutes, data = player_data)

# Apply the fix to the new player
new_player$position <- clean_position(new_player$position)
predict(perf_model, new_player) 
# Now it works! "Wing" is treated as "Other".

Weighting Performance by Sample Size

Once you've solved the crashing issues, you'll run into the "Small Sample Size" trap. If a player has played only 2 games and averaged 30 points, a basic linear model will treat them as a superstar. I usually recommend using a weighted least squares (WLS) approach. In R, lm() has a weights argument that allows you to tell the model: "Trust this data point more because the player has played 82 games, and trust this one less because they've only played 3."

By passing the number of games played into the weights parameter, you penalize the volatility of bench players and give more gravity to established starters. This prevents your performance model from being skewed by "fluke" statistical anomalies.

Evaluating with Root Mean Square Error

You can't just trust the R-squared value. In player modeling, I care about the actual margin of error. If my model predicts a player will score 20 points and they score 12, that's a massive miss in a betting or scouting context. I always calculate the RMSE (Root Mean Square Error) on a hold-out test set.

I prefer RMSE over MAE (Mean Absolute Error) here because RMSE penalizes large misses more heavily. In sports, being "slightly off" is okay; being "wildly wrong" about a player's impact is what gets you fired.

📋 Practical Task

Build a Weighted Player Efficiency Predictor

You are provided with a dataset of 10 players. Some have played many games, others very few. Your goal is to build a model that predicts efficiency_score based on minutes_per_game and position, ensuring the model is robust to new positions and weighted by games played.

# Dataset provided
players <- data.frame(
  name = c("A", "B", "C", "D", "E", "F", "G", "H", "I", "J"),
  position = c("Guard", "Guard", "Forward", "Forward", "Center", "Center", "Guard", "Forward", "Center", "Guard"),
  mpg = c(34, 31, 28, 35, 30, 22, 15, 20, 33, 12),
  games_played = c(80, 75, 60, 82, 70, 10, 5, 12, 78, 2),
  efficiency_score = c(24.5, 22.1, 18.4, 19.2, 21.0, 15.1, 12.0, 14.5, 23.1, 10.2)
)

# New player to predict (Note: position "Utility" is not in the training set)
new_player <- data.frame(position = "Utility", mpg = 25)

Your Task:

• Create a valid_positions vector containing "Guard", "Forward", "Center", and "Other".
• Write a function to coerce the position column in both the players dataframe and the new_player dataframe to these levels (mapping unknowns to "Other").
• Build a weighted linear model (lm) where efficiency_score is predicted by position and mpg, using games_played as the weight.
• Predict the efficiency_score for the new_player and print the result.