I've always found that the hardest part of dealing with high-dimensional data isn't the math—it's the visualization. When you have two variables, you have a scatter plot. Three variables, and you can maybe get away with a 3D plot if you're feeling fancy. But once you hit four or five variables, you're essentially flying blind. That's where Principal Component Analysis (PCA) comes in. It's basically a way of asking R, "If I had to squash all this data down into two dimensions without losing the 'soul' of the dataset, how would I do it?"

The Curse of Too Many Columns

Let's take a look at the built-in USArrests dataset. It tracks crime statistics across different US states. It's small, but it has four variables: Murder, Assault, UrbanPop, and Rape. I want to see if there are any natural groupings of states, but I can't plot four axes.

# Let's see what we're dealing with
data(USArrests)
head(USArrests)
pairs(USArrests)

The pairs() plot gives us a matrix of scatter plots. It's useful, but it's a lot of ocular noise. I can see some correlations—like Murder and Assault usually move together—but I can't see the "big picture" of the states as single points in space.

Trying a Naive Compression

I'll use prcomp(), which is the standard workhorse for PCA in R. I'll just throw the data in and see what happens.

# First attempt at PCA
pca_result <- prcomp(USArrests)
summary(pca_result)

Wait. Look at the importance section of that summary. The first principal component (PC1) is explaining almost 99% of the variance. That seems... suspicious. Usually, it's not that clean. Why is PC1 dominating everything?

The Scaling Trap

I suspect it's a units problem. If you look at USArrests, the Assault column has values in the hundreds, while Murder is in the single digits. PCA works by looking at variance. Because Assault has a much larger numerical range, R thinks it's the most "important" variable simply because its numbers are bigger. It's essentially ignoring the other columns because their variance is tiny by comparison.

This is a classic mistake. Whenever you're doing PCA, you almost always need to scale your data so that every variable has a mean of 0 and a standard deviation of 1. Luckily, prcomp has a built-in argument for this.

# Re-running with scaling enabled
pca_scaled <- prcomp(USArrests, scale = TRUE)
summary(pca_scaled)

That looks much more realistic. Now PC1 explains about 62% of the variance, and PC2 explains about 23%. Together, they cover 85% of the "story" the data is telling, but we've reduced our dimensions from four down to two.

Mapping the Noise

Now that we've squeezed the data, I want to see it. The biplot() function is the best way to do this because it shows us both the samples (the states) and the variables (the original columns) as vectors.

biplot(pca_scaled)

When you look at this plot, notice how the arrows for Murder, Assault, and Rape are all pointing in roughly the same direction (to the right). This tells me those three variables are highly correlated—they are essentially measuring the same "thing" (violent crime). UrbanPop is pointing in a different direction, meaning it doesn't track the same way as the crime stats.

If a state is far to the right on the x-axis (PC1), it's high in all those violent crime categories. If it's high on the y-axis (PC2), it likely has a higher urban population relative to its crime rates. We've successfully turned a 4D problem into a 2D map.

📋 Practical Task

Dimensionality Reduction of the Motor Trend Car Road Tests

Using the built-in mtcars dataset, perform a Principal Component Analysis to identify which cars are most similar based on their performance and design specifications.

• Run a PCA on the mtcars dataset, ensuring that the data is centered and scaled.
• Generate a summary of the PCA to determine how many principal components are required to explain at least 80% of the total variance.
• Create a biplot of the results.
• Identify which variables are most strongly correlated with each other based on the direction of the vectors in your biplot.