Imagine you're trying to predict how many bugs will be found in a new software release. You might start with a standard linear regression, but you'll quickly run into a problem: linear regression assumes your output can be any number, including negative ones. But you can't have -3 bugs in a release. That just doesn't happen in the real world.

Linear regression is like a rigid ruler. It's great for measuring things that grow in a straight line, but it fails when the data "bends" or is restricted to specific types of values (like whole numbers or yes/no answers). A Generalized Linear Model (GLM) is more like a flexible measuring tape. It allows us to keep the "linear" part of the math—the relationship between your inputs and your output—but it adds a "translator" (which we call a link function) to make sure the final prediction actually makes sense for the data you're dealing with.

Here is how that maps to the actual concepts in R:

• The Linear Predictor: This is the "rigid ruler" part. It's the weighted sum of your variables (beta0 + beta1*x).
• The Link Function: This is the "translator." It takes that linear result and maps it to the scale of your data. For counts, we often use a "log link" so the result is always positive.
• The Error Distribution: This is the "shape" of your data. Instead of assuming a Bell Curve (Normal distribution), we might assume a Poisson distribution for counts or a Binomial distribution for binary outcomes.

When to stop using lm() and start using glm()

You've used lm() in previous lessons for standard linear regression. But the moment your dependent variable isn't a continuous, normally distributed number, lm() is the wrong tool. I've seen a lot of engineers try to force lm() to predict binary outcomes (0 or 1), and while it technically runs, the predictions often end up being 1.2 or -0.4, which is useless for a "yes/no" question.

If you're predicting a count (e.g., "How many tickets will this user open?"), use family = poisson. If you're predicting a probability (e.g., "Will this build fail?"), use family = binomial.

Fitting a Poisson Model for Bug Counts

Let's look at a concrete example. Suppose we have a dataset where bugs is the number of defects found and kloc is the thousands of lines of code changed. Since bug counts are discrete and non-negative, a Poisson GLM is the way to go.

# Let's create a quick dummy dataset
dev_data <- data.frame(
  kloc = c(10, 20, 30, 40, 50, 60, 70, 80),
  bugs = c(2, 5, 8, 12, 15, 22, 28, 35)
)

# We use glm() instead of lm()
# Note the 'family' argument—this is the most important part!
bug_model <- glm(bugs ~ kloc, data = dev_data, family = poisson(link = "log"))

summary(bug_model)

I want to point out that link = "log" is actually the default for the Poisson family, so you can just write family = poisson. But being explicit about it helps you remember that R is taking the log of your expected count to make the linear math work behind the scenes.

Dealing with the Output

When you run summary(bug_model), you'll notice it looks a lot like lm(), but the coefficients are on the scale of the link function, not the original data. Since we used a log link, the coefficients are logarithmic.

To get a prediction that actually looks like a number of bugs, you can't just use predict(bug_model). By default, that gives you the result in "log-space." To get it back to the original scale, you specify type = "response":

# Predict bugs for a change of 45 kloc
new_code <- data.frame(kloc = 45)
predict(bug_model, newdata = new_code, type = "response")

If you forget type = "response", you'll get a number like 2.4, which represents the log of the bugs. You'd then have to manually exponentiate it (exp(2.4)) to get the actual count. Just save yourself the headache and use the argument.

📋 Practical Task

Predicting Server Downtime Incidents

You have been handed a dataset containing server metrics from the last year. Your goal is to predict the number of downtime incidents per month based on the average CPU load and the age of the hardware (in years). Since downtime incidents are counts, you must use a Generalized Linear Model.

Dataset:

server_stats <- data.frame(
  incidents = c(0, 1, 0, 2, 1, 4, 3, 5, 2, 6),
  cpu_load = c(20, 35, 25, 50, 40, 70, 65, 85, 55, 90),
  age = c(1, 2, 1, 3, 2, 5, 4, 6, 3, 7)
)

Your Task:

1. Fit a GLM to predict incidents using cpu_load and age as predictors. Ensure you use the correct family for count data.
2. Extract the summary of the model to see which variable (CPU load or age) has a more significant impact on downtime.
3. Predict the expected number of incidents for a server with a cpu_load of 75 and an age of 4 years. Make sure your result is in the original scale (the actual number of incidents), not the log scale.