Skip to Content
Course content

226: Application Performance Monitoring with Scout or New Relic

Click on the "Edit" button in the top corner of the screen to edit your slide content.

A few years ago, I was on call for a Ruby on Rails app that handled a decent amount of traffic. Around 2:00 PM on a Tuesday, our Slack alerts started screaming. Users were complaining that the "Order History" page was taking ten seconds to load, but only for some of them. I jumped into the production logs, and everything looked... fine. Every request was returning a 200 OK. I tried to reproduce it in staging, but my test account only had three orders, so the page loaded in 50ms. I spent three hours guessing and checking, digging through controllers and views, before I realized that one specific power user had 4,000 orders, and we had a hidden N+1 query in a helper method that was firing a database call for every single row in that table.

That was the day I stopped relying solely on logs and started using Application Performance Monitoring (APM). When you're running Ruby in production, you're essentially flying a plane; logs are like your altimeter, but an APM tool is your entire radar system. It tells you not just that the plane is slowing down, but exactly which engine is failing and why.

Seeing Beyond the Rails Log

Standard logging tells you that a request happened and how long it took in total. But in a complex Ruby app, a "slow request" is a vague symptom. Was it a slow SQL query? Was it a sluggish external API call to Stripe or Twilio? Or was it just a computationally expensive piece of Ruby code looping over a massive array? This is where tools like Scout or New Relic come in.

These tools use "instrumentation." When you add their gem to your Gemfile, they hook into the Ruby runtime and the Rails middleware. They wrap around your database adapter and your HTTP clients. This means every time your app calls ActiveRecord or Net::HTTP, the APM tool records exactly how long that specific call took and associates it with the current web request. Instead of guessing, you get a "trace"—a waterfall chart showing that of your 10-second request, 9.2 seconds were spent in a single SELECT statement.

Hunting N+1s and Bottlenecks in Production

One of the biggest wins with APM—especially with a tool like Scout—is the automatic detection of N+1 queries. We've all learned about includes and joins in earlier lessons, but it's easy to miss one in a complex view partial. Scout, for instance, monitors your production traffic and flags exactly which line of code is triggering a loop of queries. It doesn't just say "you have an N+1"; it tells you "this specific action is calling this query 100 times per request."

I also want you to stop looking at "Average Response Time." Averages are a lie in software engineering. If nine users have a 100ms experience and one user has a 10-second experience, your average is roughly one second. That doesn't accurately describe anyone's experience. APM tools give you percentiles—specifically p95 and p99. If your p95 is high, it means 5% of your users are having a miserable time. That's the number you should be optimizing for.

Choosing Between Lightweight and Enterprise Monitoring

You'll generally see two paths here: the "comprehensive" route and the "focused" route. New Relic is the industry giant. It provides deep JVM/Ruby metrics, server health, browser monitoring, and complex alerting. It's incredibly powerful, but it can be overwhelming—and expensive—for a small team. You can spend hours just configuring the dashboard.

Scout takes a different approach. It focuses heavily on the developer experience, specifically targeting the things that kill Rails apps: N+1s and slow endpoints. It’s generally easier to set up and gets you to the "aha!" moment faster. If you're a solo dev or in a small startup, I usually recommend starting with something lightweight. You don't need a cockpit with 50 dials if you only need to know why your database is screaming.




📋 Practical Task

Exercise: Finding the Ghost in the Machine

In this exercise, you will simulate a production performance leak and use an APM tool to identify the culprit. Since you likely don't have a high-traffic production site, you will use a local environment to generate "bad" data.

  • The Setup: Create a simple Rails application with a User model and a Post model (one-to-many). Create a seed file that generates 10 users, each with 500 posts.
  • The Leak: Create a controller action called #index that renders a list of users and their last post's title. Intentionally write this using an N+1 pattern (e.g., calling user.posts.last.title inside a loop in the view without using includes).
  • The Implementation: Sign up for a free trial of Scout or New Relic. Add the respective gem to your Gemfile, run bundle install, and configure the API key in your environment variables as per the tool's documentation.
  • The Hunt: Start your server and refresh the index page a few times. Log into your APM dashboard and locate the "Slowest Transactions" or "N+1" section.
  • The Fix: Use the trace provided by the tool to find the exact line in your view or controller causing the issue. Fix it using .includes(:posts) and verify in the APM dashboard that the query count for that action has dropped from N+1 to a constant number.
Rating
0 0

There are no comments for now.

to be the first to leave a comment.