How do I stop users from jumping between versions?

The biggest mistake I see developers make when first hacking together an A/B test is forgetting about "stickiness." If a user refreshes the page and the CTA changes from "Start Free Trial" to "Join the Community," you've just ruined the user experience and polluted your data. You need a way to ensure that once a user is assigned to a variant, they stay there.

For logged-in users, a column on the users table is the easiest route, but for guest traffic, you'll want to rely on cookies. I usually implement a simple helper that hashes a unique identifier (like a session ID) to determine the group. This keeps it deterministic without needing to store every single guest's assignment in a database.

# app/helpers/experiment_helper.rb
module ExperimentHelper
  def in_variant?(experiment_name, variant)
    # We use a digest of the session ID and experiment name 
    # to consistently assign the user to a bucket.
    identifier = "#{session.id}-#{experiment_name}"
    bucket = Digest::MD5.hexdigest(identifier).to_i(16) % 2
    
    # 0 = Control, 1 = Treatment
    variant == (bucket == 1 ? :treatment : :control)
  end
end

It's simple, it's fast, and it doesn't require a database hit every time you want to check a flag.

Is there a cleaner way to handle this than messy if/else blocks?

If you start putting if in_variant?(:new_signup_flow, :treatment) all over your views and controllers, your codebase will become a nightmare to maintain. Six months from now, you'll be terrified to delete those blocks because you won't remember which experiments are still running.

I prefer wrapping this logic in a dedicated Experiment class. This allows you to centralize the definition of your tests and makes it trivial to "roll out" a winner by simply changing a return value in one file rather than hunting through twenty views.

# app/models/experiment.rb
class Experiment
  EXPERIMENTS = {
    signup_cta: { 
      status: :active, 
      variants: [:control, :treatment] 
    },
    checkout_layout: { 
      status: :completed, 
      winner: :treatment 
    }
  }.freeze

  def self.variant_for(name, user_id)
    config = EXPERIMENTS[name]
    return nil unless config
    return config[:winner] if config[:status] == :completed

    # Deterministic assignment logic
    Digest::MD5.hexdigest("#{user_id}-#{name}").to_i(16) % 2 == 0 ? :control : :treatment
  end
end

Now, in your view, you just call Experiment.variant_for(:signup_cta, current_user.id). When the experiment ends and the treatment wins, you just update the status to :completed and set the winner. The rest of your app updates instantly.

How do I actually track the "win" without bloating my controller?

An A/B test is useless if you can't prove which version performed better. But you don't want your create action in the SignupsController to be ten lines of analytics code. I like to handle this via a lightweight tracking event that fires asynchronously or via a simple database log.

The key is to record the variant the user was in at the moment they performed the target action. If you only check the current variant during the conversion event, you might miss users who saw the control version but converted after you've already flipped the experiment to "completed."

# app/controllers/signups_controller.rb
def create
  @user = User.new(signup_params)
  if @user.save
    # Log the conversion and the variant they were exposed to
    track_conversion(:signup_cta, current_variant)
    redirect_to dashboard_path
  end
end

private

def current_variant
  Experiment.variant_for(:signup_cta, @user.id)
end

def track_conversion(experiment_name, variant)
  # This could be a call to Segment, Mixpanel, or a local table
  ExperimentConversion.create!(
    experiment: experiment_name, 
    variant: variant, 
    user_id: @user.id
  )
end

By decoupling the assignment (what they saw) from the conversion (what they did), you keep your data clean and your controllers readable.

📋 Practical Task

Implement a "Price Display" A/B Test

You are tasked with testing whether showing a price as "$49/mo" (Control) or "Only $1.63/day" (Treatment) increases conversion on a pricing page.

• Create an Experiment class (or extend the one from the lesson) to include a :price_format experiment.
• Implement a helper method or a service that determines which string to return based on the user's assigned variant.
• Modify a mock PricingController or view to display the correct price string.
• Create a migration and a model for ExperimentConversion to track when a user clicks the "Buy Now" button, ensuring the specific variant they saw is recorded.