A/B Testing in Google Analytics with Event Tracking

by Rostyslav Mykhajliw Founder of TrueSocialMetrics.com ~ 7 min

As you know, by default, Google Analytics experiments feature works only with separate pages and it works well when you are starting advertising campaign. So you can control destination url. But what to do, if you don't control your traffic sources: search, referral, social media postings. That's the main issue. But we have a simple solution for it. The solution requires:

Let's start testing

/ index page has an image on it by default, but we want to replace it with video for 50% visitors and measure sign up rate for each case. I've created a simple js for A/B testing with saving allocation in cookies.

function allocation(name, cells) {
    cell = $.cookie('ab-testing-' + name);
    if (cell) return cell;
    // allocate
    rand = Math.random();
    if (rand<0.5) {
        cell = cells[0];
    } else {
        cell = cells[1];
    }
    $.cookie('ab-testing-' + name, cell, { expires: 90, path: '/' });
    return cell;
}

This code allocates 50% of users to test cell randomly and saves allocation in cookie and returns allocation cell name. The first parameter is the "test name", the second - cells names list

Use Case:

allocation('ImageVsVideo', ['Image', 'Video']); // Image or Video

In order to replace image on the page with video in 50% cases we have to add unique identificator to the element. I've named it: ab-testing-ImageVsVideo.

That's great, we're ready for replacing logic. I've uploaded video on vimeo.com as the result, after convertation I got the following exporting code

<iframe src="http://player.vimeo.com/video/45516767" width="460" height="253" frameborder="0" webkitAllowFullScreen mozallowfullscreen allowFullScreen>

I prefered html5 code (but if you want you can always step back and use legacy flash object).

Our logic is simple: if the cell name equals "Video" update element id "ab-testing-ImageVsVideo" by given player html code.

$(function(){
    cell = allocation('ImageVsVideo', ['Image', 'Video']);
    if (cell=='Video') {
        html = '<iframe src="http://player.vimeo.com/video/45516767" width="460" height="253" frameborder="0" webkitAllowFullScreen mozallowfullscreen allowFullScreen>';
        $('#ab-testing-ImageVsVideo').html(html);
    }
});

Everything works well, but we haven't tracked our goals. How do we know which test variation wins?

Google Analytics provides great feature called Event Tracking. It's mostly used for tracking user action on the page like button press or some ajax action tracking. But we will use it for tracking our test's cells allocation. There are 2 required arguments: Category and Action, in our case "AB-Testing" category for all a/b tests and action "ImageVsVideo-Image" (or ImageVsVideo-Video) as identificator. Working script for our page.

<script type="text/javascript">
  _gaq.push(['_trackEvent', 'AB-Testing', 'ImageVsVideo-' + allocation('ImageVsVideo', ['Image', 'Video'])]);

You can find sample at landing page source code, I also have put allocation function into the file ab-testing.js for easy usage.

Google Analytics reports is the next step

Our original goal was to increase "SignUp rate", but also we wanted to know an economic outcome, general conversion rate and goal completions. So we selected following metrics:

Unique visitors
Sign Up (Goal11 Completions)
Sign Up (Goal11 Conversion Rate)
Per visit goal value
Goal Convertion Rate
Goal Completions

Also it's nice to split "New" and "Returning" visitors, because mostly when we're talking about "SignUp" we care about new users. So let's go to custom report and create a new "A/B testing"
It's great! But we still have a full list of visits without the difference "Image" or "Video", so we have to create 2 custom segments. Go to "Advanced Segments" and create "New Custom Segment" - "Image".
Find metric "Event Action" contains "ImageVsVideo-Image".
Repeat the same for "Video" segment with "Event Action" equals "ImageVsVideo-Video".

Finally, apply both segments "Image" and "Video" and you will see something like this:

As a result: Video conversion rate 13% vs 10% with Image, higher economic value, but Goal Conversion Rate is a bit lower. So we need more statistics. Let's select secondary dimension "Traffic type".

For direct traffic results are almost the same, but for search traffic "SignUp rate" and Economic value are much higher. But from another perspective, for existing users conversion is lowering. It shows that for people who already visited our website "quick response" is more important. So it makes sense to show "Video" to the new clients who come from search engines.

Links

Event Tracking Guide

Update from 05/07/2013

I've updated "allocation code" by adding the multi-tests support for the same page. For example, on a current blog page - 2 tests: - There are 4 different "singup" forms, with a different messages (but the rest of the 3 are hidden). If you open page in a source mode you will find them.

The second test is a message under the form with a link to "How-it-works" or "Singup" pages.

The full code is here, also you can download our latest version (remember we're using specialized async js loader, if you don't use it, you have to replace head.ready(function(){ for $(document).ready(function(){.

function allocationTestCell(name, cells) {
    cell = $.cookie('ab-testing-' + name);
    if (cell) return cell;
    // allocate
    rand = Math.random();
    start = 0;
    delta = 1/cells.length;
    cell = cells[0];
    for (i=0;i<cells.length;i++) {
        if (i*delta<=rand && rand<(i+1)*delta) {
            cell = cells[i];
            break;
        }
    }
    $.cookie('ab-testing-' + name, cell, { expires: 90, path: '/' });
    return cell;
}
$(document).ready(function(){
    var AbTestting = {};
    $('.ab-testing').each(function(index, element) {
        test = $(element).attr('data-ab-testing-test');
        cell = $(element).attr('data-ab-testing-cell');
        if (!(AbTestting[test] instanceof Array)) {
            AbTestting[test] = [];
        }
        AbTestting[test].push(cell);
    });
    // allocation
    for (test in AbTestting) {
        allocation = allocationTestCell(test, AbTestting[test]);
        _gaq.push(['_trackEvent', 'AB-Testing', test + '-' + allocation, 'ab-testing-'+test+'-'+'allocation', 0, true]);
        $('.ab-testing').each(function(index, element) {
            _test = $(element).attr('data-ab-testing-test');
            _cell = $(element).attr('data-ab-testing-cell');
            if (_test!=test) return;
            if (_cell!=allocation) {
                return $(element).hide();
            }
            $(element).show();
        });
    }
});

The main difference is that new code doesn't require to write any piece of code on JS. You only need to add the class "ab-testing" to the any element you want to test and specify the testname and the cellname e.g. <div class="ab-testing" data-ab-testing-test="signupblogtext" data-ab-testing-cell="how-it-works">some code for testing</div>.

Let's look on a real sample of applying this testing from the scratch.

The source code of this element looks like:

<div>
  Analyze and improve your social media presence <a href="/how-it-works">Get started now</a>
</div>

On the first stage you have to copy the element, change a message and make it hidden.

<div>
  Analyze and improve your social media presence <a href="/how-it-works">Get started now</a>
</div>
<div style="display:none;">
  Analyze and improve your social media presence <a href="/signup">Get started now</a>
</div>

Rigth now, if you refresh the page nothing will be changed: you have 1 visible and 1 invisible element with the link you want to test.
At the next step, we need to choose a testname and a cells names. In my example:

signupblogtext - test name
how-it-works - cell name for the first block with link to "how it works"
sign-up - cell name for the hidden block with link to "sign-up"

Also we will add class "ab-testing" as a marker for our JS. So the full working code looks like this:

<div class="ab-testing" data-ab-testing-test="signupblogtext" data-ab-testing-cell="how-it-works">
  Analyze and improve your social media presence <a href="/how-it-works">Get started now</a>
</div>
<div class="ab-testing" data-ab-testing-test="signupblogtext" data-ab-testing-cell="sign-up" style="display:none;">
  Analyze and improve your social media presence <a href="/signup">Get started now</a>
</div>

As a result, 50% of users will see one block with "how-it-works" link and another 50% - the second one.

Testing

My code is storing allocation in users cookies, which means allocation is happening only once, so if you want to test it you need a clean browser without cookies. But much a simplier solution is using Google Chrome feature Incognito Window. It provides absolutelly clear new browser window without any cookies. As a result, you can open the page several times and check if everything works properly.