/ index page has an image on it by default, but we want to replace it with video for 50% visitors and measure sign up rate for each case. I've created a simple js for A/B testing with saving allocation in cookies.
function allocation(name, cells) { cell = $.cookie('ab-testing-' + name); if (cell) return cell; // allocate rand = Math.random(); if (rand<0.5) { cell = cells[0]; } else { cell = cells[1]; } $.cookie('ab-testing-' + name, cell, { expires: 90, path: '/' }); return cell; }
This code allocates 50% of users to test cell randomly and saves allocation in cookie and returns allocation cell name.
The first parameter is the "test name", the second - cells names list
Use Case:
allocation('ImageVsVideo', ['Image', 'Video']); // Image or Video
In order to replace image on the page with video in 50% cases we have to add unique identificator to the element.
I've named it: ab-testing-ImageVsVideo.
That's great, we're ready for replacing logic. I've uploaded video on vimeo.com as the result, after convertation I got the following exporting code
<iframe src="http://player.vimeo.com/video/45516767" width="460" height="253" frameborder="0" webkitAllowFullScreen mozallowfullscreen allowFullScreen>I prefered html5 code (but if you want you can always step back and use legacy flash object).
$(function(){ cell = allocation('ImageVsVideo', ['Image', 'Video']); if (cell=='Video') { html = '<iframe src="http://player.vimeo.com/video/45516767" width="460" height="253" frameborder="0" webkitAllowFullScreen mozallowfullscreen allowFullScreen>'; $('#ab-testing-ImageVsVideo').html(html); } });
Everything works well, but we haven't tracked our goals. How do we know which test variation wins?
Google Analytics provides great feature called Event Tracking. It's mostly used for tracking user action on the page like button press or some ajax action tracking.
But we will use it for tracking our test's cells allocation.
There are 2 required arguments: Category and Action, in our case "AB-Testing" category for all a/b tests
and action "ImageVsVideo-Image" (or ImageVsVideo-Video) as identificator.
Working script for our page.
<script type="text/javascript"> _gaq.push(['_trackEvent', 'AB-Testing', 'ImageVsVideo-' + allocation('ImageVsVideo', ['Image', 'Video'])]);You can find sample at / source code, I also have put allocation function into the file ab-testing.js for easy usage.
Our original goal was to increase "SignUp rate", but also we wanted to know an economic outcome, general conversion rate and goal completions. So we selected following metrics:
So let's go to custom report and create a new "A/B testing"
It's great! But we still have a full list of visits without the difference "Image" or "Video", so we have to create 2 custom segments.
Go to "Advanced Segments" and create "New Custom Segment" - "Image".
Find metric "Event Action" contains "ImageVsVideo-Image".
Repeat the same for "Video" segment with "Event Action" equals "ImageVsVideo-Video".
Finally, apply both segments "Image" and "Video" and you will see something like this:
As a result: Video conversion rate 13% vs 10% with Image, higher economic value, but Goal Conversion Rate is a bit lower.
So we need more statistics. Let's select secondary dimension "Traffic type".
For direct traffic results are almost the same, but for search traffic "SignUp rate" and Economic value are much higher.
But from another perspective, for existing users conversion is lowering. It shows that for people who already visited our website
"quick response" is more important.
So it makes sense to show "Video" to the new clients who come from search engines.
Links
head.ready(function(){
for $(document).ready(function(){
).
function allocationTestCell(name, cells) { cell = $.cookie('ab-testing-' + name); if (cell) return cell; // allocate rand = Math.random(); start = 0; delta = 1/cells.length; cell = cells[0]; for (i=0;i<cells.length;i++) { if (i*delta<=rand && rand<(i+1)*delta) { cell = cells[i]; break; } } $.cookie('ab-testing-' + name, cell, { expires: 90, path: '/' }); return cell; } $(document).ready(function(){ var AbTestting = {}; $('.ab-testing').each(function(index, element) { test = $(element).attr('data-ab-testing-test'); cell = $(element).attr('data-ab-testing-cell'); if (!(AbTestting[test] instanceof Array)) { AbTestting[test] = []; } AbTestting[test].push(cell); }); // allocation for (test in AbTestting) { allocation = allocationTestCell(test, AbTestting[test]); _gaq.push(['_trackEvent', 'AB-Testing', test + '-' + allocation, 'ab-testing-'+test+'-'+'allocation', 0, true]); $('.ab-testing').each(function(index, element) { _test = $(element).attr('data-ab-testing-test'); _cell = $(element).attr('data-ab-testing-cell'); if (_test!=test) return; if (_cell!=allocation) { return $(element).hide(); } $(element).show(); }); } });
The main difference is that new code doesn't require to write any piece of code on JS. You only need to add the class "ab-testing" to the any element you want to test and specify the testname and the cellname e.g. <div class="ab-testing" data-ab-testing-test="signupblogtext" data-ab-testing-cell="how-it-works">some code for testing</div>
.
Let's look on a real sample of applying this testing from the scratch.
The source code of this element looks like:<div> Analyze and improve your social media presence <a href="http://www.truesocialmetrics.com/how-it-works">Get started now</a> </div>On the first stage you have to copy the element, change a message and make it hidden.
<div> Analyze and improve your social media presence <a href="http://www.truesocialmetrics.com/how-it-works">Get started now</a> </div> <div style="display:none;"> Analyze and improve your social media presence <a href="http://www.truesocialmetrics.com/signup">Get started now</a> </div>Rigth now, if you refresh the page nothing will be changed: you have 1 visible and 1 invisible element with the link you want to test.
At the next step, we need to choose a testname and a cells names. In my example:
<div class="ab-testing" data-ab-testing-test="signupblogtext" data-ab-testing-cell="how-it-works"> Analyze and improve your social media presence <a href="http://www.truesocialmetrics.com/how-it-works">Get started now</a> </div> <div class="ab-testing" data-ab-testing-test="signupblogtext" data-ab-testing-cell="sign-up" style="display:none;"> Analyze and improve your social media presence <a href="http://www.truesocialmetrics.com/signup">Get started now</a> </div>
As a result, 50% of users will see one block with "how-it-works" link and another 50% - the second one.