Ep # 74



Why you should split test

Source: Optimizely

A/B Split Testing

Want to run a successful marketing campaign? You might be a well-trained marketing student, a seasoned marketing professional, or even Neil Patel himself – your marketing campaign may not go right the first time. The only way to create a successful marketing campaign is to conduct various tests. This is where A/B testing comes in. A/B testing is a formulaic method of running various tests on your marketing campaign to see what works and what doesn’t.

Conversion rate optimization

How does A/B Testing work?

I’ll give you a simple explanation below.

  • An A/B test is usually a format of running two sets of your marketing campaigns.
  • There will be a difference in Variation A and B. The difference could be the design, text placement, or button color.
  • You then assess the data of two variation A and B.
  • Based on metrics such as better engagement, clicks and sales, you pick the variation that did better – this could be variation A or B.
  • The end product – You have a better campaign than you ran before.


Why should you Run an A/B?



Source: Oktopost

The alternative to A/B testings is this:

  1. You run variation A
  2. You run variation B
  3. Then, you compare the data
  4. Select the Marketing Campaign which does better

The problem with this is:

  • You waste money running two campaigns
  • You may run Marketing Campaign 1 first, and Campaign 2 later. This is a time consuming process.

Or, you may just run a single marketing campaign and then wring your head on why it is not working.

6 Ways to Improve Conversion Rate For PPC & Google Adwords

Where can use A/B Testing?

Rather, you should ask – where can’t you use A/B testing? A/B testing can be used for literally any marketing campaign. Here’s a quick list.

  • Email marketing
  • Google Adwords
  • Facebook campaigns
  • YouTube ads
  • LinkedIn InMails
  • Lead form
  • Product Pricing
  • Website landing page

What Elements can you A/B Test?

The answer to this question varies depending on which marketing medium you are dealing with, however, here’s a brief look:


An example of what you can do

Content Change the subject line, the title, the main content body.
Image Instead of using a person’s image, use a cartoon character. Or, use more visuals and less content.
Design Alter the color usage and flow.
Button Change the placement of the button or try using a different color.
Call to action Instead of using ‘Shop Now’, use ‘Buy Now’.


These are just some of the many elements you can modify when running your A/B test. I highly recommend, between Variation A and B, you only change one or two elements (not more than two). If you make major changes in your A/B test, for example, you modify the content, image, and content placement – it will be difficult to assess what did the audience like or dislike – Was it the content, image or content placement?

Because you don’t know what element the audience liked, you cannot implement the modification to the other campaigns.

How do you plan and execute A/B Testing?

This brings us to the crux of the article – How do you plan and execute A/B Testing?

Identify the Problem

Don’t run an A/B test just because you are running an email campaign or having a product launch. There should be an objective for the A/B test, which is usually to solve a problem. The problem could be:

  • Your email has a high open rate, but poor click through
  • Your Google ads get poor clicks
  • Your social media promotions don’t get good engagement

Once you identify the problem, the solution is simple – Use A/B testing to resolve the problem.


Conversion Rate Optimization 101 (CRO)

Build a Hypothesis

To conduct an A/B test, you need to create a hypothesis on what’s wrong with the marketing campaign. For this purpose, let’s take an example.

The Platform: Google Adwords
The Problem: Your ad has a low click through rate

A Google ad is fairly simple – it has a title and it has a body.

Why is no one clicking on your ad? A possible reason is that there is something wrong with your title or body text.

What could be the problem? Your text does not convince the reader to click on the link.

Why does your text not convince the reader to click on the link? Maybe it is not informative enough, the text is too long, or the sentence formation is wrong.

You do not know the exact reason, but you come up with a hypothesis – it could be that the title is too long.

Create the Variation

The next step is to create the ad variation.


Variation A

Variation B

Google Ad Title:
Plumbing Repair Service – First Call Heating in Edmonton
Google Ad Title:
Plumbing Repair Service


Set a Timeline

An A/B test cannot run for a long period. Variation B should run for a short period of time. This way, the A/B test is not an expensive expenditure. Before you launch the A/B test, decide how long to run the campaign for.

The ideal timeline is anywhere between 2 days to a week. However, honestly, this depends on the type of A/B test, the audience, the reach, your targets, etc.

You can use an A/B Split and Multivariate Test Duration Calculator. However, if you are a novice at A/B testing, stay away from the calculator.

Set up

Depending on which platform you are using, you can either use the platform to run the A/B test, use a tool or run it manually.

Social Media

Without altering the target audience, you launch the two advertisements.

Google AdWords

With the Google AdWords platform, you have two options.

  1. Under the Ad Group, you launch the two ads. This is similar to social media A/B testing.
  2. Another option, which I prefer, is using Google Experiments. Google Experiments allows you to run a short experiment which delivers quick results and helps you make quick decisions.


To conduct an email A/B test, you have two options:

  1. Manual: Running an email campaign manually is tedious and it can become even more difficult when you launch an A/B test. You have to divide your email recipients into two groups and send the email. Creating the emails, sending them out and tracking the data becomes tedious without the use of a tool.
  2. Email Tool:
    Email tools like MailChimp completely streamline your email campaign. They offer various features that allow easy execution of an A/B test.

Landing Page

A landing page is created when you run a marketing campaign. The user clicks on an advertising link or an email link and arrives on your website. So, when you run a marketing campaign – how do you ensure that your traffic is split and goes to variation A and B? You have two choices:

  1. Create variation A & B with two different URLs. And, instead of launching a single online advertisement (or email), launch the same advertisement twice – with two different URLs.
  2. The other option is to use a tool. If your website is WordPress based, there are A/B testing plugins such as Nelio AB testing. It simplifies the process and helps you track data better.

Multi variant testing

Insert Data Tracking

Ensure that you are tracking your test on all levels. This means right from sharing the link on Facebook, to the traffic visits to your website, to clicks on your landing page, you need to make sure that no level is missed. In the end, you must have data on:

  • Open Rate
  • Impressions
  • Click through rate
  • Engagement Rate
  • Conversion rate
  • Interaction
  • HeatMap of landing page

Certain platforms offer you enhanced tracking tools like:

  • Facebook Pixel
  • Google Analytics
  • Crazy Egg
  • Heap

These tools will provide a tracking code which needs to be inserted into your website HTML code. If your A/B testing is going to succeed, you need to have access to data that tells you how your audience interacts with the A/B variation.

Decide which One did Better

So – do you choose Variation A or B to continue your marketing campaign? The answer to this lies with the data.


Variation A

Variation B

Google Ad Title:
Plumbing Repair Service – First Call Heating in Edmonton
Google Ad Title:
Plumbing Repair Service


What is your hypothesis? Going back to the previous example, the hypothesis is that the title is too long.

What does the data tell you?

  • Variation B has more impressions and a higher click through rate

In this case, the choice is obvious – Variation B is a better choice. Your hypothesis holds true. However, let’s takes things a step further.

What is your hypothesis? There could be multiple of possibilities why Variation A is not working. It could be:

  • The title is too long
  • The title is not informative enough
  • The title does not have a strong call to action

The reality is that when you run a marketing campaign and it fails to get the response you want, it is very difficult to discern a single cause. By comparing the data of the two ads, you are able to get a clearer picture but even then – there is no single metric that will tell you what works. This is why it is critical that you keep experimenting. Your marketing campaign can always do better.

Real Life Examples of A/B Testing

EA Games: Selling Sims City

In 2011, EA games released Sims City 5. In an attempt to increase online downloads, EA games conducted A/B testing on the Sims City 5 landing page. The current landing page had a discount banner across the top of the page for pre-orders. However, the sales were not sufficient.

The marketing team decided to create a variation B – a landing page without the discount banner. Variation B saw a significant increase in sales and downloads.

Why did it work? People who visited the page were ready to buy the game and did not require an incentive. The discount banner was distracting and hindered their buying experience.

Design Hill: Email Campaign

Design Hill is a peer-to-peer site where designers and employers can find each other. They ran an email campaign to showcase relevant information to their readers.

However, the email did not receive a good open rate. The problem was clearly the subject line.

Variation A: Check out my recent post – Top 10 Off-page SEO strategies to for startups in 2015

Variation B: Top 10 Off-page SEO strategies to for startups in 2015

Variation B had a higher open rate of 2.57%. A subject line is the deciding factor for whether the email is opened or not. Variation A subject line was too long and didn’t instantly tell the reader what the email was about.

AdEspresso: Selling Advertisements

AdEspresso is an online service that helps users enhance their Facebook ad campaigns. They ran a Facebook ad campaign to create awareness and attract more customers. They decided to experiment with the long copy of the ad.

Variation A: LIKE us for pro tips on how to improve your Facebook ad campaigns today!

Variation B: LIKE us for daily tips on how to improve your Facebook ad campaigns & increase your ROI!

Variation A did better, while Variation B did not even get one like. There is a huge difference in the information being communicated between Variation A and B. Variation A is much more crisp, precise, straightforward, and even has a time factor.

Running an A/B test is no easy task. However, the more you do it, the better you get at it. If you think your marketing campaign is not running well, go ahead and conduct an A/B test. It is a much better and quicker alternative to getting results and you discover what’s working and what’s not. The end product is a more successful marketing campaign.


Split Testing Tools