What is A/B Testing? – A Beginner’s Guide

It doesn’t matter if you’re a part-time blogger, a full-time marketing professional, or the owner of a small and medium-sized (SMB) business looking to increase your website’s traffic, in the end, you want to get the most bang for your buck.

Maximising your returns comes down to testing multiple variables in order to find that all-important ingredient to increasing revenue.

[Tweet “Never tried A/B split testing? Check out this beginners guide…”]

While it may seem like an involved process, it’s really just a matter of making small changes and measuring results.

If you’re looking to drive more traffic to your blog, or looking to improve the effectiveness of your online pay-per-click (PPC) campaigns, or even increase your conversion rates by turning more website visitors into customers, then understanding the process involved in A/B split testing is your roadmap moving forward.

Related: 5 landing page split tests you should be using

 

What is A/B Testing?

So, what exactly is A/B split testing? Simply put, it’s running two or more separate campaigns or comparisons where different variables are adjusted in order to optimise results.

One campaign is your control sample, while the other is your test sample. As you make minor changes to the campaign you begin to track the impact of those changes in order to determine whether they help increase or decrease your returns.

 

HubSpot split tested the same two pages, but with different colour CTAs – Image Source

 

A/B split testing is a simple and straightforward experiment that allows marketers to test different fonts, images, advertisements, and links in order to pinpoint that ideal configuration.

It’s a process that can be used to optimise a website, increase revenue on a blog or forum, and or increase conversion rates on separate email marketing campaigns, PPC campaigns and SMS mobile campaigns.

However, it’s wrong to assume that you can only run a split test with one control sample and one test sample. In fact, it’s common for tests to involve multiple samples. Of course, this depends entirely on the size of your target audience and the length of time you run these tests.

 

Why is A/B Testing useful?

A/B testing gives you the data you need to make informed decisions about which direction to take in the future, and if used consistently, it will improve user experience and increase conversion rates, both in the long and short term.

 

A/B Testing examples:

Let’s assume you own a blog with the following layout (below). The advertisements that generate revenue are in red.

 

 

Over one month (30,000 visits) your click-through rates (CTR) on these advertisements was 0.8% which means that out of every 1000 visitors to your website, 8 of them are clicking on your advertisements.

The revenue you generated averaged about £6.50 on these 8 clicks. In essence, you generate £6.50 for every 1000 page views. However, you think you can do better. 

First, it’s important to understand that you must only change one variable at a time in order to get a true indication of results.

Changing multiple variables at one time will not give a true indication of the benefit of any one change. 

In order to see whether you can increase your revenue, you decide to run an A/B split test.

You are going to change the configuration of the advertisements on your blog and see what effect it has on your click-through rate and your revenue in the next month.

In essence, you are going to compare one month of 30,000 visitors to another month of 30,000 visitors.

 

Make a Change…

Now let’s assume you made the following changes to your blog. You moved the link advertisement in-between the posts and instead positioned it right below the navigation bar.  

 

 

In making this change, you saw that your click-through rate steadily increased to 1.0% over a single month comparison of 30,000 views.

Your revenue has increased to £7.25 for every 1000 page views. Now you aim to build upon that again. In doing so, you make another single change.

With this change you keep the link advertisement below the navigation bar and decide to create a skyscraper advertisement on the right side of your blog.

 

 

After another month of 30,000 views, click-through rate increased to 1.5%. Now, for every 1000 visitors to your blog, 15 are clicking on advertisements and your revenue is now £8.50 for every 1000 page views. 

This same strategy can easily be employed on email marketing campaigns and PPC campaigns.

However, the focus in those instances is to gradually change fonts, images and or links and then measure results at each interval.

 

Testing over a Time Period vs Segmenting Your Audience

In our example above, we’ve run a test over three consecutive months. In this case, we took it one step at a time.

However, in the case of an email marketing campaign, an SMS mobile text message campaign, or a PPC campaign, this type of strategy can be run in unison.

If the company has enough customers to send different advertising campaigns to, then it can run an A/B split test at the same time then choose the result that best meets their expectations.

Of course, this means having the ability to send separate campaigns to the same number of customers in the same customer segment.

It’s critical that the customer segment remain similar in order to nail down or define any improvement.

 

Conclusion

Split testing is vital in order to determine what works best for your audience and marketing campaigns, and it doesn’t have to be complicated. 

However, our experiment above was just a simple example of what you can test for. Now you know what A/B split testing is, check out this more in depth guide by Smashing Magazine.

 

Your Say!

Have you tried split testing? Did you see some positive or unexpected results? What did you change? We’d love to hear from you.