When it’s done well, A/B testing can maximise your efforts, giving you the knowledge and understanding that one simple change can be the difference between reaching your KPI and surpassing it completely.

Let's take Black & Decker's famous example: by changing the call-to-action text on its product pages from ‘Shop now’ to ‘Buy now’, they improved conversion by 17%. But before you start changing all of your product page buttons to ‘Buy now’, you should spend some time working out what works for your customer base. No two businesses are the same, so why should your customers be?

This guide is designed to bring you up to speed on the basics on A/B testing - the Dos and Don'ts, as well as some tips on how to run an A/B test.

For more advice about the different types of testing you can run in your business, speak to the FINALLY team and we’ll be only too happy to help.

What is A/B testing?

A/B testing – or split-testing – is a method of optimising the success of your KPIs by changing a single variable (yes, that’s right - single - but more on that later) on your landing page or email.

Put simply, it’s an experiment, not unlike the ones you had in school science lessons, where you test which outcome is more successful, based on a particular change.

This could be a simple copy change, such as the one in the Black & Decker example, or a change to the design and lay-out of your website.

How to run an A/B test on a website?

Step one: Come up with a theory

It’s important to have a reason or theory behind the test to give its results context. Black & Decker assumed that users making it to the product page wanted a more definitive command to help them on their journey. ‘Shop now’ seems to imply a number of actions could occur by clicking this button, whereas there’s no mistaking what ‘Buy now’ will do.

Before you do anything, get your team together and chat over a few theories. Perhaps you’re wondering if personalisation should be included in your email newsletter or maybe you want to test the order or language used in your top nav bar? Take inspiration from other websites, PPC ads and emails and come up with a test to see if what they're doing can work for you too.

Step two: Create identical versions of your page – bar one single variable

You now have something to test – create two versions of this page, email or advert. You’ll need to ensure that this will be seen by roughly the same number of people. If you’re testing an email, split your data select into two groups; if you’re testing a landing page, you can ensure that both versions are freely accessible to all traffic this via your CMS or by using Google Optimise.

One will be your Control version – or the version which doesn’t include the element that you want to test – and the other will be your Test version – or ‘Challenger’. It’s absolutely vital that these two versions are identical, apart from the variable that you are testing.

So, if you’re testing the number of clicks a red call-to-action button gets versus a green one, you need to ensure everything else on the page is the same – the copy, the design and how each page is linked to. When it comes to testing, fidelity is everything.

Step three: Decide on a time-frame for your test

This is important. Unless you have substantial data to back up your claim, you can’t be sure of the accuracy of your findings.

If your monthly website traffic is in the hundreds rather than the thousands, creating two versions of the same landing page and running these simultaneously is unlikely to yield any conclusions, at least in the short term. Running your test over a few months may combat this issue, giving you time to accumulate enough visits to draw meaningful conclusions from your test.

Types of A/B tests

As with most things in marketing, you’re only limited by your own ambition. If you’re new to split testing, make a small change which you believe will deliver the biggest disparity.

Subject lines in emails or PPC advertising is a great place to start. Why? Well, the sample size of your test group is likely to be pretty substantial here, so you can reach your conclusions relatively quickly, turning your learnings into some easy ‘quick wins’ for future campaigns.

Related: Your basic guide to email marketing

At-a-glance: Dos and Don’ts for A/B testing

Do test a single variable, ensuring everything else is the same.

Do begin your test with a theory of how things will play out.

Don’t play it safe. Testing is about turning your hair-brained ideas into the new normal.

Don’t forget what you're testing. Keep a record of your outcomes and ensure you implement the wins into your day-to-day.

Do explore time-frames for your testing. Not enough visits in a single week? Run your test over a month.

To find out how our marketing team can help with your split testing, get in touch.