This is a transcript of episode 19 of the OMGrowth podcast, published on March 24, 2021



A/B Testing can be done with a lot of things: email subject lines, your graphics, heck, you can split-test anything on your website. But when should these become part of your digital marketing and optimization strategies?

Today, we’re talking about when A/B TESTING is a useful research method for you to use, the best practices for implementing these experiments, and when you can skip them altogether.

WHAT IS A/B TESTING?

A/B Testing – which is also known as split-testing – is a user-focused statistical experiment that randomizes and tests two variants – so variant A and variant B – to test a hypothesis.

Which is a very hoity-toity way of saying…

When you want to know what will get you the best response and engagement from your audience, A/B testing can help you figure that out.

This is a method of optimization that you have available to you as a digital marketer and it can be applied to many different areas.

But like any tool or method you have available to you, “available” doesn’t always mean it’s useful or it will best serve you.

So let’s take a look at some of the most popular use cases that online entrepreneurs such as your fine self can use split-testing as a means of optimizing your own marketing campaigns.

ABOUT A/B TESTING EMAIL SUBJECT LINES

Let’s start with the place that you’ve likely already encountered an option to split-test content: your email subject lines.

Most email marketing services offers the ability for you to use 2 different subject lines on the same email broadcast or newsletter to see if the use of one subject line would have a bigger impact on your open rates than another.

First thing is first:

If you have less than 1,000 people on your email list, you can skip A/B testing your subject lines.

I say this because unless you have at least 1,000 subscribers, your sample size won’t be statistically significant enough for you to truly see anything meaningful regarding the performance of different subject lines.

Let’s do the math for an email list of less than 1K subscribers: if you have a typical open rate of 25%, that means less than 250 people will be opening your email and when you split that, you’re looking at fewer than 125 people receiving each subject line.

This would be a minimum amount of people you would need to be assessing to really make any meaningful or statistically significant analysis about your subject line trends.

But having a smaller list doesn’t mean you can’t optimize your email marketing metrics.

You can look through past email subject lines you’ve used and compare which ones performed best, then test your assumptions with the new subject lines you generate to see if there are trends or commonalities your audience responds to.

Another thing you can work on improving are your click-through rates and again, identifying which emails got the best click-throughs to your website or sales pages, and why.

If you want to start seeing WHAT elements of your emails people are clicking through on – like, are people more likely to click a button, some in-line text or an image? – you can use UTM parameters to start seeing those patterns.

Now, let’s say you do have a large enough audience to generate statistically significant results: I still recommend you use UTM parameters to see what people click-through on because most email marketing services can’t help you test and optimize for that.

Any time you’re comparing data sets – which is exactly what split-testing is all about – you aren’t starting with the test; you’re starting with the question you’re trying to answer.

So when it comes to your email subject lines, you want to be intentional about what question you’re trying to answer:

  • Do my subscribers enjoy or are they turned off of emoji use?
  • Does including a “time sensitive” header actually improve or does it hurt my open rates?
  • Do informative subject lines perform better than cheeky ones?

The list of things you could be testing for in your subject lines goes on and on… but you do have to hone in on specifically WHAT you’re testing for if you want results you can start building an optimization strategy from.

Because remember: this IS an optimization strategy and optimization is never a one-and-done process. You’ll have to repeat this whole process over multiple emails you send with that same question and intention in mind to ensure that this actually is a trend and of significance to your audience.

ABOUT A/B TESTING GRAPHICS

Another area you can test – and this one may actually be more widely accessible because you’ll be able to reach a much larger sample size – but your graphics and visuals can be split-tested.

Two places you can split-test your brand visuals is with your YouTube thumbnails and on your Pinterest feed.

Why bother? There are plenty of reasons why you would want to do this:

  • Most obvious is that by understanding which visuals work and which don’t, you can improve your viewership and click-throughs; but
  • You can also take the insights from how these visuals perform and apply them to other areas, such as your ads, your social posts and your promotional materials.

So how do you do this?

With YouTube, you could swap out the image you use at 12-24 hour intervals and see which performs best. However, you can also automate this by using a platform called TubeBuddy that – much like your email marketing service will split test your subject lines – it will split-test your images and let you know what the highest performers are.

As for Pinterest, you would create two or more different pins to promote the same post – which is recommended that you do in any case because Pinterest loves when we use what it calls “fresh pins” meaning new images to re-promote content – but then you would login to your Pinterest analytics to see if maybe a theme emerges for what your top performers are in terms of shares and clicks.

ABOUT A/B TESTING WEBSITE CONTENT

Another area you could A/B test would be, well, your entire website – no biggie, right?

A lot of landing page builders like LeadPages will allow you to A/B Test different pages to see if maybe button color impacts your performance, or the language used in your headline, or perhaps the positioning of your images.

There’s no lack of things you can and may want to test… and you’re not limited to sales pages or platforms, either.

You can use a free service called Google Optimize to do this.

With Google Optimize, you can A/B test anything on your website – from button colors to fonts to navigation bar placement to the copy you’re using. Anything!

But with great power comes great responsibility… or in this case, with this many options, you had better be coming at split-testing with great intention!

I love examples so let’s look at what would be a good one for you to use Google Optimize with:

—> Let’s say you recorded a webinar and you want to promote FREE WEBINAR in your navigation bar… but you’re not sure if you should use FREE WORKSHOP or FREE WEBINAR. <– Provided you get enough traffic to collect some statistically significant information, this can be a useful option for you to split-test and start using in your promotions for other areas, like your ads and social media.

With so many things available for you TO improve, you’ll only be in a position to make those improvements if you’re super-mega-clear and intentional with what you’re testing for and how you’ll be applying your findings.

Which brings us to this week’s ACTION ITEMS:

Because when it comes to split-testing, well, anything in your business… because, hey! we’ve established that you CAN test just about anything but let’s make sure it’s worth your time, energy and effort to do so, amirite?

1) Have something to prove.

Whether you prefer to call it A/B or split testing, it’s all testing… and this means you’re putting a thought or a theory you have to the test.
You’re not doing this just “because you can” or for funsies; you’re A/B testing when you’re looking to prove or disprove a specific outcome.

—> Are my subscribers most likely to click-through on funny subject lines or question-based subject lines?
—> Do I get more YouTube views or Pinterest clicks from images with font A or font B?
—> Does changing the color or text on my sales page button increase the likelihood of visitors to initiate checkout?

The more specific you are about the elements which you’re testing and the outcomes you’re investigating, the more useful and applicable the results of your experiment will be… not only to the thing you’re testing but to all areas of how you’re promoting yourself.

2) Check in with your choice of optimization strategy.

Because look, if you aren’t getting enough traffic to your website for the numbers to tell you anything meaningful, split-testing isn’t the optimization strategy for you right now; but getting more traffic to your website is a great thing for you to focus on optimizing!

Samesies for your email list: if you have a small list, you’re better off putting your focus on growing that list and THEN you can delve into the world testing what those subscribers respond best to.

A/B testing is like a lot of tools and strategies you have available to you: they can be all-kinds-of useful when you use them with intention, and they can also be yet another shiny object that is distracting you from actually making meaningful improvements.

If you have a question to answer or a hunch to confirm regarding how your audience relates to your content – by all means! – implement A/B testing to get your answers and confirmations.

But if you’re using them without a strategy or intent, or you don’t have the reach to gain valuable insights from your data set, try using and applying your limited time, energy and focus into improving something you’ll make bigger gains and return from.