How to AB Test in 2024

UPDATED APRIL 2024

This article is part two of a four-part comprehensive course on AB Testing, which includes:

1- An introduction to AB testing and how it may help your business

2- How to do AB Testing the right way in 2024 (this article)

3- Tools that will assist you with AB Testing

4- See the magic as it happens with a free live demo of our WordPress AB testing plugin.

The Right Way to Perform AB Testing in 2024

Now that you understand the basics of AB testing, it’s time to dive into strategy. Yes, AB testing is a fairly straightforward concept. However, AB testing works best if you’re trying to answer a specific question.

For instance, you might guess that more people will follow your CTA if you use a different verb in the copy. In this instance, you’d create an AB test campaign to try out various CTAs to see which gets the most responses.

On the other hand, you might ask yourself why your landing page isn’t converting, although you have significant traffic to the page. In this case, you’d launch an AB test to collect data regarding your visitors’ experience on the page and see which versions lead to a higher conversion rate.

Ultimately, an AB testing strategy boils down to which version of your web page/email/marketing asset will work better for your target audience.

However, running an AB test in 2024 is pointless if you don’t already know how your content is performing. You need to have some insight into how customers are interacting with your pages on the whole before you try to compare individual components.

To get the insights you need to perform an AB test expertly, you can try:

• using Google Analytics/Matomo to monitor traffic and referral sources then run heat maps, scroll maps, and other tests to evaluate how people interact with your site

• employing individual visitor session recordings using things like Mircosoft Clarity to track precisely what a person does when they land on a page

When you have a better understanding of where people are getting hung up on your page, you can begin drafting an AB test to see if you can eliminate the traffic jam.

You can create a solid AB test in ten easy steps.

1. Pick an Element to Test

Remember that the primary difference between AB testing and split testing is that AB testing compares extremely specific elements while split testing compares two entirely different options. For an AB test, you’ll want to isolate a particular aspect and compare two different versions of it.

For example, if you’re doing AB testing for a WordPress page, you might want to start by AB testing two versions of a page header. Or, if increasing your conversion rate is the goal, you can compare two versions of an above-the-fold design.

Whatever you decide to test should be relevant to the metric you want to improve.

1. Set Your Goals

You probably already have goals in mind, so this step should be easy. There’s a reason you set out to learn more about AB testing in the first place, or why else would you be here? Are you looking to increase your conversion rates, sales, or time on page?

In all honesty, you might have a more challenging time picking one metric to focus on at a time. It’s incredibly tempting just to throw all of your goals onto one test, but we guarantee that you’ll end up with clearer results if you can focus on one goal at a time.

Let’s say you decide to run an AB test in Beaver Builder. Your overall goals are to increase your click-through rates (CTRs) and generate more sales. You choose to specifically focus on your call to action at the bottom of the landing page. But if you’re focused on improving CTRs and sales, you won’t be able to tell whether the changes you made impacted CTR or sales.

Instead, you can start by focusing on CTRs and play with headline length to see if a shorter headline encourages more people to sign up for your mailing list. Once you’ve figured out which option works better for improving CTR, you can turn your attention to getting more sales (especially now that you have more people on your mailing list).

1. Gather Your Current Data

To make an accurate comparison, you’ll need to know your baseline for a given metric. Note your current data for the KPI you want to monitor, then keep it handy, so you can see if there was any change after you run the test.

Remember, even a small change means something. We’re isolating elements to put together a puzzle of the best-performing aspects, and it’s highly unlikely that one minor change is going to have an overwhelming effect on your numbers. And if it does—run with it.

1. Choose Which Page You’ll Test

The best place to start is your most important page, such as your home page or a popular landing page. Choosing a highly trafficked page will demonstrate the most significant effect and will have the best chance of providing a nuanced insight.

For example, if you’re looking to run your first split test on Gutenberg Blocks, you aren’t going to start with the page that people visit the least. You want to begin by optimizing the page that’s front and center and that minor changes will have the most impact.

1. Identify the Elements You’ll AB Test

Now it’s time to isolate the parts of your page that you want to compare. You can pick an element based on your goal (like the call to action if your goal is to improve CTRs) or based on your current traffic patterns (like playing with the length of copy in an area that visitors don’t seem to scroll past).

If you’re unsure where to start, choose the elements that you think are more likely to influence the target metric. Elements to AB test include (and are certainly not limited to):

• Headlines
• Calls to Action
• Buttons
• Forms
• Featured Images

Although there are endless opportunities for AB testing, your tests will be the most effective if you only focus on one factor at a time. For example, if you’re doing an AB Test on Bricks Builder and want to focus on the above-the-fold section of your home page, it’s best to perform separate tests for your headline, image, and call to action button instead of comparing all three at the same time.

1. Create Your Variant

The good news is that half of your test is already set up, as it’s the page that is currently in action. For your test, you’ll alter one aspect of the page as it is.

For example, if you’re performing an AB test on a call to action on your WordPress home page, you might create a variant that has a different background color, font color, or button size. Everything else will remain the same so you can see if the change is effective.

1. Employ an AB Testing Tool

You can administer the test yourself, but let’s be honest. You have a billion other things to do while running your business outside of sitting on the computer monitoring metrics.

And because AB testing is a highly nuanced procedure, you don’t want to run every single test manually. At this point, you’ll want to bring in a tool that can automate the process.

The best testing tools operate as a plugin so you can seamlessly integrate them with your preferred website builder. You’ll want a tool that is easy to use, focuses on privacy and compliance, offers unlimited tests, and automatically implements the winners from your tests on your site.

Our AB Split Test Plugin comes with all of these features and is compatible with the top site builders, including:

• WordPress
• Beaver Builder
• Elementor
• Bricks Builder
• Gutenberg Blocks
• Shortcodes

But we’re getting ahead of ourselves. In part four of this course, you’ll have the opportunity to try our app in real-time.

1. Design Your Test

With your testing tool ready to go, it’s time to set up your test. You’ll pit your current site against the variant and configure the details, such as how long the test will run and which devices you’ll use to collect data.

For instance, if you decide to run a split test in Elementor, you might set up your tool to gather intel from mobile site visitors for four weeks.

1. Let Your Tool Gather the Data

Now comes the part where you have to be patient. Once your test is up and running, you have to allow your AB testing tool to do what it does best: collect comparison data.

If you get antsy and want to take a sneak peek, you can always check on the test analytics, but once the test concludes, you’ll get a comprehensive report on the consumer engagement with the two versions.

1. Evaluate the AB Testing Data

Now that you’ve run your test, you can review the results to see which version performed better. In many cases, your tool will help you analyze the results and provide a crystal clear picture of which variation is better for your site.

And voila! You’ve completed your first AB test! Now you get to repeat the process over again with a different goal, element, or metric. Since consumer habits change frequently, AB testing is an ongoing procedure. Think of it as a cycle rather than a one-and-done technique.

How Long Should You Run an AB Test in 2024?

When it comes to AB testing, quantity is more important than duration. This is why it’s crucial to run AB tests with a prominent site page. If you run an AB test for multiple months, but you only get five visitors to each version of the site, you won’t be able to gather enough information to conclude which variation works best.

At the same time, you want to run the test for long enough to ensure that there isn’t any evidence of results convergence. Results convergence occurs when there appears to be a significant difference between two variations at the beginning of the test, but the difference decreases over time. No convergence means that the change had a notable impact on the metric you’re evaluating.

As long as you have steady traffic on your site, AB testing for several weeks will give you the information you need to identify which version performs better.

You Concluded Your AB Test—Now What?

Once your AB test is over, you want to hyper-analyze each component of the tests. You’ll evaluate aspects such as:

• How many visitors each variation received
• How many visitors were on the desktop vs. mobile site
• How long each visitor stayed on the site
• How often a specific action was taken
• By how much the winner outperformed the loser

If you’re using an AB testing tool, the analytics portion of testing is a breeze. Your tool will automatically compile this data for you, and in many cases, will implement the better-performing variation at the end of the test.

As you move forward with different AB tests, you’ll be able to put together a gorgeous site that’s optimized for the ultimate user experience from end to end.

Conclusion

Setting up an effective AB testing campaign is easy, but creating your test with specificity is critical to get the maximum benefit out of your time.

Like we said, doing all of this on your own becomes incredibly time-consuming, and using an elite tool will save you a great deal of time and effort as you work to optimize your site. In part three of our course, we’ll discuss various tools to help you execute expert AB tests.

If you’re ready to get started with AB Testing today, you can skip ahead to our hands-on demo here.

See you next time.