FROM THE BLOG

AB Testing With WordPress – a complete guide

Note: The video below is from 2022. The core CRO principles still apply, but the updated guide below reflects current best practices.

Most people think split testing doesn't work. They're wrong, but they're also not entirely to blame.

Here's the problem: most people start at step 3.

They jump straight to "let's test a blue button vs. a red button" without understanding why they're testing it, what problem they're solving, or whether it even matters.

That's not split testing. That's guessing with extra steps.

Real conversion rate optimization (CRO) has three steps:

  1. Analyze what you've got
  2. Create theories based on data
  3. Split test those theories

Miss the first two, and your tests will fail. Follow all three, and you'll consistently improve conversions.

Let's walk through the exact playbook we use in our agency to repeatably create tests that work.

Why We Built This Process

We run a website agency for IT service providers. Once a site goes live, we're basically a standard website management service. Nothing special.

But we wanted to be more valuable. So we focused on:

  • Improving customer conversion rates (if they make more money because of us, they stay)
  • Increasing agency stickiness (we wanted to be irreplaceable, not just another WordPress host)
  • Reducing dependency on ads (make more money with less traffic instead of constantly buying more)
  • Privacy (collect as little data as possible while still delivering results)

We tried every CRO tool out there. None did what we needed. So we built AB Split Test and used it to develop this repeatable, data-driven process.

And here's the key: This works whether you're an agency, a freelancer, or running your own business.

What the Hell is CRO, Anyway?

Conversion Rate Optimization isn't just split testing. It's the entire process of improving your website.

Split testing is step 3. Most people skip steps 1 and 2, then wonder why their tests fail.

Here's what real CRO looks like:

Step 1: Find Real Problems

Not your problems. Not what you think needs fixing. Real problems your visitors are experiencing.

Step 2: Suggest Solutions

Create theories based on data. Use AI, copywriters, or your own ideas. But base them on actual visitor behavior, not gut feelings.

Step 3: Test Those Solutions

Let science tell you what works. Don't get in the way of facts.

Step 4: Implement and Repeat

Roll out winners. Find the next problem. Keep optimizing.

Notice what's missing? Feelings. Creativity for its own sake. Guessing.

This is a repeatable process. That makes it perfect for agencies with teams, or solo operators who want consistent results.

What Does AI Have to Do With This?

AI is a shortcut out of your own bias.

Here's the problem with creating split test variations yourself: you already have a favorite.

Let's say you want to test button text. Deep down, you believe "Buy Now" will win. So you create:

  • A really compelling "Buy Now" button
  • A mediocre "Checkout" button

You've already ruined the test before it even starts.

With AI:

  • You say: "I want more checkouts"
  • AI suggests 5 variations
  • None of them are yours, so you're not emotionally attached
  • You test them objectively

You can also use a copywriter or your own skills. But AI is fast, unbiased, and doesn't get tired after writing 50 variations.

Step 1: Ask for Problems

Most people start with an idea. "Maybe I need more buttons in my footer!"

But what if no one's even scrolling to your footer? You just wasted time testing something irrelevant.

Instead, ask three questions:

  1. "Did you find everything you needed today?"
  2. "What could we have done better?"
  3. "Is there anything I can help with?"

These questions make it easy for people to respond quickly.

You'll get gold like:

  • "Where's your pricing?"
  • "Can I book you weeks in advance?"
  • "Your search sucks."
  • "Do you accept credit cards?"

These are either things you have on your site but visitors can't find, or things you're missing entirely.

This is where real optimization starts.

Step 2: Watch for Problems

After getting feedback, watch people use your site.

We love Microsoft Clarity (it's free). Install a snippet, and you get:

  • Session recordings (watch real visitors navigate your site)
  • Heatmaps (see where people click, scroll, and rage-click)
  • Click-through rates and scroll depth

What to look for:

Red Flag #1: People Not Scrolling

If visitors aren't scrolling past the fold, your content isn't compelling enough. Move key info higher or make your opening stronger.

Red Flag #2: Mouse Flicks

When people think they know everything, they flick their mouse wheel and scroll fast. They're missing your important info.

Red Flag #3: Click Rage / Tap Rage

Rapid clicking or tapping means frustration. Something's broken or confusing.

Red Flag #4: Weird Navigation

If someone hits checkout, then goes back to the homepage, then pricing, then checkout again... something's wrong in your funnel.

Red Flag #5: Low Click-Through Rates

If your CTA isn't getting clicked, it's either:

  • Not compelling enough
  • Not visible enough
  • Not trustworthy enough

Other tools you can use:

  • Google Analytics
  • Fathom
  • Matomo
  • Plausible

Or just watch real people:

  • Buy someone a coffee and screen-share
  • Sit next to them at a coffee shop (if you're braver than us)
  • Ask a stranger to use your site

Pro Tip: If It's Obvious, Just Fix It

If someone's clicking a button that goes to the wrong page, you don't need to split test that. Just fix it.

Split testing is for things you're unsure about. If the problem is obvious, skip the test and move on.

Step 3: Theorize Solutions

Now you've got data. Let's create a test.

Example from our site:

The Problem:

People weren't clicking our footer CTA: "Test it yourself. Fire up a free sandbox site."

How we found it:

  • Asked: People were asking in live chat if they could try the software
  • Watched: Microsoft Clarity showed mouse hovers over the button, then people moved on
  • Theory: The CTA felt like too much commitment (people thought "demo" meant Zoom call or credit card)

The Solution:

Test different CTAs that feel lighter and easier:

  1. "Test it yourself" (original)
  2. "Sample data included"
  3. "Interactive demo"
  4. AI-generated: "Try it out now, risk free"

Step 4: Measure It

Keep goals simple.

If you're eCommerce:

  • More products sold
  • More items per cart
  • Higher average order value

If you're SaaS or service-based:

  • Demo views
  • Email signups
  • Call bookings

In our example: We measured clicks through to the demo page, because we know that's the key step in our funnel.

Step 5: Create the Test

Now, and only now, do we create the actual split test.

Tools you can use:

  • AB Split Test (our plugin, best for WordPress agencies)
  • Google Optimize (free)
  • Crazy Egg
  • Split Hero
  • Nelio

Here's how we did it:

  1. Created 4 variations of the footer CTA
  2. Used Bertha AI to generate one variation (saved us from creative burnout)
  3. Set the conversion goal: clicks to demo page
  4. Let it run

The AI version?

  • Headline: "Try it out now, risk free"
  • Description: "See how much you can save on your conversion rate optimization"

We didn't have to think. AI gave us something fresh. We liked it. Done.

Step 6: Wait and Watch

Here's the hard part: Split testing takes time.

  • Not days. Weeks. Sometimes months.
  • Life is cyclical (weekends aren't like weekdays, end of month isn't like start of month)
  • You need hundreds or thousands of visits before you have enough data

Don't jump the gun. A couple early conversions don't mean you have a winner. Let the test run until it's statistically significant.

Good tools (like AB Split Test) will:

  • Track statistical significance automatically
  • Email you when there's a clear winner
  • Auto-implement the winner so all visitors see the best version

Step 7: Implement and Repeat

Once you have a winner:

  1. Roll it out site-wide
  2. Go back to Step 1
  3. Find the next problem

This is a continuous process, not a one-time project.

Real Results

Here's what happens when you follow this process:

  • 10% increase in demo conversions (if a demo is worth $500, that's huge)
  • 23% conversion lift by adding testimonials to checkout (Whistler Bag Storage case study)
  • Better customer retention (clients stay when you're making them more money)

From a customer:

"There's too much data in marketing. This helps me focus on one thing without distraction."

That's the whole point. Simple. Repeatable. Fast.

Your Turn

Stop guessing. Start optimizing.

  1. Ask for problems (3 questions to your visitors)
  2. Watch for problems (Clarity, GA4, real user testing)
  3. Theorize solutions (data + AI = unbiased variations)
  4. Measure clearly (one simple goal per test)
  5. Create the test (use any tool, we recommend AB Split Test)
  6. Wait for significance (weeks, not days)
  7. Implement and repeat

That's it. No magic. No guessing. Just a repeatable process that consistently improves conversions.