Do I need to read stats or use math to calculate the winning test result?
No. AB Split Test handles all of the statistical analysis for you. You do not need to understand statistics, run calculations, or use external tools to read your results.
How winner detection works
AB Split Test uses a Bayesian statistics engine called Autocomplete to monitor your test data continuously. When a variation reaches the required confidence threshold, the test completes automatically.
Three conditions must all be met before a winner is declared:
- One variation has reached 95% confidence or above
- The test has been running for at least 1 week
- Each variation has received at least 50 visits
The minimum visit requirement was added in v2.5.1 to prevent underpowered tests from declaring a winner too early on low-traffic pages.
What you see in the results
While the test is running, AB Split Test shows you:
- Visit count per variation
- Conversion count and conversion rate per variation
- Uplift percentage compared to the control
- Confidence level, updated continuously as new data arrives
You do not need to interpret these numbers yourself. The plugin tells you when the result is statistically ready.
What the Underpowered badge means
If your test shows an Underpowered badge, it means there is not yet enough data to make a reliable decision. This is normal on lower-traffic pages or early in a test run.
The right response is to let the test keep running. Stopping early and picking a winner based on limited data is one of the most common mistakes in split testing.
When a winner is found
When Autocomplete detects a winner it hides the losing variations, shows the winning variation to all visitors, sends an email notification to your admin address, and marks the test as Complete.
As of v2.6.0 the test list and edit screen also display a notice when a winner has been found but autocomplete is off, so nothing is missed.
Optional: optimize continuously with Multi-Armed Bandit
If you prefer not to wait for a definitive winner and want the plugin to shift traffic toward the better-performing variation in real time, you can enable Dynamic Traffic Allocation (Multi-Armed Bandit) in Settings > Advanced.
This is an alternative to Autocomplete, not an addition to it. It is best suited to high-traffic sites where continuous optimization matters more than a clean statistical conclusion.
Still unsure about your results? Contact support and we will walk you through them.