Running Your First A/B Test: Step-by-Step

This class is your first full end-to-end experiment. The goal is not perfection-your goal is a clean test, a correct read, and a repeatable workflow you can run every month.

Step 1: Pick one page and one goal. Choose a high-traffic, high-leverage page (pricing, checkout, lead form). Choose one conversion goal (purchase, trial signup, lead submit).

Step 2: Diagnose the problem with evidence. Use AB Split Test heatmaps + session replays + analytics to find the friction point. Write the observation in plain language.

Step 3: Write a hypothesis.

• Because: (evidence)
• If we: (change)
• Then: (expected outcome)
• We’ll measure: (primary metric + guardrail)

Step 4: Build the variation. Start with the smallest viable change (copy, layout, proof, CTA). Keep everything else constant.

Step 5: Set traffic and duration. Use 50/50 for your first test. Pre-commit to a minimum run time and sample size. Don’t stop early because it “looks good.”

Step 6: QA and launch. Check both versions on mobile/desktop. Confirm tracking and the conversion goal.

Step 7: Read results correctly. Look at conversion rate, sample size, and chance of winning/confidence. Decide: ship winner, iterate, or learn-and-move-on.

AB Split Test workflow: Create the test (element or page). Add goals (and subgoals if needed). Use built-in reporting + AI CRO chat to summarize: “What happened, why might it have happened, and what should we test next?”