Understanding AB Split Test Logging: Why Only the First Visit and Conversion Are Recorded

In AB Split Testing, consistency and accurate data representation are crucial to derive reliable results. One key aspect of this methodology is how data is logged. Specifically, AB Split Test logs only the first visit and the first conversion in a split test for each visitor. Let’s break down why this is the case.

1. Preserving Visitor Uniqueness

Every visitor on your website is considered a unique entity. When they first land on your website and enter a test, we log this as a ‘visit’. This visit is the starting point for that unique visitor within a particular test. Logging the first visit ensures that each visitor is counted only once, preventing skewed results due to repeat visits from the same visitor.

2. Maintaining Conversion Integrity

Similar to visits, we also log the first ‘conversion’ event for each visitor. A conversion could be any predetermined goal such as a click on a button, filling a form, or completing a purchase. By logging only the first conversion, we ensure that the test results are not biased by multiple conversions from the same visitor. It helps maintain the integrity of the conversion rate, providing a clear picture of how effectively a variation prompts users to take the desired action.

3. Accurate Data Representation

The practice of logging the first visit and conversion ultimately leads to a more accurate representation of data. It helps prevent data distortion and provides a true reflection of how each variation performs against the other. This allows you to make informed decisions based on reliable and unbiased results.

While it may seem limiting to only log the first visit and conversion, it is a necessary measure to ensure the accuracy and integrity of your split testing results. This way, you can trust the results of your split tests and confidently make data-driven decisions for your website.