Seven mistakes that tend to be averse in the A / B test

ByLauren Macdonald

It is a method of determining more excellent design by preparing different designs and comparing user's reactionsA / B test"Is widely used at the site of website construction. In executing such A / B tests, seven mistakes that are prone to odds are being made public.

Seven A / B testing mistakes you need to stop making in in 2013 - Visual Website Optimizer Blog
http://visualwebsiteoptimizer.com/split-testing-blog/seven-ab-testing-mistakes-to-stop-in-2013/


Provide tool dedicated to A / B testVisual Website Optimizer (VWO)I will explain the failure that tends to be a cause for poor results in the A / B test on the company's official blog. The seven mistakes that you are accidentally inadvertent of doing A / B tests are as follows.

◆1: Do not consider the number of samples

ByAn & amp;

Many people have not considered at all what the required number of samples is before executing the A / B test. However, as it is natural that a certain number of population is necessary to generate statistically meaningful results, before executing the A / B test, even if a dedicated tool is used, the appropriate number of samples is calculated It is recommended that you prepare.

The graph below shows the results of a certain A / B test, the green design is the first in the beginningconversionAlthough it was high, when the number of samples increased as time passed, the result was reversed. It is often that you misread the results of the A / B test if you do not assume an appropriate number of samples in advance.


◆2: to value HiPPO's opinion


Google, who created the opportunity to inform the world of the existence of the A / B test, contempts the opinion of the boss that is not backed up by data as "HiPPO (Highest Paid Person's Opinion)". In the IT industry not only in web design, it is common for top opinion to ruin the project. The A / B test, which is a method of analyzing objective data, should originally have the role of eliminating harmful HiPPO, and from the results of the A / B test it is necessary to avoid placing emphasis on HiPPO It is.

◆3: Capturing the test results "conveniently"

ByFrits Ahlefeldt-Laurvig

It is common to be tempted to want to twist the conclusion by making a convenient interpretation without trusting the test results. However, VWO criticizes such a posture, "For what purpose did you test if you do not plan to accept the test?" Do not adopt the test result means that it means stopping the progress and stopping the progress.

◆4: Being trapped by small changes

ByImages Money

To listen frequently about the power of the A / B test is "a great success is born from a small difference." However, if you believe this word too much, you will try only small design changes. However, VWO points out that it is important to challenge major changes sometimes, as it is possible that a completely new design may result in far greater results.

A thing that can not see the "big mountain" rising in the distance if you are looking at only nearby.


◆5: Testing without limiting target

ByPete

Although there is no dust if you mention the subjects for A / B tests, it is important to test which part is to be changed, targeted.

◆6: Test without making a strategy

ByJoriel "Joz" Jimenez

A / B testing should be avoided without setting a strategy of "what kind of results are the goals, what to improve and how to improve"? And a good way for strategy planning is to collect all the steps one by one by consolidating the opinions of all stakeholders.

◆7: To be unsatisfied

ByDixie Lawrence

There is no end to the A / B test, so there is no goal of satisfaction with this. Therefore, it tends to forget to evaluate achievement and rejoice success. However, being pleased with the results obtained is a key to effective and continuous A / B testing.