What is the usage of the A / B test which the technology manager of the net service talks in light of the actual example?

ByRicardo Alguacil

Service comparing travel sites "KAYAKThe technical director ofA / B testI explain the way of approach to the example with the actual example.

Apptimize | KAYAK'S Most Interesting A / B Test
http://apptimize.com/blog/2014/03/kayaks-most-interesting-ab-test/

Kayak's technical manager Vinyak Larnadey will provide A / B test service for mobile applicationsAPPTIMIZEIn a dialog with Nancy Hua 's CEO, he talks about KAYAK' s approach to A / B testing. Mr. Lunadey and Mr. Hua are MIT alumni.

Mr. Hua asked, "What is the most interesting A / B test?" Mr. Larnade talked about the unexpected result of the A / B test done with Kayak's mobile terminal application. According to Mr. Larnadee, when using mobile applications, people are cautious about purchasing products and services more than when using PC. When putting the item in the cart and tapping the button "Purchase" at the end, "Is there a communication error when tapping this?" "Is information leaked out?" "Wi-Fi is hijacked As security concerns will arise such as "Do not do it?", Mr. Laraday says that he / she is hesitant to give up and go abroad from there.

ByLab 604

In order to eliminate the concern of this user, there was discussion inside Kayak regarding whether to display the phrase "This communication is protected by encryption and is secure" under the button "Subscribe". In general, since users are not always concerned about security, it is a matter of reminding the user of security concerns rather than displaying such information, so it is better not to display them He said that he was deeply rooted.

However, since it is also the significance of the A / B test to check whether such customs and common sense in the industry is correct, Mr. Lunaday uses two types of things, / B test was conducted. Then, the result showed that security safety was betterConversion rateIt was high.


It is natural that such a result does not apply to every service, and Mr. Laraday thinks it is important to trial and error and actually try out the A / B test.

Also, the opinion that Larnadee asked, "What kind of problem are you conducting the A / B test? What are the standards?" , Senior executives, engineers, sales representatives, users, and so forth, and said, "After collecting all of them, it is a big difference that can make a big difference between small and small small differences We will try it while considering the balance between the two, and we will look for something with significance. "As for the criteria for A / B testing, it affects the profitability of the company We respond that they are focusing on three perspectives: things that have impact, those that are on an extension of the improvement that has been ongoing so far, and those that can bring something fundamentally new to the company by trying it I will .

ByTorsten Werner

Meanwhile, Mr. Larnadey is asked about a scene of design change that does not rely on the A / B test and introduces a development example of KAYAK application of iOS 7 version which was recently. Changes from iOS 6 to iOS 7 are very big, and because the announcement of official release was urgent, KAYAK's iOS application developers were forced to make quite large-scale design improvements. When you encounter such a situation, it usually takes time to change each design individually, so it will take time to change the design approximately for about three months, whereas KAYAK believes in the excellence of traditional design Lahnady said that he rebuilt the design at once in half the six weeks. The result is that KAYAK made a big profit thanks to the release of the application ahead of other companies, with a lot of developers taking time to change the design of the application.

In addition, Mr. Lunadey said that "It is a lot of fun to see designs that survived survival by A / B test," he said that they refined the application design by repeating the A / B test for several months afterwards .

Occasionally, you may come across scenes where bold design changes should be made without relying on the A / B test data. However, even in the case of A / B tests, even in the scenes that believe in experience and intuition, "what seems to be good for the user" "what seems to be good for the company overall" "ultimately want to do this I am thinking that it is important to make a design change based on the criteria that "it is suitable for goal".