Sep 04, 2023 12:10:00

I tried using ``GrowthBook'', which allows free A/B testing and gradual rollout of new features, and is open source and self-hosted.

When several website design proposals come out, it is difficult to think which one is the best. ``

A / B testing '' that displays two pages at random and measures changes is useful for optimizing such elements. GrowthBook not only allows you to easily implement A/B testing, but it is also open source and can be self-hosted, so I actually tried it out.

GrowthBook - Open Source Feature Flags and A/B Tests
https://www.growthbook.io/

Visit the official website and click 'Get started for free'.

Choose whether to deploy the Cloud version or the self-hosted version. Even with the free plan, there seems to be no limit on the amount of traffic or the number of A/B tests that can be set up, so this time I decided to try using the Cloud version. Click 'Create a free account'.

Click Register.

This time I will register using a Google account. Click 'Sign up with Google'.

Select the account you use to log in.

Now you can register with GrowthBook. Since it is necessary to create an organization, enter a suitable name and click 'Create organization'. The name entered here can be edited at any time later.

Click 'Step 1: Create a Feature Flag'.

Create a Feature by entering an appropriate name in the Feature Key field. I named it 'first-feature' this time. Click 'Next: Install an SDK'.

Select the app name and the language and framework you are using and click 'Continue'.

Since the installation method is displayed, it is OK if you refer to the displayed code and introduce it to the application.

If you are using Javascript and are not using the bundled tool, you can install it using the code provided on

the JavaScript page of the documentation .

After installation, click 'Next: Check Your Connection'.

If the initialization code is working properly, it will show 'Connected'. Click Next: Add a Data Source.

Some people may have guessed this from the fact that GrowthBook's fee structure does not charge for traffic volume, but GrowthBook does not manage its own data, but instead acquires and analyzes data from access analysis tools such as Google Analytics and

Matomo . to hold. GrowthBook's job is to select features on and off, distribute A / B tests, and analyze results. This time, we will use a sample data set to see how the analysis will be performed, so click 'Use Sample Dataset'.

Click Create Demo Datasource Project.

'GrowthBook Demo Project' is added to the Project list on the upper left, so click to switch.

When you look at the 'Features' tab, the Feature 'gbdemo-checkout-layout' is displayed, so click it.

Two rules are set in 'gbdemo-checkout-layout'. The first rule is ``If you are an employee, set the value of gbdemo-checkout-layout to dev'', and the second rule is to test all accesses by dividing them into three patterns.

According to

the guide , GrowthBook allocates Feature values, so the app needs to record the values and change the display. For example, when using JavaScript with Google Analytics, set ``trackingCallback'' when initializing GrowthBook as shown below, and send data to Google Analytics ``what feature value occurred''.
[code]let gb = new growthbook.GrowthBook({
apiHost: 'https://cdn.growthbook.io',
clientKey: 'sdk-abcd1234',
attributes: {
id: gbuuid
},
trackingCallback: function(experiment, result) {
gtag('event', 'experiment_viewed', {
event_category: 'experiment',
experiment_id: experiment.key,
variation_id: result.variationId,
gb_user_id: gbuuid,
});
}
});[/code]

Then, get the value of Feature with the code below and change the page display based on this value. If the acquisition fails, the value of the second argument is returned.
[code]const featureValue = gb.getFeatureValue('gbdemo-checkout-layout', 'current');[/code]

You can also view the analysis results by clicking 'View details and results'.

Mean values and 99% confidence intervals for 'Dev-Compact' and 'Dev' are shown with 'current' as the baseline. In this way, multiple novel patterns can be tested simultaneously and analyzed without problems.

If you look at the 'Overview' tab, you will see a page where you can write a summary of the test and attach a screenshot. Even if you perform a long-term test, such as when the number of accesses is small, you can record what kind of test it was, so you don't have to forget it.

In addition to A/B testing, there is also a function called ``Percentage Rollout'' that allows you to introduce and test new features only to a certain number of users, so let's set it up. Go to the 'Features' page and click 'Add feature'.

Enter the Feature Key. This time I chose 'rollout-test'. After that, set Value Type to 'boolean' and click 'Create'.

Click Add Rollout Rule.

The standard value is set to off, so turn on 'Value to Rollout', set 'Percent of Users' to 20%, and click 'Save'. Now 'on' is delivered to 20% of users, and 'off' is delivered to the remaining 80% of users.

Place the following code on your website. The if statement is executed only when 'on' is delivered.
[code]if (gb.isOn('rollout-test')) {
// Enter the processing when it is on here
}[/code]

To use data from your own site or app in GrowthBook, select 'All Projects' from the project selection in the upper left, and click 'Add Data Source' from the 'Data Sources' page under 'Metrics and Data.'

Click 'Guided Setup'.

As you can see below, it supports a number of event trackers. You can select the one you are using on your site or app from among these and follow the guide to import the data.

The event trackers supported by GrowthBook are as follows. Even if it is not described here, you can import data into GrowthBook by creating some SQL queries on your own .

・Segment
・RudderStack
・Mixpanel
・Snowplow
・Amplitude
・Google Analytics 4 (BigQuery only)
・Firebase
・Fullstory
・Freshpaint
・Matomo
・Heap Analytics
・Jitsu
・MParticle
・Keen IO
・CleverTap