WHAT IS AB TESTING?

There are 2 products you use that do the same thing. You only need to choose one. Which one should you choose? Of course, the product that gives you the most profit should be chosen. So how do we discover the product that earns us the most? We cannot choose the product we need to choose just by looking at its values and saying that it will yield higher returns for us. For accurate results, we have to calculate statistically.

This is where the AB Test comes into our lives.

The AB Test measures whether there is a significant difference between two independent samples.

Let’s get an idea by practicing.

P-VALUE

P-Value is the criterion of “Availability”. There is always a certain margin of error in transactions. Can be selected at 0.01, at 0.05. It depends on the person who will do the analysis. But the P-Value value gives us the amount of possible error in our operation. If the P-Value value is greater than the maximum error level we selected, H0 cannot be rejected. If less, H0 is rejected.

STEP 1: Defining the Hypothesis

We define a Hypothesis to clarify an unknown situation.

H0: There is no significant difference between the two products. H1: There is a significant difference between the two products.

On the above hypotheses, we accept that if p-value > 0.05, H0 cannot be rejected. This means that there is no statistical difference between the two products.

But there is a place we should pay attention to.

When the P-value is < 0.05, the interpretation that we will accept the H1 hypothesis cannot be made because H0 is rejected. This just indicates that H0 is not accepted.

STEP 2: Check Assumptions

1- Normality Assumption

Ensuring the Normal Distribution means that parametric tests can be applied. Thus, we can obtain results according to the distribution.

The pictures we see above are the Normality Assumption Tests and Results for the two products we have to choose from. We use the “Shapiro-Wilks” test for this measurement. H0 CANNOT BE REJECTED because the p-value for both is greater than 0.05. Thus, the distribution is Normal.

2- Variance Homogeneity

If the variances are Homogeneous, we can deduce the meaning that the deviation rate of our data from the mean is close to each other and regular.

The picture above measures the Variance Homogeneity of two products with each other. The “Levene” test is used for this measurement. As a result of this test, H0 CANNOT BE REJECTED, since the p-value is greater than 0.05. Thus, it is said that variance homogeneity is ensured.

Hypothesis tests vary according to the fulfillment of these two conditions. For the AB Test we are talking about, these two conditions must be fulfilled. Thus, since both of our conditions are met, we can perform our AB Test.

STEP 3: Application of AB Testing

When we come to our final stage, we have our test run after obtaining our Conditions.

Looking at our result, our p-value is greater than 0.05. So we say H0 CANNOT BE REJECTED. In other words, there is no statistically significant difference between the two products.

So how do we decide which product to leave when there is no significant difference between the two products?

Was the data we used while applying this test taken in common periods?

How are the products going lately?

It can be decided by searching for answers to questions such as, and the products can be tested again by giving a little more time.

THANK YOU

To Vahit Keskin for His Valuable Information.

REFERENCES

V🧡O — Veri Bilimi Okulu

[embed]Veri Bilimci Yetiştirme Programı Ön-Hazırlık Eğitimi Veri Bilimci Yetiştirme Programı - Ön Hazırlık eğitimine hoş geldiniz! Bir ön-hazırlık olarak tasarlanan bu eğitim…courses.miuul.com