Into the World of Hypothesis Testing Part 2: T-Test in Python

In my previous article I wrote about the key concepts of Hypothesis test. If you haven’t read part 1 of this article series check it here:

[embed]Into The World of Hypothesis Testing — Part 1: Introduction Have you ever wondered how to determine whether something you believe is actually true?medium.com

In this second part of the series, we are diving into the t-test, a simple but powerful tool that tells you whether the difference you see in your data is real. And don’t worry — we’ll walk through a super practical example using Python so it all makes sense.

When to use a T-test?

A t-test can only be used when you want to compare the means of two groups. However, it’s important that your data meets the following assumptions:

1. The samples are independent
2. The data are (approximately) normally distributed
3. The variance within each group is roughly equal

If your data doesn’t meet these assumptions, consider using a non-parametric alternative.

What type of t-test should we use?

There are three t-test types. The appropriate one depends on your hypothesis. Let’s briefly review each:

Image created by author

• One sample t-test: Compare a group’s mean to a known value.

Example: Are students’ test scores higher than 75 on average?

• Two sample (independent) t-test: Compare two independent groups.

Example: Do customers spent more before or after a discount coupon?

• Paired t-test: Compare two related groups (before/after).

Example: Did a weight-loss program reduce participants’ weight?

Let’s Make It Practical with an Example

But it’s always easier to understand with a clear example.

Imagine we’re working at a retail store, and the owner has just launched a new website for the online shop. Now, he wants to find out whether the new design is encouraging customers to make more purchases.

Our question is: Did the new website design increase the average customer purchase amount compared to the old one?

1. Setting Up the Hypotheses

• Null Hypothesis H0: “There is no difference in average purchase amounts between the old and the new website.”
• Alternative Hypothesis H1: “The average purchase amount increased on the new website.”

We will use our data to test if we can reject the null hypothesis — but only if we have strong evidence.

➯ Is t-test the right choice?

📌 We are comparing means (Averages): We want to check if the average amount spent is different between two groups.

This means that t-test is approprate for our hypotheses.

📌 Each customer saw either the old website or the new one: That means that our group are independent (not related or paired).

This means that two sample t-test is the one that we are going to use.

📌 The data is numeric (continuous): Purchase amounts are like 50$, 40$ etc.

📌 The sample size is reasonably small: The t-test is ideal when sample size is small to medium (typically around 30 or 50 per group) and population standard deviation is unknown.

📌 We are assuming a normal-ish distibution: The t-test assumes the data is roughly normally distributed, which is often okay for small samples, especially if you don’t see extreme outliers.

2. Collecting the Data

To run our t-test we need to import the necessary package, and create two lists with the purchase amounts of the old and the new website. Let’s say we collected the following sample data from 10 customers for each website version (just to keep it simple):

# Import necessary package
import scipy.stats as stats

# Old website purchase amounts (in dollars)
old_website = [45, 50, 47, 49, 46, 52, 48, 50, 49, 51]

# New website purchase amounts (in dollars)
new_website = [55, 58, 60, 53, 57, 59, 54, 56, 60, 58]

You may already notice the new website group looks higher, but let’s test this statistically to be sure.

3. Run the t-Test in Python

Now we are ready to perform an indepent t-test. We’ll use SciPy, a Python library with a function to run the t-test. It’s important to add the new_website list element first in the stats.ttest_ind() function because we want to check if the mean value of the new website is greater that the old one.

# Perform an independent t-test
t_stat, p_value = stats.ttest_ind(new_website, old_website, alternative='greater')

If you switch the order of the groups, change the alternative parameter accordingly:

# Perform an independent t-test
t_stat, p_value = stats.ttest_ind(old_website, new_website, alternative='less')

If you just want to test if they are different, regardless of which is higher:

# Perform an independent t-test
t_stat, p_value = stats.ttest_ind(old_website, new_website)

4. Interpret the Results

Our final step is to examine the T-Statistic and P-Value to determine whether we can reject the null hypothesis.

# Show results
print(f"T-Statistic: {t_stat:.7f}")
print(f"P-Value: {p_value:.7f}")

Output:

T-Statistic: 7.9499582
P-Value: 0.0000001

T-Statistic and P-value help us to to see if there is a real diference in our groups or the difference we observe is just due to chance.

• The T-Statistic is how different the two groups are, compared to the “noise” (spread of the data).
• The P-Value tells us the probability that this difference is just random If the null hypothesis is true.

T-Statistic Explained

The difference between the average purchases on the new website and the old website is 7.95 times larger than what we’d expect just by random chance or natural variability.

That’s a huge difference in statistical terms.

The bigger the t-statistic:

• The less likely the difference is due to random chance.
• The more confident we can be that something real is happening.

Note: These estimates vary based on sample size (degrees of freedom).

• With small samples (like 10 per group), a t = ±2.1 might just barely be significant at 0.05.
• With large samples (like 100+ per group), even t = ±1.98 can be significant.

P-Value Explained

• If p-value < 0.05 → We have strong evidence to reject H₀.
• If p-value ≥ 0.05 → Not enough evidence to reject H₀.

In our example the matching p-value is 0.0000001 way more smaller than 0.05 and confirms that it’s super unlikely this difference happened by chance. This strongly supports rejecting the null hypothesis, meaning the new design likely improved purchase behavior.

Image created by author

Would a t-test still work with bigger samples?

Yes! If both groups are large (say, 10000+), you’d usually get very similar results using a z-test, but in practice, t-tests are still widely used, even for large samples. They’re very flexible.

If you’re interested in learning more, feel free to follow and subscribe!

Thank you for reading! If you found this article helpful, don’t forget to give it a clap to show your support!

The contents of external submissions are not necessarily reflective of the opinions or work of Maven Analytics or any of its team members.

We believe in fostering lifelong learning and our intent is to provide a platform for the data community to share their work and seek feedback from the Maven Analytics data fam.

*Submit your own writing here if you’d like to become a contributor.*

Happy learning!

-Team Maven