Gaussian Mixture Models (GMM) Explained: A Simple Guide

Learn how Gaussian Mixture Models work in machine learning, how they use soft clustering, and why they go beyond simple normal distributions.

GMM explained | Image generated by AI. Google, 2026. https://gemini.google.com

A Gaussian distribution is what we also know as the Normal distribution. You know, that well spread concept of a bell shaped curve with the mean and median as central point.

Given that, if we look at a dataset, even if it does not follow a Normal distribution, we have ways to make it resemble one. Or, even better, we could say that within a dataset, there could be many Normal distributions where each data point comes from.

Now imagine that we are getting a dataset to create three clusters. One way to do that is to suppose we have three Gaussian (Normal) distributions within it. Then, we will look at each data point and analyze in which one of those distributions the data point fits better.

Three Normal Distributions. Image by the author.

Looking at the picture above, let me illustrate the concept. What the GMM algorithm does is to consider each Gaussian Distribution as one cluster. Therefore, it will take each data point and check what is the probability of that point being in each of the 3 distributions. The higher will be the cluster chosen for it.

GMM considers each cluster as a different Gaussian distribution. Then it will tell, based on probability, out of which distribution that data point came out.

Probably the most known and used algorithm for clustering is K-Means. But it has its limitations. One of them is that the clusters will be separated based on an optimal radius value from the cluster center, calculated based on the Euclidean distance to the point. Ergo, if your cluster is not defined as a circular shape, you can find trouble to separate it properly. GMM, on the other hand, works with other formats, being the elliptical shape the most common.

How GMM Works

Right, after we’re done with some initial concepts, let’s understand basically how the algorithm works behind the scenes.

Just like the K-Means algorithm, GM model has to start somewhere, using a random parameter and build it from that. Once that initial parameter is chosen, the algorithm starts a series of calculations trying to find the best weights and means needed to converge the estimated Gaussian Distributions to meet the ones with the initial parameters. In other words, choosing under what distribution each data point will fall.

For that, there’s this hyperparameter called init_params where you can choose between ‘random’ points or use the default 'kmeans' . Using K-Means can help the convergence to happen faster, thus I would leave the default on. I will suggest some reference posts at the end for those who want/need to go deeper in the crazy math behind this algo.

The next interesting parameter here is the covariance_type , where you can choose between ‘*full’, ‘tied’, ‘diag’, ‘spherical’.* To help you understanding this parameter, below is the best explanation I could find for it. But before you go there, remember that K-Means clusters points using only the circular shape.

• Full means the components may independently adopt any position and shape.
• Tied means they have the same shape, but the shape may be anything.
• Diagonal means the contour axes are oriented along the coordinate axes, but otherwise the eccentricities may vary between components.
• Spherical is a “diagonal” situation with circular contours (spherical in higher dimensions, hence the name).

Let’s get Coding!

Enough of the talk, let’s now do something.

Starting with the dataset, I will use the toy dataset native from Seaborn: tips.

import seaborn as sns
df = sns.load_dataset('tips')

Other imports needed for this example.

# Basic
import pandas as pd

# Viz
import matplotlib.pyplot as plt
import seaborn as sns

# KMeans
from sklearn.cluster import KMeans

# Gaussian Misture Model (GMM)
from sklearn.mixture import GaussianMixture

I will input the variables total_bill and tip to the algorithms and see how K-Means and GMM make the clustering, so we can assess differences.

Notice that none of them is not normally distributed.

Distributions of the tip and total_bill variables. Image by the author.

Creating the input dataset for clustering.

X = df[['total_bill', 'tip']].copy()

Clustering with K-Means.

kmeans = KMeans(n_clusters=2, max_iter=600)
fitted = kmeans.fit(X)
prediction = kmeans.predict(X)

Clustering with Gaussian Mixture Model.

gmm = GaussianMixture(n_components=2, covariance_type='full').fit(X)
prediction_gmm = gmm.predict(X)

Now let’s plot both results and compare.

GMM Full

# Add predictions to the original dataset
df['kmeans_cluster'] = prediction
df['gmm_cluster'] = prediction_gmm

# Plot K-Means
sns.scatterplot(data=df, y='tip', x='total_bill', hue='kmeans_cluster');

K-Means result. Image by the author.

K-Means is working as expected, with circular shaped division. We can see a cluster up to $23 or so and tips up to $5.5. The rest is the cluster with higher total bill.

#Plot GMM
sns.scatterplot(data=df, y='tip', x='total_bill', hue='gmm_cluster');

GMM Full result. Image by the author.

The ‘full’ covariance type gives us a tighter cluster 1, with very proportional tips against total bill and a cluster 0 with more spread values. They have different shapes.

Using the 'full' type, each cluster can be shaped like any tilted ellipse, giving the model complete flexibility to stretch and rotate in any direction.

Let’s see the graphics for the other types of covariance for the GMM.

GMM Tied

# Rerun the model
gmm = GaussianMixture(n_components=2, covariance_type='tied').fit(X)
prediction_gmm = gmm.predict(X)

# Replace the predictions
df['gmm_cluster'] = prediction_gmm

# Plot
sns.scatterplot(data=df, y='tip', x='total_bill', hue='gmm_cluster');

Tied means I want the cluster with the same shape. All clusters must have the exact same shape and tilt, but they can be placed in different locations across your data.

GMM Tide result. Image by the author.

GMM Diagonal

# Rerun the model
gmm = GaussianMixture(n_components=2, covariance_type='diag').fit(X)
prediction_gmm = gmm.predict(X)

# Replace the predictions
df['gmm_cluster'] = prediction_gmm

# Plot
sns.scatterplot(data=df, y='tip', x='total_bill', hue='gmm_cluster');

Here, the shape follow the coordinate axes, but some outliers can vary by component. Clusters can be stretched into ellipses, but they must align perfectly straight along the axes without any rotation or tilt.

GMM Diagonal result. Image by the author.

GMM Spherical

# Rerun the model
gmm = GaussianMixture(n_components=2, covariance_type='spherical').fit(X)
prediction_gmm = gmm.predict(X)

# Replace the predictions
df['gmm_cluster'] = prediction_gmm

# Plot
sns.scatterplot(data=df, y='tip', x='total_bill', hue='gmm_cluster');

This one is pretty similar to the K-Means result, as the clusters are defined in a spherical shape. Every cluster must be a perfect circle (or sphere), meaning the data stretches equally in all directions.

GMM Spherical result. Image by the author.

Before You Go

This subject is not trivial. It may look simple, but it has much more depth than this. I recommend that you read the posts I am leaving as reference for a better understanding of the GMMs.

Bottom line is, there’s no easy path. You will have to try different types of models to be sure which one is the best for your case.

One pager explanation of GMM. Image generated by AI. Google Gemini, 2026.

And don’t forget to follow my blog Code Applied.

[embed]Code Applied Code Applied delivers practical, bite-sized tutorials on data science, AI agents, automation, and more. Each post packs…medium.com

References

Sklearn Documentation : GMM

Explanation of the covariance types from Stack Exchange

[embed]Gaussian Mixture Models Explained In the world of Machine Learning, we can distinguish two main areas: Supervised and unsupervised learning. The main…towardsdatascience.com

[embed]How to code Gaussian Mixture Models from scratch in Python GMMs and Maximum Likelihood Optimization Using NumPytowardsdatascience.com

[embed]Gaussian Mixture Modelling (GMM) Making Sense of Text Data using Unsupervised Learningtowardsdatascience.com

[embed]Gaussian Mixture Models Clustering Algorithm Explained Gaussian mixture models can be used to cluster unlabeled data in much the same way as k-means.towardsdatascience.com