Comparative Analysis of KNN and Gaussian Naive Bayes Through a Crop Recommendation Project

Introduction

Recently, I started learning some fundamental Machine Learning classification algorithms, especially:

• K-Nearest Neighbors (KNN)
• Gaussian Naive Bayes

After learning the theoretical foundations of K-Nearest Neighbors (KNN) and Gaussian Naive Bayes, I decided to apply them to a practical real-world problem: Crop Recommendation. This project goes beyond simply achieving high accuracy. It demonstrates a complete, professional machine learning workflow from problem understanding and rigorous Exploratory Data Analysis (EDA) to proper preprocessing, feature scaling, cross-validation, hyperparameter tuning, and insightful model comparison.

Building a Crop Recommendation System

The idea was simple, a smart agricultural tool that recommends the best crop to plant based on environmental and soil conditions. I specifically chose the Crop Recommendation dataset because it is beginner-friendly and perfect for practicing classification algorithms.

Farmers often face difficulty selecting the best crop for their land due to changing environmental conditions.

A poor crop decision can lead to:

• low productivity,
• economic loss,
• and inefficient use of resources.

Machine Learning can help by analyzing environmental data and making intelligent predictions.

Key Concepts Practiced in This Project

While implementing this project, I practiced several important concepts that I had recently learned:

• Feature Scaling
• Standardization vs Normalization
• Label Encoding
• Cross Validation
• Hyperparameter Tuning
• Model Evaluation
• Confusion Matrix Analysis

Why I Chose KNN and Gaussian Naive Bayes

I selected KNN and Gaussian Naive Bayes because they represent two very different approaches to classification.

• KNN is a distance-based algorithm that classifies samples based on neighboring data points.
• Gaussian Naive Bayes is a probabilistic algorithm based on Bayes’ Theorem and feature independence assumptions.

By comparing both algorithms on the same dataset, I was able to better understand:

• how preprocessing affects different models,
• why scaling is critical for KNN,
• and how model assumptions influence performance.

Dataset Overview

Dataset: Crop Recommendation Dataset (Kaggle)

The dataset contains:

• 2200 rows
• 7 input features
• 22 crop categories

Key Features:

• Nitrogen (N), Phosphorus (P), Potassium (K)
• Temperature (°C), Humidity (%), pH, Rainfall (mm)

Target: 22 crop labels (Rice, Maize, Chickpea, etc.)

Importing Required Libraries

Before building the models, I imported the necessary libraries for:

• data analysis,
• visualization,
• preprocessing,
• model building,
• and evaluation.

# Data analysis
import pandas as pd
import numpy as np

# Visualization
import matplotlib.pyplot as plt
import seaborn as sns

# Preprocessing
from sklearn.preprocessing import LabelEncoder
from sklearn.preprocessing import StandardScaler
from sklearn.preprocessing import MinMaxScaler

# Model Selection
from sklearn.model_selection import train_test_split
from sklearn.model_selection import GridSearchCV
from sklearn.model_selection import cross_val_score

# Models
from sklearn.neighbors import KNeighborsClassifier
from sklearn.naive_bayes import GaussianNB

# Evaluation
from sklearn.metrics import accuracy_score
from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report

Loading the Dataset

df = pd.read_csv("data/raw/Crop_recommendation.csv")
df.head()

Before moving to model training, I explored the dataset to understand:

• the number of rows and columns,
• missing values,
• feature distributions,
• and class balance.

print(df.shape)
print(df.isnull().sum())

Observation

The dataset is clean and balanced:

• no missing values,
• balanced crop classes,
• and numerical features suitable for classification algorithms.

Exploratory Data Analysis (EDA)

EDA helped me understand the structure and behavior of the dataset before preprocessing.

Statistical summary

key statistical insights such as mean, standard deviation, minimum, maximum, and quartile values of the dataset.

df.describe()

Feature Distribution Visualization

sns.set_style("whitegrid")

# Feature Distribution with KDE
features = df.drop('label', axis=1).columns

plt.figure(figsize=(15, 10))
for i, col in enumerate(features, 1):
    plt.subplot(3, 3, i)
    sns.histplot(data=df, x=col, kde=True, bins=25, color='#2E8B57', alpha=0.75)
    plt.title(f'{col}', fontsize=13, fontweight='bold')
    plt.xlabel('')
    plt.ylabel('')

plt.suptitle('Feature Distributions', fontsize=18, fontweight='bold', y=0.98)
plt.tight_layout()
plt.show()

Correlation Analysis

plt.figure(figsize=(11, 8))

sns.heatmap(
    df.drop('label', axis=1).corr(),
    annot=True,
    fmt='.2f',
    cmap='YlGnBu',
    linewidths=1,
    linecolor='white',
    square=True,
    cbar_kws={'shrink': 0.8}
)

plt.title('Feature Correlation Heatmap', fontsize=16, fontweight='bold', pad=15)
plt.xticks(rotation=45, ha='right')
plt.yticks(rotation=0)
plt.tight_layout()
plt.show()

What I Observed

I noticed that:

• Some features like rainfall (range: ~20–300) have much larger values compared to pH (range: ~4–9).
• feature distributions are not identical,
• heatmap showed a strong positive correlation (0.74) between Phosphorus (P) and Potassium (K),
• Other features had weak to moderate correlations, indicating low multicollinearity and supporting the independence assumption of Naive Bayes.

These observations strongly influenced my preprocessing strategy (especially standardization) and helped explain the performance difference between KNN and Gaussian Naive Bayes.

Label Encoding

The target labels are crop names such as: rice, maize, cotton, chickpea, etc.

Since machine learning models work with numbers, I converted the labels into numerical values using Label Encoding.

X = df.drop("label", axis=1)

y = df["label"]

encoder = LabelEncoder()

y_encoded = encoder.fit_transform(y)

Train-Test Split

To evaluate the models properly, I divided the dataset into:

• training data (80%)
• testing data (20%)

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y_encoded,
    test_size=0.20,
    random_state=42,
    stratify=y_encoded
)

I used stratify=y_encoded to ensure that all crop categories are proportionally distributed across both training and testing sets.

Feature Scaling

Feature scaling became one of the most important concepts I learned during this project.

KNN is a distance-based algorithm, meaning:

features with larger numerical ranges can dominate predictions.

For example:

• rainfall values can exceed 200,
• while pH values usually stay between 4 and 9.

Without scaling:

• rainfall would heavily influence distance calculations,
• reducing the contribution of smaller-range features.

Standardization

I used StandardScaler to standardize the features.

scaler = StandardScaler()

X_train_scaled = scaler.fit_transform(X_train)

X_test_scaled = scaler.transform(X_test)

Standardization transforms data so that:

• mean = 0
• standard deviation = 1

This allows all features to contribute more equally.

Before vs After Scaling Visualization

• Before Scaling : Features had drastically different ranges

sns.boxplot(data=X_train)

• After Scaling : All features are now centered around 0 with similar scales.

sns.boxplot(data=pd.DataFrame(X_train_scaled))

Standardization vs Normalization

I also explored normalization using MinMaxScaler.

minmax = MinMaxScaler()

X_train_norm = minmax.fit_transform(X_train)

X_test_norm = minmax.transform(X_test)

What I Learned

• Standardization worked slightly better for this project.
• KNN performance improved significantly after scaling.
• Gaussian Naive Bayes was less sensitive to scaling.

This helped me understand that:

preprocessing choices directly affect model behavior.

Building the KNN Model

KNN classifies data points based on their nearest neighbors.

The intuition is simple:

similar data points are likely to belong to the same class.

Hyperparameter Tuning with GridSearchCV

Instead of manually choosing parameters, I used GridSearchCV to automatically search for the best combination.

param_grid = {
    'n_neighbors': [3,5,7,9,11],
    'weights': ['uniform', 'distance'],
    'metric': ['euclidean', 'manhattan']
}

grid_search = GridSearchCV(
    KNeighborsClassifier(),
    param_grid,
    cv=5,
    scoring='accuracy',
    n_jobs=-1
)

grid_search.fit(X_train_scaled, y_train)

Why Hyperparameter Tuning Matters

Different parameter combinations can dramatically change model performance.

For example:

• smaller K values may overfit,
• larger K values may oversmooth predictions.

GridSearchCV helped me systematically find the best configuration.

Final KNN Model

knn_model = grid_search.best_estimator_

knn_model.fit(X_train_scaled, y_train)

knn_predictions = knn_model.predict(X_test_scaled)

Building Gaussian Naive Bayes

Unlike KNN, GaussianNB is a probabilistic algorithm.

It predicts classes using Bayes’ Theorem and assumes:

• feature independence,
• and Gaussian distributions.

nb_model = GaussianNB()

nb_model.fit(X_train, y_train)

nb_predictions = nb_model.predict(X_test)

Cross Validation

Initially, I evaluated the models using only a train-test split.

However, I learned that:

a single split may produce misleading results.

To improve reliability, I used 5-Fold Cross Validation.

cv_knn = cross_val_score(
    knn_model,
    X_train_scaled,
    y_train,
    cv=5,
    scoring='accuracy'
)

cv_nb = cross_val_score(
    nb_model,
    X_train,
    y_train,
    cv=5,
    scoring='accuracy'
)

Why Cross Validation Is Important

Cross-validation:

• evaluates the model across multiple subsets,
• reduces evaluation bias,
• and gives more trustworthy performance estimates.

This was one of the most valuable concepts I practiced during this project.

Confusion Matrix

To better understand prediction behavior, I visualized the confusion matrix.

# Model Comparison Chart

models = ['KNN', 'GaussianNB']

accuracies = [
    knn_accuracy * 100,
    nb_accuracy * 100
]

plt.figure(figsize=(8,6))

bars = plt.bar(
    models,
    accuracies
)

plt.ylim(90, 100)

plt.ylabel('Accuracy (%)')

plt.title(
    'Model Accuracy Comparison',
    fontsize=16,
    fontweight='bold'
)

for bar in bars:

    height = bar.get_height()

    plt.text(
        bar.get_x() + bar.get_width()/2,
        height + 0.2,
        f'{height:.2f}%',
        ha='center',
        fontsize=12,
        fontweight='bold'
    )

plt.tight_layout()

plt.savefig(
    'reports/figures/model_comparison.png',
    dpi=300
)

plt.show()

Results and Model Comparison

| Metric                    | KNN      | GaussianNB |
| ------------------------- | -------- | ---------- |
| Test Accuracy             | 98.18%   | 99.55%     |
| Cross Validation Accuracy | 98.12%   | 99.43%     |
| Training Speed            | Moderate | Very Fast  |
| Scaling Sensitivity       | High     | Low        |

Key Observation

Gaussian Naive Bayes achieved slightly higher accuracy than KNN on this dataset.

One possible reason is that many features in the dataset show relatively independent behavior with near-Gaussian distributions, which aligns well with the assumptions of Gaussian Naive Bayes.

In contrast, KNN relies heavily on distance calculations and can still be affected by local feature variations even after scaling.

This taught me an important lesson:

simpler algorithms can perform extremely well when their assumptions align with the dataset.

Making Predictions on New Data

Finally, I tested the model on new input values.

new_data = np.array([
    [90, 42, 43, 21, 82, 6.5, 202]
])

new_scaled = scaler.transform(new_data)

prediction = knn_model.predict(new_scaled)

crop = encoder.inverse_transform(prediction)

print("Recommended Crop:", crop[0])

Key Learnings

This project helped me strengthen several important Machine Learning concepts:

• Importance of preprocessing
• Why feature scaling matters
• Difference between normalization and standardization
• Proper evaluation using cross-validation
• Hyperparameter tuning using GridSearchCV
• Understanding algorithm assumptions

Conclusion

This project helped me move beyond theory and apply Machine Learning concepts practically.

Rather than simply training models, I learned:

• how preprocessing affects performance,
• why validation matters,
• and how different algorithms behave under the same dataset.

This project became an important step in transforming theoretical understanding into practical implementation.

GitHub Repository

You can find the complete implementation, notebook, and project structure here : **Crop_recommendation_project**