Comparing Classification Algorithms on Iris Dataset

A practical comparison of KNN, SVM, Decision Tree and Logistic Regression using scikit-learn.

1. Introduction

What is Classification?

Classification is one of the most fundamental problems in machine learning. In many real-world applications such as spam detection, medical diagnosis, and customer segmentation, the goal is to assign an input to one of several predefined categories.

In this article, I compare several popular classification algorithms using the well-known Iris dataset and evaluate their performance using scikit-learn.

The models compared in this study are:

• K-Nearest Neighbors (KNN)
• Logistic Regression
• Decision Tree
• Support Vector Machine (SVM)

The aim is not only to see which model performs best, but also to understand why.

2. The Iris Dataset

The Iris dataset is a classic dataset in machine learning introduced by Ronald Fisher. It contains 150 samples of iris flowers with 4 features:

• Sepal length
• Sepal width
• Petal length
• Petal width

The target variable consists of 3 classes:

• Setosa
• Versicolor
• Virginica

This dataset is small but very useful for experimenting with classification algorithms.

3. Data Preparation

First, the dataset is loaded from scikit-learn and split into training and test sets.

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split, cross_val_score

iris = load_iris()
X = iris.data
y = iris.target

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

4. Models

-K-Nearest Neighbors (KNN)

KNN classifies a data point based on the majority class of its nearest neighbors.

-Logistic Regression

Despite its name, Logistic Regression is widely used for classification problems.

-Decision Tree

Decision Trees split the data based on feature values and create a tree-like structure.

-Support Vector Machine (SVM)

SVM tries to find the optimal hyperplane that separates the classes.

5. Training and Evaluation

Each model is trained using the training set and evaluated using accuracy on the test set.

from sklearn.neighbors import KNeighborsClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score

models = {
    "KNN": KNeighborsClassifier(),
    "Logistic Regression": LogisticRegression(max_iter=200),
    "Decision Tree": DecisionTreeClassifier(),
    "SVM": SVC()
}

model_names = []
accuracies = []
for name, model in models.items():
    model.fit(X_train, y_train)
    y_pred = model.predict(X_test)
    scores = cross_val_score(model, X, y, cv=5)
    model_names.append(name)
    accuracies.append(scores.mean())
    print(name, scores.mean())

6. Discussion

From the results, it can be observed that all models achieved very similar accuracy values. This is mainly because the Iris dataset is relatively small, clean, and well-separated, which makes it an easy classification problem for most algorithms.

Since the dataset is simple, even basic models are able to perform very well, and therefore the performance differences between models are not very significant.

Among the compared models, KNN and Logistic Regression achieved the highest average accuracy, while SVM performed slightly worse in this experiment. However, the differences are quite small and may vary depending on the data split and cross-validation folds.

These results highlight an important point: on simple datasets, model choice may not have a large impact on performance, and factors such as interpretability, computational cost, and scalability can become more important than raw accuracy.

Finally, although accuracy is a commonly used evaluation metric, in real-world problems it may not always be sufficient. Metrics such as precision, recall, F1-score, and confusion matrix can provide deeper insights, especially when dealing with imbalanced or noisy datasets.

7. What I Learned

This project helped me understand:

• How different classification algorithms behave on the same dataset
• The importance of fair model comparison
• That model selection depends not only on accuracy, but also on interpretability and scalability

Even with a simple dataset like Iris, meaningful insights can be obtained by comparing multiple models.

8. Conclusion

In this article, I compared four popular classification algorithms using the Iris dataset.

For real-world problems, dataset size, complexity, and business requirements should always be considered when choosing a model.