Linear Regression vs Ridge vs Lasso: Understanding Regularization Using Wine Quality Data

I compared Linear Regression, Ridge Regression, and Lasso Regression on the Wine Quality dataset to understand how regularization changes model behavior in practical machine learning.

So instead of only reading theory, I decided to compare three regression models on the Wine Quality dataset:

• Linear Regression
• Ridge Regression
• Lasso Regression

At first, all three models looked almost the same to me. But after training them and comparing the results, I finally understood why regularization matters.

About the Dataset

For this experiment, I used the Wine Quality dataset from the UCI repository. The dataset contains different chemical properties of wine such as:

• Fixed acidity
• Volatile acidity
• Citric acid
• pH
• Alcohol
• Sulphates
• Density

The target column is the wine quality score.Predict wine quality using all these chemical features.

Importing Libraries

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression, Ridge, Lasso
from sklearn.metrics import mean_squared_error, r2_score

Loading the Dataset

df = pd.read_csv("winequality-red.csv")

print(df.head())

After checking the dataset, I separated input and output columns.

X = df.drop("quality", axis=1)
y = df["quality"]

Splitting and Scaling the Data

I split the dataset into training and testing data.

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

Then I scaled the features.

scaler = StandardScaler()

X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

Scaling is important here because Ridge and Lasso are sensitive to feature values. If features are on different scales, the models can behave inconsistently.

Linear Regression

I started with simple Linear Regression.

lr = LinearRegression()
lr.fit(X_train_scaled, y_train)
y_pred_lr = lr.predict(X_test_scaled)

Linear Regression tries to fit the best possible line to the data.

The problem is that it focuses only on minimizing prediction error. Sometimes that leads to overfitting.

Especially when features are highly related to each other.

Ridge Regression

Next, I trained Ridge Regression.

ridge = Ridge(alpha=1.0)
ridge.fit(X_train_scaled, y_train)
y_pred_ridge = ridge.predict(X_test_scaled)

Ridge Regression adds a penalty term to the model.

Instead of allowing coefficients to become too large, it shrinks them.

That helps the model stay more stable.

Lasso Regression

Finally, I tested Lasso Regression.

lasso = Lasso(alpha=0.1)
lasso.fit(X_train_scaled, y_train)
y_pred_lasso = lasso.predict(X_test_scaled)

This model was the most interesting.

Unlike Ridge, Lasso can completely remove some features by making their coefficients zero.

That means it can perform feature selection automatically.

Evaluating the Models

To compare the models, I used RMSE and R² score.

models = {
    "Linear Regression": y_pred_lr,
    "Ridge Regression": y_pred_ridge,
    "Lasso Regression": y_pred_lasso
}

for name, pred in models.items():
    print(name)
    print("RMSE:", np.sqrt(mean_squared_error(y_test, pred)))
    print("R2 Score:", r2_score(y_test, pred))
    print("-" * 40)

Visualizing Coefficients

This part made the differences much easier to understand.

coef_df = pd.DataFrame({
    "Feature": X.columns,
    "Linear": lr.coef_,
    "Ridge": ridge.coef_,
    "Lasso": lasso.coef_
})
coef_df.plot(x="Feature", kind="bar", figsize=(12,6))
plt.title("Coefficient Comparison")
plt.xticks(rotation=45)
plt.show()

After plotting the coefficients, the behavior of all three models became clear.

• Linear Regression produced larger coefficients.
• Ridge reduced the coefficient values.
• Lasso pushed some coefficients close to zero.

This was the point where regularization finally made practical sense to me.

Correlation Heatmap

This is one of the best visuals for regression problems.

It shows:

• which features are strongly related
• multicollinearity
• feature relationships

Code:

plt.figure(figsize=(12,8))

sns.heatmap(df.corr(), annot=True, cmap="coolwarm")

plt.title("Feature Correlation Heatmap")

plt.show()

The heatmap helped identify relationships between features and showed why regularization becomes useful when variables are correlated.

What I Observed

A few things stood out during this experiment.

1. Linear Regression is simple but unstable

It works well, but coefficients can become large when features are correlated.

2. Ridge Regression gives more balanced predictions

The model felt more stable because coefficients were controlled.

3. Lasso creates a simpler model

Some features became unnecessary and were automatically removed.

4. Scaling matters a lot

Without scaling, Ridge and Lasso performance was not reliable.

Final Thoughts

Before doing this project, Ridge and Lasso felt like small modifications of Linear Regression.

But after training and comparing them on the same dataset, the differences became much more practical.

Linear Regression focuses only on fitting the data. Ridge focuses on reducing model complexity. Lasso reduces complexity and can even remove unnecessary features.

For this dataset, regularization definitely helped make the models more stable and generalizable.

This project also taught me something important:

Understanding machine learning becomes much easier when you compare models practically instead of only reading theory.

And honestly, that is where most of the learning happens.

# Connect With Me

The complete notebook, dataset, and project files are available below.

• GitHub : [My GitHub link]
• Google Drive : [My Drive link]
• LinkedIn : [ My LinkedIn profile]

Feel free to connect or share feedback about the project.