What is multicollinearity and how do you detect it?

**Multicollinearity occurs in regression analysis when two or more independent variables are highly correlated with each other**. This means they carry similar information, making it hard for the model to understand their individual impact on the target variable.

🔹 Why It’s a Problem

• Coefficients become unstable (they change a lot with small data changes)
• Hard to interpret which variable is actually important
• Can lead to **incorrect conclusions** in models

🔹 Simple Example

If you’re predicting salary using:

• Years of experience
• Age

👉 These two variables are often highly correlated → causing multicollinearity.

🔹 How to Detect Multicollinearity

1. 🔸 Correlation Matrix

• Check correlation between independent variables
• If correlation > 0.8 or 0.9, it’s a warning sign

👉 Easy and quick method

2. 🔸 Variance Inflation Factor (VIF)

Most popular method

VIF=11−R2VIF = \frac{1}{1 — R²}VIF=1−R21​

• VIF = 1 → No correlation
• VIF between 1–5 → Moderate
• VIF > 5 or 10 → High multicollinearity 🚨

3. 🔸 Tolerance

Tolerance=1−R2Tolerance = 1 — R²Tolerance=1−R2

• Low tolerance (< 0.1) → High multicollinearity

4. 🔸 Regression Signs

• High R² but insignificant variables (high p-values)
• Coefficients flip signs unexpectedly

🔹 How to Fix It

• Remove one of the correlated variables
• Combine variables (feature engineering)
• Use techniques like **Principal Component Analysis (PCA)**
• Use regularization methods (Ridge, Lasso)