Compare regularization methods (L1, L2, ElasticNet) and explain when you’d use each.

L1 Regularization (Lasso)

• Adds a penalty equal to the absolute values of the coefficients.
• Encourages **sparsity**, meaning some feature weights become exactly zero.
• Works well when you believe only a few features are important, effectively performing feature selection.
• Can behave unpredictably if features are highly correlated.
• Example: Selecting key genes from thousands in a medical dataset.

L2 Regularization (Ridge)

• Adds a penalty equal to the squared values of the coefficients.
• Shrinks coefficients toward zero but rarely makes them exactly zero.
• Handles correlated features well by distributing weights among them.
• Does not perform feature selection but reduces overfitting.
• Example: Predicting house prices with many correlated numeric features like size and number of rooms.

ElasticNet Regularization

• Combines L1 and L2 penalties, balancing sparsity and stability.
• Can select features while still handling correlated features better than L1 alone.
• Requires tuning two hyperparameters for L1 and L2 contributions.
• Example: High-dimensional data such as text data with many **correlated **features.

Rule of Thumb:

• Use L1 if you want sparse models with automatic feature selection.
• Use L2 if features are correlated and you want stability.
• Use ElasticNet if you want both sparsity and stability, especially with correlated features in **high-dimensional data.**