What are hyperparameters?

Hyperparameters are the settings or configurations of a machine learning model that are set before training and control how the model learns.

🔍 In simple terms:

They are like “knobs” you tune manually to get the best performance.

🧠 Example:

In Linear Regression, there aren’t many hyperparameters, but in more complex models like:

• **Neural Network** → number of layers, learning rate
• Random Forest → number of trees, depth of trees

These values must be chosen before training starts.

⚙️ Common Hyperparameters:

• Learning rate → how fast the model updates
• Number of epochs → how many times the model sees the data
• Batch size → number of samples per update
• Number of layers / neurons (in neural networks)
• Regularization strength

❗ Hyperparameters vs Parameters:

• Parameters → learned automatically during training (e.g., weights in a neural network)
• Hyperparameters → set manually before training

🛠️ How to Choose Them:

• Trial and error
• Grid search
• Random search
• Advanced methods like Bayesian optimization

📌 Quick Analogy:

Think of cooking:

• Ingredients & quantities learned while cooking → parameters
• Recipe settings (temperature, time, method) → hyperparameters