Hyperparameter Optimization Technique

What are these hyperparameters? For each logarithm, there will be model parameters which will be learnt from given data set. For example, loss function is used for a & b in (y = a + bx). Model will find patterns between x & y. In this process, a & b parameter values will be updated from loss function (Squared Error Loss) by finding best values.

Hyperparameters are specific to algorithm itself. These cannot be learnt from dataset, we actually use these hyperparameter to calculate model parameters. Example: Hyperparameters are used to calculate a & b parameters.

Hyperparameters are different to each algorithm. Hyperparameter tuning is to find set of optimal values for hyperparameter for an algorithm for any dataset such that relationship between x and y.

Methods to tune these Hyperparameters:

1. Manual Method: Here, we manually adjust the values and monitor the model performance. During initializing the model, we would explicitly provide hyperparameter values.

With changes in hyperparameter, the values of f1 score changes. However, manual change is not feasible as we need to check for multiple hyperparameter.

An automated for hyper parameter tuning.

2. GridSearchCV: It's a brute force method for trying all the combination of hyperparameters. Grid is created with discrete values and fit the model with all possible combination of the dataset. Record all of them and select the best parameters.

3. RandomSearchCV : Random combinations from the hyperparameters. Among the random selection, the best parameters are returned.

In this way, hyperparameter can be performed on any ML algorithms.