Handling categorical and numerical values in the data

Collection of data — One of the ways to collect real life data is through conducting surveys. As being a student, the closest subject is academic data. So I created a google form and gathered responses from students relating to their academic performance.

About the data:

We can classify the data columns into numerical/categorical values:

1. Numerical values: age, family size,study time(in hours), failures(past backlogs), going out with friends(1-very low to 5- very high),health status(1-very bad to 5-excellent),average attendance(in percentage),interest in subjects(1-very low to 5-very high),difficulty level of subjects(1-very low to 5-very high),GPA(1st semester),GPA(2nd semester),GPA(3rd semester)
2. Categorical values: gender, mother’s education, father’s education, mother’s job, father’s job, travel time, extra paid classes for courses, extra_curriculum activities , time spent on internet/social media.

Observation from heat map for missing values:

From heat map we can see that family size, father’s job, failures(past backlogs), extra paid classes for courses, difficulty level of subjects(1-very low to 5-very high), GPA(1st semester),GPA(2nd semester),GPA(3rd semester) these columns contain missing values.

For numerical data:-

Replacing with mean/median/mode:

We can replace missing values with mean/median/mode as they represent the central tendency of the data.For columns family size and difficulty level of the subject we can replace missing values with the median of the columns.

Code for replacing with median:

Dropping the rows:

For column failures(past backlogs),GPA(1st semester), GPA(2nd semester), GPA(3rd semester) we can remove missing values by dropping the rows which contain missing values.

For categorical data:-

Label encoding :

To see when we use label encoding let’s take the example of the column time spent on internet/social media

We can say that (> 5 hours) > (1 to 5 hours) >(<1 hour).

In such cases of categorical variables where we can arrange the output in order we can use label encoding.

Code for label encoding:

One hot encoding:

The categorical variables which we can not arrange in order we can use one hot encoding. In these categorical variables we can’t give priority to one group of data over the other group of data.

Code for one hot encoding:

You can see the complete code on my github repository:

https://github.com/saumyadeshmukh/dataanlysis-studentperformacesystem