Deciphering India through data visualization: Indian Education System

tldr; Why did I do this and what did I find?

As I began my journey into Data Science, I always knew I wanted to work on data with social impact. In search of a topic for my first project, I started reading a lot of research papers, projects, newspaper articles and searching related data. This is how I got into reading ASER education reports and learnt about DISE publications. As I investigated the data, various questions popped into my mind, ‘What are the learning levels?’, ‘What percentage of kids do not go to school?’, ‘What is the percentage of private school enrolment’, ‘How about the Girls enrolment?’, ‘What is the status of school infrastructure?’, ‘How are the parameters changing over time?’, ‘How does each state compare against each other?’ etc. So, I decided to take it up for analysis and explore the data through 4d-visualization. Why 4d-visualization? Because it enables the exploration and comparison of multiple dimensions.

Summary of my findings as below:

Read on if you find this interesting…

ASER and DISE to the rescue

Annual Status of Education Report (ASER) is an annual survey that has reported on children’s schooling status and their ability to do basic reading and arithmetic tasks since 2005. Facilitated by the NGO Pratham, the survey is carried out by volunteers from local partner organizations in almost all rural districts of India.

The District Information System for Education (DISE) collects information on enrolment, availability of infrastructure, teachers and other facilities available in all schools in the country. The database is developed and maintained at the National University for Educational Planning and Administration. DISE report have been published annually since 2002.

Preparing to Decipher

What are the data analysis parameters? I chose DISE and ASER data for the years between 2007 and 2016. Information pertaining to Enrolment, Infrastructure, Pupil Teacher Ratio and Student Classroom Ratio is included from DISE. Learning Levels, School Type and Out of School information is from ASER. The parameters are chosen based on the relevance to the study and consistency over time. Parameters relating to medium of instruction and teachers can be considered for a future study.

R, d3 and dimple were awesome! The data exploration and preparation are performed using R programming language, an open source software for statistical computations and graphics. Bubble chart used for 4d-Visualizations is developed using d3 and dimple javascript libraries. And they proved to be very relevant and very useful to perform the analysis

Thawing the data to the right level: State is the unit of analysis here. Summary functions are used to detect outliers and missing values. No outlier treatment is deemed necessary and rows with missing values are deleted. ASER provides combined information for the newly formed states Telangana and Andhra while DISE provides separate data. The DISE data for the two states has been combined using averages.

Data Structure: Among the variables used for this study, all the values are percentages except Total Enrolment, Parent Teacher Ratio and School Classroom Ratio. The region of each state is derived by merging state to region mapping file with the file containing DISE and ASER data.

How does the visualization look?

Select drop down boxes are provided for dynamic 4d-visualization. There is a select drop down box for each dimension in the bubble chart.

Changing the option in select drop down updates the corresponding axis. The x-axis, y-axis and size dropdowns have the same options pertaining to enrolment, infrastructure and learning level information. The Colour drop down has two options ‘State’ and ‘Region’. ‘State’ option displays a different colour for each state and ‘Region’ has a colour for each region the state belongs to. This helps to analyse the information by state and region. The storyboard is set to ‘Year’. With each tick of the year, the chart is updated to show information for that year. Play\Pause button controls the animation.

What did I discover?

Private School enrolment catching up big

Though majority of children (5–16) continue to get their education from Government Schools, the private school enrolment is gaining in recent years in spite of their high fees. Even in a state like Uttara Pradesh which is highly populated and backward, the private school enrolment is around 50% in 2016. What is the reason for this? Is it because facilities and teachers are better than in Government school?

Pupil Teacher Ratio (PTR) and Student Classroom Ratio (SCR) are declining

The number of students per teacher (PTR) and number of students in a classroom (SCR) are declining. Does that mean number of schools are increasing? Have private schools helping in bringing down the ratios?

School infrastructure such as toilets, drinking water facilities are improving

Most of the states have hit 100% in providing these facilities, though the condition of these can be a topic for another study. It is worth noting that grants as a result RTE Act which came into force in 2009 has played a role in improving these facilities.

So, the learning levels inching higher?

Absolutely Not! The classic example is that of the neighbouring states of Uttara Pradesh and Bihar, both highly populated and socially and economically backward. While Uttara Pradesh has a private enrolment of almost 50%, Bihar is well behind with around 10%. But when it comes to learning levels, both perform equally bad with around 20% of the children (5–16) unable to read and similar number unable to identify numbers in 2016. So, what are the other factors influencing the learning levels?

Steep decline in learning levels since 2012

Most of the states have shown a marked increase in the percentage of children (5–16) who cannot read and do math since 2012. During the same time, there has also been a decline in the percentage of children (5–16) who can do division (highest level math test) and read std 2 text (highest level reading test). What has triggered such decline since 2012?

Private enrolment the only regional trend

Interestingly, there are no regional trends among the data analysis parameters. The only exception being private school enrolment. The private school enrolment is low among eastern states of Bihar, Jharkhand, Orissa and West Bengal. The highest private school enrolment can be seen in northern states of Uttara Pradesh, Punjab and Haryana.

Girls Enrolment holding steady with no regional barriers

Another interesting aspect is the Girls Enrolment which has remained steady since 2007. Most of the states have Girls Enrolment between 45% and 48% of the total enrolment. The backwardness of a state, both economic and social, has not impacted the percentage. Even the low female ratio states like Haryana and Rajasthan have girls enrolment above 45% of the total enrolment.

5% children (5–16) are out of school

Around 5% of children (5–16) are not in school. It has remained steady since 2007. Have they not gone to school at all? If they were in school before, at what stage did they drop out? If not in school, what do they do?

Conclusion

I have listed some of the observations that I found interesting. Can you find more? Please visit my *github *page for the visualization.

To conclude, learning levels are the most important measure of the health of education system in a country. From the data, it is apparent that India is not doing particularly well. This despite of the implementation of RTE Act in 2009, growing middle class, increase in grants and other resources. What are the reasons for this? Is the quality of teachers affecting the outcome? Are the socio-economic factors playing a role? How about adult literacy and urbanisation?

I am sure to come back with further updates..keep checking.