Remedying Education with Personalized Homework: Evidence from a Randomized Field Experiment in India

Abstract

Can Information and Communication Technology (ICT) enabled personalization remedy the educational production in resource-strapped schooling systems? We conduct a randomized field experiment on a group of residential schools in Hyderabad India to examine this question. In a school setting, students first learn concepts through class room instructions and then reinforce their learning by doing homework. In our experiment, students were first taught different topics in mathematics through classroom instructions, and then a randomly selected one half of them were assigned computer-generated adaptive homework (CGAHW) and the other half were offered paper-based traditional homework (PBTHW). In a PBTHW, a pre-decided fixed number of easy and hard questions were offered from different topics. In a CGAHW, first half of the total questions were offered in the easy category, and based on a student’s performance on these questions, later questions were adaptively generated such that: (1) more questions were offered on the topics in which student incorrectly answered questions and (2) hard questions on a topic were offered when the student correctly answered easy questions on that topic. Thus, while all PBTHW students received the same number of easy and hard questions on different topics, CGAHW students received different numbers and difficulty levels of questions on different topics based on their individual learning needs. A total of 50 homework in each category were offered to students between October 2014 and April 2015, and their learning was assessed in two standardized exams offered in this period.We found that CGAHW students on average obtained lower homework scores than PBTHW students, but they obtained 4.28 percent higher scores in exams than PBTHW students. Lower homework scores could be attributed to students receiving more questions in their weak areas in CGAHW. However, by doing more questions in their weak areas and less in their strong areas, students achieved personalized learning in CGAHW, and hence obtained higher exam scores. To provide evidence that personalized learning in CGAHW resulted in improvement in their exam scores, we show that students that were offered higher levels of personalization in CGAHW, obtained higher exam scores. To further understand the differential effect of CGAHW on students of different abilities, we categorized students in low, medium, and high categories of ability based on their mathematics scores in standardized exams at the beginning of experiment. We found that personalized learning through CGAHW helped the students in low and medium ability categories but not in high ability category. Overall, we developed and deployed an adaptive homework generation application in a field set up to show how ICT-enabled personalized learning could improve educational production with existing school resources.