Improving Student Surveys with Natural Language Processing

Stakeholders from academic institutions across the world employ surveys to assess the quality of their work. With surveys these stakeholders attempt to obtain quantified, structured, and directed data in order to make decisions. Often these stakeholders employ long, directed Likert scaled surveys to gain this information. We propose an alternate construction for academic surveys, where stakeholders provide 1-3 open ended "free text" questions, allowing students to lead the discussion. We call this survey methodology "Student Directed Discussion Surveys" (SDDS). SDDS retain the ability to provide quantified, structured, and directed results by employing Natural Language Processing (NLP). We confirm the accuracy of SDDS in relation to traditional Likert scaled surveys with a permutation test, assessing a negligible statistical difference between SDDS and Likert surveys using real data. We then show the utility of SDDS by employing word frequency and sentiment analysis, providing important unbiased decision making information, which is limited when traditional Likert scaled surveys are administered.