Measuring Peer Mentoring Effectiveness in Computing Courses: A Case Study in Data Analytics for Cybersecurity

Abstract

Computing courses often suffer from lack of diversity. In this paper we evaluate an intervention method of peer mentoring to help increase interest in data analytics in cybersecurity. We present a text mining approach to analyze student assignments while they undergo a peer mentoring exercise. In our prior work, we have shown that the peer mentoring approach is effective at improving the students' interest in cybersecurity careers and contributes to an overall better knowledge gain throughout the semester. This was also reflected by an improvement in grades with two years of anonymous survey results. Across the years we also observed that peer mentoring is particularly effective in diverse groups. In this paper, we perform text mining of the written assignments for analyzing the group behavior of the control and experiment sections of a class while also documenting the effectiveness of intervention methods such as peer mentoring. We employ a few text mining techniques, namely Text Frequency Analysis, Lexical Diversity, Readability Analysis, Word Cloud Visualization, Hyperlink usage and Objectivity Analysis on the text assignments submitted by the students and show that students who receive peer mentoring are able to express more complex ideas with fewer words and thereby receive higher grades by the end of the semester. Based on these results, we also discuss how our methodology would be applicable in increasing reachability and diversity in other specialized computing courses such as Big Data and distributed systems.