{"title":"Predicting student success with and without library instruction using supervised machine learning methods","authors":"Karen Harker, Carol Hargis, Jennifer Rowe","doi":"10.1108/pmm-12-2023-0047","DOIUrl":null,"url":null,"abstract":"<h3>Purpose</h3>\n<p>The main purpose of this analysis was to demonstrate the value of predictive modeling of student success and identify the key groups of students for which library instruction could provide the most impact.</p><!--/ Abstract__block -->\n<h3>Design/methodology/approach</h3>\n<p>Data regarding the attendance of library instruction associated with a first-year writing course were combined with student demographic and academic data over a four year period representing over 10,000 students. We applied supervised machine learning methods to determine the most accurate model for predicting student outcomes, including course outcome, persistence and graduation. We also assessed the impact of library instruction on these outcomes.</p><!--/ Abstract__block -->\n<h3>Findings</h3>\n<p>The gradient-boosted decision tree model provided the most accurate predictions. The impact of library instruction was modest but still was second only to the previous grade point average (GPA). The value of this metric, however, was greatest for students who were struggling, especially those who were first-generation students, regardless of ethnicity. More notably, the impact of library instruction was substantially greater for specific student demographics, including students with lower cumulative GPAs.</p><!--/ Abstract__block -->\n<h3>Research limitations/implications</h3>\n<p>Features of the models were limited to high-level academic metrics, some of which may not be very useful in predicting outcomes. Measures more closely related to learning styles, the course or course of study could provide for greater accuracy.</p><!--/ Abstract__block -->\n<h3>Practical implications</h3>\n<p>Prediction modeling could allow for a more selective approach to outreach and offers information that the librarian can use to customize instruction sessions and reference interactions.</p><!--/ Abstract__block -->\n<h3>Social implications</h3>\n<p>Targeting students who may be at risk of not succeeding in a course has ethical implications either way. If used to bias the subjective assessments, these predictions could produce self-fulfilling prophecies. Conversely, to ignore indicators of possible difficulties the student may have with the material is a disservice to the education of that student.</p><!--/ Abstract__block -->\n<h3>Originality/value</h3>\n<p>There are few studies that have incorporated library instruction into models of predicting student outcomes. Library resources and services can play a major role in the success of students, particularly those who have had less exposure to the resources and skills needed to use these resources.</p><!--/ Abstract__block -->","PeriodicalId":44583,"journal":{"name":"Performance Measurement and Metrics","volume":null,"pages":null},"PeriodicalIF":1.8000,"publicationDate":"2024-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Performance Measurement and Metrics","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1108/pmm-12-2023-0047","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"INFORMATION SCIENCE & LIBRARY SCIENCE","Score":null,"Total":0}
引用次数: 0
Abstract
Purpose
The main purpose of this analysis was to demonstrate the value of predictive modeling of student success and identify the key groups of students for which library instruction could provide the most impact.
Design/methodology/approach
Data regarding the attendance of library instruction associated with a first-year writing course were combined with student demographic and academic data over a four year period representing over 10,000 students. We applied supervised machine learning methods to determine the most accurate model for predicting student outcomes, including course outcome, persistence and graduation. We also assessed the impact of library instruction on these outcomes.
Findings
The gradient-boosted decision tree model provided the most accurate predictions. The impact of library instruction was modest but still was second only to the previous grade point average (GPA). The value of this metric, however, was greatest for students who were struggling, especially those who were first-generation students, regardless of ethnicity. More notably, the impact of library instruction was substantially greater for specific student demographics, including students with lower cumulative GPAs.
Research limitations/implications
Features of the models were limited to high-level academic metrics, some of which may not be very useful in predicting outcomes. Measures more closely related to learning styles, the course or course of study could provide for greater accuracy.
Practical implications
Prediction modeling could allow for a more selective approach to outreach and offers information that the librarian can use to customize instruction sessions and reference interactions.
Social implications
Targeting students who may be at risk of not succeeding in a course has ethical implications either way. If used to bias the subjective assessments, these predictions could produce self-fulfilling prophecies. Conversely, to ignore indicators of possible difficulties the student may have with the material is a disservice to the education of that student.
Originality/value
There are few studies that have incorporated library instruction into models of predicting student outcomes. Library resources and services can play a major role in the success of students, particularly those who have had less exposure to the resources and skills needed to use these resources.
期刊介绍:
■Quantitative and qualitative analysis ■Benchmarking ■The measurement and role of information in enhancing organizational effectiveness ■Quality techniques and quality improvement ■Training and education ■Methods for performance measurement and metrics ■Standard assessment tools ■Using emerging technologies ■Setting standards or service quality