Predicting Programming Success: How Intermittent Knowledge Assessments, Individual Psychometrics, and Resting-State EEG Predict Python Programming and Debugging Skills

Abstract

Computer programming requires fluid application of acquired chunks of declarative knowledge to accomplish a defined goal. This raises the question-how strongly do declarative knowledge assessments collected during training predict individual learners' future programming capabilities, and how might neurocognitive measures expand these predictions? The current study explored this by using stepwise regression to determine whether neurocognitive characteristics of individual learners and post-module declarative assessments collected in the Codecademy learning platform explain unique or overlapping variance when predicting real-world coding outcomes. Based on data from 80 participants over 16 one-hour Python training sessions, we found that post-module declarative knowledge assessments explained the most variance in each of our seven learning outcomes: multiple-choice test accuracy, programming accuracy, and debugging accuracy (collected at two time points) plus learning rate. However, neurocognitive measures also contributed unique variance, with total variance explained varying across outcomes. Our preliminary results suggest that declarative knowledge and neurocognitive indices combine in different proportions to predict different types of programming outcomes.