Assessing individual contributions to software engineering projects: a replication study

Abstract

ABSTRACT Background and Context Assessing team members’ indivdiual contributions to software development projects poses a key problem for computing instructors. While instructors typically rely on subjective assessments, objective assessments could provide a more robust picture. To explore this possibility, In a 2020 paper, Buffardi presented a correlational analysis of objective metrics and subjective metrics in an advanced software engineering project course (n= 41 students and 10 teams), finding only two significant correlations. Objective To explore the robustness of Buffardi’s findings and gain further insight, we conducted a larger scale replication of the Buffardi study (n = 118 students and 25 teams) in three courses at three institutions. Method We collected the same data as in the Buffardi study and computed the same measures from those data. We replicated Buffardi’s exploratory, correlational and regression analyses of objective and subjective measures. Findings While replicating four of Buffardi’s five significant correlational findings and partially replicating the findings of Buffardi’s regression analyses, our results go beyond those of Buffardi by identifying eight additional significant correlations. Implications In contrast to Buffardi’s study, our larger scale study suggests that subjective and objective measures of individual performance in team software development projects can be fruitfully combined to provide consistent and complementary assessments of individual performance.