Measuring shared understanding in software project teams using pathfinder networks

[Context] Software engineering teams must have a shared understanding of the system design in order to work independently but successfully integrate their code. Success also depends on the differentiated skill and experience of the team members. These issues of understanding are important to project success but difficult to investigate with current approaches. [Goal] To investigate this problem, we developed and evaluated a technique to measure the degree of shared understanding and identify areas of similarity and difference. Adapted from the Pathfinder technique for evaluating Team Mental Models, this is a quantitative analysis of paired comparisons of design concepts as understood by the team. [Method] We performed an empirical, mixed-methods pilot study of the technique with 5 student teams developing a semester long project. We used questionnaires and interviews to evaluate the effectiveness of the technique in measuring areas of similarity and difference. We also investigated the association between differences in understanding and problems during development. [Results] Our results support the ability of the technique to identify and measure areas of similarity and difference. There is limited support for the association between differences and poor project outcomes. [Conclusions] We find these pilot results encouraging. We will use them to refine the technique and plan to re-evaluate it with a professional software development team.