Systems Thinking Assessment: A Method Through Computer Simulation

Abstract

This paper presents a novel assessment method for Systems Thinking and its supporting competencies. Systems Thinking is a key component in engineering education, providing students with the means to explore, understand, and design engineered systems both holistically and in terms of the relationships between their components, which can be both technical and non-technical. The assessment method described in this paper is implemented as a software simulation of a domain-agnostic system to support integration of Systems Thinking into engineering education. The simulation was tested with a group of beta-testers and then fully deployed online as a freely available tool for a 6-month experimental period. The pool of volunteer participants included students and mixed professionals from a diverse set of geographical, educational, and career backgrounds. Results of the assessment show success at both evaluating Systems Thinking Maturity as a whole, and at assessing complex facets of Systems Thinking that have eluded assessment in prior methods. The tool shows promise at evaluating competencies within all four Systems Thinking domains — Mindset, Content, Structure, and Behavior. These domains contain key systemic skills such as the ability to recognize interconnections and feedback loops, see non-linear causal relationships, and understand dynamic behavior. When examined holistically through multiple regression analysis, participants’ scores in the 11 assessed competencies show a moderate to high ability to predict their levels of overall Systems Thinking performance in the simulation. The results also reveal previously unknown dependencies and strengths of relationships between Systems Thinking competencies.