Development and analysis of a Numbas diagnostic tool for use in a mathematics refresher program

Abstract

We describe the development and analysis of an online diagnostic tool implemented in the Numbas e-learning system and used in an Australian university mathematics refresher program. Following the rapid transition to online delivery of the refresher program caused by COVID-19, the diagnostic instruments and methods used within the pre-pandemic, in-person, version of the program were lost. In 2022, we undertook to revive them in a way that would honour their original diagnostic purpose but offer a more sophisticated approach utilising the Numbas diagnostic exam type. Improvement of the tool after its initial deployment has involved the use of Rasch-based item analysis and recursive refinement of the knowledge map underlying the items.