Challenges for introducing artificial intelligence to improve the efficiency of a next generation assessment approach

Abstract

The U.S. Army sought to develop capabilities that allow for the automated or semi-automated, with greatly reduced human involvement, creation of tests and assessments. In recognizing the potential for an assessment approach that goes beyond multiple-choice, the Army chose our team to introduce and evaluate automated capabilities to author concept mapping-based assessments. This paper describes our initial approaches toward introducing efficiencies into the authoring process for concept map-based assessments. We are developing and evaluating methods to automatically generate concept maps from a knowledge domain and convert the maps into assessments for formative and summative purposes. Our initial work has sought to overcome challenges as we introduced artificial intelligence into the authoring process. In this paper, we describe our emergent approach and the challenges we have faced in seeking efficiencies in the conversion of text to concept maps.