The evaluation of an intelligent tutoring system

Abstract Numerous intelligent tutoring systems (ITSs) have been developed to date. However, one aspect of their development which is consistently ignored, judging from the literature, is the crucial activity of evaluation. It is also ironical that researchers generally agree that the benefits of this activity could be far-reaching. ITSs are often described to some degree of detail but an appraisal as to their (potential) usefulness is seldom given. The relative novelty of doing this was the main motivation for this paper, which starts by attempting to shed some light on why evaluation has become such a taboo activity, not only in intelligent tutoring, but in artificial intelligence (AI) research in general. It then overviews a tutor, the fractions intelligent tutoring system (FITS), and reports on how it was appraised. The approaches used are neither ideal nor generally accepted, but may well provide an adequate starting point in the belief that an attempt at an honest evaluation of any sort is better tha...