(Un)Discounted Usability: Evaluating Low-Budget Educational Technology Projects with Dual-Personae Evaluators

Abstract

The discounted usability inspections typically employ usability-expert evaluators set to evaluate the target system situated out of the real field. Nonetheless, usability-experts work hard to disguise under an end-user persona, and imitate the context of the original domain and workplace to its maximum, but it hardly is an imitation of the original settings. Hence, these techniques are consistently reported to miss usability problems related to real task scenarios. The problem is especially evident in the particular case of Heuristic Evaluation, where the set of rules used for evaluations weighs domain and task orientation less than other attributes, contributing further to negligence of task and domain related problem. It is thus advised to supplement usability-expert based discounted inspections with expensive end-user based testing---a privilege otherwise unavailable to low-budget projects. The authors in this work, however, propose and test using dual-personae evaluators employed in discounted inspections. The results of the experimentation are promising. The evaluators identify a large number of task and domain related problems, as compared to the number of problems identified in other dimensions during the activity. The authors conclude that such evaluators are especially useful in the case of low-budget projects where expensive testing is not possible.