Usability testing of a conceptual model through retrospective cross-case analysis

Abstract

Qualitative conceptual models are commonly used in the scientific literature to make complex phenomena easier to understand. However, the effectiveness and usability of conceptual models to serve as analytical tools is rarely explored and tested, and there is a lack of guidelines for such analyses. This paper adapts and combines the methods of usability testing and cross-case analysis to describe a systematic protocol to facilitate the qualitative evaluation of conceptual models. Usability testing is an established method for identifying problems or shortcomings within a product and for assessing different dimensions of product usability: suitability, accessibility, relevance, and integrity. Cross-case analysis, on the other hand, is a qualitative research method for systematically comparing information from individual case studies and identifying commonalities and patterns that apply across cases. Taken together, these methodological approaches provide a structured way of retrospectively applying a conceptual model to existing literature and thereby evaluating its effectiveness in meeting its intended purpose.

• •

  We show how researchers can prepare, conduct, and synthesise the results of a usability test of a conceptual model.

• •

  We provide recommendations for the practical implementation of each step.

• •

  We outline the benefits, limitations, and ethical considerations that researchers should be aware of.