Improving the Identification of Hedonic Quality in User Requirements — A Controlled Experiment

Context and Motivation Systematically engineering a good user experience (UX) into a computer-based system under development demands that the user requirements of the system reflect all needs, including emotional, of all stakeholders. User requirements address two different types of qualities: pragmatic qualities (PQs), that address system functionality and usability, and hedonic qualities (HQs) that address the stakeholder's psychological well-being. Studies show that users tend to describe such satisfying UXes mainly with PQs, and that some users seem to believe that they are describing a HQ when they are actually describing a PQ. Question/Problem The problem is to see if classification of any user requirement as PQ-related or HQ-related is difficult, and if so, why. Principal Ideas/Results We conducted a controlled experiment in which twelve requirements-engineering and UX professionals, hereinafter called "classifiers" classified each of 105 user requirements as PQ-related or HQ-related. The experiment shows that neither (1) a classifier's involvement in the project from which the requirements came nor (2) the classifier's use of a detailed model of the qualities in addition to the standard definitions of "PQ" and "HQ" has a positive effect on the consistency of the classifier's classification with that of others. Contribution The experiment revealed that classification of user requirements is a lot harder than initially assumed.