Methodologies for subjective performance assessment of collaboration

Our goal is to develop methodologies for measuring and analyzing subjective performance of users collaborating in immersive 3D spaces such as Virtual Collaboration Arena (VirCA). The collaborative task in the study was to plan a schedule for a foreign student group spending a weekend in Budapest, Hungary with shared digital representations of documents and other information. The performance is measurable with objective indicators, but a weekend program experience is not an objective category, because includes preferences, aspects, judgement and personal attitudes. Conventionally, users' subjective performance is difficult to measure in collaborative tasks. Q-sorting technique was applied to rank the programs, analyzing them with simple descriptive statistics and Q-methodology.