Effective Design for Experiments on Small-Group Conversation: Insights From an Example Study.

Abstract

With current advances in experimental techniques, there is a renewed interest in studying communication behavior, reflecting a desire to improve our understanding of hearing disability and the effects of treatment interventions at the level of in-the-moment behaviors. Group conversations are among the most challenging situations for people with hearing loss. Experiments on group conversations are increasingly common and disproportionately more demanding than dyad studies to design and execute. Thorough design and planning are critical for successfully capturing valid behavioral data, highlighting the value of sharing behind-the-scenes experiences with the researcher community. We have completed a laboratory study of four-way group conversations involving people with and without hearing loss. This article describes the goals and compromises involved in our design choices and evaluates their effectiveness through participant feedback. Aspects covered include contrasts and covariates, group composition and physical arrangement, participant characteristics, hearing devices, participant experience, physical environment, conversational task, and measurement modalities. Next, we briefly describe the experiment's execution. Finally, we analyze and discuss participants' feedback and reflect on what proved effective, what did not, and what design "worries" proved founded or unfounded. We hope thereby to provide support and inspiration for others who may be faced with similar design challenges. The main message is that such an experiment can be carried out successfully and in such a way that the behavioral and self-report data collected are likely to carry a relatively high degree of ecological validity while still supporting experimental and statistical control.