Enhancing online speech and language assessment: Item development for the remote adult language experiment (ReAL-E) tool

Abstract

Purpose: Recent technological advancements in psychology have yielded multiple unsupervised online tools for measuring cognitive skills. However, these tools do not measure key features of communication abilities in adults, such as productive speech or oral language. To address this gap, we developed the Remote Adult Language Experiment (ReAL-E) using a three-step approach that incorporates domain identification, expert evaluation, and piloting testing.

Method: Step 1 focused on domain and task identification by collaborating with experts and reviewing the literature. Step 2 included two rounds of evaluation by content and lay experts to assess the initial tool design, identify areas for refinement, and solicit feedback for desired features. Step 3 involved piloting the tool in a sample of adults recruited online.

Results: The ReAL-E tool adapted three speech planning and production tasks and four oral and written language tasks into online versions. Feedback from content and lay experts led to significant improvements, including enhanced task instructions, presentation modifications, and the incorporation of features to address potential confounds. Pilot testing demonstrated that adults perform within expected ranges, and five out of seven tasks had good to excellent preliminary reliability.

Conclusion: The development of the ReAL-E tool contributes to the field by offering a scalable, efficient method for assessing a broad range of communication skills in adults, paving the way for future research and clinical applications in developmental communication disorders. Our study underscores the collaborative effort and innovative approach required for creating a digital assessment tool that promises to expand research capabilities and improve diagnostic processes in the discipline.