A preliminary psychometric evaluation of the activity ordering task with a metacognitive facet (AOT-M).

Abstract

Purpose: The Activity Ordering Task with a metacognitive facet (AOT-M) was developed, in our previous work, to address the disconnect between traditional working memory (WM) tasks and everyday WM demands, the lack of culturally sensitive, context-based WM tasks in India and enhance participant engagement. The present study aims to provide preliminary evidence of the AOT-M's psychometric properties among a non-clinical adult population, evaluate its sensitivity to cognitive and metacognitive changes with aging, establish construct validity, ecological validity, concurrent validity and test-retest reliability.

Methods: Ninety neurotypical adults, evenly distributed across three age groups, participated in the study. Descriptive statistics examined the distribution of performance spans and estimation discrepancies across age groups and the age-related statistical differences were evaluated using the Kruskal-Wallis Test. Construct validity was assessed using Rasch analysis, while ecological validity was evaluated with the Multidimensional Assessment of Research in Context (MARC) tool. Concurrent validity with sentence ordering and digit letter ordering tasks, was determined through Pearson's correlation coefficient and test-retest reliability was assessed using the Intraclass Correlation Coefficient and Bland-Altman plots.

Results: The patterns observed in WM performance spans and estimation discrepancies highlighted the task's sensitivity to aging related cognitive and metacognitive changes. Evidence from the MARC tool substantiated ecological validity, and concurrent validity was demonstrated through significant correlations with established WM tasks. While Rasch analysis supported construct validity, moderate person reliability indicated some limitations in task sensitivity. The AOT-M demonstrated good test-retest reliability.

Conclusion: Overall, the study provides preliminary evidence of the AOT-M's good psychometric properties within a neurotypical adult sample, suggesting it to be a promising addition to the cognitive communicative toolbox for Speech Language Pathologists.