Exploring the measurement validity of the Your Experience of Service survey.

Objective: To provide critical feedback to public mental health services, Australian governments developed and implemented the Your Experience of Service survey to capture consumers' experiences of mental health care. The grey literature reports several domain structures of the survey, but these neither used data collected during routine service delivery nor have been peer-reviewed. This study sought to confirm a widely used six-domain structure of the Your Experience of Service and examine for other structures, including one informed by the CHIME five-domain framework based on the concept of personal recovery. Personal recovery is considered best mental health service practice.

Methods: Consumer-reported Your Experience of Service data for years 2019-21 were obtained from Queensland (n = 8564) and South Australia (n = 2536) Health Departments. Exploratory and confirmatory factor analyses on subsets of data were conducted to determine the best fit domain structure. Means, Cronbach's alphas and correlations were used to examine structure and reliability of the measured factors.

Results: The four factors, respectful environment, making a difference, providing information and support and supporting active participation of consumers, carers and family, were found to statistically best represent the data sets. The original six domains were partly supported within the four domains however the hypothesised five-factor CHIME personal recovery structure was not supported by the analyses.

Conclusion: The findings highlight that a four-domain structure be reported in future and illustrate opportunities to improve the Your Experience of Service survey's ability to more accurately measure consumers' experiences with mental health services, including items measuring personal recovery, and therefore maximise its value and utilisation in practice.