Usability and Acceptability of a Symptom Checker App by Patients Seeking Urgent Care.

Abstract

Abstract: BACKGROUND: Diagnostic errors are common in medical care and can significantly affect patient outcomes. Symptom checker apps promise to help patients understand their symptoms, possible diagnoses, and actions to take, but lack evidence on accuracy, safety, and usability.

Abstract: OBJECTIVES: This study aimed to evaluate the usability and acceptability of the Ada symptom checker app for patients in urgent primary care (PC) settings, and extended it with more diverse data from an earlier emergency department (ED) study.

Abstract: METHODS: Patients entered their symptoms into Ada before assessment by a physician. Both the PC study and the ED study compared Ada's diagnoses to the treating physicians' diagnoses. Usability and acceptability were evaluated using a questionnaire based on the Technology Acceptance Model (TAM). Factors influencing usability were analyzed using linear and logistic regression. Quality of clinical history was also evaluated.

Abstract: RESULTS: The study included 214 urgent PC patients and 40 ED patients. Usability scores were generally high; 73% of PC patients agreed or strongly agreed with the seven TAM usability questions, and ED patients scored higher at 84% (with more direct assistance). Predictors of lower usability scores were age over 50 and limited technology experience. Sex, race, ethnicity, socioeconomic status, and educational level were not significant. In the PC study, Ada's top three diagnoses matched at least one treating physician diagnosis in 109/172 (63.3%) of cases with clear final diagnoses (sensitivity); three independent physicians had median matching of 124/172 (72.1%) on Ada summaries. Clinical history was rated complete in two-thirds of cases.

Abstract: CONCLUSION: The usability and acceptance of Ada were rated highly except by people older than 50 or with lower technology experience. Extensive research is needed to validate the usability of different diagnostic tools, including symptom checkers and large language models (used widely for symptom checking), across diverse populations, and their effects on patient decision-making and care-seeking behavior.