Development of a personalized conversational health agent to enhance physical activity for blind and low-vision individuals.

Abstract

Background: With the advancements in mobile health (mHealth) technologies, sighted individuals can benefit from mobile apps and wearable devices to more easily manage their physical activity (PA) and wellness data through intuitive touch gestures and effective data visualizations. However, for blind and low-vision (BLV) individuals, these conventional interaction methods are often challenging, not only limiting their ability to use these technologies but also potentially diminishing their motivation to adopt them to support health-promoting behaviors. We aimed to develop a health monitoring application called Personalized and Conversational Health Agent (PCHA) that supports BLV individuals with self-monitoring and management of their PA and wellness data (e.g., step count, exercise duration, calories burned, heart rate).

Methods: Drawing on social cognitive theory and insights from prior needs assessment research, five key design goals were established to guide the development of the app's core features and functionalities. PCHA leverages a large language model (LLM) to enable a conversational health agent that can be installed on iPhone and Apple Watch devices. This conversational interface is designed to ensure accessibility and inclusivity, offering PA management tools through a voice user interface (VUI) that minimizes the navigation challenges often associated with traditional touchscreen-based systems. To ensure evidence-based PA guidance, a thorough review of scientific literature and published PA guidelines was conducted. Finally, two blind accessibility experts conducted the accessibility testing.

Results: Accessible user interface (UI) designs, featuring high color contrast, large buttons, and a simple layout, were created using Figma. The main features and functionalities include: (I) a voice health interview to assess users' basic health information; (II) PA recommendations to guide users toward achieving their PA goals; (III) a chat feature enabling human-like conversations with the app; (IV) a PA scheduling and reminder feature with haptic feedback on the Apple Watch; and (V) an in-exercise mode that provides audible updates on heart rate, PA duration, and walking speed. The app's mobile accessibility was found to be satisfactory.

Conclusions: A follow-up study involving BLV research participants will be conducted to improve the app's accessibility and usability, and to update its features and functionalities. More research is needed to fully harness the potential of LLMs in the new mHealth system to motivate PA behaviors for BLV populations. To deliver truly personalized PA feedback for BLV individuals, mHealth app developer should incorporate PA and wellness data specific to the BLV population, along with their unique personal and contextual factors that influence PA behaviors.