From Prototype to Practice: Mixed Methods Study of Human-Centered Design and Usability Evaluation of Augmented Reality for Complex Oncology Surgical Workflows.

Abstract

Background: Hepato-pancreato-biliary (HPB) oncology involves complex surgical planning and multidisciplinary decision-making that present significant cognitive, spatial, and communicative challenges. While augmented reality (AR) technologies are emerging as valuable tools in surgical planning, most existing systems are developed in a technologically deterministic manner, often without consideration of patients' lived experiences or the collaborative needs of multidisciplinary clinical teams. This limits their integration into complex workflows and their potential for meaningful clinical impact.

Objective: This study aimed to co-design and evaluate a human-centered AR system that supports spatial understanding, emotional sensitivity, and collaborative decision-making in HPB surgical oncology.

Methods: This mixed methods human-centered design study was conducted between February 2024 and September 2025 and comprised 2 phases. In Phase 1, convenience sampling was used to recruit 14 oncology health care professionals (surgeons, oncologists, and psycho-oncology specialists) and 10 patients with cancer from multidisciplinary oncology care settings to participate in co-design workshops and semistructured interviews exploring the cognitive, communicative, emotional, and workflow requirements for an AR system supporting HPB surgical oncology. Qualitative data were analyzed using reflexive thematic analysis and informed iterative prototype development. In Phase 2, the resulting AR prototype was evaluated through usability testing with 10 hepatobiliary clinicians using simulated multidisciplinary tumor board planning scenarios. Usability was assessed using the System Usability Scale (SUS), supplemented by observational data and qualitative feedback.

Results: The co-design process identified intersecting spatial, cognitive, emotional, and communicative challenges in HPB surgical oncology. These findings informed the iterative development of an AR prototype that functioned as a shared cognitive and communicative workspace. The system enabled collaborative manipulation of 3D anatomical models, emotionally calibrated visualizations, and seamless integration with existing planning workflows. The prototype achieved a mean SUS score of 67.25 (SD 7.95), indicating acceptable usability. In simulated multidisciplinary tumor board scenarios, clinicians reported improved spatial understanding, reduced cognitive load, and enhanced interdisciplinary communication. Patients with cancer participating in the co-design phase valued the ability to engage with their own anatomy in an emotionally manageable way and reported that interactive visualization improved their understanding and sense of agency. Overall, the findings suggest that AR systems designed with human-centered methods can function not only as visualization tools but also as boundary objects that support communication and distributed cognition in complex cancer care settings.

Conclusions: A human-centered co-design process can guide the development of AR systems that extend beyond technical novelty to become emotionally attuned, cognitively supportive, and workflow-compatible tools in surgical oncology. By foregrounding the needs of both patients and professionals, AR can function as a boundary object that facilitates collaborative decision-making, empathetic communication, and multidisciplinary coordination in complex oncology care. This study provides actionable design recommendations for developing AR technologies that are meaningfully integrated into complex clinical contexts.