Appealing but Potentially Biasing - Investigation of the Visual Representation of Segmentation Predictions by AI Recommender Systems for Medical Decision Making

Abstract

Artificial intelligence (AI)-based recommender systems can help to improve efficiency and accuracy in medical decision making. Yet, it has been shown that a recommendation given by an algorithm can influence the human expert responsible for the decision. The strength and direction of this bias, induced by a computer-aided diagnosis workflow, can be influenced by the visual representation of the results. This study focuses on evaluating four frequently used visualization types (bounding box, segmentation mask, segmentation contour, and heatmap) for displaying segmentation results of medical data. A group of 24 medical experts specializing in pathology and radiology participated in the evaluation, assessing the subjective appeal of these visualizations. The study evaluated the pragmatic and hedonic quality of the visualizations based on a standardized questionnaire and specific criteria relevant to medical decision making. The findings indicate that the heatmap received the highest ratings for non-task-oriented aspects of the user experience. However, it exhibited significant inconsistencies among experts concerning task-oriented aspects and was perceived as the most biasing visualization type. On the other hand, the segmentation contour consistently received high ratings across various subscales. The results of the study contribute to better alignment between visualization techniques and user requirements for the development of future AI-based recommender systems.