Clinician's Artificial Intelligence Checklist and Evaluation Questionnaire: Tools for Oncologists to Assess Artificial Intelligence and Machine Learning Models.

Abstract

Advancements in oncology are accelerating in the fields of artificial intelligence (AI) and machine learning. The complexity and multidisciplinary nature of oncology necessitate a cautious approach to evaluating AI models. The surge in development of AI tools highlights a need for organized evaluation methods. Currently, widely accepted guidelines are aimed at developers and do not provide necessary technical background for clinicians. Additionally, published guides introducing clinicians to AI in medicine often lack user-friendly evaluation tools or lack specificity to oncology. This paper provides background on model development and proposes a yes/no checklist and questionnaire designed to help oncologists effectively assess AI models. The yes/no checklist is intended to be used as a more efficient scan of whether the model conforms to published best standards. The open-ended questionnaire is intended for a more in-depth survey. The checklist and the questionnaire were developed by clinical and AI researchers. Initial discussions identified broad domains, gradually narrowing to model development points relevant to clinical practice. The development process included two literature searches to align with current best practices. Insights from 24 articles were integrated to refine the questionnaire and the checklist. The developed tools are intended for use by clinicians in the field of oncology looking to evaluate AI models. Cases of four AI applications in oncology are analyzed, demonstrating utility in real-world scenarios and enhancing case-based learning for clinicians. These tools highlight the interdisciplinary nature of effective AI integration in oncology.