Racial/ethnic reporting differences in cancer literature regarding machine learning vs. a radiologist: a systematic review and meta- analysis

Abstract

Background: Machine learning (ML) has emerged as a promising tool to assist physicians in diagnosis and classification of patient conditions from medical imaging data. However, as clinical applications of ML become more common, there is concern about the prevalence of ethnoracial biases due to improper algorithm training. It has long been known that cancer outcomes vary for different racial/ethnic groups. Methods: We reviewed 84 studies that reported results of ML algorithms compared to radiologists for cancer prediction to evaluate if algorithms targeted at cancer prediction account for potential ethnoracial biases in their training samples. The search engines used to extract the articles were: PubMed, MEDLINE, and Google Scholar. All studies published before May 2022 were extracted. Two researchers independently reviewed 115 articles and evaluated them for incorporation and inclusion of demographic information in the algorithm. Exclusion criteria were if an inappropriate imaging type was used, if they did not report benign vs. malignant cancer results, if the algorithm was not compared to a board-certified radiologist, or if they were not in English. Results: Of the 84 studies included, 87% (n=73) reported demographic information and 38% (n=32) evaluated the effect of demographic information on model performance. However, only about 11% (n=9) of the articles reported racial/ethnic groups and about 4% (n=3) incorporated racial/ethnic information into their models. Of the nine studies that reported racial/ethnic information, the specified racial/ethnic minorities that were included the most were White/Caucasian (n=9/9) and Black/African American (n=8/9). Asian (n=4/9), American Indian (n=3/9), and Hispanic (n=2/9) were reported in less than half of the studies. Conclusions: The lack of inclusion of not only racial/ethnic information but also other demographic information such as age, gender, body mass index (BMI), or patient history is indicative of a larger problem that exists within artificial intelligence (AI) for cancer imaging. It is crucial to report and consider demographics when considering not only AI for cancer, but also overall care of a cancer patient. The findings from this study highlight a need for greater consideration and evaluation of ML algorithms to consider demographic information when evaluating a patient population for training the algorithm.