Dependence of observer task on conclusions drawn from in silico trials evaluating the performance of full-field digital mammography and digital breast tomosynthesis.

Abstract

Purpose: We aim to refine the task-based evaluation of full-field digital mammography (FFDM) and digital breast tomosynthesis (DBT) through in silico trials (ISTs). Previous ISTs mostly employ lesion detection tasks for task-based performance evaluation, which differ from clinical practice where the task normally involves the radiologists both detecting whether a suspicious lesion is present and rating how likely it is that the lesion is malignant. We hypothesize that differing conclusions may result from ISTs based on the defined task.

Approach: The shape of the masses was employed as a surrogate indicator for malignancy, with spiculated masses representing malignant lesions and lobular masses representing benign lesions. A convolutional neural network (CNN) model observer was then trained to differentiate between spiculated and nonspiculated masses using Monte Carlo-simulated breast images. This approach leverages prior research demonstrating that CNN-based frameworks can approximate the performance of an ideal observer. We systematically evaluated the effects of varying dose levels, detector pixel size, and projection angular range on the CNN model observer's performance in both detection and classification tasks, assessing the performance of both FFDM and DBT systems.

Results: Our findings demonstrate significant variations in conclusions drawn from IST models depending on whether the task is lesion detection or classification. Specifically, we observed that varying average glandular dose levels from 2.0 to 0.5 mGy had little effect on the detection of masses, whereas a small but significant decrease in performance with reduced dose was observed with the classification task across FFDM and DBT. Similarly, reduced spatial resolution resulted in a small but significant decrease in performance with the classification task for FFDM. For DBT ISTs, we also observed that the preferred angular range varies depending on whether the task is detection or classification.

Conclusions: Integrating classification tasks into ISTs and potentially physical phantom studies can provide additional information in the evaluation of clinical breast imaging systems. This methodology can enhance the reliability of performance assessments for new breast imaging technologies. Depending on the study's objective, ISTs and physical phantom studies should aim to employ tasks that closely model actual clinical scenarios.