AAPM Task Group Report 332: Verification of vendor-provided data, tools, and test procedures in radiotherapy

Abstract

The purpose of this document is to provide the qualified medical physicist (QMP) with guidance on the critical evaluation and independent validation of “closed” or “black box” systems in a radiation oncology setting. Radiotherapy delivery systems and their associated subsystems are highly sophisticated and complicated. In recent years, vendors have worked closely with QMPs, information technology (IT) personnel, and clinical engineers to develop and ultimately provide an entire package of resources at the time of purchase of radiotherapy delivery equipment. Commissioning and routine quality assurance (QA) on these new, black-box systems is not necessarily more difficult or time-consuming; however, there are different factors to consider with black-boxes. Independence in the vendor's own validation techniques now becomes critical to establish. For vendor-provided tools, the user must understand the implications of working with a service tool or a true QA tool. Lastly, the user must determine which components of the system are unknown and differ from a classic white-box system. Understanding these characteristics of the black-box will guide the user in determining which quality management (QM) tools are applicable and which verification and validation (V&V) procedures to follow. Black-box systems are becoming more and more prevalent in the radiation oncology setting. For example, true black-box systems in the form of machine learning (ML) algorithms are already widely used within common treatment planning systems (TPS). These systems present a unique challenge to the QMP who is responsible for conducting independent V&V performance measurements on such systems. Although these systems require a different approach from classic treatment delivery systems, we present new terms for characterizing black-box systems and a methodology for using alternative methods of independent validation.