Editorial Commentary: The Importance of Rigorous Testing for Artificial Intelligence Solutions: Don't Forget to Flip Over the Page and Answer All the Questions!

Abstract

Manual Current Procedural Terminology coding for billing purposes is labor-intensive, error-prone, and imposes a substantial administrative burden on physicians and care teams. As interest in artificial intelligence applied to health care inefficiencies grows, appropriately designed solutions may optimize billing and coding workflows by improving efficiency and accuracy. Recent locally deployed models have shown promise in providing scalable function while protecting patient data. However, these models have shown success only when provided contextual clues in the setting of verification, with performance dropping to near chance when such context is withheld. In addition, the current evaluation has been limited to verification rather than prospective code generation from operative reports. These models also have not been shown to distinguish closely related codes or apply multiple codes to a single procedure, severely limiting their applicability to real-world practice. Future investigations must therefore focus on developing models that prospectively generate Current Procedural Terminology codes, independently verify and flag incorrect codes, and achieve external validity across multiple institutions, disciplines, and documentation styles before they are ready for clinical implementation.