Toward the Comprehensive Evaluation of Medical Text Generation by Large Language Models: Programmatic Metrics, Human Assessment, and Large Language Models Judgment

Abstract

This commentary discusses three evaluation approaches for assessing large language models' generation in healthcare: programmatic metrics, human assessment, and large language models judgment. No single approach can address all challenges; however, the combination of these three methods provides a pipeline toward the comprehensive evaluation of medical text generation.