MedImgGuard: evidence-verified LLM code review for data leakage and evaluation contamination in medical imaging AI code

Abstract

Data leakage and evaluation contamination can substantially inflate reported performance in medical imaging artificial intelligence systems, yet these risks are often introduced through concrete repository-level implementation decisions rather than explicit methodological statements. Existing static analysis, machine learning bug detection, and large language model (LLM)-based code review methods are limited in detecting such protocol-semantic defects because they usually lack medical identity modeling, evaluation-stage reasoning, traceable evidence construction, and claim-level verification. This paper presents MedImgGuard, an evidence-verified LLM code review framework for detecting data leakage and evaluation contamination in medical imaging AI repositories. MedImgGuard constructs a Protocol Evidence Graph from source code, configurations, and metadata schema to represent program-flow dependencies, medical identity semantics, evaluation-stage relations, fit-apply dependencies, metric-decision relations, and evidence anchors. It then checks protocol invariants for identity-disjoint splitting, training-only fitting, and test-stage isolation, and extracts compact evidence slices for candidate findings. The LLM is used as an evidence-to-review translator rather than an unconstrained detector, and its generated atomic claims are verified against repository evidence before reporting. We evaluate MedImgGuard on controlled defect and real-world repository benchmarks. The results show that MedImgGuard achieves the best overall detection performance, with a precision of 0.86, recall of 0.82, and F1-score of 0.84, while also improving localization quality, evidence recovery, review faithfulness, and actionability. These results demonstrate that protocol-aware evidence construction and claim-level verification can make LLM-assisted review more reliable for medical imaging AI code.