TurkRadLLM: A Compact MedGemma-Based Model for Structuring Turkish Lumbar and Cervical Spine MRI Reports-A Multi-reader Validation Study.

Abstract

To develop and validate a compact, locally deployable MedGemma-based model converting unstructured Turkish spine MRI reports into structured six-field output. In this retrospective single-institution study, 5210 lumbar and 5017 cervical Turkish spine MRI reports were used to fine-tune MedGemma 4B IT with QLoRA. Silver-standard labels from a proprietary large language model served as training targets and as the reference for automated fidelity analyses, but not as the clinical reference standard. Evaluation comprised multi-reader validation of 180 authentic reports interleaved with 20 adversarial cases, a second round in which the readers scored the silver labels themselves, and comparison on the full held-out set against a rule-based baseline, the non-fine-tuned base model, and a much larger model. Every output was valid, schema-conformant JSON, and the grouped acceptance rate was 100% (95% CI 97.9-100), though one reader judged negative-finding lists incomplete in 57.8% of reports. On the full held-out set (1025 reports), fine-tuning increased agreement with the silver-label impression from 17.7% to 67.1% and urgent-flag sensitivity against the silver reference from 18.8% to 62.5% (both p < .001); the larger non-fine-tuned model reached 27.5% and 50.0%, and a rule-based baseline 99.0% on body region. When the readers scored the silver labels themselves, the model was rated higher on negative findings and lower on the impression. Deployment required 14.75 GB of GPU memory on a consumer card. Fine-tuning a compact medical language model produced structured output rated acceptable by clinicians, though negative-finding completeness remained reader-dependent. The gain lay in content-level fidelity rather than output validity.