Fine Tuned LLM With Lora-Q for Enhanced Health Literacy

Abstract

This study describes the implementation of sophisticated parameter-efficient strategies for fine-tuning the LLaMA-2-7b model on a carefully selected, Web-scraped medical dataset targeted at increasing health literacy. Designed to improve the contextual accuracy of medical dataset, the dataset consists of important fields: “question,” “answer,” “source,” and “focus area.” Using 4-bit quantization and Low-Rank Adaptation (LoRA), the model was tuned for low computational overhead and high-performance deployment. Post-optimization, the model showed a notable rise in linguistic metrics: the BLEU score rose from 0.1397 to 0.1486, the ROUGE score improved from 0.0510 to 0.0599, and the Translation Edit Rate (TER) dropped from 0.8714 to 0.8440, so highlighting the model’s increased capacity in producing accurate and contextually relevant medical information. The results highlight the effectiveness of using innovative NLP techniques to increase the accessibility and understanding of medical knowledge, therefore supporting the main objective of higher global health literacy.