Development of a context-aware integrated training module based on large language models for continuous education in radiation protection

Abstract

Purpose

Reliability of LLM-based AI chatbots can be enhanced by using domain-specific models with a controlled knowledge base, especially relevant in the context of education and training of healthcare professionals and researchers. The aim was to produce an LLM data model with the goal of continuous education in radiation protection, that allows users to access reliable scientific information by querying it on specific topics, eliminating the need for manually perusing educational materials.

Method

A domain-specific LLM data model was developed and trained using custom knowledge from several domains, tested applying various test scenarios, and fine-tuned to ensure optimal selections of hyperparameters including top k, chunk size, temperature, max tokens etc.

Results

The final model produced reliable and accurate answers to a variety of users and queries based on controlled educational materials. Embedding model and similarity cutoff had the greatest impact on model performance.

Conclusion

The developed model was trained and validated on radiation protection training material, allowing users to access information on topics including radiobiology and radiation protection in a quick and reliable manner.