A novel guidance framework for nasal rapid antigen tests with improved swab keypoint detection

The global impact of the COVID-19 pandemic has placed an unprecedented burden on healthcare systems. In this paper, we present a novel deep learning-based framework to guide individuals in performing nasal antigen rapid tests, with a particular focus on improving swab keypoint detection. Our system provides real-time feedback to participants on the correct execution of the test and may issue a certificate upon successful completion. While initially developed for COVID-19 antigen rapid tests, our versatile framework extends its applicability to various nasal screening tests, eliminating the need for specific information about the liquid solvent. To implement and evaluate our framework, we curated a comprehensive dataset with rapid test components and trained an object detection model to identify the position and size of all objects in each video frame. Addressing the challenge of swab depth classification, we propose a novel approach to locate and classify crucial swab points by a self-defined decision tree for depth assessment within the nasal cavity. The robustness of the proposed framework is validated with COVID-19 antigen rapid tests from various manufacturers. Experimental results demonstrate the remarkable performance of the framework in classifying the nasal placement of the swab, achieving an $F_1$-Score of 89.78%. Additionally, our framework attains an $F_1$-Score of 99.37% in classifying final test results on the test device.