ClinicYA: An Application for Pill Identification Using Deep Learning and K-means Clustering

Abstract

Pill Identification is one of the most important tasks to assure medication safety. With a high-quality smartphone camera, we can create a mobile-based application to identify unknown pills automatically. However, most existing studies can fail to detect and identify pills under unconstrained real-world conditions. To overcome the difficulty in identifying pills in practical usage, we present the design, implementation, and evaluation of a mobile-based application called ClinicYA. The development of ClinicYA involves key processes: a pill recognition model based on the Mask-RCNN algorithm that extracts the shape of pills and a color clustering and matching template in the RGB and HSV color model. The proposed application, ClinicYA, achieves over 99.27% accuracy in the localization and recognition of pill shapes. For color detection, our approach achieves 93.85% accuracy in the HSV color model for single color identification and up to 90.5% in the HSV color model for two color identification.