Cognitive Assistance for Dementia Patients

Abstract

This paper addresses the growing global health concern of dementia through the introduction of an innovative Cognitive Assistance System to locate personal items. Our proposed system utilizes smart spectacles equipped with a camera, driven by an ESP 32 CAM to capture the user's field of vision and Live stream. This Real-time video feed is processed using YOLO v8 for object detection, and the extracted labels data is stored in a MongoDB database. This data is then used to find the objects and recognize the location of the objects, using a list of object labels seen at that time. The proposed methodology involves continuous video streaming, object detection, and scene recognition. Notably, a Random Forest Classifier, trained on a custom dataset, attains an average accuracy of 91 % in recognizing indoor scenes based on label data. A DialogFlow-integrated chatbot on Telegram assists users in locating personal belongings and retrieves details on the scene detected and time of objects. Hardware development focuses on creating compact, comfortable, and lightweight spectacles tailored for regular use. Results showcase the effectiveness of the Random Forest Classifier and YOLO v8 in scene detection and object recognition. The seamless integration of the chatbot with Telegram enhances user accessibility, representing a significant advancement in providing practical support for dementia patients and addressing challenges for caregivers. Future work involves integrating voice assistants, refining accuracy, and expanding capabilities for indoor navigation, aiming to extend the solution's reach and enhance the lives of those affected by cognitive decline.