CVRT: Cognitive Visual Recognition Tracker

Abstract

Studies on visual attention of patients with Alzheimer's disease and Dementia is a promising way for keeping track of the individual patient's image recognition ability over. This research seeks to expand upon the current applications of combining the Android operating system with TensorFlow by providing a visual question answering platform for image analysis. This application, Cognitive Visual Recognition Tracker (CVRT), provides an entry point by which the user can ask questions concerning any image of their choosing, and then receive cumulative metrics over time to better assess any diminishing cognitive ability (i.e. Alzheimer's patients). In this work, recurrent neural networks as well as semantic analysis are leveraged to provide an interactive VQA experience. One of the main objectives of CVRT is for physicians to be able to determine trends from patient data that could either be applicable to the individual patient, or to many patients if an aggregate is formed from many individual datasets. On an individual level, these metrics would provide a way for the physician to monitor daily cognitive capability, whereas on a grander scale, these joint datasets could be used to provide better overall treatment for the disease with the future inclusion of predictive analytics. The final contribution is an interactive metrics platform by which other users can assess the primary user's cognitive capacity based on features of their questioning, and to then provide them with accurate trending or possible remediation plans based on their condition.