Enhancing visual speech perception through deep automatic lipreading: A systematic review

Abstract

Communication involves exchanging information between individuals or groups through various media sources. However, limitations such as hearing loss can make it difficult for some individuals to understand the information delivered during speech communication. Conventional methods, including sign language, written text, and manual lipreading, offer some solutions; however, emerging software-based tools using artificial intelligence (AI) are introducing more effective approaches. Many approaches rely on AI to improve communication quality, with the current trend of leveraging deep learning being a particularly effective tool. This paper presents a comprehensive Systematic Literature Review (SLR) of research trends in automatic lipreading technologies, a critical field in enhancing communication among individuals with hearing impairments. The SLR, which followed the Preferred Reporting Items for Systematic Literature Review and Meta-Analysis (PRISMA) protocol, identified 114 original research articles published between 2014 and mid-2024. The essential information from these articles was summarized, including the trends in automatic lipreading research, dataset availability, task categories, existing approaches, and architectures for automatic lipreading systems. The results showed that various techniques and advanced deep learning models achieved convincing performance to become state-of-the-art in automatic lipreading tasks. However, several challenges, such as insufficient data quantity, inadequate environmental conditions, and language diversity, must be resolved in the future. Furthermore, many improvements have been made to the deep learning models to overcome these challenges and become a massive solution, particularly for automatic lipreading tasks in the near future.