Knowledge-driven deep learning approaches for computer vision tasks: A survey

Abstract

Hybrid artificial intelligence aims to integrate data-driven techniques with knowledge-based systems, offering a promising avenue to enhance artificial intelligence systems accuracy, interoperability, and explainability. Within this domain, neuro-symbolic artificial intelligence represents a sub-field focusing on merging specifically deep neural networks with knowledge-based systems for improved effectiveness. This paper provides a comprehensive overview of recent advancements in the field, specifically focusing on knowledge-driven training approaches for computer vision tasks where knowledge-based systems are deeply integrated into the deep neural networks training process. This integration takes advantage of structured domain knowledge to guide feature extraction. It improves robustness against noisy and incomplete data, allows more reliable and interpretable decision-making mechanisms, and facilitates better generalization in diverse and complex scenarios. These enhancements ultimately improve the overall performance of the neural networks. The presented approaches in this survey are categorized based on the integration level of knowledge within deep neural networks, including input integration, intermediate-level integration, and integration into the loss function. Additionally, the methodologies are sub-categorized based on the knowledge representation extracted from the knowledge-based systems before integration into the deep learning model. The integration methodology for each approach is highlighted to provide a comprehensive comparison between the different contributions. Through a survey of the literature, this paper identifies gaps in understanding the collaboration of knowledge-based systems and deep neural networks in the computer vision field. State-of-the-art approaches are analyzed and compared, evaluating their methodologies, integration knowledge strategy, and application domain. Our work also highlights the strengths and weaknesses of the approaches, discusses the challenges, and provides a critical review of their effectiveness. The paper concludes by exploring potential improvements and outlines future research directions to advance the integration of knowledge-based systems and deep neural networks.