PlantView: Integrating deep learning with 3D modeling for indoor plant augmentation

Abstract

Indoor plant recognition poses significant challenges due to the variability in lighting conditions, plant species, and growth stages. Despite the growing interest in applying deep learning techniques to plant data, there still needs to be more research focused on the automatic recognition of indoor plant species, highlighting the need for real-time, automated solutions. To address this gap, this study introduces a novel approach for real-time identification and visualization of indoor plants using a Convolutional Neural Network (CNN)-based model called PlantView, integrated with Augmented Reality (AR) for enhanced user interaction. The proposed PlantView model not only accurately classifies the plant species but also visualizes them in a 3D AR environment, allowing users to interact with virtual plant models seamlessly integrated into their real-world surroundings. We developed a custom dataset comprising over 28,000 images of 48 different plant species at various growth stages, captured under diverse lighting conditions and camera settings. Our proposed approach achieves an impressive accuracy of 98.20 %. To validate the effectiveness of PlantView model, we conduct extensive experiments and compared its performance against state-of-the-art methods, demonstrating its superior accuracy and processing speed. The results indicate that our method is not only highly effective for real-time indoor plant recognition but also offers practical applications for enhancing indoor plant care and visualization. This research offers a comprehensive solution for indoor plant enthusiasts and professionals, combining advanced computer vision techniques with immersive AR visualization to revolutionize the way indoor plants are identified, visualized, and integrated into living spaces.