WCE with real time polyp detection and segmentation using deep neural networks

Capsule robots are developing rapidly. The capsule robots perform non-invasive gastrointestinal endoscopy on patients. With the application of artificial intelligence in the capsule robot, the images collected by the capsule can be read efficiently and accurately offline. However, since the previous capsules either lacked wireless data transmission or only used Bluetooth data transmission, real-time interaction was not yet possible. It is very helpful for the diagnosis of the patient’s condition. However, due to the lack of real-time interaction, it is still only an early diagnosis and cannot be applied to real-time treatment. In other words, patients still need to undergo invasive surgery after the gastrointestinal capsule endoscopy. The capsule robot proposed in this paper transmits data through Wi-Fi, detects and locates the lesion in real-time using a deep neural network named SEG-YOLOv5. It provides real-time feedback of the endoscopy, which shows a potential of non-invasive treatment or surgery. The experimental result shows that the model detection mAP@0.5 of 10 fold cross-validation on the open-access dataset Kvasir-SEG reaches 97.77% and the dice coefficient is 85.03% for lesion segmentation.