Enhanced accuracy for classification of video capsule endoscopy images using multiple deep learning convolutional neural networks

Abstract

Background and aims

Video capsule endoscopy (VCE) is widely used in the detection of abnormalities in the small intestine. However, it remains challenging to correctly identify a limited number of possible abnormal images from tens of thousands of total images, and this impediment has limited expansion of the technology. More recently, artificial intelligence (AI) technology has been used in classifying VCE images from patients, but clinical-grade diagnostic accuracy (>99%) has not been achieved.

Methods

This study proposes a system for the automatic classification of a number of categories of unbounded VCE images with high accuracy by means of a transfer learning approach using multiple convolutional neural networks (CNNs). With this new approach, it is not necessary to implement image segmentation; thus, the feature extraction becomes automatic, and the existing models can be fine-tuned to obtain specific classifiers.

Results

More than 16,000 VCE GI images from normal individuals, including those with normal clean mucosa, the pylorus, the ileocecal valve, a reduced mucosal view due to luminal contents and lymphangiectasia (a normal variant), and patients with 5 pathologic states (angioectasia, bleeding, erosions, ulcers, and foreign bodies), were obtained from a publicly available data set. These were used in building, testing, and validating AI models for evaluating the diagnostic accuracy of our combined 17-CNN deep learning approach. Compared with a single CNN approach used by other research groups, our AI method, using 17 CNNs, achieved an overall diagnostic accuracy of 99.79%, with an accuracy of 100% for identifying bleeding and foreign bodies. The high accuracy was further shown in the confusion matrices, precision, recall, and F1 score.

Conclusions

We have developed accurate AI deep learning models for unbounded VCE image classification of various medical conditions in medical practice.