Classification of Video Capsule Endoscopy Images Using Visual Transformers

Abstract

Cancers related to the gastrointestinal tract have a high incidence rate in the population, with a high mortality rate. Videos obtained through endoscopic capsules are essential for evaluating anomalies that can progress to cancer. However, due to their duration, which can reach 10 hours, they demand great attention from the medical specialist in their analysis. Machine learning techniques have been successfully applied in developing computer-aided diagnostic systems since the 1990s, where Convolutional Neural Networks (CNNs) have become very successful for pattern recognition in images. CNNs use convolutions to extract features from the analyzed data, operating in a fixed-size window and thus having problems capturing pixel-level relationships considering the spatial and temporal domains. Otherwise, transformers use attention mechanisms, where data is structured in a vector space that can aggregate information from adjacent data to determine meaning in a given context. This work proposes a computational method for analyzing images extracted from videos obtained by endoscopic capsules, using a transformer-based model that helps diagnose of gastrointestinal tract abnormalities. Preliminary results are promising. The classification task of 11 classes evaluated on the publicly available Kvasir-Capsule dataset yielded an average value of 99.70% of accuracy, 99.64% of precision, 99.86% of sensitivity, and 99.54% of f1-score.