P269 Spaciotemporal machine learning analysis of complete small bowel capsule endoscopy videos for prediction of outcomes in Crohn’s disease

Capsule endoscopy (CE) is a prime modality for diagnosis and monitoring of Crohn’s disease (CD). However utilization of CE for monitoring of CE is hampered by prolonged reading time and interobserver variability. Machine learning (ML) techniques such as convolutional neural networks (CNN) are capable of accurate detection and grading of inflammatory findings on isolated images from CE in CD patients, with accuracy above 95%. However, ML-based analysis of complete CE films have not yet been reported; moreover, the predictive utility of ML of CE in CD for disease outcomes has not been examined so far. The study cohort included treatment-naïve CD patients that have performed CE (SBIII, Medtronic) within 6 months of diagnosis of CD and had a minimal follow-up of 6 months. Complete small bowel videos (first duodenal to first cecal image) were extracted using the RAPID reader software, V9.0. Prior to extraction, CE videos were scored using the Lewis score (LS). Clinical, endoscopic and laboratory data were extracted from the electronic medical records. All patients were classified as per start of biological therapy during follow-up. Machine learning analysis was performed by Intel Inc. using TimeSformer computer vision algorithm developed by Facebook. Timesformer algorithm utilizes spaciotemporal The patient cohort included 103 patients (54 (52%) female, median age- 25 years, interquartile ratio (IQR) 21–40). The median duration of follow-up was 941 (363–1644) days. The median LS was 450 (IQR-225–900). Biological therapy was initiated by 41(39.8%) of the patients (adalimumab -20 (48.7%), infliximab-14 (34,1%), vedolizumab- 7 (17%) patients, respectively). TimeSformer algorithm achieved training and testing accuracy of 91% and 79% respectively, with area under the curve (AUC) of 0.79 for prediction of the need for biologic therapy. In comparison, the AUC for LS was 0.69. The required time for analysis per complete video was 440ms on a single GPU. Spaciotemporal analysis of complete small bowel CE videos of newly diagnosed CD patients achieved accurate prediction of the need for biological therapy. The accuracy was superior to that of human reader index. Following future validation, this approach will allow for fast and accurate personalization of treatment decisions in CD.