Deep Learning and Image Generator Health Tabular Data (IGHT) for Predicting Overall Survival in Patients With Colorectal Cancer: Retrospective Study.

Abstract

Background: Recent advances in artificial intelligence (AI) have contributed to improved predictive modeling in health care, particularly in oncology. Traditional methods often rely on structured tabular data, but these approaches can struggle to capture complex interactions among clinical variables. Image generator for health tabular data (IGHT) transform tabular electronic medical record (EMR) data into structured 2D image matrices, enabling the use of powerful computer vision-based deep learning models. This approach offers a novel baseline for survival prediction in colorectal cancer by leveraging spatial encoding of clinical features, potentially enhancing prognostic accuracy and interpretability.

Objective: This study aimed to develop and evaluate a deep learning model using EMR data to predict 5-year overall survival in patients with colorectal cancer and to examine the clinical interpretability of model predictions using explainable artificial intelligence (XAI) techniques.

Methods: Anonymized EMR data of 3321 patients at the Gil Medical Center were analyzed. The dataset included demographic details, tumor characteristics, laboratory values, treatment modalities, and follow-up outcomes. Clinical variables were converted into 2D image matrices using the IGHT. Patients were stratified into colon and rectal cancer groups to account for biological and prognostic differences. Three models were developed and compared: a conventional artificial neural network (ANN), a basic convolutional neural network (CNN), and a transfer learning-based Visual Geometry Group (VGG)16 model. Model performance was assessed using accuracy, sensitivity, specificity, precision, and F1-scores. To interpret model decisions, gradient-weighted class activation mapping (Grad-CAM) was applied to visualize regions of the input images that contributed most to predictions, enabling identification of key prognostic features.

Results: Among the tested models, VGG16 exhibited superior predictive performance, achieving an accuracy of 78.44% for colon cancer and 74.83% for rectal cancer. It showed notably high specificity (89.55% for colon cancer and 87.9% for rectal cancer), indicating strong reliability in correctly identifying patients likely to survive beyond 5 years. Compared to ANN and CNN models, VGG16 achieved a better balance between sensitivity and specificity, demonstrating robustness in the presence of moderate class imbalance within the dataset. Grad-CAM visualization highlighted clinically relevant features (eg, age, gender, smoking history, American Society of Anesthesiologists physical status classification (ASA) grade, liver disease, pulmonary disease, and initial carcinoembryonic antigen [CEA] levels). Conversely, the CNN model yielded lower overall accuracy and low specificity, which limits its immediate applicability in clinical settings.

Conclusions: The proposed IGHT-based deep learning model, particularly leveraging the VGG16 architecture, demonstrates promising accuracy and interpretability in predicting 5-year overall survival in patients with colorectal cancer. Its capability to effectively stratify patients into risk categories with balanced sensitivity and specificity underscores its potential utility as a clinical decision support system (CDSS) tool. Future studies incorporating external validation with multicenter cohorts and prospective designs are necessary to establish generalizability and clinical integration feasibility.