Deep learning-based automated detection, genetic signature identification and classification of oral lesions for early diagnosis of oral cancer

Abstract

Oral cancer, particularly oral squamous cell carcinoma (OSCC), remains a major global health burden where early detection is critical for improving patient outcomes. Conventional diagnostic approaches, such as biopsy and histopathological evaluation, are invasive and may not always facilitate the timely identification of early-stage lesions. In this study, we propose a multimodal computational framework that integrates deep learning–based image analysis with transcriptomic profiling to improve the early detection and biological understanding of oral cancer.

The proposed system employs an ensemble of three convolutional neural network architectures—DenseNet201, InceptionResNetV2, and MobileNetV2, to analyze histopathological and clinical oral images. Model predictions are integrated using Dempster–Shafer Theory (DST) to address uncertainty and enhance classification robustness. The framework was evaluated using two publicly available datasets: an OSCC histopathology dataset containing 5192 images and a Dental Condition dataset comprising 13,172 clinical images across seven oral disease categories. The ensemble model achieved an accuracy of 92% for binary OSCC classification and 70% for multiclass oral disease classification, outperforming individual base models.

To complement imaging-based predictions, transcriptomic analysis of the GEO dataset GSE37991 was performed to identify differentially expressed genes associated with OSCC. Network and pathway enrichment analyses revealed significant involvement of extracellular matrix (ECM)–receptor interaction and cytochrome P450–mediated metabolic pathways, highlighting molecular mechanisms relevant to tumor progression and drug metabolism.

Together, these findings demonstrate the potential of integrating artificial intelligence–driven image analysis with transcriptomic insights to support improved characterization of oral cancer. While the results highlight the promise of this approach, the framework should be considered a proof-of-concept, and further validation using larger, multi-center clinical datasets will be necessary before translation into routine diagnostic practice.