This study introduces an approach to classifying histopathological images for detecting dysplasia in oral cancer through the fusion of support vector machine (SVM) classifiers trained on deep learning features extracted from InceptionResNet-v2 and vision transformer (ViT) models. The classification of dysplasia, a critical indicator of oral cancer progression, is often complicated by class imbalance, with a higher prevalence of dysplastic lesions compared to non-dysplastic cases. This research addresses this challenge by leveraging the complementary strengths of the two models. The InceptionResNet-v2 model, paired with an SVM classifier, excels in identifying the presence of dysplasia, capturing fine-grained morphological features indicative of the condition. In contrast, the ViT-based SVM demonstrates superior performance in detecting the absence of dysplasia, effectively capturing global contextual information from the images. A fusion strategy was employed to combine these classifiers through class selection: the majority class (presence of dysplasia) was predicted using the InceptionResNet-v2-SVM, while the minority class (absence of dysplasia) was predicted using the ViT-SVM. The fusion approach significantly outperformed individual models and other state-of-the-art methods, achieving superior balanced accuracy, sensitivity, precision, and area under the curve. This demonstrates its ability to handle class imbalance effectively while maintaining high diagnostic accuracy. The results highlight the potential of integrating deep learning feature extraction with SVM classifiers to improve classification performance in complex medical imaging tasks. This study underscores the value of combining complementary classification strategies to address the challenges of class imbalance and improve diagnostic workflows.

The authors have declared no competing interest.

This study did not receive any funding.

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

NDB-UFES: An oral cancer and leukoplakia dataset composed of histopathological images and patient data. URL: https://data.mendeley.com/datasets/bbmmm4wgr8/4

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes