Endoscopic neoplasia detection in inflammatory bowel disease (IBD) remains challenging. We developed and validated a novel artificial intelligence (AI) model for lesion detection and characterisation in 478 images from 30 patients with IBD, 10 of whom had a total of 25 neoplastic lesions (including 8 sessile serrated polyps); sensitivity and specificity for lesion detection were 93.5% and 80.6%, respectively. The IBD model was then further validated during a real-time endoscopic assessment of a further 30 consecutive patients with 25 neoplastic lesions found in 11/30 of them and achieved lesion detection rate, lesion per colonoscopy and neoplasia per colonoscopy of 90.4%, 4.6% and 0.96. respectively. The sensitivity and specificity of lesion characterisation were 87.5% and 80.6%, respectively.

Deep learning (DL) is a subset of AI that uses multilayered computer algorithms (also called deep artificial neural networks) to automatically learn representations of data with multiple levels of abstraction, thus avoiding some of the limitations of more traditional AI techniques that rely on hand-crafted feature extraction.1

The IBD-dedicated DL model in this study was developed using RetinaNet architecture with a ResNet-101 backbone for deep feature extraction. This is a one-stage detector that uses a focal loss function to eliminate the accuracy gap between this one-stage detector and two-stage detectors while running at a faster processing speed.2 Figure 1 illustrates the structure of the DL model.

Shows the RetinaNet deep learning architecture used to develop the IBD deep learning model in this study.

The characterisation function in this study is a binary classification of lesions into neoplastic or non-neoplastic. Neoplastic category includes adenoma (low or high grade), cancer and any IBD-associated dysplastic lesions. Non-neoplastic category includes all other lesions (ie, hyperplastic, inflammatory and pseudopolyps).

In all stages of this study (training, validation and testing), images were classified as containing lesions (including both …