Improving detection of impacted animal bones on lateral neck radiograph using a deep learning artificial intelligence algorithm

We herein present a deep learning artificial intelligence algorithm for the detection of impacted animal bones on lateral neck radiography. The algorithm demonstrated a non-inferior detection rate as compared to human readers in the testing set. We further investigated the potential application of this algorithm in a real-world clinical setting with a simulation set consisting of patients enrolled during a different time period and evaluated in a direct comparison with radiologists’ reports. The direct comparison revealed that the deep learning algorithm correctly identified 3 more animal bones than the radiologists on lateral neck radiographs.

Several previous studies have investigated the value of lateral neck radiography in patients with impacted foreign bodies, with reported sensitivities ranging from 10% to more than 90% [4,5,6]. The diverse sensitivities may be attributed to different components of the foreign bodies and their locations. Of note, studies having reported a higher sensitivity did not distinguish cases according to the specific type of foreign body [4, 5, 18], whereas those that reported a lower sensitivity included only impacted animal bones [6, 19, 20]. For animal bone impaction, studies have indeed suggested that plain radiograph is of little value, while CT demonstrates the highest accuracy [19, 20]. Meanwhile, more recent studies have suggested that a lateral neck radiograph be performed only after a negative laryngeal fiberscope examination, as this examination is well-tolerated for patients and the detection rate of lateral neck radiograph for animal bones located in the oropharynx is poor [6, 18]. In our institution, all patients presenting with suspected foreign body impaction will initially receive a laryngeal fiberscope examination. Therefore, the main value of a lateral neck radiograph is to detect foreign bodies that are inaccessible by the laryngeal fiberscope, while impacted bones detected on the lateral neck radiograph will exempt the patient from a further CT scan. The positive identification of impacted animal bones on plain radiograph will effectively act to accelerate the diagnostic and management processes while decreasing the radiation dosage and medical fee.

Radiographic signs for impacted animal bones on lateral neck radiography include direct visualization of radiopaque density and indirect signs, including presence of abnormal air column lucency, loss of cervical lordosis, and increased prevertebral soft tissue thickness [21]. However, since the indirect signs may merely reflect local soft tissue irritation [21, 22], unless the animal bone is directly visualized, further study, such as NECT, is often performed before a definitive treatment can be determined. Similar to a previous report [8], the interpretation accuracy of lateral neck radiography for doctors with different years of experience and subspecialties varies in this study. Although with the aid of the deep learning AI algorithm, every doctor exhibited improved accuracy. The ability of an interpreter to accurately identify animal bones of various sizes and in variable locations on lateral neck radiograph gradually improves with experience, thus our deep learning AI model may effectively act to accelerate and enhance this acquired ability.

There was a decrease of more than 30% in sensitivity between the test set and the simulation set, for both the deep learning AI model and radiologists. The main explanation for this result was likely the different cohorts comprising the two sets. More specifically, the test set, in addition to the training and validation sets, included only cases in which the animal bones were identifiable on lateral neck radiography. By contrast, the simulation set had no such exclusion criteria, and thus included cases which would have been excluded from the testing set. Therefore, the decreased detection rate observed in the simulation set was potentially due to the intrinsic limited effectiveness of the lateral neck radiograph to detect impacted animal bones.

Included in the simulation set were 14 cases with animal bones which were retrospectively deemed as identifiable on lateral neck radiography, with or without reference to CT imagery. Among these 14 cases, 5 were missed by the radiologists and received subsequent CT scans. Our deep learning model accurately detected 3 more cases as compared to the radiologists, which would translate into 3 fewer CT scans performed if the model was applied in clinical practice. Furthermore, as most of the false positives made by the AI model and the radiologists did not overlap, the AI model could act to complement the interpretation of the lateral neck radiograph, thereby achieving a lower false positive rate.

The radiographic evaluation of patients with animal bone impaction varies across institutions, with plain radiograph being the first-line radiological investigation [17, 23] to completely abandoning plain radiograph in the evaluation process [24, 25]. Although lacking sensitivity, a positive result on the plain radiograph is sufficiently specific to warrant direct treatment without the need for further imaging [23]. As many missed cases were retrospectively identifiable, our AI model may enhance the interpretation process of lateral neck radiographs for the detection of animal bone impaction, thereby decreasing the need for further imaging and accelerating the clinical workflow. However, the radiograph interpreter should be aware of the factors which may affect the interpretation of lateral neck radiograph. In clinical practice, it is often challenging to interpret lateral neck radiographs in older patients due to complex calcification and ossification structures in the neck which can obscure the image or be mistaken for swallowed animal bones. In this study, no animal bones were missed by the AI algorithm among pediatric patients. This could be attributed to better soft tissue penetration with no obscuring calcification or ossification structures in the neck. However, since pediatric patients only made up a small proportion of the samples (3 cases out of 50 positive cases in the testing data set, and 2 cases out of 20 positive cases in the simulation data set), further studies involving larger numbers of pediatric patients are needed to reach more definitive conclusions.

One of the main challenges in the integration of AI in radiological practice is the need for radiologists to be trained in the use of AI algorithms and to understand the decision-making processes of the AI models [26]. Another challenge is the need for collaboration between radiologists and AI developers to ensure that the AI algorithms are properly validated and the results are properly interpreted [27]. Lastly, the integration of AI in radiological practice also requires the development of infrastructure and the integration with the existing Radiology Information System (RIS) and Picture Archiving and Communication System (PACS) in the hospital. The AI model in this study is a relatively straightforward application aimed at a very specific clinical scenario for which the training of its use would be simple and fast. However, users must note that the algorithm was trained in a single institution, such that the accuracy of the model may be affected by distinct varieties of ingested animal bones in cultures with different diets.

The strength of this study lies in the fact that the labeling and classification was not based on radiologists’ reports, but rather retrospectively referenced to the CT, endoscopy, and photograph of the specimen to ensure the quality of the data used for developing the algorithm. Meanwhile, there are indeed several limitations. First, the data used to train the model were from a single institution. Although the data were attained by different brands and models of x-ray machines in a time period of 10 years, external validation is still needed for further verification. Since not only the brand and model of x-ray machines may affect the final results, animal bones from different species of animals (particularly different species of fish) may also impact detection rates [20, 28]. Therefore, the results may vary in different geographic zones with different diets. Second, the simulation section of the study was conducted in a relatively short period of time, while clinical efficacy may be better evaluated by a prospective clinical trial. Third, although our results demonstrate the potential benefits of AI-assisted detection on plain radiograph to decrease the need for CT imaging, the detection rate is limited by the intrinsic limitations of plain radiography, particularly for bones impacted in the thoracic esophagus. Lastly, the deep learning AI model was trained to specifically identify animal bones on lateral neck radiographs and is not intended to replace a formal radiological report. Rather, the purpose of this AI model is to assist the interpreter to quickly identify impacted animal bones on lateral neck radiograph, while the interpreter should still scrutinize the imagery for other potentially critical findings, such as abnormalities of the cervical spine, airway, or other soft tissue lesions of the neck.

In conclusion, our deep learning AI model demonstrates a superior sensitivity for the identification of impacted animal bones on lateral neck radiograph without an increased false positive rate. The application of our AI model in clinical practice may accelerate the diagnostic process, thereby improving workflow and decreasing the need for CT imagery.

留言 (0)

沒有登入
gif