Background: Several conditions exist that do not have their own unique diagnosis code in widely-used clinical terminologies, making them difficult to track and study. Acute severe ulcerative colitis (ASUC) is one such condition. There is no automated method to identify patients admitted for ASUC from observational data, nor any specific billing or diagnosis code for ASUC. Accurate, automated, large-scale identification of hospital admissions for non-coded conditions like ASUC may enable further research into them. Methods: We performed a retrospective cohort study of patients with a history of ulcerative colitis (UC) admitted to a single academic institution from 2014-2019. Clinicians at our institution performed a chart review of these admissions to determine if each was due to a true episode of ASUC or not. Logistic regression, random forest (RF), and support vector machine (SVM) models were trained upon administrative claims data for all admissions. Results: 268 ASUC admissions and 3,725 non-ASUC admissions among UC patients were included. Our RF model exhibited the best performance, correctly classifying 95.5% of admissions as either ASUC or non-ASUC, with a validation AUROC of 0.96 (95% CI 0.94-0.98; AUPRC 0.73). The model had a sensitivity of 81.5% and specificity of 96.5%. The five most important features in the model were endoscopy of sigmoid colon, length of stay, age, endoscopy of rectum, and abdominal x-ray. Conclusions: There is currently no modality by which ASUC, which does not have its own unique diagnosis code, can be identified from claims databases in a scalable fashion for research or clinical purposes. We have developed a machine learning-based model that identifies clinically significant ASUC and reliably distinguishes them from admissions for non-ASUC reasons among UC patients. The ability to automatically curate large, accurate datasets of non-coded conditions like ASUC episodes can serve as the basis of large-scale analyses to maximize our ability to learn from real-world data, enable future research, and better understand these diseases.

The authors have declared no competing interest.

JM is supported by T15LM007092 from the NLM/NIH and the Biomedical Informatics and Data Science Research Training (BIRT) Program of Harvard University. WY is supported by T32HD040128 from the NICHD/NIH.

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

This study was reviewed and approved by the Institutional Review Board (IRB) of Beth Israel Deaconess Medical Center (BIDMC).

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

The data underlying this article cannot be shared publicly for the privacy of the patients who were included in this study, due to the data containing protected health information on these patients.