Identifying immunosuppressed patients using structured data can be challenging. Large language models effectively extract structured concepts from unstructured clinical text. Here we show that GPT-4o outperforms traditional approaches in identifying immunosuppressive conditions and medication use by processing hospital admission notes. We also demonstrate the extensibility of our approach in an external dataset. Cost-effective models like GPT-4o mini and Llama 3.1 also perform well, but not as well as GPT-4o.

T.L.W. has received research funding from Gilead Sciences to support investigation of the relationship between immunosuppressive conditions and COVID-19 outcomes. Gilead personnel had no involvement in this research. All other authors declare no financial or non-financial competing interests.

The NU SCRIPT Study is funded by NIH NIAID U19AI135964. This work was also supported by NUCATS, SQLIFTS, and the Canning Thoracic Institute of Northwestern Medicine. S.D.T. was supported by the NIH (grant no. 1F31LM014201). A.A. was supported by NIH (grant nos. U19AI135964 and R01HL158139). A.V.M. was supported by the NIH (grant nos. U19AI135964, P01AG049665, P01HL154998, U19AI181102, R01HL153312, R01HL158139, R01ES034350, and R21AG075423). G.R.S.B. was supported by a Chicago Biomedical Consortium grant, a Northwestern University Dixon Translational Science Award, the Simpson Querrey Lung Institute for Translational Science, the NIH (grant nos. P01AG049665, P01HL154998, U54AG079754, R01HL147575, R01HL158139, R01HL147290, R21AG075423, and U19AI135964), and the Veterans Administration (award no. I01CX001777). R.G.W. was supported by the NIH (grant nos. U19AI135964, U01TR003528, P01HL154998, R01HL149883, R01LM013337). T.L.W was supported by Gilead Sciences (award no. CO-US-540-6435) and the NIH (grant nos. U19AI135964, U19AI181102, and R21HD107571). C.A.G was supported by the NIH (grant no. K23HL169815), a Parker B. Francis Opportunity Award, and an ATS Unrestricted Grant.

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

IRB of Northwestern University gave ethical approval for this work (STU00204868).

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

The annotated SCRIPT notes dataset analyzed during the current study is available from the corresponding author upon reasonable request to those who sign a Data Use Agreement. MIMIC-III is available at https://physionet.org/content/mimiciii/1.4/ to credentialed users. The annotated MIMIC-III notes corpus developed and analyzed during the current study has been submitted to PhysioNet as a derived dataset to make it readily available to credentialed users of MIMIC-III. In the meanwhile, it is available to credentialed users from the corresponding author upon reasonable request.