Amblyopia is a neurodevelopmental visual disorder that affects approximately 3-5% of children globally and it can lead to monocular vision loss if it is not diagnosed and treated early. Traditional diagnostic methods, which rely on subjective assessments and expert interpretation of eye movement recordings presents challenges in resource-limited eye care clinics. This study introduces a new approach that integrates the Gemini large language model (LLM) with eye-tracking data to develop a classification tool for diagnosis of patients with amblyopia. The study demonstrates that, (i) LLMs can be used to analyze fixation eye movement data to diagnose patients with amblyopia; and (ii) integration of medical subject matter expertise improves the performance of LLMs in medical applications. Our LLM-based classification tool achieves an accuracy of 80% in diagnosing patients with amblyopia using a combination of few shot learning and multiview prompting with expert input from pediatric ophthalmologist. The model classifies amblyopic patients with moderate or severe amblyopia from control subjects with an accuracy of 83% and mild or treated amblyopic patients from control subjects with an accuracy of 81%. Finally, the model achieves an accuracy of 85% for classifying amblyopic patients with nys-tagmus from control subjects. The results of this study demonstrate the feasibility of using LLMs in ophthalmology application and highlights the essential role of medical expert input in LLM-based medical applications.

The authors have declared no competing interest.

This research was funded in part by grants from the US National Institutes of Health (NIH), U24EB029005, R01DA053028, the US Department of Defense (DoD) grant W81XWH2110859, and the Clinical and Translational Science Collaborative of Cleveland, which is funded by the NIH, National Center for Advancing Translational Sciences, Clinical and Translational Science Award grant, UL1TR002548. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

This study was reviewed and approved by the Cleveland Clinic Institutional Review Board (IRB). Written informed consent was obtained from each participant or parent/legal guardian for this study as mandated by the Declaration of Helsinki. The patient data with protected health information (PHI) elements was securely stored at Cleveland Clinic and only the deidentified data was subsequently shared with Case Western Reserve University for analysis by the transformer deep learning model.

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

Available upon reasonable request.