Introduction

Diabetic retinopathy (DR) remains a major cause of vision loss. It affects up to 25% of patients with diabetes. This burden should increase: an estimated 700 million people worldwide will have diabetes in 2045 vs 463 million today (http://www.idf.org/diabetesatlas).1 2 In France, approximately 1 million people have DR.3 This number is increasing, but the number of ophthalmologists is decreasing. Even with full coverage of expenses and follow-up by ophthalmologists, DR can evolve to blindness. DR remains the first cause of blindness in the working-age population.4

One major problem with DR management is that it is still based on the Airlie House classification using old imaging techniques that is insufficient to accurately predict blinding complications (Davis MD, 1968).5 This classification is based on standard 45° retinographies and was updated in the 1991 Early Treatment of Diabetic Retinopathy Study (ETDRS).6 Practitioners consider the classification too complex to use, so they use a simplified version based on the international DR and diabetic macular oedema disease severity scales.7 Unfortunately, decisions based on this classification have poor predictive values, and the management is challenging: a pivotal stage such as ‘severe non-proliferative DR’ evolves to proliferative complication in 51.5% of cases, with only 17.1% of cases evolving to high risk of blindness.6 However, because the classification cannot be more precise in its prediction, ophthalmologists apply close monitoring to all patients and evaluate them with fluorescein angiography (available in some centres only) or even use laser treatment for fear of complications. Laser treatment in these patients is iatrogenic and alters their visual field when actually it may have been useless. Indeed, data from several recent studies suggest that the evolution to DR complications is lower than what is predicted by the Diabetic Retinopathy Severity Scale, and high-risk eyes are no longer properly detected by this classification.8 Classification of the other complication of DR, diabetic macular oedema, is even less effective. Because we cannot currently identify people at high risk of vision loss from diabetic macular oedema and because they are treated once the vision has decreased as recommended (European Society of Retina Specialists), they only recover 50% of their lost vision.9 A better risk prediction could allow for better preserving vision in people at high risk in the short term.

New imaging techniques are available. They generate massive valuable data despite no new adapted classification based on these data. Ultra-widefield fundus photography (UWF-FP) covers three times the area of standard 45° photography. It provides new critical information on the periphery of the retina. Optical coherence tomography (OCT) produces cross-sectional imaging with a resolution of a few microns and has been recently enriched by OCT-angiography (OCTA), which allows for non-invasively visualising normal and abnormal vasculature of the retina. The techniques can allow for early and accurate evaluation of DR. Therefore, a new classification of DR is needed to include these new imaging modalities and to be suitable for treatments that were not available a few decades ago, when the ETDRS classification was established.

New imaging modalities produce an expanding amount of data, including three-dimensional data. Only very high human expertise can handle such analysis. As such, any clinical score based on a new classification will be more complex than current practice and challenging for most ophthalmologists. In addition, the ETDRS classification did not take into account systemic data (eg, glycaemic and blood pressure control, lipid status), having a major role in the incidence and progression of DR.10–12

Artificial intelligence (AI) is the ideal solution for monitoring this evolution. Retinal imaging relies on digital imaging, so deep learning algorithms will allow for massive information processing. AI has been found effective for DR screening of fundus images.13–17 The US Food and Drug Administration approved at least two AI software systems to detect DR (IDx-DR18 and EyeArt19), and we also developed a similar system, OphtAI, with high sensitivity and specificity (CE marked).14 20 21 AI has also been applied to OCT image analysis, including retinal layer segmentation and detection of intraretinal or subretinal fluid,22 23 with applications in referral recommendations.24

More recently, AI has also been applied to the prediction of pathological evolution in patients. This includes predicting the need for antivascular endothelial growth factor treatment25 or outcomes26 using OCT; the conversion to wet age-related macular degeneration within 6 months using OCT27; the development of DR within 2 years using standard FP28 or the progression of DR severity by at least two ETDRS Diabetic Retinopathy Severity Scale steps within 1 year using FP.29 Prediction performance is promising in the latter DR-related applications, but there likely is room for improvement because input data used for prediction is limited to only one, and outdated, imaging modality.

We hypothesise that AI could integrate the massive amount of data provided by modern available fundus imaging devices, notably OCT-A and UWF-FP, with other medical data for the patient and allow for the development of an AI-based system capable of reliably predicting the risk of progression towards severe forms of DR.

Aims and objectives

The main objective of the Evaluation Intelligente de la Rétinopathie Diabétique, Intelligent evaluation of DR (EviRed) project is to develop and validate an AI-based system assisting the ophthalmologist by improving the prediction of DR evolution and decision-making during DR follow-up. This main objective will be to validate the prognostic tool and evaluate how accurately the algorithm can predict the progression to ‘severe retinopathy’ (defined by the presence of proliferative DR and/or significant macular oedema involving the centre of the macula, or need for laser photocoagulation, vitrectomy or intravitreal injection depending on updated indication of these treatments) in the following year.

Secondary objectives will be to (1) evaluate how accurately the algorithm can assess DR severity, (2) evaluate how accurately the algorithm can assess individual components of severe DR to demonstrate the superiority of the algorithm to predict DR progression towards a severe form of DR compared with human prediction by comparing the prediction by an algorithm to the one by the ophthalmologist based on the current DR classification and their clinical experience, (3) compare the DR evaluation by the ophthalmologist in daily clinical practice with the DR evaluation by ophthalmologists in the imaging reading centre, (4) estimate the 2-year cost of DR management and (5) estimate the generic health-related quality of life of patients with DR.

Methods and analysis

Design

A cohort of up to 5000 patients with diabetes will be recruited and followed for 1–4 years depending on the time of recruitment to the end of the study. This cohort will be stratified to include 10% of patients with no DR or mild non-proliferative DR in both eyes; 10% of patients with complications in both eyes, complications meaning proliferative DR (untreated or treated with panretinal photocoagulation) or macular oedema involving the centre of the macula (untreated or previously treated with intravitreal injections) and 80% of patients with uncomplicated moderate-to-severe non-proliferative DR with or without macular oedema in at least one eye. Each year, systemic data as well as eye data will be collected. Retinal images of both eyes will be acquired using different imaging modalities including widefield photography, OCT and OCTA. Colour photographs, OCT and OCTA images will be acquired with a variety of machines and for each eye with two different brands of machines to allow for developing an algorithm as much as possible agnostic to the machine brand. All images and data will be collected on a common platform and centralised on a server. The EviRed cohort will be divided into two groups: one group will be randomly selected in each stratum during the inclusion period to be representative of the general diabetic population. Their data will be used for the validation of the algorithms (validation cohort). The data for the remaining patients (training cohort) will be used to train the algorithms. Since the start of the study, two amendments affecting the study population have been approved and applied to the study: the study duration has been extended by 1 year, and the last 3 months of inclusion will allow investigators to include as many patients with all degrees of DR as possible. Recruitment began in December 2020 and will end in December 2023. The last visit of the last patient in the study is scheduled for December 2024. The study is expected to be completed by mid-2025.

Types of participants and inclusion criteria

The study will include patients with both type 1 and type 2 diabetes mellitus as well as other forms of diabetes mellitus. Because the goal of the study is to train algorithms to predict the progression of DR at 1 year, we will exclude patients with diabetes with no risk of progression, such as patients with type 1 diabetes and diabetes duration <10 years. We will exclude patients with previous vitrectomy in both eyes, patients with ungradable photographies or OCT/OCTA imaging in both eyes, patients unable or unwilling to comply with study procedures, patients with gestational diabetes and patients with untreated or treated proliferative DR and macular oedema involving the centre of the macula in both eyes or with panretinal photocoagulation and untreated or treated macular oedema involving the centre of the macula in both eyes.

Patients will be recruited in France, in 14 ophthalmology departments (online supplemental appendix 1) from 1 December 2020, all specialised in managing retinal diseases, that work with 18 diabetology departments (online supplemental appendix 2) in which patients will be screened as well. There will be two sources of recruitment: the active cohort of patients with diabetes followed in the diabetology departments, and patients consulting directly in the ophthalmology departments.

Data collection

At inclusion, general and systemic data will be collected; the list of data is provided in table 1. The ophthalmology history of the patients will be collected, including cataract surgery, vitrectomy, intra-ocular pressure-lowering drugs, laser photocoagulation and intravitreal injections. At inclusion and on a yearly basis, the systematic and ophthalmological data listed in table 1 will be collected. An additional ophthalmological visit including clinical data collection and imaging will be performed prior to initiation of any treatment (including laser, intravitreal injection, vitrectomy). Retinal images will be acquired yearly and will include at least one UWF photography (with Clarus 500; Carl Zeiss Meditec, Dublin, California, USA and/or another brand—Optos; Optos, Dunfermline, Scotland, or another system) and at least one OCT and OCTA examination (with PLEX Elite 9000 Swept-Source OCTA; Carl Zeiss Meditec and/or an OCT/OCT-A device of another brand). The other OCTA brands used will be Spectralis (Heidelberg Engineering, Heidelberg, Germany), Solix (Optovue, Fremont, California, USA), Triton (Topcon, Tokyo, Japan) and Cirrus (Carl Zeiss Meditec). Depending on the device, for each eye, the patients will undergo at least one macular OCT cube, one macular OCTA scan with the highest resolution and one second OCTA acquisition with the largest field available depending on the device, in addition to the UWF retinography. The patients will be followed with the same devices during the whole study. All the images and data will be collected on the common platform and centralised on a server.

The patients will be followed according to usual clinical care, except that they will have retinal imaging with two devices of different brands (Zeiss and another brand) instead of one for colour fundus, OCT and OCTA. The study will be conducted in agreement with usual ophthalmological care with no additional risks. Patients will be seen on a yearly basis for 2–3 years. Each year, general data as well as ophthalmological data will be collected.

Data analysis

All images will be centralised and graded by several independent observers in the EviRed reading centre. The images will be annotated independently by two graders in the EviRed virtual reading centre, using the annotation software product that will be developed during the first year of the project. A super-grader will ensure the coherence and accuracy of the annotations for each image.

The LaTIM AI laboratory will develop algorithms to predict the progression of DR based on multiple imaging modalities. Multimodal information fusion will combine all images and patient data for improved diagnosis and risk prediction. The algorithms will be trained on the data for the training cohort and compared with current practice using the series of up to 1000 patient images of the validation cohort read by humans using the current classification for validation.

The main outcome measures will be sensitivity, specificity and the area under the receiver operating characteristic curve (AUC) of the algorithm to predict DR progression towards a severe form of DR (defined by the presence of proliferative DR and/or severe macular oedema involving the centre of the macula, or the need for laser photocoagulation, vitrectomy or intravitreal injection) in the following year.

We will use the data for up to 1000 patients with diabetes (validation cohort) to validate the algorithms that are developed. Algorithms will be evaluated by comparing the automatic progression prediction provided by the algorithm with the effective progression observed after 1 year. Predictive algorithm results will be expressed as a predictive score between 0 and 1 corresponding to a probability of progression. We will calculate the AUC of this score and its threshold value using the Youden index, as well as its sensitivity and specificity (complementary analyses for determining the best cut-off will use the function of costs of misclassifications). These calculations will involve the whole population and according to initial disease severity to test possible differences in performance of the algorithms by disease stage.

Sample size calculation

We calculated that the involved diabetology departments altogether have an active file of about 55 000 patients with diabetes. The prevalence of patients with moderate or more severe DR is 15%, corresponding to 8200 patients with moderate or more severe DR and 46 000 patients with mild or no DR. In addition, 10–60 patients are seen in ophthalmology consultations each week. We estimate that about 4500 patients with diabetes with severe forms of DR can be recruited directly from these consultations. We should be able to recruit up to 5000 patients including 500 with mild or no DR and 4500 with moderate non-proliferative or more severe DR.

The sample size calculation is based on the requirement to estimate the AUC of the predictive tool with good precision. For these calculations, we will take into account (1) the expected proportions at baseline of the levels of severity of retinopathy expected in the EviRed cohort and (2) the 1-year incidence of sight-threatening retinopathy according to level of severity of retinopathy due to type 1 and type 2 diabetes mellitus as measured by Younis et al.30 31 The expected 1-year incidence is about 20% for our main outcome. Considering this incidence and using the method proposed by Hanley and McNeil,32 a sample of 1000 patients will be required to estimate an AUC >85% with precision of ±2.5% or better. This sample size will also allow for (1) precision of the estimates of specificity and sensitivity corresponding to a maximum marginal error <0.03 and 0.0825, respectively33 and (2) a power >80% to detect (at a 5% alpha level) a better AUC by at least 2.5% with prediction by the algorithm versus the clinician (secondary objective).

Statistical analysis

Statistical analysis will involve using SAS V.9.4 in the intention-to-treat population. Predictive algorithm results will be expressed as a predictive score between 0 and 1 corresponding to a probability of progression. We will calculate the AUC for this score and its threshold value using the Youden index, as well as its sensitivity and specificity (complementary analyses for determining the best cut-off will use the function of costs of misclassifications). Additional subgroup analyses will evaluate the robustness of the performance of the algorithm by the different types of patients. Similar analyses will be applied to secondary outcomes.

Patients and public involvement

Two patient groups will be involved in plans to disseminate study results and provide input on new DR management, including algorithms.

Ethics and dissemination

The study protocol was approved by the French South-West and Overseas Ethics Committee 4 on 28 August 2020 (CPP2020-07-060b/2020-A01725-34/20.06.16.41433). This study is registered on ClinicalTrials.gov website under the reference NCT04624737. The study complies with the guidelines of the Declaration of Helsinki. Results of this research are expected to be disseminated in peer-reviewed publications and conference presentations. The database will be available for further study or development that could benefit patients.

All participants will give written informed consent before entry in the study and will be made aware that participation is strictly voluntary.

The study will be conducted in agreement with usual ophthalmological care with no additional risks.