INTRODUCTION

COVID-19 altered the way that health care was provided in many ways. In particular, emphasis was placed on managing patients remotely where possible.1 This is an area novel to optometrists in the UK, who typically manage patients in person, and the College of Optometrists (UK) rapidly issued advice on this.2 However, most optometrists have had little or no training on conducting remote consultations.

Unlike in neighbouring Scotland and Wales, the provision and commissioning of eye care in England is fragmented. Sight tests performed for symptoms that are refractive in nature are covered and commissioned nationally, whereas all other services (e.g., automated visual field tests, repeated tonometry and acute eye problem consultations) are provided either at a cost to the patient, the practice or in some areas, covered by local National Health Service (NHS) teams known as Clinical Commissioning Groups.3 This lack of consistent commissioning across the country leads to local variations in service provision and patient care. Therefore, while some countries of the UK already had national provision of emergency eye care in the community, England did not. Some areas of England, such as those in the present study, had existing Minor Eye Condition Services (MECS, also known as Primary Eyecare Acute Referral Schemes: PEARS), examples of which have been previously reported.4 In these services patients could self-present to, or be referred to, an accredited optometrist who was funded to provide an assessment beyond the scope of a sight test at no charge to the patient.5 Accordingly, the existing MECS across the areas of the present study (the Hull, East Riding of Yorkshire, Harrogate, Bedfordshire and Luton Clinical Commissioning Group areas) needed to be altered by each of the respective Clinical Commissioning Groups to incorporate remote consultation. Contracts were altered from March 2020, and an initial telephone triage service was added to the patient journey. This adapted service, known as COVID Urgent Eyecare Service (CUES), commenced across the Hull, East Riding of Yorkshire and Harrogate Clinical Commissioning Group areas from 17 April 2020 and is provided by Primary Eyecare North Yorkshire and Humber (PENYH). Subsequently, Bedfordshire and Luton Clinical Commissioning Groups opted to provide CUES, commissioned through Ocular Outcomes (a private company directed by some of the staff involved in PENYH). This service was commissioned in line with the national specification published by NHS England and NHS Improvement.6 Some of the aims of CUES are similar to MECS: to reduce unnecessary presentation to secondary care, with a second aim of managing patients virtually where possible. CUES provided care beyond that seen in a typical MECS by enabling the addition of remote (e.g., telephone based) consultation that could provide care for those clinically vulnerable or unable to visit an optometrist in person. Additionally, in the CUES examined by the present study, instead of offering a payment structure of cost per patient episode, the payment structure was changed to a block contract on a month-to-month basis, which provides a budget for the service to operate in and a more accurate financial forecast for the Clinical Commissioning Group. However, this could impact patient care. Specifically, a company could reduce the quality of care (i.e., less face-to-face (F2F) consultations) in order to: (a) maximise profit and/or (b) to meet the finite set budget. Overall, the amendments of the MECS were to: (a) reduce unnecessary traveling and encounters between people in the COVID-19 pandemic and (b) reduce costs to the Clinical Commissioning Groups.

The design of CUES has been published elsewhere.7 Briefly, routine F2F sight testing was suspended in England from 1 April to 17 June 2020.8 This meant that in areas without previously locally commissioned urgent eye care services, patients with an acute eye problem had either to contact their general medical practitioner (GP) or hospital in order to receive eye care. In order to alleviate unnecessary demands on ophthalmologists, who were required to be redeployed to other overburdened hospital departments,9, 10 the COVID urgent eye care service specification was published.11 This specification was not intended to replace existing services that had the ability to be adapted to provide appropriate optometric consultations. Accordingly, in part due to local commissioning, some differences exist between CUES and existing services.

In the present study, patients are initially required to telephone a free of charge central phone number (0800) where the patient is registered on a custom built system designed by the eye care company (PENYH / Ocular Outcomes) developed by PharmOutcomes (Pinnacle Systems Management, phpartnership.com). The service is only available for patients registered with a GP within the area of the Clinical Commissioning Group. Accordingly, this first step (telephoning the 0800 number) is required to ensure that the patient is registered with a GP in the area; data is verified, where possible, using the national database of patient demographic information (Personal Demographic Service) which is extracted from the NHS Spine. Once the patient has registered and their eligibility is confirmed, patient details and presenting symptoms are taken and sent electronically to an optometrist, who would then call the patient for a tele-consultation. Optometrists were able to use photos and video calling as well as the telephone to aid the consultation at their discretion. As the phone line is a central service, optometrists did not have access to the patient's clinical records; this is in contrast to a service where the telephone consultation was performed by the patient's regular optometric practice. However, as patients are able to move between different practices freely, it is not necessarily the case that a patient would visit their usual optometrist in the event of an acute eye problem. The outcome of the optometrist's tele-consultation could be patient self-management (e.g., over-the-counter medication); a direct referral to alternate care (hospital ophthalmology department, Accident & Emergency, GP) or a face to face (F2F) appointment with a participating optometrist (e.g., high street practice). Patients have free choice of which practice they visit and should their first choice practice not have an appointment available, the triaging optometrist will find an optometric practice that has availability within the specified timescale.

Optometrists for the telephone consultation service were either known to the directors of the companies or recruited via word-of-mouth and all had experience of providing MECS type services. Where possible, the optometrists should not work in the same area that they provide tele-consultations in order to reduce the likelihood of unconscious bias and directing patients to their own practice / place of work. Optometrists did not require any specific extensive local knowledge (i.e., of other locally commissioned services), as the outcome options of a consultation were contained within the service and referrals to F2F providers was done within PharmOutcomes.12

The present study aimed to determine the patient reported clinical safety and effectiveness of the scheme, specifically aiming to identify instances of patients whose problem was incorrectly managed.

METHOD

In order to meet the required minimal sample size for binomial logistic regression,13, 14 a minimum sample size of 500 was required. The present study required patients to verbally consent twice: once at the initial telephone appointment (when they ring the telephone triage line), and once more when they were contacted by the research team. Previous research on telephone-based research indicates that 80%15, 16 to 90%17 of patients who originally consent for a phone survey subsequently complete the survey (i.e., 10%–20% decline or were unable to be contacted). Moreover, Nelson and colleagues reported that when researchers require oral consent at an initial stage, when subsequently contacted, the final overall consent rate was approximately 40%.18 Accordingly, our initial sample size aimed for a minimum of 1250 patients. As the providers of the services in the present study estimated that between 1000 and 2000 patients called the service per month, to ensure minimum sample size requirements, the study ran from 1 November 2020 to 24 December 2021 (54 days) in Luton and Bedford and from 30 November 2020 to 22 January 2021 (54 days) in Harrogate, East Riding and Hull. At the time of data collection, optometric practices were considered essential services and remained open to provide F2F care to patients. Data for patients who consented were extracted from PharmOutcomes and sent via an encrypted Microsoft Excel spreadsheet (microsoft.com) to the research team who subsequently attempted to contact the participants. Optometrists were not explicitly made aware that the patients they managed would be contacted for this evaluation.

For the present study, attempts to contact patients were made by one author (AS) a maximum of three times at differing times of day, usually on different days.19 Typically, this was once Monday to Friday (10:30–18:30), once Saturday to Sunday (10:30–18:30) and then finally Monday to Friday (after 18:30). The call was made from a mobile number (not withheld) and the researcher introduced themselves at the beginning of the call.20 Clinical and demographic data were extracted from PharmOutcomes, including notes taken from the original reception staff stating the presenting symptoms of the patient. Starting approximately four weeks after the initial call, a member of the research team (AS) phoned all consenting patients to determine: (a) consent, (b) whether the patient could recall the appointment, (c) the patient's understanding of the treatment, (d) whether this treatment resolved the patient's issue, (e) whether the patient presented to another health care provider to get the issue resolved and (f) whether the patient was satisfied with the service.

For the purpose of the present study, we define the service's effectiveness at resolving patients' eye problems (correct / incorrect decisions) with respect to the definitions displayed in Figure 1a for telephone consultations and Figure 1b for F2F consultations.

The diagnosis decision tree for consultations. (a) How correct / incorrect outcomes were determined for telephone consultations. (b) The decision tree for face-to-face consultations. In order to receive a face-to-face consultation, patients had to pass through the telephone consultation

These outcomes were patient-determined following discussion with the research team. For example, if the patient reported that the optometrist recommended ocular lubricants and that treatment resolved the problem, this was defined as correct. Conversely, if the recommended treatment did not resolve the patient's eye problem, this was classified as incorrect. Patient safety is assessed with regards to the number, and outcome of incorrect decisions made by the optometrists.

For the analysis of the service's effectiveness, an increased number of appointments is likely to eventually lead to the correct outcome. Therefore, unless otherwise indicated, we included only the outcome of the patient's first appointment. We acknowledge that even the most qualified and experienced professionals may not make the correct treatment decision on one visit. Therefore, if the follow up appointment was optometrist initiated, we assessed the overall outcome. All cases where the outcome was categorically correct (e.g., optometrist recommended ‘xyz’ and it resolved the problem) were marked as correct outcomes (true positives) by one author (AS). For all patients where the outcome was not categorically correct, the outcome was assessed by two authors (AS/CD) and any disparity of opinion was discussed until agreement was reached on the outcome classification.

Incorrect diagnoses were grouped into the following categories: recommendation did not work, unnecessary referrals, incorrect diagnosis and major errors. Major errors were defined as an error or omission (as judged by the authors AS and CD) that resulted in a problem that the patient identified, and is likely to have or could have resulted in serious harm. ‘Major’ is differentiated from when the optometrist made a recommendation that didn't work by the nature of the symptom and the patient's report of how the condition deteriorated. For example, a patient who was recommended warm compresses which didn't resolve symptoms of bilateral itchy eyes would be categorised as ‘recommendation that didn't work’, rather than major error. If, on the other hand, the patient was subsequently diagnosed with scleritis, this would be classified as a major error. Incorrect diagnoses were able to be determined by the patient's account. For example, an incorrect diagnosis was determined to have occurred when: (a) the treatment partially resolved the patients symptoms, (b) the patient used a different treatment to what the optometrist recommended (which resolved the issue), (c) the patient didn't use the optometrist recommended treatment and the condition self-resolved, (d) the patient was referred to the hospital and reported that what the optometrist had suggested was incorrect and (e) where the patient reported the condition resolved with the treatment but became apparent to not be what the optometrist had described.

For analysis of socioeconomic status, data were analysed by English lower-layer super output areas. These are areas in England which have an average population size of 1500.21 Socioeconomic status was determined using the Index of Multiple Deprivation (IMD), which is the ranking of lower-layer super output areas in order from most to least deprived (i.e., 1 to 10) nationally, based upon weights of various deprivation measurements. This study utilised data from the Ministry of Housing, Communities and Local Government to convert postcodes to IMD deciles.22 Regression analyses were performed using SPSS version 25.0 (IBM, ibm.com) and the ridge plot was created in R.23

Ethics approval was granted by the Chair of the Biomedical, Natural, Physical and Health Sciences Research Ethics Panel at the University of Bradford on 21 September 2020.

RESULTS Consenting participants

In total, 2372 patients were asked to participate. Of these, 1358 (57.3%) consented at the initial stage (when the patient originally contacted the telephone line). Of the 1358 patients who originally consented, 1106 (81.4%) patients comprise the final analysis (Table 1). 187 patients did not answer the phone, 33 could not remember the appointment, 21 declined to participate and 11 were removed for other reasons (e.g., deceased patient, outside time/location of present study, telephone number not in use).

TABLE 1. A breakdown of consent rate of patients in each area and the reasons for removing patients from the data analysis Area Total, n First consent Reasons for removal (n) Second consent n % DNA Decline Memory Other n % Luton 1252 124 53.2 20 3 3 0 98 79.0 Bedford 542 84 5 17 5 431 79.5 Harrogate 346 220 63.6 31 3 6 1 179 81.4 East Riding 766 348 61.6 39 6 6 3 294 84.5 Hull 124 13 4 1 2 104 83.9 Total 2364 1358 57.4 187 21 33 11 1106 81.4 Abbreviation: DNA, did not attend.

Overall, 1106 patients had 1188 appointments included in the present study. A total of 1036 patients had one tele-consultation, 58 had two appointments and 12 had three appointments.

Age (in years) was available for all three groups: (1) those who didn't consent at stage one (when the patient rang the telephone line); (2) those that originally gave consent, but then withdrew from the study (did not attend / declined / memory failure / other / incomplete data) and (3) those included in the final analysis (consented at both stages). The median age of each group is 56 (Declined first), 53 (Declined second) and 57 (Consented). These data are presented in the ridge plot in Figure 2.

Ridge plot showing the distribution of the ages of the patients in each consent group. The median is depicted by the middle long line in each ridge. The two lines either side represent the end of the 1st (left) and 3rd (right) quartiles. The height of the ridge indicates the relative frequency of each age and the short bars at the base of each ridge represent the range of ages included in the distribution. The ridge itself is a density curve

For 26 patients, the overall outcome could not be determined – for example, the patient was referred routinely from a F2F consultation and hasn't been seen yet, or the telephone optometrist recommended treatment but the patient was unable/unwilling to adhere to the regime and, therefore, potentially may still be suffering with the same problem (e.g., recommended warm compress but the patient has not done this). Furthermore, two additional patients were removed from this analysis due to there being no date of birth recorded and four further records were removed for not specifying the optometrist. Accordingly, unless otherwise stated, subsequent analysis is based on 1074 first encounters.

Optometrists

Optometrists (eight male, three female) had a range of experience from 4 to 47 years post-qualification (median = 18 years, interquartile range 5–24.5 years). Data from nine optometrists were available for further qualifications and is provided in Table 2. An important consideration is that none of additional qualifications were in the area of telemedicine.

TABLE 2. The number of optometrists with each postgraduate qualification Further qualification Number None 1 WOPEC MECS level 1 8 WOPEC MECS level 2 6 WOPEC Glaucoma level 1 3 WOPEC Glaucoma level 2 3 WOPEC Cataract 1 WOPEC Learning Disabilities 1 Professional Certificate in Glaucoma 2 Diploma in Diabetic Retinal Screening 1 Diploma in Independent Prescribing 1 Abbreviations: MECS, Minor Eye Condition Services; WOPEC, Wales Optometry Postgraduate Education Centre.

Regarding practice setting, of the nine optometrists, none were currently working in a university although one had prior experience of that type of work. One was currently delivering eye care in a hospital and a further two optometrists had previous experience of hospital optometry. Four were currently working in a multiple (e.g., national chain) practice and a further three had experience working in that type of practice. Finally, five optometrists were working in independent practice with a further two having experience of this type of work. In total, optometrists worked between 24 and 65 h per week (median = 44 h, interquartile range 34–48 h).

Service safety

The number of patients managed by each optometrist (telephone consultation) is given in Table 3. Due to the small number of patients cared for by optometrists 8, 9, 10 and 11, these are grouped together as ‘other optometrists’ in future analysis.

TABLE 3. The number of appointments managed correctly and incorrectly by each optometrist involved in the service Management done by Number of patients Correct (n) Incorrect (n) Incorrect (%) Optometrist 1 199 141 58 29.1 Optometrist 2 25 19 6 24.0 Optometrist 3 75 53 22 29.3 Optometrist 4 55 44 11 20.0 Optometrist 5 28 16 12 42.9 Optometrist 6 99 70 29 29.3 Optometrist 7 125 99 26 20.8 Other optometrists 23 17 6 26.1 Tele-consultation Total 629 459 170 27.0 F2F optometrist 445 417 28 6.3 Total 1074 876 198 18.4 Note ‘Other’ is optometrists 8 to 11. Abbreviation: F2F, face-to-face.

Overall, 18.4% of patients phoning the service were incorrectly managed. For the patients who were solely managed by a telephone optometrist, 27.0% of patients were incorrectly managed. Telephone optometrists attempted to manage 46.9% to 72.4% of their patients solely by telephone, with 27.6% to 53.1% being referred for a F2F appointment with another optometrist. On the other hand, for patients who saw an optometrist in person, only 6.3% patients were incorrectly managed.

A binary logistic regression was used to assess the effect of area (Luton, Bedford, Harrogate, East Riding, Hull), Optometrist (1, 2, 3, 4, 5, 6, 7, other), source of referral to service (optometrist, GP, other), patient age (in years), socioeconomic status (IMD deciles 1–10) and whether the patient was seen in person [yes, no] on the outcome of the consultation (Correct / incorrect). The following equation was significant: χ2(24) = 114.09, p < 0.001, R2 = 0.16 and the results are displayed in Table 4.

TABLE 4. The results of the binomial logistic regression analysis examining predictors of whether a patient was correctly managed. Significant predictors are displayed in bold Variable Compared to β S.E. Wald df Sig. Exp (β) 95% CI for Exp (β) Lower Upper Area 9.39 4 0.05 East Riding Bedford 0.51 0.26 3.85 1 0.05 1.66 1.00 2.75 Harrogate 0.06 0.27 0.05 1 0.82 1.07 0.63 1.81 Hull −0.13 0.37 0.12 1 0.73 0.88 0.43 1.82 Luton 0.77 0.36 4.60 1 0.03 2.15 1.07 4.34 IMD decile 10.40 9 0.32 2 1 −0.20 0.55 0.13 1 0.72 0.82 0.28 2.40 3 −0.12 0.54 0.05 1 0.83 0.89 0.31 2.54 4 −0.57 0.50 1.30 1 0.26 0.57 0.21 1.51 5 −0.31 0.51 0.37 1 0.54 0.73 0.27 1.99 6 −0.29 0.50 0.33 1 0.57 0.75 0.28 1.99 7 0.16 0.50 0.10 1 0.75 1.17 0.44 3.10 8 0.07 0.49 0.02 1 0.88 1.08 0.41 2.84 9 0.44 0.50 0.77 1 0.38 1.56 0.58 4.18 10 −0.21 0.49 0.18 1 0.67 0.81 0.32 2.10 Optometrist 7.86 7 0.35 Optometrist 2 Optometrist 1 −0.10 0.42 0.05 1 0.82 0.91 0.40 2.08 Optometrist 3 −0.08 0.30 0.07 1 0.80 0.93 0.52 1.66 Optometrist 4 0.33 0.39 0.72 1 0.40 1.39 0.65 2.95 Optometrist 5 −0.78 0.40 3.76 1 0.05 0.46 0.21 1.01 Optometrist 6 −0.13 0.27 0.23 1 0.63 0.88 0.52 1.48 Optometrist 7 0.20 0.27 0.52 1 0.47 1.22 0.71 2.08 Other Optometrists