Systematic review

The rigorous search strategy used in a published systematic review of the diagnostic accuracy of routinely collected data in motor neuron disease [5] was modified and used to search MEDLINE (Ovid), EMBASE (Ovid), Web of Science (Thompson Reuters) and Cochrane (see Supplementary data). The reference lists of identified studies were also searched. To be eligible for inclusion, studies had to be full text (articles published only as abstracts or conference proceedings were excluded as methodological quality could not be determined) and report an original study that provided an analysis of the accuracy of specific ICD-10 diagnostic codes in PSP and CBS. Where available, the following information was extracted: first author, year of publication, routinely collected data source, reference or gold standard, and, measures of accuracy including sensitivity, PPV, specificity and negative predictive value (NPV).

Evaluation of ICD-10 diagnostic codesData sets

Healthcare data for individual patients in Scotland are collected as a series of Scottish Morbidity Records (SMR). SMR01 is an episode-based data set describing all inpatients and day cases discharged from non-obstetric and non-psychiatric specialties. A record is generated when a patient completes an episode of inpatient or day case care, including discharge from hospital, transfer to another clinician, a change of specialty or death. SMR04 is similarly an episode-based record relating to all inpatients and day cases admitted to and discharged from Mental Health specialties. Diagnoses are coded in these data sets using ICD-10 diagnostic codes. The NHSCR holds a record for every patient registered with a Scottish GP, everyone born in Scotland since 1985 who has not been registered with a Scottish GP, and, patients formerly registered with a Scottish GP who died after 31 December 1992. The NHSCR data set is linked to death data collected in the General Register Office for Scotland Vital Events. Causes of death are also coded using ICD-10.

Diagnostic codes of interest

PSP has a specific ICD-10 diagnostic code, “G23.1 Progressive supranuclear ophthalmoplegia [Steele–Richardson–Olszewski]”. Previous research has shown that PSP patients have also been frequently erroneously assigned the “G12.22 Progressive Bulbar Palsy” code (G12.2 codes Motor Neuron Disease) [6]. There is no specific ICD-10 diagnostic code for corticobasal degeneration (CBD) or corticobasal syndrome (CBS), and no national guidance on non-specific coding use (personal communication). Potential non-specific coding options therefore additionally explored in this analysis are detailed in Table 1.

Table 1 Candidate ICD-10 diagnostic codes for PSP and CBSPrevalence data

SMR data were used as one method of case ascertainment in a national prevalence study which estimated the prevalence of PSP and CBS at regional and national levels in Scotland, UK. This study has been reported and described in detail elsewhere [7]. In brief, nationally, multiple methods of case ascertainment were used to identify cases, including clinician and nurse specialist referral, searches of ICD-10 diagnostic coding in SMR and patient self-referral. Two additional methods—searches of GP databases and unselected hospital correspondence—were restricted to NHS Grampian due to resource and time constraints. Identified cases were verified by clinical examination (where patients had also consented to participation in a concurrent prospective national cohort study, the Scottish PSP and CBS cohort [8]), medical record review, or through contact with an individual’s named consultant. Individuals were deemed prevalent cases if they had a diagnosis of PSP or CBS and were alive and resident within the study population on the prevalence day, December 31, 2018.

AnalysisFrequency and distribution of clinical diagnoses coded using ICD-10 diagnostic codes G23.1, G25.9, G31.0, G31.8, G23.9, G23.8 and F02.0 in SMR01

As part of a preliminary evaluation of the utility of the SMR as a method of case ascertainment for the prevalence study detailed above, in NHS Grampian only, the clinical records of all individuals assigned the candidate ICD-10 diagnostic codes of interest, over a 2-year period from 1 April 2014 to 31 March 2016, were reviewed to evaluate the frequency and distribution of clinical diagnoses coded using each of these codes.

Accuracy of ICD-10-coded diagnoses (PPV, sensitivity) in SMR01 and SMR04 compared to a reference clinical standard in NHS Grampian, NHS Highland, NHS Orkney and NHS Shetland

All healthcare episodes from February 2011 to July 2019 in SMR01 and SMR04, with any of the ICD-10 codes of interest, in any diagnostic coding position (main diagnosis, secondary diagnoses), were extracted by the electronic Data Research and Innovation Service of the Information Services Division Scotland. Using postcodes and health board boundary data from the Scottish Government Ordnance Survey [9], the records of individual’s resident in NHS Grampian, NHS Highland, NHS Orkney and NHS Shetland (areas where we had access to the full medical record) were identified, and the relative frequency of healthcare episodes and individuals assigned each of the candidate ICD-10 diagnostic codes in these health boards compared to the total number of similar records from across NHS Scotland.

PPV and sensitivity were then calculated for each of the candidate ICD-10 diagnostic codes after we excluded individuals who died prior to 1 January 2018 (the start of prevalent period in the Scottish PSP and CBS prevalence study). The full electronic medical record (containing primary care referrals, hospital inpatient and outpatient correspondence and investigation results) of individuals with the candidate ICD-10 diagnostic codes in SMR01 and SMR04 were retrospectively reviewed. Reference standard clinical diagnoses were predominantly based on a clinician’s recorded diagnosis within these computerized medical records, further verified by clinical examination in people who had also consented to participation in a concurrent prospective national cohort study in PSP and CBS [8]. We estimated the PPV for two outcomes, a specific diagnosis of PSP or CBS, and, any parkinsonian disorder. The syndrome of parkinsonism was also evaluated as it is a key diagnostic milestone which, if identified, clinically narrows differential diagnoses, potentially minimising the impact of misdiagnosis or delayed diagnosis on the sensitivity of case finding of specific causes of the parkinsonian syndrome, including PSP/CBS [10]. The PPV was defined as the proportion of all coded cases that were confirmed from review of the medical record to be cases of PSP, CBS or parkinsonism, respectively. The small number of individuals in whom we were unable to access their medical record to assess their clinical diagnosis were excluded from analysis. Due to small numbers, confidence intervals for the PPV were estimated using the Wilson score interval. Sensitivity, defined as the proportion of all identified prevalent cases that were coded positive, was calculated in NHS Grampian only as this was the region where, due to active case ascertainment, we had most rigorously been able to minimise the risk of missing prevalent cases.

Consistency of G23.1 discharge diagnostic coding in SMR01 and SMR04 compared to NHSCR death certificate coding

In deceased individuals, coded with the G23.1 diagnostic code between February 2011 to July 2019 in SMR01 and SMR04, the consistency between discharge diagnostic coding and death certificate coding from linked vital events data was also evaluated.

The study was reviewed and approved by the Public Benefit and Privacy Panel for Health and Social Care (HSC-PBPP), NHS Scotland. All results were reported according to the modified Standards for Reporting of Diagnostic accuracy criteria [11].