Data collection of this study was part of the Dutch Adverse Events Monitor, a longitudinal retrospective record review study among patients who died in Dutch hospitals [3]. A stratified sample was drawn of twenty hospitals, including university (n = 4), tertiary teaching (n = 6) and general hospitals (n = 10), stratified for region. University hospitals were oversampled to allow comparison between hospital types, the results were weighted to correct for this.

Per hospital, approximately 150 electronic patient records of patients who died in the hospital in 2019 were randomly selected from the hospital information system. Records from patients admitted to the psychiatry or obstetrics department and of children younger than one year were excluded. In total 2,998 patient records were included. Comparison between characteristics of the sample and all patients deceased in Dutch hospitals showed that the sample was representative (regarding age, sex, length of admission) for the total population of deceased patients in Dutch hospitals.

The full information on the design and sample of the Dutch Adverse Events Monitor (e.g. power calculation, record selection and check for representativeness) can be found elsewhere [2, 11].

The Medical Ethical Committee of the VU University Medical Center (IRB00002991) declared that the Medical Research Involving Human Subjects Act did not apply (reference no.2020.052). The requirement of individual informed consent was exempted, as this study was conducted within the conditions of the Dutch Healthcare Quality, Complaints and Disputes Act [12].

Retrospective patient record review is a thorough and internationally widely used method to measure AE rates [1]. Patient records were reviewed from the electronic health system in a two-stage review process. In stage 1 nurses collected background characteristics such as sex, age and Charlson comorbidity index (weighted score representing comorbidity severity) [13]. Reviewing nurses furthermore screened the patient records using a trigger list (appendix Table 1), which is based on the Harvard Medical Practice Study [14]. Records with at least one present trigger were eligible for stage 2 review. In stage 2 physicians systematically reviewed the patient records for AEs.

The extensively trained reviewer group consisted of 17 nurses and eight physicians (medical specialties: internal medicine, neurology and surgery). The training consisted of one full day of training in which the trainer focused on: the consistent use of the right definition and rating for adverse events and preventability, and the overall approach of the record review. The trainer was a medical specialist who was involved in the Dutch Adverse Events Monitor since the first study in 2004. All reviewers had access to handbooks with comprehensive information and clinical examples during their record review. In addition to the training and handbooks, multiple intervision meetings and peer coaching sessions were organized throughout the data collection. Moreover, reviewers could always request a double-check/discussion meeting with external experts when in doubt regarding a record review. All reviewers had to have extensive clinical experience (minimum of 5 years for nurse reviewers and 10 years for medical specialist reviewers). Reviewers did not review patient records from the hospitals they worked at during the study or had worked at in the past.

The IRR was calculated as positive and negative agreement, for both review stages a random sample of approximately 10% of records was reviewed double blind. The IRR in both stages was deemed sufficient. Positive agreement for finding triggers in the first stage was 97.4%, negative agreement was 75.0%. Positive agreement for determining an AE was 63.0%, negative agreement was 75.4%.

The three main outcomes to answer our first research question were: AE, potentially preventable AE and potentially preventable deaths. An AE was determined based on three criteria: unintended injury; health consequence; causality. An AE was considered potentially preventable when the provided care fell below the current level of performance that can be expected of the healthcare professional and/or hospital. When a potentially preventable AE had a contribution to the patient’s death it was considered a potentially preventable death. The main outcomes with definitions and measurements can be found in Table 1.

All included patients died in hospital, however, not all these patients suffered an AE, and not every AE contributed to the patient’s death. Patients with a condition relevant for palliative care could still experience a preventable death, when it was the preventable AE that led to their death instead of their condition. In other words: their death was sooner than could be expected based on their prognosis, and caused by healthcare management instead of their condition.

When an AE was scored, the reviewers further reviewed its nature and causes. Moreover, for the potentially preventable AEs prevention strategies were determined, both categorically and through open text. These outcomes answered our second research question, the definitions and measurements can be found in appendix Table 2.

To identify the subgroups we used the selection method of Etkind (2017) based on codes from the international classification of Disease-10 (ICD-10) [15]. Patients with one or more relevant ICD-codes were classified as ‘with a condition relevant for palliative care’, all other patients were classified as ‘without a condition relevant for palliative care’. Both the main diagnosis of the index admission and co-morbidity diagnoses were taken into account. Appendix Table 3 shows the ICD-10 codes relevant for palliative care. Patients classified into the ‘condition relevant for palliative care’ group, did not necessarily have to be in the terminal phase of their palliative condition, nor did they have to be admitted to receive palliative or terminal care.

The ICD-10 codes were not available in patient records, but retrieved from a national database. Dutch Hospital Data (DHD) is a Dutch non-profit organization specialized in national hospital data [16]. DHD is processing, managing, analyzing and benchmarking the data for hospitals. DHD has multiple databases, i.e. the National Basic Register Hospital care. This database contains – among other data – ICD-10 codes of all Dutch hospitals. Registration in this national database is obliged for all hospitals. DHD states that the database is a reliable source for scientific research and analyses. The ICD-10 data is collected and coded by medical coders of the hospitals. They code the ICD-10 codes based on: the complete EHR, discharge letters and lab results. Out of the 20 included hospitals, 18 hospitals agreed to data retrieval from DHD. For the two hospitals that did not, the patients have missing ICD-10 codes (n = 380, 12.7%) and could not be classified into a subgroup. The analyses of this missing subgroup is added to the appendix.

Descriptive statistics were calculated for the total sample of deceased patients, and the subgroups of patients with and without a condition relevant for palliative care. Descriptive analyses weighed for hospital type were performed to calculate all outcomes. T-tests were performed to test for differences between subgroups on main outcomes. StataSE version 14.1 was used.