Study design and settingResearch context

The Treat-Me-ACP was developed, validated, and used in the STADPLAN project (STudy on ADvance care PLANing in care-dependent community-dwelling older persons). The purpose of the STADPLAN project was to develop an ACP intervention for elderly care-dependent people living at home and to evaluate the effects of this complex intervention in comparison to optimized usual care. The project comprised a two-arm cluster-randomized controlled trial (c-RCT) with a 12-month follow-up (German Clinical Trials Register: DRKS00016886 on 04/06/2019). The intervention was an adapted version of the patient-centered ACP program Respecting Choices® [8], and was conducted at the home care service level and the patient level [9]. Alongside effectiveness and process evaluation, a health economic evaluation of the intervention was conducted [9, 10]. Twenty-seven home care services in three German study regions (Oldenburg, Halle [Saale], and Lübeck) participated and were randomized [9, 11]. Recruitment of home care services took place from April 2019 to December 2019. First patient in was 28th May 2019, last patient out was 11th January 2021. More information about the project can be found in the study protocol [9, 10]. In addition, the development of the intervention [12] the main results [11] , and the process evaluation are published [13].

Ethical approval was obtained from the ethics committees of the Medical Faculties of the Universities of the Martin Luther University Halle-Wittenberg (no. 2019-045), the Carl von Ossietzky University Oldenburg (no. 2019-024), and the University of Lübeck, Germany (no. 19–080) in a joint approval. All methods were performed in accordance with this approval.

Study population

The study population of the c-RCT STADPLAN included 380 patients aged 60 years or older. The study was originally designed to include patients aged 65 years or older. The minimum age was lowered to 60 as a result of the previous pilot study, as the home care services stated that it was important not to exclude “younger” patients. Informed consent was obtained from all patients and their legal guardians. Clients of home care services who were assigned to a care grade (as assessed by expert raters of the Statutory Health Insurance) and rated to have a life expectancy of at least four weeks were included. In addition, adequate knowledge of German and the cognitive ability to follow the intervention and data collection were required. Cognitive ability to follow the intervention was assessed using the Dementia Screening Scale (DSS) [14]. Patients with a score < 3 were included in the study. Patients with a score of 3 to 5 were included if the trained nurse from the participating home care service considered the patient to be cognitively able to follow the intervention [9, 11].

This validation study used a convenience sample of 80 patients from the participants in the STADPLAN c-RCT: 40 patients of 13 home care services assigned to the intervention group (IG), 40 clients of 12 home care services assigned to the control group (CG). Patients were selected by the research assistants who conducted the interviews depending on whether they felt that an additional survey using the Treat-Me-ACP could be conducted without imposing a heavy burden on the patient. The analyses of this validation study included 64–80 cases each. On average, data from 69 patients (SD = 2.2) were included in the analyses. The recruitment target of the main study was n = 960. The recruitment target for this validation study was 12.5% (n = 120) of the patients included in STADPLAN. The recruitment goal was limited by the available staff resources.

Instrument development and pre-testStep 1: development of a preliminary version

The Treat-Me-ACP was developed during the STADPLAN project. The design of the Treat-Me-ACP is based on the Life Support Preferences Questionnaires [15, 16] and the Emanuel Medical Directive [17], which were identified in a literature review. The instruments are the only ACP preference instruments known to us that have been scientifically studied. Both instruments were translated into German independently by two researchers using the multi-stage translation–review–adjudication–pre-test–documentation (TRAPD) team approach [18]. Within a project meeting, each translated element was rated by interdisciplinary research team members (RS, KS, AB, ÄK, FH, GM, SK) according to its relevance, acceptability, and appropriateness for the German healthcare context. The research team was qualified to assess the elements because of its composition of an epidemiologist, health services researchers, nursing scientists, and an nurse. To reach an agreement, at least one member of the study team from each site had to vote in favor of including the translated element. Thus, an approved item always received the approval of more than 50% of the research team members. Unlike Gilbert and Prion 2016 [19], we chose this threshold in order to include a larger number of items in the testing. Elements that received consensus were used for a preliminary version of the instrument. The result was an instrument with nine scenarios. The preliminary version was ordered by increasing intensity of health limitations and translated into patient-friendly language by experts.

Step 2: Assessment of comprehensibility, acceptability, and feasibility

Pre-testing of the preliminary version of the Treat-Me-ACP was conducted in two rounds of interviews with probing questions to test for comprehensibility, acceptance, and feasibility [20, 21]. The results of the interviews were evaluated and discussed within the research team and adjustments were made as needed.

Step 3: final interview round with adapted instrument

The final step was to test the revised instrument in interviews with probing questions to test for comprehensibility, acceptance, and feasibility again. The final adjustments were made after consultation within the research team (see Additional file 1).

Final instrument

The final Treat-Me-ACP contains one global medical care goal-item and five scenarios: S1 (current health status), S2 (advanced dementia), S3 (stroke with paralysis), S4 (stroke with six weeks coma), S5 (incurable brain tumor) (see Additional file 2). Scenarios are roughly sorted by severity of the associated health limitations, with the most severe being the final scenario. Figure 1 shows a schematic diagram of the Treat-Me-ACP.

Fig. 1

The Treat-Me-ACP instrument (modified version based on Schwartz et al. 2004 [17])

The Treat-Me-ACP items were used to understand the dynamics of patient preferences over the follow-up period and the differences between both study groups. We assumed that preferences could change meaningfully through an ACP intervention. Therefore, we expected to see greater changes in preference scores in the IG of the STADPLAN c-RCT than in the CG.

For each scenario, patients are asked how they would feel if they spent the rest of their lives in this state (how would you feel-item) and what their treatment preference (TP) would be: TP1 (antibiotics in the case of a severe infection), TP2 (cardio pulmonary resuscitation in the case of cardiac arrest), TP3 (cholecystectomy in the case of a gallbladder infection), TP4 (temporary artificial nutrition if they are unable to eat and drink independently), and TP5 (permanent artificial nutrition if they are unable to eat and drink independently). Response options were presented on a 5-point Likert scale (“definitely not” to “definitely”) supplemented by “not applicable” and “I do not wish to answer”.

The global medical care goal-item includes the question of whether patients prefer care or treatment that allows them to live as long as possible but may lead to health problems or care or treatment that might shorten their life but potentially reduce the risk of major health problems.

Psychometric evaluationData collection

For psychometric evaluation of the Treat-Me-ACP, a secondary analysis of the health economic evaluation sub-study data was conducted. In addition to the Treat-Me-ACP, information was collected on patients’ sociodemographic characteristics, current health status, and use of health services in the past three months.

Data were collected at two time points of the c-RCT: T0 (baseline) and T2 (12-month follow-up). The instrument was the last part of a case report form to be completed during a structured face-to-face interview conducted by trained research assistants in patients’ homes. Each interview was scheduled individually. A family member or close friend was allowed to participate. Patients could ask comprehension questions at any time during the interviews. The research assistants trained for the STADPLAN project were qualified to answer these questions. We report methods and results of the psychometric evaluation according to the recommendations set out in Streiner et al. 2014 [22].

Preference scores

Following Schwartz et al. 2004 [17], two types of preference scores were created and used in this validation study: (1) preference scores across treatments within a scenario (scenario preference score, or SPS), indicating patients’ preference for a given intensity of life-sustaining treatments in the respective health status; and (2) preference scores for a specific treatment across scenarios (treatment preference score, or TPS), indicating patients’ overall preference for a given intensity of life-sustaining treatment across the five health states (see Table 1). Thus, a total of 10 preference scores per time point were formed by assigning values from 0 (= definitely not) to 4 (= definitely) to the answer choices and then adding them up. The value range of each preference score is 0 to 20, higher scores indicating higher preference for life-sustaining treatments (SPS) or higher preference for a particular treatment (TPS).

Table 1 Baseline patient characteristicsData analysis

Descriptive statistics of baseline data were calculated for the acceptance of the instrument, global medical care goal-item, how would you feel-items, and for SPS and TPS. Missing data were not imputed. Cases were excluded for the respective SPS if an item of the scenario was not completed. The same applied to the TPS when items of the respective treatment scale were not completed as well as the global medical care goal-item.

In accordance with other psychometric evaluations [23,24,25] the acceptance of the instrument was assessed based on data from T0 by using completion rate and frequencies of missing data per item and of the whole instrument. The response options “not applicable” and “I do not wish to answer” were also counted as missing values, since these do not provide any content-related information about patients’ preferences.

The internal consistency (Cronbach’s α) of the preference score scales was calculated based on data from T0 as in Schwartz et al. 2004 [17] to test whether it was possible to create preference scores by scenario and treatment.

For concurrent validation as part of the criterion validation [26], the association between the global medical care goal-item and the SPS, both measured at baseline, was examined using Cramér’s V as was the association between the how would you feel-items and the preference scores. Based on Schwartz et al. 2004 [17], it was expected that patients who preferred a longer life with possible health impairments would have higher preference scores than patients who preferred a shorter life without major health problems. By analogy, it was expected that patients who indicated on the how would you feel-item that living with the scenario health state would be acceptable would have a higher preference score than patients who indicated that life with the respective health state would be barely livable or not livable at all.

In addition, data from T0 and T2 were used to assess the responsiveness of the instrument to changes in preferences for life-sustaining treatments. For this, changes in preference scores from T0 to T2 were compared between and within study groups using non-parametric tests. On average, preference scores were expected to decrease in the IG, because preferences for life-sustaining treatments were expected to decrease. This assumption was made based on the results of systematic reviews indicating that ACP reduces preference for life-sustaining treatments [7, 27].