We conducted a multicentre observational study of the implementation of PROMs in palliative care units. The study was approved by the ethics committees of all participating institutions (details: Ethics and consent section).

The project was conducted in three palliative care units (Freiburg, Mannheim, Ulm) located at different university medical centres in the federal state of Baden-Wuerttemberg, southwest Germany. These are part of the Competence Centre Palliative Care (KompetenzZentrum Palliative Care), a network of the palliative care units at the university hospital centres in Baden-Wuerttemberg. Two of the units are closely linked to oncology wards, one is independent of structural links. The wards have 10–14 beds and cared for 155–335 patients in 2019. Periods of implementation and patient recruitment varied but were all within the period from April 2019 to June 2020.

In a workshop with representatives of the participating palliative care units, study objectives, implementation strategy, the selection of a common questionnaire, measurement frequency and measurement points were discussed and agreed upon. Measures for implementation were derived from relevant literature [6, 7, 20]:

Local steering groups including executives and team members were formed and regularly discussed, consented and reviewed planning and progress, employing the PDCA cycle [23] (PDCA: planning, implementing [doing], checking and, if necessary, changing processes [acting]). The steering groups received standardised forms in which they were asked to document e.g. targets of PROM use in their ward, local workflows and responsibilities for patient information, PROM collection and PROM review by staff—including the rationale and records of changes over time. Each ward had a study assistant who was trained in study conduct and the use of PROMs, and had the role of a local expert who passes on their knowledge. Study assistants were also a human resource for reliable collection of the PROMs. The plan was for them to initiate the collection of PROMs and oversee the process in the first weeks before handing it over step by step to the teams. Furthermore, the Freiburg study team visited all wards to provide information on PROM use and study conduct and the external study assistants were trained at the ongoing project at the Freiburg site before starting at their site.

Physicians consecutively screened all newly admitted adult patients to determine whether they were physically and cognitively able to participate in the project and had sufficient German language skills to communicate their needs and symptoms themselves. The screening was based on the professional experience and expertise of the physicians taking the medical history on admission to the ward. Criteria were e.g. the patient’s verbal (statements of inability to concentrate / comprehend / listen / answer, conclusiveness of statements) and non-verbal responses (reduced vigilance, disorientation). Inability to read or write was not a reason for exclusion of patients if assisted completion of PROMs was possible. Patients fulfilling these criteria were informed about the project and written informed consent was obtained if they were willing to participate. In cases of continued care by the in-house specialist outpatient palliative care teams after discharge or on readmission to hospital, patients and staff were free to continue using PROM, but this was not counted as a second study participation.

An extended form of the Integrated Patient Outcome Scale (IPOS [24]) was administered at admission, weekly and at discharge to all study patients. When patients were discharged, PROM use did not continue unless they were receiving further care through directly associated outpatient palliative care. The IPOS is a 10-item holistic, valid and reliable multidimensional assessment of needs and symptoms for patients with advanced, severe diseases. We added eight dichotomous items (yes/no) concerning practical problems (e.g. housing situation, insurance, work/school, transport, childcare, financial worries, care for others, other) that were of special interest to social workers. Patients received paper-based questionnaires from members of staff or study assistants. If study assistants or staff felt patients might need help completing PROMs, they offered support. Support could be given by a staff member/the study assistant or relatives. The implementation of the support could vary according to the situation and patient needs, e.g. explanation of terms, reading out the questions or a more informal conversation. The specific situation in which PROM completion occurred was determined locally (e.g. during anamnesis or special appointment). The completed questionnaires were collected and given to the treating physician. Further procedures for passing the questionnaires on to other staff members (e.g. inclusion in medical records or scanning for sending by email) or discussing them within the team depended on local decisions in the steering group.

A written evaluation survey for palliative care staff was conducted in the final weeks of the study. This covered the use and benefits of PROMs and the desire to continue/discontinue their use, predominantly documented through closed questions. A description of the development and pre-testing, as well as a full version of the questionnaire, are included in Additional file 1. The staff at the palliative care units were asked to complete the evaluation questionnaire if they were in any way involved in the implementation and/or use of PROMs (inclusion criteria). This included either active (e.g. assessment or review of PROMs) or passive involvement (e.g. attending team meetings where PROMs were discussed). The survey was anonymous and questionnaires were returned in closed envelopes within the respective institutions or mailed to the study group in Freiburg. In order to ensure anonymity in the small sample, gender, age and professional experience were not collected.

Patients were asked to complete a short evaluation questionnaire with closed and open-ended questions (with or without assistance) after completing at least the second PROM. This was to ensure that they had experience not only of completing PROMs but also of staff using them. A description of the questionnaire and all results are included in Additional file 3.

The following data were documented:

Feasibility of PROM completion: The proportion of patients able to complete PROMs (independently or assisted) was assessed in a two-stage procedure. Firstly, the treating physicians assessed patients’ cognitive and physical abilities and German language skills during screening at admission, which includes taking a medical history with patients (and/or relatives). Secondly, study assistants re-evaluated the same parameters on their first visit, usually 1–3 days after admission, which was necessary due to the sometimes rapidly declining health of palliative care patients. The study assistants all had a professional medical background and were experienced in conversation situations with cancer and palliative care patients. If both physician and study assistant assessments were positive, it was assumed that patients were able to complete PROMs.

Furthermore, for non-participants, the reasons for refusal were requested. For participants, the proportion of patients able and willing to complete the questionnaire without support was determined.

Benefit and use: Physicians marked unexpected information in the first completed patient questionnaire (admission) with a highlighter. To ensure that this was done, assistants stuck a Post-It Note on the questionnaires that read “Please mark the information on this questionnaire that you would not have expected” (translated from German). The evaluation also included various items concerning both benefit and use. An English translation of the questionnaire is provided in Additional file 1.

The study aimed at a descriptive and explorative analysis for all study questions. Missing data are reported as we do not expect them to be random (no imputation). Open-ended questions in evaluations were categorised by a member of the study team by content and two other members checked if their understanding of categories and the allocation of answers were comprehensible (translations of all original answers and categories are reported in Additional files 1 and 3). The following associations were explored with inferential analysis: (a) The association between distress levels and information being unexpected as well as differences between professional groups in the evaluation survey were investigated by Chi2-test (two-sided). (b) Kendall’s Tau was employed to explore the correlation between the use of PROMs and evaluation of their benefits as well as the wish to continue their use. Due to the explorative nature of the study, alpha adjustment was not applied; alpha level was set at 5% (2-sided).