INTRODUCTION

Globally, the need for palliative care services is over 40 million individuals.[1] The Ministry of Public Health of Thailand has been promoting palliative care since 2017[2] with a focus on end-stage cancer patients. The national service plan aims to improve quality of life by encouraging home-based care, supported by primary care providers, rather than hospitalisation.[3,4] However, there is a lack of an easy-to-use patient assessment tool for home-based care. Existing tools were mostly for clinical research and hospital-based applications; for example, ‘The Palliative Care Outcome Scale (POS)’[5] and ‘The Edmonton Symptom Assessment System,’[6] are primarily developed for clinical research and hospital-based applications.[7,8] Patients might have trouble understanding the questions and need help in responding to them.[9] Our communication with some Thai palliative care providers in hospitals and community settings found that POS was too complicated and had too many questions, while patients might give biased responses to the ESAS visual analogue scale, recalling their prior responses. The purpose of the study is therefore create a simple, valid, reliable and sensitive-to-change patient-reported outcome measurement for those who suffer from terminal cancer and are receiving palliative care.

MATERIALS AND METHODS

We applied three-step approaches mixed with qualitative and quantitative studies, running from February 2019 to February 2022.

Step 1 Item generation, using purposive review literature and focus groups.

Beginning in February 2019, we reviewed some key published literature based on the following selection criteria: (1) focused on cancer patients, (2) quality of life in palliative care and (3) published in Thai or English. Twenty studies were identified. They were then filtered by the exclusion criteria, including (1) not focusing on the terminal stage and (2) studying only specific areas of quality of life. Finally, twelve papers were selected.

Concurrently, we conducted two focus groups with (1) palliative medical experts and (2) highly experienced palliative care nurses. The expert group included ten palliative medicine specialists and family physicians from five leading medical schools and tertiary-care cancer centres in Thailand who had extensive experience in providing palliative care. They were asked to share ideas and experiences on the outcome of care for terminally ill patients, requirements for assessing end-stage cancer care outcomes, symptoms needing special care at home and assessment methods for advanced cancer patients receiving palliative care at home. The nurse group comprised ten nurses from a cancer hospital and general and community hospitals who had deep experience in the area of palliative care in hospitals, in-patient homes or community settings. The discussion focused on special care needs for end-of-life care at home. The nurses were also required to ask questions in a group of experts. One of the researchers acted as the facilitator, while the others joined as observers. The main ideas were captured to design the drafted version of patient-reported outcome measurement for terminally ill cancer patients (PROMs-TCP).

Step 2 Pilot-testing of the drafted PROMs-TCP for content validity and ease of use.

From October to December 2019, the drafted PROMs-TCP, originally with six questions, was tested for content validity. A panel of five experts, including the head of the palliative care centre of a university-affiliated medical centre, a family physician with a certificate in clinical fellowship (palliative care), a family physician with long-time experiences in providing palliative care at home and two palliative care nurses with hospital-based and home-based experiences, independently rated each question item using a 4-point rating scale (1 = not relevant, 2 = somewhat relevant, 3 = quite relevant and 4 = highly relevant).

The drafted PROMs-TCP was also given to 20 selected advanced cancer patients receiving palliative care in four tertiary-care and community hospitals – 12 in hospital settings and eight in home-based settings – for their feedback on relevance, ease of use, length, understandability of the items and response choices and other suggestions, if any. Collected feedback was reviewed and applied to make modifications to the question items, response choices and presentation format.

Step 3 Evaluation of the instrument properties of PROMsTCP, including validity, reliability and sensitivity to change. From January 2020 to March 2021, 169 pairs of advanced cancer patients and their caregivers were voluntarily recruited from palliative care centres of six participating tertiary-care hospitals – two in Bangkok, one in the north, one in the south and two in the central region. The inclusion criteria included (1) having been diagnosed with end-stage cancer; (2) receiving palliative care; (3) the patients and their caregivers both agreeing to participate; (4) being aged between 18 and 70 years; and (5) being able to communicate in Thai language. The number was slightly more than the calculated sample size for correlation based on α = 0.05, power = 0.9 and correlation coefficient rate of 0.25.[10] All participating hospitals assigned palliative care nurses to be project coordinators. They received training from the research team on the protocols, consent procedures and application of POS and PROMs-TCP.

After the recruitment and consent process, demographic data of all patients were collected on day 0 (baseline). The nurses assessed each patient with POS. Each patient responded to a question on overall well-being (OWB) to indicate whether OWB on that day was ‘good’ or ‘terrible.’ A good day meant that the patient felt satisfied with their health status, given the underlying health conditions. By contrast, a terrible day implied that the patient needed immediate intervention to improve their condition or to alleviate the individual’s suffering. Finally, the patient, their caregiver and the nurse independently used PROMs-TCP to assess the patient’s status. After that, assessments using the OWB and PROMsTCP measurements were repeated daily, while the POS was repeated on day 5 and day 10. Each patient was monitored for 10 days unless the patient passed away during that period, regardless of whether the patient was hospitalised or continued to receive care at home.

Data analysis

Step 1, item generation, we gathered information from both existing literature and focus groups to identify factors that contribute to the well-being of cancer patients who received palliative care. We looked for common factors that were relevant to the Thai context of palliative care and ranked them in order of importance. Our goal was to create a concise initial draft of a tool called PROM-TCPs. During the focus group discussions, we obtained permission from the participants to video record the conversations. These recordings were then transcribed word for word and made anonymous. We also took field notes to supplement the transcripts. The main facilitator checked the accuracy of the transcripts. Both the primary and secondary facilitators conducted a traditional qualitative content analysis. They independently read and re-read the transcripts, identified key themes based on their observations, and created a set of codes and count of frequency. They then met to discuss and resolve any differences. They independently read and re-read the transcripts, identified key themes based on their observations, created a set of codes and counted their frequency. This approach aligns with the goals of our study. In Step 2, pilot testing was conducted with an expert panel of five experts to calculate the content validity index (CVI) score of each question.[11] The feedback from pilot patients was also assessed for any issues with the questionnaire. In final Step 3, the key instrument properties of PROMsTCP were measured, including internal consistency, inter-rater reliability, discriminant validity, criterion validity and sensitivity to change. STATA 16.0 was used for analysis of all statistical information.

The assessment of internal consistency was conducted through the utilisation of item-total correlations[12] and Cronbach’s alpha coefficients.[13] A satisfactory level of item-total correlation was deemed to be >0.3,[14] while alpha values of 0.70 or higher were considered acceptable.[15] The study assessed inter-rater reliability using PROMs-TCP on day 0, day 5 and day 10 among patients, non-health-professional caregivers and six nurses. Weighted kappa (k) statistics were utilised, with confidence intervals (CIs) calculated by bootstrapping (1000 repetitions). Moderate, substantial, almost perfect and perfect agreement was indicated by k scores of 0.40–0.59, 0.60–0.79, 0.80–0.89 and ≥0.90, respectively.[16,17] Discriminant validity was tested by assuming patients’ OWB assessment as a ‘good day’ or a ‘terrible day’ comparison. We applied the receiver operating characteristic (ROC) analysis to determine the most appropriate PROMs-TCP cut-off point. The areas under the curve (AUC) were estimated using a point estimate and 95% CI, with an AUC of 0.5–≥0.6 indicating poor, >0.6–≥0.7 acceptable, >0.7–≥0.8 good and >0.9 perfect discrimination, respectively.[18] The criterion-validity test was based on Spearman’s correlations between the PROMs-TCP score and the gold-standard POS score at baseline, day 5 and day 10. The coefficients were graded as poor (r < 0.30), fair (0.31 < r < 0.70), good (0.71 < r < 0.90) or excellent (≥0.90) correlation.[19] Sensitivity to change of the PROMs-TCP was evaluated by calculating correlations of score changes between day 0 and day 5 and between day 0 and day 10, as measured by PROMs-TCP and POS. Effect size (ES) and standardised response means were used to calculate the magnitude of change using Spearman’s correlation coefficient.[20]

RESULTS Step 1 – Item generation

We deduced from the literature review that we should focus on four major domains of well-being for patients with end-stage cancer: (1) The physical domain, Reduced suffering symptoms; (2) the mental domain, presence of loved and to whom patients were able to talk to reduce fear caused by whatever reasons; (3) spiritual domain: Self-fulfillment and happiness in life, not limited to religious beliefs and (4) the economic domain: Concerns about the cost of medical care. Pain, shortness of breath, fatigue, constipation, insomnia, loss of taste and anxiety were among the symptoms identified in a Thai study on the symptoms and signs of patients with late-stage cancer.[21-32]

The focus groups, in summary, found that outcome assessment in terminally ill patients should primarily include physical symptoms such as pain, fatigue and insomnia. The secondary issues comprised the mental state of a patient, that is, whether the patient felt bored, depressed and/or hopeless. Patients and their family members experienced anxiety caused by inadequate confidence in self-care at home, pressure from neighbours (who, seeing the patient’s suffering, pushed for hospital care over home care) and caregiver burnout. Assessing the patient’s well-being should consider daily variations and rapid changes in symptoms during the end-of-life period. Prompt and effective symptom management plays a critical role in alleviating patient suffering. Because it is critical to have the ability to detect patient pain and other critical symptoms as early as possible and then call the palliative-care provider for help, having a tool to assess these symptoms easily as needed might raise the confidence of caregivers. Any assessment tool must be easy to understand and to respond to, i.e., not having too many response choices, and it should be as concise as possible – preferably having around five questions.

Based on the literature review and the focus groups, we identified common factors determining the well-being of patients and chose associated keywords in drafting PROMsTCP. The first draft comprised five questions addressing key physical symptoms and psychological well-being, including pain, dyspnoea, fatigue, adequate sleep and feeling worthlessness, with three simple response choices. They were prioritised by both the patients and palliative care experts as determinants of urgent needs for care assistance [Table 1]. Additionally, there was a question (Question #6) that asked about any other symptoms that patients might be experiencing. Spiritual and economic issues, despite their effects on quality of life, were excluded to keep focus on the prioritised needs for urgent care assistance.

Table 1: Domain and item generated

Domain Item Data categorisation Ref Patients No. nurses No. experts Physical functioning Pain √ 8 8 [21-23,25,27-32] Dyspnoea √ 8 6 [22,25-27,29-31] Fatigue √ 6 4 [22,23,26,27,32] Loss of taste √ [22,26,29,32] Insomnia √ 4 2 [23,28,29,32] Constipation √ [27] Nausea and vomiting √ 2 [22,27,29,31] Seizure 1 Bleeding 1 Skin rash [27] Emotional function Anxiety √ 4 [27,32] Depress √ 2 2 [23,27,28,30] Worthlessness or burden √ 4 4 [21,24,27] Finances Economic √ 2 [21] Spiritual Loss of self-esteem √ [21,27] Step 2 –Pilot-testing

Based on the experts’ evaluation, the S-CVI of the drafted PROMs-TCP was 0.97, while I-CVIs were ≥0.8, indicating good content validity. The feedback from the 20 patients in both hospital and home settings showed that most of them comprehended the questions and the response choices well, and the number of questions was appropriate. Overall, they were satisfied with PROMs-TCP. The only noted comment was on Question#6, asking about any other disturbing symptoms that could vary considerably by their severity, for example, nausea and vomiting. As their possible impacts were captured by the first five questions, we thus modified by keeping Question#6 but not assigning any score point [Table 2].

Table 2: PROMs-TCP assessment (translated into English; the original version is in Thai)

A simple PROMs-TCP who received home-based palliative care 1)Within the past day, did you experience any pain? 0
No pain or mild pain which pain medication was not required. 1
Moderate pain or pain which required pain medication every 3–4 hours. 2
Severe pain or pain resulting in an inability to sleep or pain which, even after taking the pain medication, did not help 2)Within the past day, did you experience any dyspnoea/difficulty breathing? 0
No sign of dyspnoea or mild dyspnoea without disrupting daily life activity 1
Moderate dyspnoea or fatigue, which impacted daily life activities 2
Severe dyspnoea or dyspnoea, which disabled daily life activities 3)Within the past day, did you experience any fatigue or weakness? 0
No sign of fatigue or mild fatigue without disrupting daily life activities. 1
Moderate fatigue that led to being unable to do any hard work but still had no impact on daily life activity 2
Severe fatigue or fatigue that causes you to rest all the time and be unable to do any work. 4)How did you sleep last night? 0
Got enough sleep 1
Slept fitfully. 2
Did not get any sleep at all 5)Within the past day, did you feel sad, bored, discouraged, hopeless, worthless or a burden to others? 0
Not sad or felt sad sometimes but did not feel hopeless. 1
Felt worthless and burdensome to others. 2
Felt very discouraged and hopeless that I did not want to live my life anymore. 6)Within the past day, did you experience any other symptoms that made you feel unwell? No Yes.
Please describe………………………………………. Step 3 – Measurement of key instrument properties

This step-3 study included 169 terminally ill cancer patients—72 males (42.9%) and 96 females (57.1%), with a mean age of 58.8 ± 8.7 years. Most of these patients (94.5%) practised Buddhism, with education lower than a bachelor’s degree (82.7%). Around two-thirds of the patients (66.5%) had two to four caregivers, of which some 44.2% were spouses, followed by family members.

Internal consistency

The item-total correlations for Question#1 to Question#5 with the total scores were 0.52, 0.67, 0.74, 0.69 and 0.65, respectively. The overall Cronbach’s alpha value of the tool was 0.85. This indicated a sufficient level of internal reliability.

Inter-rater reliability

The average weighted kappa values of each question in PROMs-TCP between the patients and their caregivers, as well as between the patients and their nurses, were between 0.69 and 0.87, indicating substantial to almost perfect agreement [Table 3].

Table 3: Interrater reliability for each PROMs-TCP question item between assessors PR and CR and between PR and NR

Day N Rater Item 1 k (95%CI) Item 2 k (95%CI) Item 3 k (95%CI) Item 4 k (95%CI) Item 5 k (95%CI) Baseline 158 PR: CR 0.84 (0.76–0.90) 0.87 (0.80–0.92) 0.78 (0.69–0.86) 0.80 (0.70–0.87) 0.73 (0.61–0.81) PR: NR 0.85 (0.77–0.92) 0.83 (0.75–0.90) 0.76 (0.68–0.84) 0.84 (0.76–0.91) 0.74 (0.63–0.85) Day 5 140 PR: CR 0.80 (0.69–0.88) 0.74 (0.62–0.84) 0.75 (0.64–0.84) 0.69 (0.58–0.79) 0.74 (0.63–0.83) PR: NR 0.78 (0.68–0.87) 0.81 (0.69–0.89) 0.77 (0.67–0.86) 0.79 (0.68–0.87) 0.79 (0.69–0.88) Day 10 133 PR: CR 0.87 (0.78–0.94) 0.80 (0.71–0.90) 0.85 (0.76–0.92) 0.76 (0.66–0.87) 0.75 (0.64–0.87) PR: NR 0.85 (0.77–0.92) 0.82 (0.72–0.91) 0.81 (0.72–0.89) 0.82 (0.73–0.91) 0.75 (0.65–0.87) Discriminant validity

The ROC analysis revealed that AUC was 0.91 (95% CI: 0.90, 0.93). The cut-off value that demonstrated the maximum accuracy for distinguishing patients’ ‘good day’ and ‘terrible day’ was four. With this cut-off value, PROMs-TCP correctly identified of the patients with good days of 76.91% and 88.67% of the patients with terrible days, respectively [Figure 1].

Figure 1: Different cut-off points for the PROMs-TCP (n = 1,709). CI: Confidence interval, ROC: Receiver operating characteristic, AUC: Areas under the curve, PPV: Positive predictive value, NPV: Negative predictive value. PROMs-TCP: Patient-reported outcome measurement for terminally ill cancer patients. The bold values in the cut points show how sensitive and specificity of the data.

Export to PPT

Criteria-related validity

With a total of 132 patients completing the assessments at all 3 time points (day 0, day 5 and day 10), there were moderate-to-significant associations between PROMs-TCP and POS. Spearman’s rank correlation coefficients were 0.71, 0.75 and 0.80, respectively, with a statistically significant value of 0.01 [Table 4].

Table 4: Spearman rank correlation coefficient between PROMs-TCP and POS assessment at baseline and at 5 days and 10 days after baseline assessment (n=132)

Spearman’s rho PROMs-TCP baseline PROMs-TCP day 5 PROMs-TCP day 10 POS baseline Correlation 0.71** 0.53** 0.48** coefficient Sig. (two-tailed) <0.01 <0.01 <0.01 POS day 5 Correlation 0.60** 0.75** 0.68** coefficient Sig. (two-tailed) <0.01 <0.01 <0.01 POS day 10 Correlation 0.48** 0.72** 0.80** coefficient Sig. (two-tailed) <0.01 <0.01 <0.01 Sensitivity to change

Spearman’s rank correlation analysis of score changes for each period between day 0 and day 5 and between day 0 and day 10 of PROMs-TCP and POS revealed a moderately favourable association [r = 0.5–0.7, Figure 2]. The PROMsTCP and POS score variations over 10 days were small, however. The ES values were 0.31 and 0.30 for day 0–day 5 and 0.36 and 0.33 for day 0–day 10 for PROMs-TCP and POS, respectively.

Figure 2: The change score between PROMs-TCP and palliative care outcome scale assessment (n = 132). PROMs-TCP: Patient-reported outcome measurement for terminally ill cancer patients.

Export to PPT

DISCUSSION

The purpose of this study is to develop a patient-reported outcome measurement tool (PROMs-TCP) for assessing the well-being of terminally ill cancer patients who received palliative care at home. The ultimate purposes were to be able to differentiate between a ‘good day’ and a ‘terrible day’ and for the professional palliative care team to be alerted if the home-based patients needed urgent attention or care. The study findings demonstrate that PROMs-TCP was reasonably valid, reliable and sensitive to change.

PROMs-TCP covers the major determinants of well-being of terminally ill cancer patients receiving palliative care at home to indicate their urgent needs for our team attention. Although it was not designed to be a comprehensive tool or to capture broad areas of quality of life of palliative-care patients, the criterion-validity test demonstrated the PROMs-TCP scores correlated well to POS. PROMs-TCP addresses some items of POS, namely ‘pain’ and ‘feelings of hopelessness’[5] and some items in ESAS including ‘tiredness,’ ‘shortness of breath,’ ‘drowsiness’ and ‘feeling sad.’[33] Compared to POS, PROMs-TCP asks about feeling depressed and sleeping, instead of feeling anxious or worried about the illness or treatment and sharing feelings with family or friends. Other POS questions, such as feeling that time was wasted on health-care appointments or practical matters resulting from illnesses, seemed irrelevant to our aims. Moreover, using PROMs-TCP to determine ‘good day’ or ‘terrible day’ statuses corresponded well with the patients’ perception of their own statuses.

PROMs-TCP was satisfactorily reliable – considering the possibility or likelihood that patients might not always be able to use the tool by themselves and would need others, such as their personal caregivers, to perform the assessment. Inter-rater agreements between those of the patients and those of the caregivers, as well as between those of the patients and the nurses, were at a moderate-to-good level (Cohen’s weighted kappa = 0.69–0.87), whereas the POS achieved values of 0.31–0.35.[5,34]

Comparing to the other assessment tools available in Thai such as POS (11 items), ESAS (10 items), Functional Assessment of cancer therapy-General scale (27 items)[35] or the tool from The European Organization for Research and Treatment of Cancer (30 items),[36] our six-question PROMsTCPs were more concise and user-friendly, making it better fit for home-based care. Most of the patients and caregivers were able to complete the assessment in <1 min although it took up to 2 min in some cases with reading difficulties. Moreover, with the cut-off point to differentiate the patients with terrible days from those with good days, PROMs-TCP allows the palliative care teams to focus their attentions to the patients who need more and urgent assistance from them. This enables allocation of limited palliative and primary care resources more effectively.

Given the promising findings, we recognise some limitations. First, asking patients directly to indicate a good day or a terrible day may not be a perfect choice for discriminant validity testing. However, the use of this overall self- indicating assessment was aligned with the purpose of the tool to be patient centric in identifying a home-based patient who needed assistance from professional palliative care teams with validity and reliability. It is also worth noting that the approach of using a single overall question for assessing quality of life has been used in many studies and found to correlate with multi-question tools.[37-39] Second, the sensitivity to change of our tool was tested against POS based on specific timing, not against changes in patients’ clinical and health symptoms at exact moments of increase or relief of suffering. We encourage a study that addresses this issue in more detail. Finally, our study was conducted in a limited number of palliative cares patients and was limited to health-care settings in Thailand. This might impact some generalisability of PROMs-TCP. Further studies in different settings, or even different countries, are needed, it is crucial to develop electronic patient-reported outcome measures. These measures enable researchers and health-care professionals to collect valuable data on patients’ health outcomes and experiences in a more efficient and standardised manner.[40]

CONCLUSION

PROMs-TCP was proven to be reasonably valid, reliable, sensitive to change and simple to use, making it suitable for supporting home-based palliative care by primary care or palliative care providers. It enables effective communication between care teams and their patients and families and allows for appropriate responses to the patients’ needs for medical attention as well as efficient allocation of resources.