Project governance (including patient involvement)

The research has three governance groups who will be actively involved at key stages: the research team (comprising a core group of researchers including PROM developers, health economists, and clinical academics); the Clinical Advisory Group (CAG) (comprising a diverse group of clinicians and clinical academics who specialise in ALS); and the Patient Advisory Group (PAG) (comprising adults with ALS). The CAG and PAG will collaborate with the research team at critical times throughout the research project (Fig. 1), informed by an existing framework designed to ensure lived experience is integrated fully into PROM development [9]. Representatives of the PAG were first engaged in the research via input into the conceptual framework underlying item generation in Stage 1 [7]. While the methods/study design are already defined, the PAG (and CAG) will continue to have input into the development of PROQuALS throughout the project, including in content decisions, interpreting results, and helping to select which items are taken forward for preference weighting.

Fig. 1

Overview of project. ALSAQ-5 = Amyotrophic Lateral Sclerosis Assessment Questionnaire; ALSFRS-R = Revised Amyotrophic Lateral Sclerosis Functional Rating Scale; CAG = Clinical Advisory Group; CTT = Classical test theory; DCE = Discrete choice experiment; IRT = Item response theory; PAG = Patient Advisory Group; PlwALS = People living with ALS; PROQuALS = Patient Reported Outcome measure of Quality of life in Amyotrophic Lateral Sclerosis; PROM = Patient reported outcome measure; PWM = Preference-weighted measure; QuALS = Health-related quality of life in amyotrophic lateral sclerosis

Project aims

This project has three aims:

1.

To generate draft PROM content for measuring HRQoL in plwALS based on a comprehensive literature review and conceptual framework of HRQoL in ALS (QuALS).

2.

To produce a finalized novel PROM for measuring HRQoL in plwALS, utilising qualitative methods (cognitive debriefing interviews), quantitative methods (psychometric analyses), and expert input from collaborators (PAG and CAG).

3.

To value a set of items from the novel PROM to generate an associated preference-weighted measure (PWM) that will enable utility values to be generated in order to derive QALYs for use in cost-effectiveness analyses.

The research is split into three iterative stages that map onto the aims above (Fig. 1).

Stage 1: concept elicitation

The purpose of Stage 1 is to generate draft content for the novel PROM. The underlying conceptual framework for the PROQuALS instrument is the QuALS framework. QuALS was developed following a systematic review of the literature on HRQoL in ALS, and was ratified by plwALS and clinical experts [7]. The QuALS framework was used to inform the development of draft PROM content (including items, instructions, and response options) for the PROQuALS. A total of 101 draft items, across seven QuALS domains (Activities; Physical Health; Autonomy; Cognition; Feelings and Emotions; Self-identity; and Relationships) have been drafted by the core research team. Draft instructions were developed based on a structure that has been received well in other PROMs developed by the research team [10, 11]. The draft PROM features a 2-week recall period to balance issues of having a sufficient time period to capture different aspects of HRQoL with issues of recall. Two weeks is also the modal recall period used in ALS-specific PROMs assessing aspects of HRQoL [7]. An initial draft 4-point response scale that assesses subjective impact of each aspect of HRQoL (i.e., “how much of a problem for you was…”) was developed for testing and differentiates this PROM from other condition-specific measures in ALS. The draft PROM content will be further refined through collaboration with the PAG prior to Stage 2a (cognitive interviews).

Stage 2: refining the descriptive system

The purpose of Stage 2 is to revise the draft content developed in Stage 1 to produce a final HRQoL PROM for use in ALS. Stage 2 has two substages (Fig. 1).

Stage 2a – cognitive debriefing interviews

Draft PROM content will be assessed through qualitative methodology (cognitive debriefing interviews). The purpose of the interviews is to determine the content validity of the draft PROM content (i.e., to assess its relevance, comprehensiveness, and comprehensibility [12]). This includes the instructions, items, response options, and recall period. Interviews will be informed by a topic guide based on published guidance from The Professional Society for Health Economics and Outcomes Research (ISPOR) and COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) [13, 14]. Interviews will be conducted online to facilitate inclusivity, and transcribed intelligent verbatim. Due to the number of draft items generated in Stage 1, and reflecting upon response burden, interviews will be conducted over a minimum of two iterative waves. Each participant will be asked to comment upon a maximum of 40 items only (incorporating full sets of items from each HRQoL theme). The methodology outlined, including sample size requirements, are based on COSMIN guidance [15]. A minimum number of 42 interviews will be conducted in Stage 2a. Potential participants will be recruited by advertising through relevant charitable organisations based in the United Kingdom (UK) (e.g., MND Association). Adaptations will be made to allow for participation from individuals whose speech is impaired. This includes (but is not limited to) the use of augmentative communication devices and/or ‘chat’ function in the online interview platform. Purposive sampling will be used, based on a predefined target sampling framework to ensure representation across age, gender, and years since diagnosis of ALS (Table 1).

Table 1 Target sampling framework for each wave of the cognitive interviewing

The qualitative data will be used to assess each component of the draft PROM, including items, response options, and/or completion instructions against existing content validity criteria for relevance, comprehensiveness within themes(s), and comprehensibility. Analysis will be conducted wherever possible alongside data collection to best reciprocally inform each process. Analysis will follow best practice guidance [15], with 20% of transcripts dual-coded. Wave 2 will encompass the same approach. Following Wave 1, interview results will be discussed with the CAG to help ratify any decisions of modification to the PROM as a result of cognitive debriefing. Following Wave 2, the interview results will be discussed with the PAG in the same manner. If necessary, a further (and final) wave of interviews will be conducted to test any final modifications to the PROM content. At the end of Stage 2a, proposed PROM content is to be taken forward to Stage 2b following consultation with both the CAG and PAG (Fig. 1).

Stage 2b – Quantitative surveying

The refined PROM content will be included in a quantitative survey online alongside a selection of existing validated PROMs, including a generic PWM (EQ-5D-5L [16]), a condition-specific measure of disease progression (ALSFRS-R [17]), and a condition-specific measure of health status, Amyotrophic Lateral Sclerosis Assessment Questionnaire (ALSAQ-5 [18]). These PROMs were selected based on frequency of use in the field and to minimize participant response burden (alongside the draft PROM and other questions in the survey). The inclusion of additional PROMs allows for the psychometric performance of the draft PROM to be assessed (i.e., its construct validity).

The survey will be hosted online on Qualtrics, with opportunity sampling of plwALS via relevant charitable organizations based in the UK (e.g., MND Association). As well as self-reported (by plwALS) subjective measures of health, basic clinical (to determine King’s staging [19] and type of ALS) and sociodemographic data will be collected to enable descriptions of the sample and for additional psychometric analyses (e.g., construct validity via predicted relationships and known-groups analyses against underlying characteristics). Whilst prior PROM development studies have used samples of > 300 participants [10, 20], it is important to reflect that ALS is a rare condition and adaptations need to be made. Other studies have demonstrated that smaller sample sizes are appropriate when developing PROMs in rare conditions [11]. We will adopt a pragmatic approach and set a minimum sample size of 100 participants. Recruitment for the survey will be open for at least 4 months and until the minimum sample size has been achieved.

Prior to analysis, data will be cleaned and a priori data quality checks will be performed, similar to those outlined in Carlton et al., 2024 [10]. Analysis will be conducted in R, with descriptive analysis of item responses including item distributions, floor/ceiling effects, and missing data values explored. Classical test theory (CTT) and item response theory (IRT) approaches will be undertaken, with analyses adapted for the resultant sample size. Assuming a sufficient sample size, a combination of confirmatory factor analysis (CFA) and exploratory factor analysis (EFA) will be used to understand the dimensionality of the scale. Items will also be considered for potential redundancy based on the strength of their correlation with other items. Following any necessary revisions to the scale through CTT, and again assuming a sufficient sample size, IRT on unidimensional scales will be used to assess individual item performance (in terms of item fit, ordering of thresholds, and differential item functioning). Assumptions of IRT (i.e., unidimensionality, monotonicity, and local independence) will be assessed prior to use.

All decisions on final item selection for the PROM will incorporate and balance evidence from Stage 2a (qualitative interviews) and Stage 2b (psychometric survey). Trade-offs may have to be made between achieving sufficient content validity (particularly comprehensiveness) of the measure by including content that is important to plwALS and achieving optimal psychometric performance. All decisions on final item selection will be made consultatively within the research team, PAG, and CAG, and documented via an ‘item tracking matrix’.

Once the final PROM is confirmed, exploratory analysis will be undertaken to assess the PROM’s internal consistency reliability (using Cronbach’s alpha) and construct validity. The reliability and validity of subdomain scores (e.g., physical, psychological, social functioning) will be explored where possible. We expect scores on the PROQuALS to correlate significantly (i.e., at > 0.3) with scores on the ALSFRS-R, ALSAQ-5, and EQ-5D-5L, in decreasing order of magnitude. We will explore known-group validity by investigating how condition severity (as determined by King’s Staging) affects HRQoL (including subdomain scores). We expect disease severity to be negatively associated with PROQuALS scores, with a stronger relationship for physical functioning domain score. The content validity of the final PROM will be re-assessed through cognitive interviews with seven plwALS, following similar sampling and methodological procedures to that described in Stage 2a.

Stage 3: Valuation and econometric modelling to generate an ALS-specific preference-weighted measure

The refined PROM generated and finalised in stage 2 cannot be used to directly inform cost- effectiveness analyses and therefore an ALS-specific PWM is generated for this purpose. A PWM consists of a) a classification system comprised of dimensions of HRQoL and associated severity levels that can be used to categorise the HRQoL of all patients, and b) a scoring system derived from preferences which enable a utility value to be generated for every state defined by the classification system.

Stage 3.1 Deriving a classification system from the PROM developed in Stage 2

The PROM developed in Stage 2 will potentially contain multiple items reflecting the same or similar underlying constructs within each domain and is expected to contain over 10 items. A PWM requires a parsimonious number of items since a larger number of items with multiple items tapping identical constructs are not feasible to value in Stage 3.2. Psychometric analyses will be conducted using the quantitative data generated in Stage 2b, and published criteria for developing PWM [8] will be applied, and results will be considered by the research team, consisting of PROM developers, health economists and clinical professionals. The PAG will also be consulted and collaborate in selecting items for the PWM. This will enable selection of the best-performing items most appropriate for valuation that reflect the underlying constructs and dimensions of the PROM deemed important to patients in Stage 2a.

Stage 3.2 Eliciting and modelling preferences to generate utility values

An online discrete choice experiment (DCE) will be conducted to elicit preferences for the states defined by the classification system. DCEs are commonly used for this purpose to enable the generation of utility values for PWMs and condition-specific PWMs in particular [21]. Due to the large number of states defined by the classification system, it is infeasible to include all and hence the DCE tasks will be selected statistically taking into account the modelling that will be undertaken on the DCE data. Participants will be representative of the UK general public in accordance with NICE recommendations for utility values to generate QALYs for cost-effectiveness analyses [22]. Participants will be recruited from existing panels of participants willing to answer surveys by a market research company. The sample (N = 1000) will be with participants of the UK general population representative for age and gender. The DCE results will be modelled using regression analyses to generate a utility value for every health state that can be directly used to inform cost-effectiveness analyses.