The science around evidence-informed medicine has developed exponentially over the last three decades, with trials and systematic reviews becoming an integral part of the knowledge ecosystem machinery in the domain. For trials and systematic reviews to be policy and practice relevant, and for meaningful comparisons of benefits and harm to be made, it is essential to choose the right outcomes which should be measured consistently across studies. There is a lack of consistency in which outcomes are measured in trials, arising due to differences in chosen outcomes, specific measures, and methods of measurement1. Heterogeneity and inconsistency of outcomes makes it difficult to compare and pool data for evidence synthesis, thereby reducing the scope for meta-analyses2. Furthermore, without clear and pre-defined outcomes, authors may selectively report outcomes with significant results (reporting bias)3. Core Outcome Sets (COS) are a consensus-derived, standardized sets of outcomes that should be minimally measured in trials on a specific health condition or disease4-6 thus enabling the evidence base to be built faster and be comparable globally. Development of COS usually follow a standardized process involving multiple interest-holders, including patients, to maximize usefulness of outcomes1.

Despite the increasing availability of COS, its uptake has been sub-optimal and varies widely across areas of health7. For instance, in a review of late-phase trials published in 2019-2020; 98 per cent of the included studies did not use COS8. The Core Outcome Measures in Effectiveness Trials (COMET) initiative, a collaborative of people involved in COS development has conducted two surveys on the use of COS, both with a small number of participants (62 and 81 participants, respectively) despite multi-national scope9,10. In this study, we aimed to fill this gap by examining the use, knowledge, and attitudes about COS by clinical trialists and systematic reviewers in India, with the view of enhanced generalisability through a greater number of participants. We took a country-level approach because different factors would influence issues in different countries, and initiatives for capacity building or uptake are better targeted at country-level.

Materials & Methods

An online cross-sectional study was conducted by the Meta-research and Evidence Synthesis Unit of The George Institute of Global Health, India between August to November 2023. The study protocol was approved by the Institutional Ethics Committee of The George Institute for Global Health, and there was no protocol deviation.

Setting and participants

Adult participants (≥18 yr of age) working in healthcare and related fields, involved in conducting trials and/or systematic reviews, and residing in India or who are Indian citizens were included. There were no other restrictions, and the study was conducted online in English.

Recruitment of participants

Participants were recruited by emailing the listed ‘Principal Investigator (PI) or Trial Coordinator’ of clinical trials from the ‘Clinical Trial Registry- India’ database, registered after Jan 1, 2019 (2019 was chosen as cut-off considering e-mail IDs before that might be either irrelevant or inactive), or through social media posts with links to online survey(example through X (Twitter) and LinkedIn) or by snowballing (participants were requested to forward the emails or links to others, eligible in their network). For those invited by email, one reminder email was sent, at least two weeks after the initial mail.

Sample size

A minimum of 100 participants were planned initially for inclusion, to get a better understanding of the knowledge and perceptions towards COS, than prior studies with 62 and 81 participants, respectively, despite them being multi-country in nature9,10. We planned to maximize the number of participants (with no upper limit on recruitment numbers) in order to increase generalisability.

Data collection

Data was collected in an anonymous manner, with no personal information collected from the participants, through a questionnaire instituted through REDCap (online application for managing database and surveys ( https://redcap.georgeinstitute.org.in/ ). The questionnaire included brief demographic questions, followed by questions on awareness, use and attitudes on COS through a mix of continuous, categorical, Likert-scaled (5-point) items and open-ended questions. The questionnaire was pre-tested on a group of seven individuals internally within the institution. Data from tool testing was not included in our final analysis.

The actual questionnaire was preceded by the informed consent page. The consent page described participant rights and researcher contact information was displayed. The survey was terminated for those who did not consent to participate. The survey was anonymous, and no identifiable information was collected.

The questionnaire was designed for maximal efficiency of time for respondents such that participants after filling brief demographic information were asked if they were aware of COS before this survey (a yes/no question). Only those participants who were aware of COS before the survey were asked other questions, on use and attitudes about COS. For others, the survey ended at that point.

Data analysis

Survey responses were collated and analyzed in Microsoft Excel, which included standard descriptive statistics. Descriptive statistics (percentages for COS awareness and familiarity, and median for attitudes towards COS) were used. Open-ended questions were descriptively reported, with content analysis of the free text responses in the open fields, noting frequencies (as noted in other parameters).

Results Characteristics of the survey participants

There were 533 responses out of which 10 (1.9%) did not give consent to participate in the study (thus terminating the survey at the consent page). The mean age of the 523 participants was 42.91 yr (19-90 yr; median 42 yr). The survey indicated a good gender balance [male (259, 49.52%), female (261, 49.90%), others (0, 0%); prefer not to say (3, 0.57%)].

Most participants were from Karnataka (89, 17.02%) followed by Maharashtra (70, 13.38%) and Delhi (44,8.40%). There were no participants from Dadra and Nagar Haveli, Daman & Diu, Ladakh, Mizoram, Nagaland, and Sikkim. There were 10 (1.91%) participants currently residing outside India (USA-2, Germany-1, Oman-2, Canada-1, UAE-2, UK-1, did not respond-1). Participants reported diverse expertise in conducting trials and systematic reviews within different specialities. Detailed characteristics are presented in supplementary material.

Of the total participants, 358 (68.5%) were PIs (Principal Investigators) or co-PI or site lead of a trial. There were 212 (40.8%) lead/senior authors of systematic reviews of interventions.

Awareness about COS

A little more than half of the participants (269, 51.4%) were aware of COS, while the rest were hearing about it for the first time from this survey. Around 55.02 per cent of PI / co-PI/ site lead of a trial, were aware of COS, compared to 43.63 per cent of people who were not PI/Co-PI/site leads of a trial. Furthermore, 59.43 per cent of lead/senior authors of systematic reviews of interventions were aware of COS, compared to 46.25 per cent of people who were not lead / senior authors of systematic reviews.

Familiarity and use of COS

Familiarity and use of COS among those aware (n=267), is presented in Box 1.

Box 1.

Familiarity about, and use of COS among trialists and systematic reviewers who are aware about COS

Familiarity around use of COS

Have seen COS reported in a trial (148, 55.4%),

Have seen a COS reported in a systematic review of interventions (111, 41.6%),

Have attended training on COS (16, 6.0%),

Have attended a conference presentation/seminar/talk on COS (40, 15.0%),

Other (10, 3.7%)- read about COS in journals and co-authored paper on importance of COS

Use of COS by participant

Have used or is currently using a COS in a trial (37, 13.9%),

Have used or is currently using a COS in a systematic review of interventions (28, 10.5%),

Development of COS by participant

Availability and relevancy of existing COS

Have been wanting to use COS but could not identify COS relevant to disease condition/population under investigation (27, 10.1%),

Available COS of interest was not relevant/valid to the Indian setting/ context (10, 3.7%)

COS, core outcome sets

Participants were asked about the conditions for which they wanted to use COS but could not identify one. The list of these conditions and their actual availability which were unavailable to the participants (identified later from COMET database) is shown in table I. Overall, we found that COS were not available for many conditions of relevance to trialists and systematic reviewers of India. Whereas, in few cases, there was lack of awareness about their availability.

Table I. Conditions for which participants could not identify COS and the actual availability status

Condition for which participant to use COS but could not identify one Status of COS availability in COMET database (last searched and links accessed on August 21, 2024) Communicable disease 1 COVID-19 findings in the lung – typical and atypical ones during the initial first wave. Not available specific to COVID-19 lung manifestations 2 Tuberculosis Not available, but being developed ( http://www.comet-initiative.org/studies/searchresults?guid=c72bde98-56b9-4195-95ba-a07ae954861e ) Injuries 3 Amputation of limbs Not available, but being developed ( https://www.comet-initiative.org/studies/searchresults?guid=6bf0cb02-07bb-4444-baac-dd0e4ff0d2a9 ) Non-communicable disease 4 Acute intracerebral haemorrhage Not available 5 Childhood epilepsy Not available for all types of interventions. COS is available only for specific intervention types ( https://www.comet-initiative.org/studies/searchresults?guid=995dfd44-32aa-4e0e-8a1d-9997a82f641d ) 6 Chronic kidney disease (multiple responses) COS available ( https://www.comet-initiative.org/studies/searchresults?guid=d3beb5d4-5d03-4c02-884c-1e29bc0343f1 ) 7 Chronic low back pain COS available ( https://www.comet-initiative.org/studies/searchresults?guid=08e19bb5-a554-49b4-b28e-c99666ae882a ) 8 COPD COS available ( https://www.comet-initiative.org/studies/searchresults?guid=daefbd7e-a062-4ca6-8f2b-7bb1be766e81 ) 9 Degenerative conditions of musculoskeletal system Broad domain mentioned- not searched. 10 Diabetic retinopathy Not available, but being developed ( https://www.comet-initiative.org/studies/searchresults?guid=60cbfea6-916c-4b01-9630-e1875bb88dd5 ) 11 Endocrine disorder Broad domain mentioned- not searched. 12 Iron Deficiency anaemia Not available, but being developed ( https://www.comet-initiative.org/studies/searchresults?guid=2330d43a-d6dc-45fb-bf36-8642e79df1e8 ) 13 Knee osteoarthritis COS available ( https://www.comet-initiative.org/studies/searchresults?guid=6eed012e-f6ad-4840-b75d-81217fb6f528 ) 14 Multiple sclerosis COS available ( http://www.comet-initiative.org/studies/searchresults?guid=cdc5a4fb-f692-416d-af88-7261130bd2c4 ) 15 Palliative medicine Broad domain mentioned- not searched. Several COS in the domain is available 16 Pancreatitis Not available, but being developed ( https://www.comet-initiative.org/studies/searchresults?guid=fc8048f2-8777-444d-ba96-6704ed231927 ) 17 Polycystic ovarian syndrome COS available ( https://www.comet-initiative.org/studies/searchresults?guid=1ccbbfb5-699b-40b2-b0d7-5475f328670e ) 18 Pregnancy Available for many complications of pregnancy but not for normal pregnancy ( http://www.comet-initiative.org/studies/searchresults?guid=1b75f4c4-7ee5-4d52-9dd3-2f04ae6a2590 ) 19 Reproductive health Broad domain mentioned- not searched. Several COS in the domain is available 20 Rheumatoid arthritis COS available ( http://www.comet-initiative.org/studies/searchresults?guid=d88d65d7-76b9-4885-bd72-d0502b88e796 ) 21 Sickle cell anaemia COS available ( http://www.comet-initiative.org/studies/searchresults?guid=6adca623-1651-4697-90b5-02b1b9eea4d5 ) Oral health 22 Dental caries (multiple response) Not available, but being developed ( https://www.comet-initiative.org/studies/searchresults?guid=1de72fc1-9f96-4806-bdbe-a24b5c113429 ) 23 Gingivitis Not available ( https://www.comet-initiative.org/studies/searchresults?guid=57896287-1a99-423b-bb9a-73f1e39de748 ) 24 Oral health Broad domain mentioned- not searched. Some COS in the domain is available

Participants mentioned that for the following diseases/health conditions available COS was not relevant or valid in the Indian context/setting:

Communicable disease like Tubercular meningitis;

Non communicable diseases including kidney diseases (multiple responses including one mentioning CKD specifically), childhood epilepsy; youth mental health interventions to improve positive mental health among young people in the school/college/community settings; comparing two drugs for analgesic efficacy; other conditions reported were rhinosinusitis; and systemic sclerosis.

Knowledge about COS concepts were assessed among individuals who were aware of COS. We asked several statements with respect to COS concepts, but deliberately refrained asking it in a true-false format (in which guesswork could lead to a 50% correct response). Instead, we used a 5-point Likert scale of agreement to understand if the participants to choose the correct response (strongly agree to the statement). We found that majority of the participants had some misgivings about the core concepts of COS (Table II).

Table II. Knowledge about COS concepts among those who were aware about COS

Statement in survey to assess knowledge of COS Participants who said strongly agreed to statement (n)/Total number of respondents (N); % All outcomes in the COS should be measured in a trial 71/214; 33.2 All outcomes in the COS should be used in a systematic review 61/213, 28.6 Other outcomes can be measured in addition to outcomes in COS in a trial/systematic review 58/211, 27.5 COS reduce heterogeneity in how outcomes are measured 48/211, 22.7 COS reduces outcome reporting bias 72/213, 33.8 Development of a COS involves multiple stages 86/213, 40.4 Using COS increases comparability of findings across trials 78/213, 36.6

Attitudes about COS among participants who were aware were also recorded. Majority of participants who were aware of COS, had positive attitudes towards COS, in terms of improving transparency, openness, interest-holder relevance and being relevant to routine care. However, substantial proportion of participants believed that COS were restrictive in nature, added to the cost of trial, led to patient burden, and prevented trialist innovation (Table III).

Table III. Attitude about COS among those who were aware about COS (attitudes measured through agreement with statements on a Likert scale)

Statement in survey to assess attitudes about COS Strongly disagree; n,% Disagree; n,% Neither agree nor disagree; n,% Agree; n,% Strongly agree; n,% COS increase transparency and openness of research 5, 2.3 3, 1.4 12, 5.6 99, 46.5 94, 44.1 COS is restrictive in nature 8, 3.8 64, 30 79, 37.1 52, 24.4 10, 4.7 Using COS adds to cost of trial 13, 6.1 46, 21.6 68, 31.9 70, 32.9 16, 7.5 Using COS adds to patient burden (quantity and repetition) 14, 6.6 69, 32.4 60, 28.2 57, 26.8 13, 6.1 Use of COS increases relevance of outcome to key stakeholders (including patients) 2, 0.9 12, 5.6 13, 6.1 101, 47.4 85, 39.9 COS prevent innovation for trialists 24, 11.3 64, 30.2 65, 30.7 49, 23.1 10, 4.7 COS is relevant to routine care 4, 1.9 12, 5.6 34, 16 114, 53.5 49, 23 Discussion

This study demonstrated that almost half of Indian trialists and systematic reviewers were not aware of COS at all. Even among those aware, uptake of COS in trials and systematic reviews was poor (13.9% and 10.5%, respectively). This study also identified four barriers to COS uptake, namely, a) unavailability of COS for a number of conditions of relevance to trialists and systematic reviewers of India or available COS perceived to have low relevance; b) low knowledge about core concepts around COS; c) negative attitudes around COS being restrictive, preventing innovation, adding to patient burden and trial costs; d) for many health conditions relevant to India, there was no COS. Furthermore, it was observed that participants were not aware of the COS-despite its availability, or available COS was thought to be irrelevant or invalid in the context.

The first study on this topic, which had just 62 respondents (with >75% participants from Europe), identified poor knowledge about COS and negative perceptions (particularly about COS being restrictive and often containing too many outcomes) as a major barrier to COS uptake10. While we found around 50 per cent participants were aware of COS in this study, and the uptake of COS in trials was only 13.9 per cent, the corresponding figures in the aforementioned study were higher at 65 per cent and 50 per cent, respectively. However, the small sample size of the previous study limits interpretative comparison. Both the present study as well as Bellucci et al10 study identified similar negative perceptions, but our study identified additional issues around non-availability of COS for health conditions of relevance to India and the low relevance of available COS. These aspects are related to interest-holders from low- or middle-income countries (LMICs) (including India) not leading to the development of COS and being excluded in COS development despite its supposedly global scope. The poor involvement of interest-holders from LMICs in COS development has been documented earlier11-14.

The more recent COMET study was among 81 LMIC interest-holders9. The study used a case-study approach preventing direct comparison. However, the quantitative part of the study reported lesser awareness (32.09%), but similar COS uptake (12.34%). The study found that usefulness, availability, and practicability of COS in LMIC settings and poor involvement of patients and carers in their development were key barriers to COS uptake. The present study using a quantitative approach, found similar challenges around usefulness, availability, and practicability of COS in the Indian setting, as well as additional aspects. Although these barriers do highlight drawbacks of COS, some perceived barriers are also reflective of decreased knowledge regarding COS. For instance, although many participants indicated the COS were restrictive, only 27.5 per cent of participants understood that other outcomes can be measured in addition to outcomes in COS in a trial/systematic review.

As per our knowledge, the present study is the largest survey so far which provides insights about barriers for COS uptake, despite its scope being limited to a single country (India), unlike previous studies which had multi-national scope9,10. This is a key strength of the survey. Considering the fact that English is the lingua franca of academic medicine and healthcare in India, the choice of language had no major implications on representativeness of the survey results in the context of this study. Many e-mails sent to trialists’ e-mail addresses from CTRI (Clinical Trial Registry of India), bounced back or were undelivered. This method of electronic recruitment may have resulted in researchers who were already familiar with COS being more likely to respond to the survey. We thus envisage that our results paint a more optimistic picture than the reality. This is a perceived limitation. Implications of the study for policy, practice and future research is summarized in Box 2.

Box 2.

Implications of our study on policy, practice, and future research

Funding agencies encourage the use of COS in clinical trials through mandates in funding calls. Where COS is not available, these should be developed before initiating clinical trials.

Ethics committees should also mandate use of COS where available. Non-usage of COS, when available should be justifiable. There is a need to evaluate awareness and perceptions around COS in funding agencies and ethics committees too. The mandate for such action falls within the domain of Central Drugs Standard Control Organisation, which registers ethics committees which approve trials in India.

There is need for streamlining more funding and future research in developing new and high-quality COS for relevant areas and conditions.

More training should be organized for researchers, health professionals to increase engagement and exposure to the concept of COS in research.

We identified many health conditions relevant to India, where there is no COS or trialists identified available COS that were not relevant or valid in the Indian context. Indian Council of Medical Research should take the lead in funding development of such COS relevant to India (and more widely to South Asia and other low-middle-income-nations). Indian medical researchers should be encouraged to lead development of COS for diseases which are high burden in India. The Indian Clinical Trial and Education Network (INTENT) might play a key role around development of COS, such that Indian researchers move beyond the passive role of COS users. There is also a need to fund meta-research on understanding adequacy and relevancy of existing COS in the Indian context

Future COS development should involve wide range of people including patients and caregivers representing geographic diversity from endemic nations. Core outcome set which are developed with limited participation of people from endemic nations serve limited utility and have little relevance. The COS-STAD standards for developing COS should be revised to have nuance in representativeness. This has been highlighted previously, while developing a COS on snakebite for South Asia14,15.

There is research needed to understand awareness, use and attitudes at country-level for both high, middle, and low-income nations to inform capacity building initiatives specific nations accordingly.

Applicability of existing COS for AYUSH systems of medicine is an area of meta-research which the Ministry of AYUSH might focus on.

In conclusion, there is a need to enhance the awareness of the concepts and importance of COS to facilitate higher uptake in research. Initiatives can be taken to increase understanding of how COS can improve data comparison to drive evidence-based policy and practice. COS development should also promote inclusion of interest-holders across countries if COS are to be relevant in a global context.