INTRODUCTION

Patient safety (PS) is an important dimension of quality of care alongside accessibility, acceptability, effectiveness, efficiency and people-centredness. In 2002, the WHO Member States agreed on a PS resolution during the World Health Assembly. In 2015, during the 68th WHO Regional Committee for SouthEast Asia, all Member States of the Region, including India, endorsed the ‘Regional Strategy for Patient Safety in the WHO South-East Asia Region (2016–2025)’ aiming to support the development of a national quality of care and PS strategies, policies and plans and committed to translate six objectives of the regional strategy into actionable strategies at the country level. In this context, a multistakeholder PS Expert Group constituted by the Ministry of Health and Family Welfare, Government of India, developed the National Patient Safety Implementation Framework (NPSIF).1

The strategic objective 3 of the NPSIF emphasizes a ‘catch them young’ approach, where PS principles are incorporated in medical and nursing education, and then the accumulated skills are sustained through continual medical/nursing education and on-the-job training.

The competency-based undergraduate curriculum for the Indian Medical Graduate 2018 states that at the time of graduation, the clinician should demonstrate knowledge of policies that influence PS. He/she should also be a leader and member of the healthcare team and system who shall participate appropriately and effectively, inter alia, in measures that will advance PS within the healthcare system.2

The Australian Patient Safety Education Framework (2005)3 and the WHO4 proposed the following topics in the PS curriculum: communicating effectively; identifying, preventing and managing adverse events and near misses; using evidence and information; working safely; being ethical; continuing learning and other specific issues such as preventing wrong site, wrong procedure and wrong patient treatment and medication safety. The WHO also incorporated infection control in its curriculum.4 In 2009, Canada released a framework entitled ‘The Safety Competencies: Enhancing patient safety across the health professions’ that proposed six PS competencies (Fig. 1).5

FIG 1.: Safety Competencies Framework into Health Professions Education Program (modified from reference 5)

Export to PPT

The Safety Competencies Framework is the theoretical framework which underpins the Health Professional Education in Patient Safety Survey (H-PEPSS).6 The H-PEPSS comprising 23 items was developed by Ginsburg et al.6 from the six PS competencies framework and designed to measure health professionals’ (HPs) self-reported PS competence around the time of entry to practice. The H-PEPSS focuses primarily on the sociocultural aspects of PS including culture, teamwork, communication, managing risk and understanding human factors. The safety competencies were designed to be relevant across health disciplines. The six domains of the safety competencies are: (i) contribute to a culture of PS; (ii) work in teams of PS; (iii) communicate effectively for PS; (iv) manage safety risks; (v) optimize human and environmental factors and (vi) recognize, respond to and disclose adverse events.

In India, there are no scales that measure the perception of students and residents of their training with respect to PS like the H-PEPSS which measures the perception of students and residents with respect to their training in PS in the Canadian setting. This study aimed to do a confirmatory factor analysis (CFA) and psychometric validation of the H-PEPSS in the Indian context.

METHODS

The study was done at a public sector healthcare institute in New Delhi, India. The bed complement of the hospital attached to the institute is 2362. During 2016–2017, the hospital catered to about 4.14 million outpatients and 0.23 million inpatients and performed over 0.17 million surgical procedures. The average bed occupancy was close to 86%, the average hospital stay was 9 days and the net death rate was <2%. Patient care is mostly provided by resident doctors at various levels of training under the overall supervision of the relevant specialty’s faculty. In the context of a high patient load and hospital bed utilization, it is necessary that PS aspects are addressed rigorously.

Sample size

The study population comprised all the resident doctors of the clinical departments in the hospital attached to the institute. The literature recommends a sample size of 200 to perform a CFA.7 The recommended ratio of sample size to the number of items is 10 to 1 to perform a CFA.8 The H-PEPSS has 23 items. Therefore, the minimum sample size required was 230. We used a non-probability convenient sampling.

Tool

The H-PEPSS6 comprises 23 items. All items are answered using a 5-point Likert scale. For each item, participants were required to respond separately about their confidence in what they learnt in the classroom setting versus the clinical setting. Residents with at least 1-year experience were included and medical students were excluded.

Statistical analysis

The data were checked for the presence of univariate outliers, skewness, kurtosis and any multivariate outliers. Kaiser– Meyer–Olkin (KMO) test and Bartlett’s test of sphericity were carried out to assess the adequacy of the sample. Harman’s single-factor test was done to rule out common method bias. Amos V. 20 (SPSS Inc.) was used to carry out a CFA of the sample. The maximum likelihood estimation method was used. Four CFAs were carried out. The first and second CFA tested the 6-factor 23-item model of PS competency in the classroom setting and in clinical settings, respectively. The third and fourth CFA tested the 6-factor 16-item model of PS competency in the classroom setting and in clinical settings, respectively. The normed fit index (NFI), Tucker–Lewis index (TLI), comparative fit index (CFI), root mean square residual (RMR), standard RMR (SRMR) and root mean square error of approximation (RMSEA) were used to evaluate the model fit. Internal consistency and reliability of the six factors of the model was examined using Cronbach’s alpha. Convergent validity of the model was examined using average variance extracted (AVE) and composite reliability (CR). Discriminant validity was assessed using the Fornell and Larcker criterion and the heterotrait–monotrait (HTMT) method. External validity was assessed by comparing the study results with the Canadian sample results.

Ethical considerations

Permission was obtained from the institution to conduct the survey for statistical validation of the tool. The survey was conducted in accordance with the Indian Council of Medical Research ethical guidelines for biomedical and health research on human participants.9 Written informed consent was obtained from all the participants. The responses were anonymized. No personal details of any kind were sought or recorded from any participant.

RESULTS

Of the 240 filled-in questionnaires that were received, 11 responses were incomplete and were excluded from the study. The data were checked for the presence of univariate outliers, skewness, kurtosis and any multivariate outliers, and 22 samples were again excluded based on this. KMO test and Bartlett’s test of sphericity were carried out to assess the adequacy of the sample. Harman’s single-factor test was done to rule out common method bias. Four CFAs were carried out on the final sample comprising 207 respondents. The results of Cronbach’s alpha, AVE and CR are given in Tables I and II. The results of the fit indices are given in Tables III and IV. The results of the HTMT method to demonstrate discriminant validity are given in Table V. The path diagrams for the 6-factor 16-item model are shown in Figs 2 and 3.

TABLE I.: Results of Cronbach’s alpha, average variance extracted and composite reliability for the items tested in the classroom

Factor 23-item model Cronbach’s alpha 16-item model Cronbach’s alpha AVE CR Teamwork >0.870 >0.747 >0.522 >0.84 Communication >0.834 >0.834 >0.634 >0.90 Safety risk management >0.868 >0.868 >0.688 >0.92 Human and environmental factors >0.827 >0.799 >0.675 >0.88 Recognizing adverse events >0.853 >0.864 >0.763 >0.92 Safety culture >0.812 >0.781 >0.566 >0.86 Total >0.921 >0.894 >– >–

TABLE II.: Results of Cronbach’s alpha, average variance extracted and composite reliability for the items in clinical settings

Factor 23-item model Cronbach’s alpha 16-item model Canadian model (16 items) Cronbach’s alpha AVE CR Cronbach’s alpha AVE CR Teamwork >0.845 >0.731 >0.49 >0.74 >0.848 >0.564 >0.885 Communication >0.815 >0.815 >0.78 >0.82 >0.816 >0.731 >0.891 Safety risk management >0.848 >0.848 >0.81 >0.85 >0.850 >0.769 >0.909 Human and environmental factors >0.833 >0.762 >0.78 >0.66 >0.833 >0.75 >0.900 Recognizing adverse events >0.866 >0.821 >0.84 >0.82 >0.868 >0.716 >0.910 Safety culture >0.803 >0.755 >0.51 >0.76 >0.807 >0.631 >0.872 Total >0.926 >0.899 >– >– >– >– >–

TABLE III.: Results of the model fit for the model in the classroom setting

Model Cmin/df RMR NFI TLI CFI RMSEA SRMR 23-item model >2.55 >0.075 >0.822 >0.861 >0.882 >0.085 (0.076–0.094) >0.06 16-item model >1.75 >0.068 >0.917 >0.949 >0.962 >0.059 (0.043–0.074) >0.05

TABLE IV.: Results of the model fit for the model in clinical settings

Model Cmin/df RMR NFI TLI CFI RMSEA SRMR 23-item model >1.574 >0.038 >0.872 >0.939 >0.806 >0.0539 (0.042–0.063) >0.048 16-item model >1.549 >0.330 >0.914 >0.956 >0.967 >0.0520 (0.034–0.068) >0.042

TABLE V.: Results of heterotrait–monotrait method

Factor Culture Recognizing adverse event(s) Human environment Risk Communication Recognizing adverse event(s) >0.21 >– >– >– >– Human environment >0.29 >0.71 >– >– >– Risk >0.83 >0.80 >0.97 >– >– Communication >0.49 >0.81 >0.74 >1.04 >– Team >0.54 >0.74 >0.71 >0.79 >1.17

FIG 2.: Path diagram of the 6-factor 16-item confirmatory factor analysis model in clinical settings

Export to PPT

FIG 3.: Path diagram of the 6-factor 16-item confirmatory factor analysis model in classrooms

Export to PPT

DISCUSSION

In the context of an increasingly complex health system, all HPs are required to be educated and trained in PS concepts and principles if they are expected to maintain a safe environment for patients and manage them. In this regard, the H-PEPSS is an important tool to measure the subjective perception of residents and students with respect to their confidence in PS issues after obtaining training in the classroom and in clinical settings.

In the study by Ginsburg et al.,6 the initial 6-factor 23-item model was reduced to a 6-factor 16-item model. Seven items were removed on theoretical grounds such as redundancy or the item being distal to the remaining items in the construct. We did a CFA of both the models in the two settings.

A comparison of the results of this study with that of Ginsburg’s study suggests a generalizability of the PS competencies raised in the H-PEPSS. The internal consistency reliability of the factors was above 0.70 for all the factors in both the settings. An alpha value of 0.60 and 0.70 or above is the criterion for demonstrating internal consistency of new scales and established scales, respectively.10

The relative chi-square values were between 1 and 3 in our study, which are indicative of a satisfactory model fit.11 The extent to which the model is supported by data was evaluated with the help of fit indices. The closer the values of the fit indices are to 1.00, the better is the fit. The fit measures are classified as absolute fit measures, relative fit measures, fit measures based on non-central chi-square distribution and information theoretic fit measures. Absolute fit measures judge the fit of a model per se without reference to other models. Some of the absolute fit measures are the goodness-of-fit index, RMR, SRMR and adjusted goodness of fit index (AGFI). The relative fit measures such as NFI, NNFI and CFI measure the fit of the study model relative to a standard. Fit measures based on the non-central chi-square distribution such as RMSEA state that a model can only be approximately correct.12 Usually, RMSEA around 0.05 is considered as a sign of good fit.12 Alternately, RMSEA results between 0.08 and 0.10 provide a mediocre fit and values close to 0.06 provide a good model fit.13 In our study, the RMSEA values of the 16-item 6-factor model in the classroom and clinical settings were below 0.06, suggesting a good model fit. The lower limit of the confidence intervals should be close to 0 and the upper limit should be <0.08 in order to indicate a good fit.14 A SRMR value of 0 indicates a perfect fit and values <0.05 indicate a good model fit.15 Values were below 0.05 in our study, which are indicative of a good model fit. Models with TLI and CFI values >0.95 are indicative of a good model fit.13 In our study, the TLI and CFI values of the 16-item 6-factor model in the classroom and clinical settings were above 0.95, suggesting a good model fit.

The AVE value for five factors was >0.50, which indicated that the convergent validity of H-PEPSS was met. Overall, H-PEPSS produced good reliability with a CR score equaling 0.77.

In our study, the square root of the AVE exceeded the intercorrelations of the factor with other factors, which ensured discriminant validity as per the Fornell and Larcker criterion.16 As per Henseler et al., HTMT criterion is defined as the mean value of the item correlations across constructs (i.e. the heterotrait correlations) relative to the geometric mean. A threshold value of 0.90 is suggested if constructs are conceptually similar and 0.85 if the constructs are conceptually more distinct.17 However, in our study, the values were more than 0.90 for the construct pairs: teamwork/communication and managing risk/communication (Table V).

Since the 23-item model was already reduced to a 16-item model in the study by Ginsburg et al., a second sample was not selected for assessing the model fit of the 16-item model in this study.

The major limitation of this tool is that it is a subjective self-report measure that highlights the sociocultural aspects of PS. Other measures such as objective structured clinical examinations are also required for a holistic assessment of residents.

Conclusion

The competency-based undergraduate curriculum for the Indian Medical Graduate 2018 states that at the time of graduation, the clinician should demonstrate knowledge of policies that influence PS. He/she should also be a leader and member of the healthcare team and system who shall participate appropriately and effectively, inter alia, in measures that will advance PS within the healthcare system.2 The H-PEPSS may serve as a vital validated instrument for formative assessment of PS and for remediation of PS topics in the Indian setting. It may serve as a starting tool to gauge the responses of doctors and other HPs before initiating a major curriculum reform in HP education as envisaged in the NPSIF. The results may be used to test a correlation between residents’ confidence in managing PS issues and the rate of adverse events in the hospital. However, the tool highlights only the sociocultural aspects of PS.