Enhancing nursing competency through virtual reality simulation among nursing students: a systematic review and meta-analysis

1 Introduction

Nursing education is an applied discipline in which theory and practical education are combined; prospective nurses prepare to become competent by applying the knowledge learned in theoretical education to the practical education process. The need for nursing education to train professionals who provide nursing and medical services to humans utilizing digital-based, non-face-to-face media such as artificial intelligence (AI) and big data has recently become more urgent (1). In nursing education, there has been an increasing interest in virtual-reality simulation (VRS) education as an alternative and complementary method to traditional simulation education, providing students with new learning experiences in a reproduced clinical environment and enhancing clinical adaptability (2). Virtual reality (VR) is defined as “the use of partial immersion through a digital learning environment (computer, tablet, phone, screen, etc.) to foster a perceived lived experience for an intended outcome (e.g., learning and entertainment)” (3). This study defines VRS to include VR and its derivatives, augmented reality (AR), and mixed reality (MR), using the terminology consistently. In VRS, learners can collaborate with other healthcare professionals to provide interventions, such as solving patients’ problems or practicing simple skills (4, 5). Improved clinical performance skills, knowledge, and metacognition, as well as enhanced learning satisfaction, communication, self-efficacy, confidence, and teamwork have been reported as effects of these VR programs (4, 6). In addition, studies on the effectiveness of nursing education using VR have been conducted on learning effectiveness, emotional engagement and immersion, learner confidence, and satisfaction (7, 8). Reportedly, VRS programs for nursing skills are effective in improving skills (9) and have the advantage of enabling safe and repetitive training without time and space constraints (10). Thus, learning through VRS has demonstrated improvement in various factors related to clinical nursing competency, albeit often assessed in a fragmented manner. As various forms of VRS are being applied in nursing education, and diverse elements contributing to nursing competency are considered, there is a need to comprehend the holistic outcomes of these studies. Consequently, this study aims to comprehensively review the results, considering nursing competency in a broader sense that encompasses collaboration, interpersonal relationships, communication, professional development, and the nursing process, skills, and education (11).

Moreover, a systematic review and analysis of nursing students’ outcomes are essential for determining specific factors that are deemed effective. Systematic reviews and meta-analyses can amalgamate the results of included studies, accounting for differences in sample size, variations in research approaches, and intervention effects among independent studies. We believe that the systematic review and meta-analysis in this study will enable an assessment of the overall effect of VRS-based education on nursing students’ nursing competency. Consequently, this study aims to provide foundational data on VRS by conducting a systematic literature review and meta-analysis, investigating the improvement effect of VRS on nursing students’ nursing competency as a primary outcome, and examining knowledge, self-efficacy, problem-solving skills, confidence, and satisfaction as secondary outcomes.

This study aims to acquire and analyze evidence regarding the enhancement of nursing students’ nursing competency through VRS. The primary outcome focuses on nursing students’ self-reported feelings and reactions, while the secondary outcome assesses nursing students’ nursing competency following exposure to VRS.

2 Materials and methods 2.1 Search strategy and data sources

The search was jointly conducted by two researchers, Cho, M.-K. and Kim, M.Y., across nine electronic databases or e-journals: PubMed, Cochrane, EMBASE-OVID, CINAHL, World of Science, SCOPUS, PQDT, APA PsycArticles, and Research Information Sharing Service. The primary search, conducted from July 18, 2023, to August 20, 2023, targeted articles published in English and Korean from January 1, 2003, to April 30, 2023. A secondary search was carried out from April 6, 2024, to April 9, 2024, focusing on articles published from May 1, 2023, to March 31, 2024, also in English and Korean. The search strategy and formula, following the PICO-SD framework (population, intervention, comparison, outcome, study design), are detailed in Table 1. The keywords employed in search terms across the nine databases included combinations and variations of “nursing students,” “virtual reality,” “augmented reality,” “extended reality,” “metaverse,” “competency-based education,” “clinical competence,” “competency,” and “controlled clinical trial.” These keywords were chosen to comprehensively capture studies relevant to the impact of virtual reality simulation on nursing competency.

Table 1. Search strategy according to PICO.

2.2 Inclusion and exclusion criteria

The reporting of the results adhered to the PRISMA 2020 checklist. Inclusion criteria comprised nursing students aged 19 years or older (Population), interventions involving VRS (Intervention), with conventional learning methods or no intervention as the control (comparison). The primary outcome was nursing competency, and secondary outcomes included knowledge, self-efficacy, problem-solving, confidence, satisfaction, and other variables, which were concurrently measured. If multiple measurements were conducted post-intervention, the first measurement was used to calculate the effect size. Only studies presenting subject numbers, means, and standard deviations in the results were selected for precise effect-size calculation. The study designs included randomized controlled trials (RCTs) and quasi-experimental studies. Exclusion criteria included studies encompassing students from majors other than nursing, interventions using conventional simulation-learning methods instead of VRS, the absence of nursing competency as an outcome variable, studies not reported in Korean or English, studies with inaccessible original texts, and single-group studies lacking a control group.

2.3 Data extraction

Two researchers, Cho, M.-K. and Kim, M.Y., independently conducted searches and selected studies for analysis based on the predefined inclusion and exclusion criteria. The selected studies were extracted, incorporating information such as author, year of publication, country, publication language, number of schools, institutional review board (IRB) approval, funding details, number of participants, study design, intervention characteristics (type, facilitator, duration, session, time/session, pre-briefing, debriefing, other activities, outcome measurement time, and measurement method), quality assessment score, and dependent variables. This information was meticulously recorded in a coding book created using the Microsoft Excel spreadsheet software. Any disparities in coding were addressed by revisiting the original text to ascertain and input the final coding values (Table 2).

Table 2. Descriptive summary of the included studies.

2.4 Quality assessment

The quality assessment of selected articles was independently performed by Cho, M.-K. and Kim, M.Y. using the Joanna Briggs Institute (JBI) Checklist for RCTs and the Checklist for Quasi-Experimental Studies. Five RCTs were assessed using the 13-question JBI Checklist; the average score was 8.40, and all five studies lacked clear reporting on “Q2. Was allocation to treatment groups concealed?” and “Q4. Were participants blind to treatment assignment?” Quasi-experimental studies comprised eight articles, and on evaluation using the 9-item JBI Checklist for Quasi-Experimental Studies (32), the average score was 8.50, with generally well-reported items (Table 3).

Table 3. Quality assessment of the included studies.

2.5 Statistical analyses

MIX 2.0 Pro (Ver. 2.0.1.6, BiostatXL, 2017) was used to calculate and merge effect sizes for both the primary outcome of nursing competency and secondary outcomes. The overall effect was determined using a random-effects model, considering between-subject variability and heterogeneity between studies. Hedge’s g was employed for effect-size calculation, and significance was assessed using 95% confidence intervals (CIs), Z tests, and p-values. The weight of each effect size was determined using the inverse of variance (33). Heterogeneity was evaluated using Higgin’s I2 (34), with an I2 of >50% indicating heterogeneity (35). Subgroup analysis, meta-regression, and exclusion-sensitivity analysis were conducted for nursing competency to identify factors contributing to heterogeneity. Publication bias was examined using funnel plots, trim-and-fill plots, Begg’s test, Egger’s regression, and the trim-and-fill method to correct for the overall effect (36).

3 Results 3.1 Characteristics of the included studies

A total of 579 articles were initially identified from 9 databases following the search strategy. After excluding duplicates, 373 articles were extracted. Following the application of the inclusion and exclusion criteria, 21 research articles were ultimately selected. The research by Sahin Karaduman and Basak (28) was designed using two experimental groups and was analyzed as two separate studies, resulting in 22 studies being analyzed (Figure 1). Of these, six studies were published before 2022; three were conducted in the United States (USA), twelve studies were published in English; nineteen were conducted at a single university; nineteen and nine studies had IRB approval and funding, respectively. The study designs included seven RCTs, fifteen quasi-experimental studies, and eight studies with fewer than 60 participants. Interventions comprised 18 VR/AR simulations and four metaverse. Eight studies had a professor as a facilitator, four had an intervention duration of more than 4 weeks, two had four or more intervention sessions, eight had an intervention time of more than 1 h per session, 19 had a pre-briefing, and nine had a debriefing. Dependent-variable measurements were taken immediately after the intervention in 20 studies, the measurement method was observational measurement in 12 studies, 19 studies had no additional activities, such as reflection, besides the simulation, and 14 studies had an above-average quality assessment score (Table 2).

Figure 1. PRISMA flow diagram. An article by Sahin Karaduman and Basak (28), designed using two experimental groups, was divided into two studies.

3.2 Effect of VRS-based intervention on nursing competency

The overall effect of nursing competency, as the primary outcome for the 22 VRSs, was found to be Hedge’s g = 0.88 (95% CI: 0.47 to 1.29). This was interpreted as a large effect based on the criteria provided by Brydges (37) for interpreting effect sizes (Figure 2). The high degree of heterogeneity among the studies, indicated by Higgins’s I2 of 91.8% in the heterogeneity test, prompted subgroup and meta-regression analyses to explore factors contributing to this heterogeneity.

Figure 2. The effect of virtual reality simulation-based intervention on nursing competency. ES, effect size; CI, confidence interval. An article by Sahin Karaduman and Basak (29), designed using two experimental groups, was divided into two studies (18a and 18b). *Removal of the number of duplicate subjects in the 18th study.

In subgroup analyses, the characteristics of studies significantly associated with improvements in nursing competency IRB-approved studies (Hedge’s g = 1.02, 95% CI: 0.57, 1.48); interventions with a duration not reported or those with a duration of less than 4 weeks (Hedge’s g = 1.05, 95% CI: 0.56, 1.53); interventions with sessions not reported or those with less than 4 sessions (Hedge’s g = 0.95, 95% CI: 0.50, 1.39); those with outcome measurement immediately after the intervention (Hedge’s g = 0.93, 95% CI: 0.50, 1.37); those with pre-briefing before the simulation (Hedge’s g = 0.71, 95% CI: 0.23, 1.20); those with debriefing after the simulation (Hedge’s g = 1.02, 95% CI: 0.57, 1.48); and those with no other activities besides the simulation, such as keeping a reflective journal (Hedge’s g = 1.03, 95% CI: 0.55, 1.50). Publication year, Country, publication language, number of schools, funding status, research design, number of participants, intervention type, facilitator, intervention time per session, measurement method, debriefing, and quality assessment score also showed statistically significant effect sizes (Table 4).

Table 4. Subgroup analysis of nursing competency according to study characteristics.

Univariate meta-regression identified factors influencing the overall effect. Publication year after 2022 (Z = 2.68, p = 0.007); having an IRB (Z = 5.17, p < 0.001); having an fund (Z = −2.61, p = 0.009); RCT (Z = 2.02, p = 0.044); intervention duration over than 4 weeks (Z = −3.33, p < 0.001); intervention session over than 4 sessions (Z = −3.01, p < 0.001); intervention time/session over than 1 h (Z = −5.20, p < 0.001); observational measurement rather than self-reporting (Z = 3.21, p = 0.001); having a pre-briefing before the simulation (Z = 3.76, p < 0.001); having a debriefing after the simulation (Z = −4.41, p < 0.001); and having other activities besides the simulation (Z = −4.41, p < 0.001) had statistically significant effects on nursing competency (Table 5).

Table 5. Meta-regression analysis to evaluate competency.

The exclusion-sensitivity test (38), excluding one study at a time, showed Hedge’s g ranging from 0.67 to 0.94, indicating a moderate to large effect size. The 95% CI (0.36 ~ 0.53, 0.98 ~ 1.36) did not include 0, signifying statistical significance. The effect sizes from the exclusion-sensitivity test were not significantly different from Hedge’s g = 0.88, which included all 22 studies (Table 6).

Table 6. Exclusion-sensitivity test of the virtual-reality simulation-based intervention.

3.3 Effect of intervention program on secondary outcomes

The secondary outcomes in this study included knowledge, self-efficacy, problem-solving, confidence, and satisfaction. Among these, knowledge, self-efficacy, confidence, and satisfaction exhibited statistically significant changes. After VRS, knowledge and self-efficacy showed significant increases, with moderate effect sizes of Hedge’s g = 0.60 (95% CI: 0.07, 1.14) and Hedge’s g = 0.53 (95% CI: 0.09, 0.97), respectively. Confidence and satisfaction exhibited substantial increases, with large effect sizes of Hedge’s g = 1.02 (95% CI: 0.48, 1.57) and Hedge’s g = 1.35 (95% CI: 0.43, 2.28), respectively (Table 7).

Table 7. Effects of virtual reality simulation-based intervention on other variables.

3.4 Publication bias

To evaluate publication bias, funnel-plot and trim-and-fill plot analyses were conducted. Represented by the black circle, the individual effect sizes of the 22 studies included in the study were asymmetrical— slightly skewed to the right—indicating some potential publication bias (Figure 3). The trim-and-fill plot suggested the addition of one study, represented by the white circle, skewing to the left (Figure 4). Further analysis, The coefficient of the bias was 8.58, indicating the initial value (intercept) and the p-value was 0.001. Thus, the null hypothesis was rejected, and the existence of a publication bias could be confirmed. Unlike Egger’s regression test result, Begg’s test for rank correlation (Tau b = 0.27, ties = 0; Z = 1.75, p = 0.080) confirmed the absence of publication bias. Moreover, the trim-and-fill method suggested adding one article; the effect size of the 23 corrected articles was 0.60 (95% CI: 0.49, 0.72). Although the effect size of nursing competency was somewhat smaller after correction than before, it remained statistically significant. In conclusion, this study was deemed free of publication bias (Table 8).

Figure 3. Funnel plot of virtual reality simulation-based intervention on nursing competency. Precision = 1/standard error; 0.05; limit line = 95% confidence limit.

Figure 4. Trim and fill plot of virtual reality simulation-based intervention on nursing competency. Precision = 1/standard error; 0.05; limit line = 95% confidence limit.

Table 8. Publication bias test of virtual reality simulation-based intervention on competency.

4 Discussion

In this study, the impact of simulation-based programs on nursing competency demonstrated a significant effect size of 0.88. It’s notable that this simulation-based program yielded encouraging results by positively enhancing nursing competency. This is consistent with similar improvements observed in self-efficacy, a factor linked to nursing competency (19), enhanced knowledge, educational satisfaction, and academic achievement through VR in a hospital environment (39); and improved nursing-process performance (40), heightened critical thinking, clinical performance, and practice satisfaction through vSim for Nursing (41). Additionally, these results partially correlate with those in a study indicating that hands-on training utilizing scenario-based admission management in VR increased learning immersion, learner confidence, and learning satisfaction (7).

In the meta-regression analysis evaluating nursing competency, several factors emerged as influential. First, in cases where the publication year was 2022 or later, nursing competency was found to be significantly improved compared to studies that received IRB approval, compared to studies published before then. In the evolving landscape of clinical practice, recent emphasis on patient safety and rights has shifted the focus toward observing nursing behavior rather than direct patient care (42). This shift underscores the active implementation of simulation-based education, suggesting a more systematic adaptation of teaching methods and educational systems to enhance nursing competency compared to previous approaches. Moreover, studies with an intervention duration not reported or one of less than 4 weeks demonstrated a significant effect on nursing competency compared to those lasting more than 4 weeks. In cases of intervention with fewer than four sessions, competency was significantly improved compared to intervention sessions with four or more sessions. Similarly, interventions with time per session not reported or those lasting less than 1 h were associated with a significant improvement in nursing competency compared to those lasting more than 1 h. These findings suggest that shorter, more intensive interventions may be more effective in enhancing nursing competency through VRS. Establishing short-term intensive courses could thus be a meaningful approach. Even in the case of pre-briefings, which are recognized for their utility, the introduction and assignment of roles and expectations during pre-briefings may not be optimal. This is because simulation anxiety is linked to higher levels of extraneous cognitive load (43). Further investigation into the timing and temporal aspects of these activities is warranted to optimize their effectiveness. Therefore, further research specifically focusing on the temporal aspect is deemed necessary to comprehensively understand its implications.

Furthermore, pre-briefing before simulation emerged as a significant factor contributing to the improvement of nursing competency compared to that in the control group. This is consistent with the recognized importance of pre-briefing in face-to-face simulations, in which it influences simulation readiness (44). Given that most included studies conducted virtual pre-briefing activities individually, such as pre-briefing lessons and quizzes, the findings imply that virtual pre-briefing can be actively utilized with comparable effectiveness in face-to-face simulations. Various pre-briefing methods, including role rubrics, are currently under development (45). Further research will be necessary to ascertain the effectiveness of these diverse pre-briefing approaches.

Moreover, this study identified that post-simulation debriefing had a more significant effect of improving nursing competency compared to non-simulation debriefing. This could be attributed to the characteristic of VRS that enables repeated and reflective learning through debriefing with immediate feedback, thus providing learner-customized learning (46). The ability to facilitate individual improvement in nursing competency through immediate feedback is consistent with previous studies emphasizing the effectiveness and importance of debriefing in simulation (47). While debriefing in a virtual setting may differ from team interaction, reflection, and discussion in a face-to-face simulation, the results underscore the crucial role of debriefing in VRS situations.

Competency improved significantly when observation was measured rather than self-report. Role assignment in nursing simulation often elicits significant anxiety stemming from uncertainty, performing in front of faculty and peers, and social evaluation (45). Moreover, many individuals perceive themselves as lacking proficiency, particularly in terms of nursing competency. Consequently, self-reported improvements in nursing competency may underestimate actual progress observed through objective evaluation. Hence, effective communication and encouragement regarding the significance of simulation are vital when implementing simulation programs.

Nursing competency was statistically significantly improved when compared to those who did not engage in any other activities other than simulation. Other activities take as much time, which suggests that core simulation activities are important for improving nursing competency. Non-simulation activities, denoting the absence of activities other than simulation, exhibited a significant effect on nursing competency. While non-simulation activities may improve competencies such as team cooperation, communication, or empathy, they were not associated with improvements in nursing competency. This suggests that clear simulation content, along with pre-briefing and debriefing activities tailored to enhance nursing competency, directly influence this competency.

Meanwhile, several variables did not demonstrate a statistically significant effect of improving nursing competency. The country, number of centers, funding status, research design, and all the variables related to the operation of the intervention program (participants, intervention type, facilitator, intervention session, and outcome-measurement time), as well as the quality score, did not show significant differences in improving nursing competency. The inconsistency in trends observed across these variables can be attributed to the diverse definitions and measurements of nursing competency utilized in the included studies. This variability in research outcomes underscores the absence of a standardized measurement tool for nursing competency, potentially leading to increased heterogeneity in results.

Furthermore, the secondary outcomes measured alongside nursing competency in this study included knowledge, self-efficacy, problem-solving, confidence, and satisfaction. Among these, knowledge and confidence demonstrated statistically significant improvement. These variables, particularly knowledge and confidence, are closely related to nursing competency and can concurrently improve with it in VRS. Conversely, self-efficacy, problem-solving, and satisfaction did not show significant improvement. This is consistent with previous research indicating that VR nursing education improves knowledge (48) and increases learning satisfaction (49) but does not enhance technical skills (48) or significantly impact self-efficacy (49). This suggests that while VRS is effective in improving knowledge-related competencies, consistent improvements in self-efficacy, problem-solving, and satisfaction may depend on its design and utilization.

Given that learning immersion through simulation has been demonstrated to impact the development of clinical-nursing competence (50), and VR-based programs have been effective in improving cognitive performance, such as theoretical knowledge, through realism (51), VRS holds promise in nursing education. However, the results of this study underscore the need to carefully consider elements that are more challenging to implement in virtual situations than in face-to-face scenarios. Therefore, further research, such as systematic reviews and meta-analyses exploring other variables in VRS, is recommended for a more comprehensive understanding of its impact on nursing education.

VR-based nursing education represents an innovative field that has not been previously explored. These simulators offer a range of environments that transcend physical constraints, enabling participants to immerse themselves within the virtual space (52). It’s crucial for educators responsible for program development to grasp the distinctions between virtual reality and reality to facilitate effective education.

This study underscores the significance of pre-briefing and debriefing elements in VR-based simulation, highlighting the importance of their organization. Rather than focusing solely on operational time or the duration of the simulation itself, the key lies in how these elements are implemented for optimal educational outcomes. Additionally, when assessing effectiveness, we advocate for a combined approach utilizing both self-reported evaluations and objective evaluations through observation or assessment.

4.1 Limitations of the study

This study acknowledges several limitations. First, there is variability in reporting randomization methods among the included studies, with some providing comprehensive discussions on the topic while others lack detailed information on the methods employed. Second, the diverse interpretations and definitions of nursing competency across the included studies may introduce variability in the study outcomes. Third, the absence of a standardized measurement tool for nursing competency could contribute to increased heterogeneity. Fourth, the selection criteria for the studies analyzed, which included only those published in English or Korean and reported precise means, standard deviations, and sample sizes, could lead to selection bias and limit the generalization of our study results. Additionally, the studies encompass sample sizes from different countries, further contributing to overall heterogeneity. To enhance the robustness of future research and validate the effectiveness of interventions for nursing students, larger sample sizes and higher-quality studies are recommended.

5 Conclusion

The meta-analysis of nursing competency in VRS revealed the latter’s effectiveness in enhancing nursing competency. Notably, the incorporation of key elements from face-to-face simulation, such as pre-briefing and debriefing, significantly improved nursing competency compared to scenarios in which these elements were absent. This study suggests the importance of reflecting core simulation elements in virtual simulations and underscores the need to enhance the quality of pre-briefing and debriefing in virtual contexts. Moreover, the findings suggest that intensively operating VRS over a short period could be more effective in improving nursing competency. This implies the significance of considering the effectiveness of short-term intensive courses for nursing-competency improvement within virtual spaces. The study findings provide valuable insights for the design of VRS aimed at enhancing nursing competency.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding authors.

Author contributions

M-KC: Conceptualization, Data curation, Formal analysis, Methodology, Visualization, Writing – original draft, Writing – review & editing. MK: Conceptualization, Data curation, Funding acquisition, Project administration, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This study was supported by the National Research Foundation of Korea grant funded by the South Korea Government (MSIT; no. 2022R1F1A1076248).

Acknowledgments

The authors sincerely thank those who participated in this study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Han, D-L . Nursing students’ perception of virtual reality (VR) and needs assessment for virtual reality simulation in mental health nursing. J Digit Contents Soc. (2020) 21:1481–7. doi: 10.9728/dcs.2020.21.8.1481

More from this channel

Enhancing nursing competency through virtual reality simulation among nursing students: a systematic review and meta-analysis

留言 (0)