Reliability of Risk Assessment for Intra-Hospital Venous Thromboembolism: An Exploratory Cross-Sectional Study

Shasha Geng,1,2,* Yang Li,1,2,* Jianli Ge,1,2 Xiaotong Guo,1,2 Yue Liu,1,2 Hua Jiang1,2

1Department of General Practice, Shanghai East Hospital, Tongji University School of Medicine, Shanghai, People’s Republic of China; 2Department of Geriatrics, Shanghai East Hospital, Tongji University School of Medicine, Shanghai, People’s Republic of China

Objective: Hospital-associated venous thromboembolism (VTE) is a major cause of unintended death in hospitalized patients. Standardized and reasonable prevention measures may reduce its occurrence effectively. This study aims to analyze the consistency of VTE risk assessment by physicians and nurses and its potential causes.
Methods: A total of 897 patients admitted to Shanghai East Hospital from December 2021 to March 2022 were recruited. The VTE assessment scores of physicians and nurses and the activities of daily living (ADL) scores within the first 24 h of admission were collected for each patient. Cohen’s Kappa values were calculated to assess the inter-rater consistency of these scores.
Results: VTE scores were fairly consistent between doctors and nurses in both surgical (Kappa = 0.30, 95% CI: 0.25– 0.34) and non-surgical (Kappa = 0.35, 95% CI: 0.31– 0.38) departments. There was moderate agreement in VTE risk assessment between doctors and nurses in surgical departments (Kappa = 0.50, 95% CI: 0.38– 0.62) while fair agreement in VTE risk assessment between doctors and nurses in non-surgical departments (Kappa = 0.32, 95% CI: 0.26– 0.40). The assessment of the mobility impairment component was fairly consistent between doctors and nurses in the non-surgical departments (Kappa = 0.31, 95% CI: 0.25– 0.37).
Conclusion: Due to the poor consistency of VTE risk assessment between doctors and nurses, it is necessary to provide systematic training and develop a standardized assessment process for healthcare professionals to construct a scientific and effective VTE prevention and treatment system.

Keywords: venous thromboembolism, risk assessment, consistency, reliability, exploratory study

Introduction

In clinical practice, venous thromboembolism (VTE) mainly includes deep venous thrombosis (DVT) and pulmonary embolism (PE). Recently, VTE incidence has been increasing annually and has become an important cause of intra-hospital unintended and peri-operative deaths for inpatients. It is a potential risk for clinical practices and patient safety,1 and standardized prevention measures can reduce VTE incidence and mortality.2–5

Numerous VTE risk assessment tools are available, including the Caprini risk assessment model (Caprini RAM), the Rogers risk assessment scale, the Padua risk assessment scale, the 4-Element risk assessment model (4-Element RAM), and the Khorana scale.6 Caprini designed the Caprini RAM in the United States in 2005, integrating clinical experience and research findings.7,8 It has been extensively validated and is currently used to evaluate the risk of VTE for surgical inpatients.9 A multidisciplinary team from Padua University in Italy designed the Padua risk assessment scale to assess the risk of VTE in internal medicine departments.10 Several studies have demonstrated the importance of the Padua scale for the early screening and prevention of VTE in non-surgical inpatients.11

In the past decade, various professional organizations worldwide have issued different guidelines or management recommendations for VTE prevention and treatment.12–17 The “National Project on Capacity Building for Prevention and Treatment of Pulmonary Embolism and Deep Venous Thrombosis”, approved by the Medical Affairs Administration of the National Health Commission, was officially launched in October 2018 in China.1 More than 1500 hospitals nationwide have participated in constructing the prevention and treatment system by June 2022.

China lacks a systematic quality assessment index for the VTE prevention and treatment system. Simultaneously, healthcare professionals (especially junior professionals) lack sufficient awareness of VTE risk and have not received adequate prior training or a standardized VTE assessment process. These reasons may lead to a significant decrease in the accuracy of VTE risk assessment, affecting the clinical decision-making and practices of early intervention for VTE high-risk groups. This study aims to examine the consistency of VTE risk assessment between doctors and nurses based on a purposive sample of retrospective data and to guide the subsequent development of systematic training and standardized VTE risk assessment processes.

Study Design

This study involved the retrospective data sampling of 897 patients admitted to Shanghai Oriental Hospital between December 2021 and December 2022. Patients were recruited from non-surgical departments (cardiovascular medicine, emergency internal medicine, oncology, respiratory ICU, neurology, geriatrics, and VIP clinical department) and surgical departments (traumatology, urology, and gastrointestinal surgery), with cardiovascular medicine, respiratory ICU, emergency internal medicine, oncology, and gastrointestinal surgery serving as priority departments for VTE prevention and treatment in the hospital.

According to the “Guidelines for Quality Evaluation and Management of Intra-Hospital Venous Thromboembolism Prevention and Treatment (2022 Edition)” in China, patients in departments such as orthopedics, intensive care unit, neurology, gynecology, obstetrics, and oncology are the focus of VTE prevention and treatment; they are called priority departments and the rest are non-priority departments. Patients in non-surgical departments should be evaluated using the Padua scale, while patients in surgical departments should be evaluated using the Caprini RAM. All patients’ demographic data, VTE scores from doctors and nurses, and ADL scores within 24 h of admission were collected. The risk of VTE was determined according to each patient’s VTE score (Padua scale: low risk: 0–3 points, moderate-high risk: ≥ 4 points, Caprini RAM: 0–2 points, moderate-high risk: ≥ 3 points).

The study was statistically analyzed with STATA 17.0 software. Normality for continuous variables is first examined by the Shapiro–Wilk test, where non-normal variables are characterized as median (interquartile range) [M(IQR)]. Categorical variables were described in terms of frequencies and percentages. Cohen’s Kappa values were calculated using the Kappa command to assess inter-rater agreement. Kappa values ranged from < 0.20 (extremely low agreement), 0.21–0.40 (fair agreement), 0.41–0.60 (moderate agreement), 0.61–0.80 (high agreement), and 0.81–1.00 (almost perfect agreement).

Results

Patients’ characteristics: A total of 881 patients were included in the consistency analysis, excluding the misuse of measurement scales by doctors or nurses (n = 16). There were 285 patients from surgical departments and 596 from non-surgical departments. The median age was 67 (55–76) for 500 males and 381 females. The median VTE score by doctors was 2 (1–4), with 639 (72.35%) in the low-risk group and 247 (27.47%) in the moderate-high-risk group. The median VTE score by nurses was 3 (1–4), with 506 (57.43%) in the low-risk group and 375 (42.57%) in the moderate-high-risk group. The median ADL score by nurses was 95 (60–100). Table 1 presents the general characteristics of patients in each group.

Table 1 General Characteristics of Patients from Surgical and Non-Surgical Departments

VTE scores were fairly consistent between doctors and nurses in both surgical (Kappa = 0.30, 95% CI: 0.25–0.34) and non-surgical (Kappa = 0.35, 95% CI: 0.31–0.38) departments. Among the non-surgical departments, VTE scores were moderately consistent between doctors and nurses in oncology (Kappa = 0.47, 95% CI: 0.34–0.60) and geriatrics (Kappa = 0.45, 95% CI: 0.31–0.59). Among the surgical departments, there was a higher degree of agreement in VTE scores between doctors and nurses in the traumatology specialty (Kappa = 0.32, 95% CI: 0.31–0.59). Table 2 displays the consistency of VET scores between physicians and nurses in each department.

Table 2 Consistency of VET Scores Between Doctors and Nurses in Each Department

There was moderate agreement in VTE risk assessment between doctors and nurses in surgical departments (Kappa = 0.50, 95% CI: 0.38–0.62), while fair agreement in VTE risk assessment between doctors and nurses in non-surgical departments (Kappa = 0.32, 95% CI: 0.26–0.40). Among the non-surgical departments, there were higher degrees of agreement between physicians and nurses in the oncology department (Kappa = 0.49, 95% CI: 0.28–0.69), VIP clinical department (Kappa = 0.50, 95% CI: 0.24–0.75), and geriatrics department (Kappa = 0.62, 95% CI: 0.36–0.89). Among the surgical departments, there was a higher degree of agreement between doctors and nurses in VTE risk assessment in traumatology (Kappa = 0.50, 95% CI: 0.29–0.71). Table 3 indicates the consistency of VET risk assessment between doctors and nurses in each department.

Table 3 Consistency of VET Risk Assessment Between Doctors and Nurses in Each Department

Table 4 demonstrates the degree of agreement between doctors and nurses in assessing mobility impairment and thrombosis potential in the non-surgical departments. The Padua scale was used for VET risk assessment in the non-surgical departments. The assessment of the mobility impairment component was fairly consistent between doctors and nurses (Kappa = 0.31, 95% CI: 0.25–0.37). A significant association (P < 0.05) was discovered between the mobility scores and ADL scores of nurses in the non-surgical departments. Doctors and nurses had no statistically significant agreement in assessing the thrombosis potential component (P = 0.73).

Table 4 Consistency of Mobility Impairment and Thrombosis Potential Assessments Between Doctors and Nurses in Non-Surgical Departments

Discussion

Regarding VTE scores, doctors and nurses in surgical and non-surgical departments approached fair agreements. The consistency of VTE scores was slightly higher in priority departments than in non-priority departments. It is suggested that there are still significant differences in VTE scores between doctors and nurses in most departments. Qualitative results are more subjective than quantitative results, and subjective factors highly influence several components of the VET risk assessment scale. Individual variances among assessors may lead to a lower inter-rater agreement without harmonized and well-defined criteria.18,19

Therefore, this study suggests that the inconsistency of VTE scores may be due to (1) different awareness of VTE risks and the specific diseases among different healthcare professionals, (2) different standardization of pre-training on VTE risk assessment, and (3) the lack of a uniform VTE risk assessment process. To address these issues, we are also considering a further expanded study to explore the major causes so as to enhance training and guide medical staff to assess and prevent VTE more accurately, thus optimizing the VTE prevention system.

According to the VTE risk level, we could identify whether patients were at high risk and whether they required pharmacological and/or mechanical interventions.20,21 There was a higher degree of agreement on VTE risk assessment between doctors and nurses in the surgical departments than in the non-surgical departments. This result may be related to the fact that different VTE assessment scales with different components were selected for the surgical and non-surgical departments. The Caprini RAM has more components than Padua scale, and the operationalization of each component is clearer. Intuitive, clearly guided assessment tools allow for greater inter-rater consistency.22,23

Further analysis of the Padua scale components in the non-surgical departments exhibited that the different scores between doctors and nurses were primarily in the mobility impairment and thrombosis potential. Physicians and nurses agreed fairly on mobility impairment, but there was no statistically significant agreement on thrombosis potential. Additional correlation analysis indicated a significant association between nurses’ mobility and ADL scores, probably due to the nurses referencing the ADL scores in their mobility scores. Nurses require the ADL score for each patient admitted to the hospital at admission, whereas doctors do not usually require it. This may lead to differences in the scores of doctors and nurses on this component. When elderly patients are admitted to the hospital, geriatricians, together with a multidisciplinary team of nurses, pharmacists and nutritionists, need to undertake a comprehensive geriatric assessment (CGA), of which ADLs are an important part.24 Therefore, geriatricians refer to the CGA results when scoring the mobility impairment component, resulting in the highest consistency of VTE risk assessment between doctors and nurses among all departments. However, according to the operational guidance of the Padua scale, there is no direct relationship between mobility and ADL scores. Therefore, there is still an urgent need for doctors and nurses to be systematically trained in VTE risk assessment.

Existing studies suggest that the risk of VTE fluctuates seasonally.25 This study’s short sampling period made it difficult to characterize VTE risk assessment throughout the year. Additionally, limitations of this study included (1) the inconsistency of raters from different departments, which made exploring intra-rater consistency difficult, and (2) the relatively small sample size, which was insufficient to support the consistency evaluation for each component of VTE risk scales.

Conclusion

This study revealed a poor consistency in VTE risk assessment between doctors and nurses in most departments, with non-surgical departments slightly less consistent than surgical ones. Therefore, standardized training for healthcare professionals on VTE risk assessment and the development of a uniform assessment process should help to improve the consistency of VTE assessment between doctors and nurses, hence building a scientific and effective VTE prevention and treatment system to narrow the gap between clinical practice and evidence-based guidelines.

Abbreviations

VTE, venous thromboembolism; ADL, activities of daily living; DVT, deep venous thrombosis; PE, pulmonary embolism; Caprini RAM, Caprini risk assessment model; 4-Element RAM, 4-Element risk assessment model; CGA, comprehensive geriatric assessment.

Data Sharing Statement

All data generated or analysed during this study are included in this published article.

Ethics Approval and Consent to Participate

The Shanghai East Hospital Ethics committee waived the need for approval and informed consent because this retrospective study was based on anonymous data from electronic health records. All experiments were performed in accordance with relevant guidelines and regulations, and comply with the Declaration of Helsinki.

Acknowledgments

We thank other departments for their cooperation and assistance in the implementation of this study.

Author Contributions

Shasha Geng and Yang Li share first authorship. The main contributions of each author were as follows: Shasha Geng – original analysis, writing-original draft, Yang Li – methodology, software, writing-original draft, Jianli Ge – investigation, data curation, visualization, Xiaoxi Guo – data curation, validation, Yue Liu – conceptualization, project administration, writing-review & editing, and Hua Jiang – funding/resources, supervision, writing-review & editing.All authors made substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; took part in drafting the article or revising it critically for important intellectual content; agreed to submit to the current journal; gave final approval of the version to be published; and agree to be accountable for all aspects of the work.

Funding

This study was supported by Municipal Health Commission of Pudong New Area (PW2019E-4) and Shanghai Municipal Health Commission (202140248).

Disclosure

None of the authors have personal, commercial, political, governmental, academic, or financial conflicts of interest. The funding organization and/or sponsor had no role in the design and conduct of the study; in the collection, analysis, and interpretation of the data; and in the preparation, editing, or censoring of the manuscript.

References

1. Pulmonary Embolism and Pulmonary Vascular Diseases Group of Chinese Thoracic Society. Working committee on pulmonary embolism and pulmonary vascular disease of Chinese association of chest physicians, National Collaborative Group for the prevention and treatment of pulmonary embolism and pulmonary vascular disease. guidelines for the management and prevention of pulmonary thromboembolism. Chin Med J. 2018;98(14):1060–1087.

2. Stockler MR. ASCO updated recommendations for preventing and treating VTE in adults with cancer. Ann Intern Med. 2020;172(2):Jc2–Jc2. doi:10.7326/ACPJ202001210-002

3. Nemeth B, Nelissen R, Arya R, et al. Preventing VTE following total Hip and knee arthroplasty: is prediction the future? J Thromb Haemost. 2021;19(1):41–45. doi:10.1111/jth.15132

4. Lim GB. Low-dose LMWH to prevent pregnancy-related VTE. Nat Rev Cardiol. 2022;20:6.

5. Henke PK, Kahn SR, Pannucci CJ, et al. Call to action to prevent venous thromboembolism in hospitalized patients: a policy statement from the American Heart Association. Circulation. 2020;141(24):E914–E931. doi:10.1161/CIR.0000000000000769

6. Rafizadeh R, Singhal S, Halkidis K, et al. Characterization of venous thromboembolism risk in medical inpatients using different clinical risk assessment models. Can J Hosp Pharm. 2016;69(6):454–459. doi:10.4212/cjhp.v69i6.1608

7. Caprini JA, Arcelus JI, Hasty JH, et al. Clinical-assessment of venous thromboembolic risk in surgical patients. Semin Thromb Hemost. 1991;17:304–312.

8. Caprini JA. Risk assessment as a guide for the prevention of the many faces of venous thromboembolism. Am J Surg. 2010;199(1a):S3–S10. doi:10.1016/j.amjsurg.2009.10.006

9. Pannucci CJ, Barta RJ, Portschy PR, et al. Assessment of postoperative venous thromboembolism risk in plastic surgery patients using the 2005 and 2010 caprini risk score. Plast Reconstr Surg. 2012;130(2):343–353. doi:10.1097/PRS.0b013e3182589e49

10. Barbar S, Noventa F, Rossetto V, et al. A risk assessment model for the identification of hospitalized medical patients at risk for venous thromboembolism: the Padua Prediction Score. J Thromb Haemost. 2010;8(11):2450–2457. doi:10.1111/j.1538-7836.2010.04044.x

11. Kahn SR, Lim W, Dunn AS, et al. Prevention of VTE in nonsurgical patients antithrombotic therapy and prevention of thrombosis, 9th ed: American College of chest physicians evidence-based clinical practice guidelines. Chest. 2012;141(2):E195s–E226s. doi:10.1378/chest.11-2296

12. Konstantinides SV, Torbicki A, Agnelli G, et al. 2014 ESC guidelines on the diagnosis and management of acute pulmonary embolism the task force for the diagnosis and management of acute pulmonary embolism of the European Society of Cardiology (ESC). Eur Heart J. 2014;35(43):3033–3080. doi:10.1093/eurheartj/ehu283

13. Konstantinides SV, Meyer G, Becattini C, et al. 2019 ESC Guidelines for the diagnosis and management of acute pulmonary embolism developed in collaboration with the European Respiratory Society (ERS): the Task Force for the diagnosis and management of acute pulmonary embolism of the European Society of Cardiology (ESC). Eur Respir J. 2019;54(3):543–603.

14. Ortel TL, Neumann I, Ageno W, et al. American Society of Hematology 2020 guidelines for management of venous thromboembolism: treatment of deep vein thrombosis and pulmonary embolism. Blood Adv. 2020;4(19):4693–4738. doi:10.1182/bloodadvances.2020001830

15. Stevens SM, Woller SC, Kreuziger LB, et al. Antithrombotic therapy for VTE disease: second update of the CHEST guideline and expert panel report. Chest. 2021;160(6):e545–e608. doi:10.1016/j.chest.2021.07.055

16. Expert Committee on the Chinese Guidelines for the Prevention and Treatment of Thrombotic Diseases. Chinese guidelines for the prevention and treatment of thrombotic diseases. Chin Med J. 2018;98(36):2861–2888.

17. Kakkos SK, Gohel M, Baekgaard N, et al. European society for vascular surgery (ESVS) 2021 clinical practice guidelines on the management of venous thrombosis. Eur J Vasc Endovasc Surg. 2021;61(1):9–82. doi:10.1016/j.ejvs.2020.09.023

18. Guinon L, Soler A, López RM, et al. Inter-rater reliability assessment for the new-born screening quality assurance. Biochemia Medica. 2022;32(3). doi:10.11613/BM.2022.030901

19. Pourasghar F, Daemi A, Tabrizi JS, et al. Inter-rater reliability of triages performed by the electronic triage system. Bull Emerg Trauma. 2015;3(4):134–137.

20. Kearon C, Akl EA, Comerota AJ, et al. Antithrombotic therapy for VTE disease antithrombotic therapy and prevention of thrombosis, 9th ed: American College of chest physicians evidence- based clinical practice guidelines. Chest. 2012;141(2):E419s-+. doi:10.1378/chest.11-2301

21. Hill J, Treasure T, Acute NCGC. Guidelines Reducing the risk of venous thromboembolism in patients admitted to hospital: summary of NICE guidance. BMJ. 2010;27:340.

22. Mitchell R, Bue O, Nou G, et al. Validation of the interagency integrated triage tool in a resource-limited, urban emergency department in Papua New Guinea: a pilot study. Lancet Reg Health West Pac. 2021;13:100194. doi:10.1016/j.lanwpc.2021.100194

23. Mitchell R, McKup JJ, Banks C, et al. Validity and reliability of the Interagency Integrated Triage Tool in a regional emergency department in Papua New Guinea. Emerg Med Australas. 2022;34(1):99–107. doi:10.1111/1742-6723.13877

24. Parker SG, McCue P, Phelps K, et al. What is Comprehensive Geriatric Assessment (CGA)? An umbrella review. Age Ageing. 2018;47(1):149–155. doi:10.1093/ageing/afx166

25. Styler M, Singhal S, Halkidis K, et al. The impact of winter months on Venous Thromboembolism (VTE) patients: a retrospective analysis of hospital outcomes in the United States. Cureus J Med Sci. 2022;14(9). doi:10.7759/cureus.29091

留言 (0)

沒有登入
gif