INTRODUCTION

Kidney failure is defined as the permanent loss of kidney function wherein kidney replacement therapy (KRT) is required to sustain life. An estimated 2.6 million people receive KRT worldwide and this is projected to double to 5.4 million by 2030 (Liyanage et al., 2015). Use of KRT through peritoneal dialysis, kidney transplantation or haemodialysis (HD) is a costly but lifesaving treatment. In Australia, HD is the treatment modality of choice for 78% (n = 9557 patients) of prevalent dialysis patients (ANZDATA Registry, 2019). Patients on maintenance HD require well-functioning vascular access (VA) to achieve effective therapy. Maintaining the patency of the VA is an important patient-centred outcome established by the international Standardized Outcomes in Nephrology initiative (Viecelli et al., 2018). Repeated missed cannulation may result in serious complications such as haematoma, infection, and aneurysm formation, which can lead to need for access revision, central venous line placement, or loss of access (Al-Jaishi et al., 2017; Harwood et al., 2017; Lee et al., 2006; McCann et al., 2009; Polkinghorne et al., 2013; Schinstock et al., 2011; Vachharajani, 2014; Van Loon et al., 2009; Wilson et al., 2010). Additionally, further cannulation attempts are painful for the patient (Wilson & Harwood, 2017). It is therefore important to develop a HD instrument that can measure VA cannulation complexity, so that cannulation of the VA can successfully occur on the first attempt, resulting in less risk of complications for the patient.

LITERATURE REVIEW

The Western Australia Haemodialysis Vascular Access Classification (WAHVAC) instrument was developed by a subgroup of the Western Australian (WA) Unit Leaders' Group, comprising of seven HD nurse experts representing the 22 rural, remote and metropolitan WA dialysis centres. The instrument was introduced as part of routine clinical care in all WA HD units from 2011 (J. Hosking, personal communication, September 2019). This instrument aims to classify the cannulation complexity of the arteriovenous fistula (AVF) or arteriovenous graft (AVG) as simple, challenging, or complex. Depending on the classification of complexity, a suitably skilled, competency-assessed nurse can be matched to perform the cannulation to minimise the risk of missed cannulation and trauma. At our HD centres, we have a rigorous program of competency assessment for access cannulation. This learning framework involves the staff member using Self-Directed Learning Packages (SLDP) together with Formative and Summative Assessments supervised by expert team members to demonstrate the advancement of their skills and associated knowledge of VA management. Nurses work their way through 3 levels (SDLP's/Assessment) of competency: simple, challenging, complex. Not all nurses will progress to Complex as not all nurses are able to develop a high level of competency.

Our previous study showed that cannulating a fistula compared with a graft, absence of a stent and matured access were associated with successful cannulation (Coventry et al., 2019). The WAHVAC includes these variables in the instrument as well as other variables that impact cannulation complexity.

Instruments that assess access complexity for the related but different scenario of peripheral intravenous catheter insertion are common (Civetta et al., 2019; Hirani et al., 2019; Pagnutti et al., 2016; Van Loon et al., 2019). There is currently no “gold standard” for grading the complexity of HD VA and no published instrument to assess HD VA complexity. Proxy measures for the complexity of HD VA may include the number of cannulation attempts, the need to ask a more experienced nurse to perform the cannulation, inability to complete the prescribed dialysis session, or use of a central venous catheter (CVC) (Coventry et al., 2019). Although the WAHVAC was developed by experts in HD nursing, the instrument had not undergone formal validity or reliability testing. For this study, a panel of VA experts were invited to participate and assess content validity testing of the instrument, and reliability testing was also conducted. Therefore, the aim of this study was to evaluate the WAHVAC instrument for content validity and interrater and test–retest reliability.

MATERIAL AND METHODS Design

A cohort study design was used to assess the reliability of the instrument.

Participants Content validity

Content validity was conducted by an international panel of experts in HD nursing (n = 8). Experts were identified by the authors through publications in the area of HD VA, the Australian New Zealand Vascular Access Nurse network, personal communication and known expertise in cannulation. An email was sent inviting the experts to participate.

Reliability

Reliability assessments were conducted using data collected from one in-centre and one satellite HD unit in Perth, Western Australia from September to November 2019. The participants were HD patients (n = 67) with a VA (AVF or AVG) and HD nurses (n = 38) who were responsible for cannulating the access. To assess interrater reliability, two nurses completed the WAHVAC instrument independently for the same patient at the same time. To assess test-retest reliability, the same nurse completed a second WAHVAC instrument for the same patient at a second time point approximately 2 weeks later.

WAHVAC

The WAHVAC (see Figure 1) has 20 items that assess access history, access assessment and relevant patient clinical history. The total instrument can be scored from 0 to 209. Each of the variable scores is added together for a total score. Total scores less than or equal to 12 are classified as simple access, scores between 13 and 20 are classified as challenging, and, scores ≥ 21 are classified as complex. The value of the score is a set value, for example, if the access was surgically created less than 3 months ago, the score is 21 and the access is then classified as a complex cannulation. We recommend the complexity of the VA be assessed monthly and reassessed after a significant change, for example, after widespread infiltration or radiological or surgical intervention.

Western Australian Haemodialysis Vascular Access Classification tool

Data collection Content validity

An international panel of experts conducted content validity assessment of the WAHVAC. Descriptive characteristics of the content validity experts (demographic data; work history, education, and HD training experience) along with the content validity scores were assessed using an electronic survey via email. In addition, the expert panel were asked to comment on the (a) clarity and wording of items, (b) comprehensiveness of the items in reflecting VA complexity, (c) any items to omit, (d) areas for possible improvements or modifications, and, (e) if they agreed with the complexity score allocated to each variable on the WAHVAC. The WAHVAC was sent to the experts on two separate occasions to review the content validity of the items on the instrument.

Reliability

A research nurse collected patient and nurse demographic data at study entry.

Patient characteristics

The patient data included demographics, medications, and access history collected via a written survey.

Nurse characteristics

The nurse data included demographics, work history, education, and HD training experience, collected via a written survey.

Episodes of cannulation

The WAHVAC was completed by the HD nurse who used nurse clinical judgement to identify if the variables were present or not present on the WAHVAC. Data on episodes of cannulation included: nurse confidence (before cannulation) with successful cannulation on a scale of 0 (no confidence) to 10 (complete confidence); if swelling, bruising or haematoma were present at the VA site; if a tourniquet or ultrasound was used; type of cannulation (area, rope ladder or both), the distance between arterial and venous needles; and if an arterial needle was inserted antegrade. Other outcome measures included if an existing CVC was used; if the allocated nurse did cannulate; if another nurse assisted with the cannulation; if dialysis was disrupted or unable to be completed due to cannulation issues; final online clearance (Kt/V); and the number of cannulation attempts. Cannulation episode success was defined as insertion of two needles (arterial and venous) without extra attempts.

The research nurse coordinated the HD nurses to enable two nurses to independently complete the WAHVAC for interrater reliability assessment. The research nurse also coordinated the same HD nurse to complete the WAHVAC for test–retest reliability assessment after a 2-week period.

Ethical considerations

Human research ethics approval was obtained from the study sites and the project team's university (Sir Charles Gairdner Osborne Park Health Care Group, HREC No: 2015-049; Joondalup Health Campus, HREC No: 1513; and Edith Cowan University, HREC No: 13153). The research nurse explained the study, provided an information sheet and obtained written consent from both patients and nurses to participate in the study.

Statistical analyses

Summary statistics, including means and standard deviations (SD) or medians and interquartile ranges, were provided for all continuous variables, and frequencies and percentages for all categorical variables. All descriptive statistical analyses were undertaken using SPSS version 27 (IBM Corp. Released 2015, IBM SPSS Statistics for Windows, Version 26.0; IBM Corp). κ analyses were performed using Microsoft Excel (version 1908; Microsoft). The Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) study design checklist recommends for studies assessing κ, sensitivity and specificity, the sample size should be >100 (Mokkink et al., 2019). For reliability assessment, 100 pairs are recommended (Mokkink et al., 2019).

Content validity

Expert reviewer characteristics were analysed using descriptive characteristics. An a priori acceptable level of interrater agreement for relevancy was set at 0.70 and higher (Mojahedi et al., 2014). The content validity index (CVI) was computed to derive the CVI for each item (I-CVI) in the scale. The I-CVI was calculated as the proportion of experts rating either a 3 or 4 (not relevant = 1, somewhat relevant = 2, quite relevant = 3, and very relevant = 4), divided by the total number of experts who rated the item. The I-CVIs between 0.7 and 1.0 inclusive were retained, the I-CVIs between 0.5 and 0.7 were further revised or clarified, and the I-CVIs <0.5 were discarded. Derivation of the overall instrument was expressed as the number of items rated three or four by at least 80% of the experts. Scale-level content validity (S-CVI), the proportion of items given a rating of three or four by all the raters, was derived from averaging across all I-CVIs. Also reported was the number of variables where there was total expert agreement and S-CVI universal agreement (S-CVI UA), and this was calculated by dividing the number of variables with total expert agreement by the number of variables in the instrument.

Reliability

To assess the interrater and test–retest reliability of each of the individual variables (present, not present) and the classifications of simple, challenging and complex on the WAHVAC we conducted observed and expected agreements, κ statistics and their 95% confidence intervals (CI), the bias index, prevalence index and the adjusted κ value. See Supporting Information 1 for formulae calculations. Multiple studies (Byrt et al., 1993; Viera & Garrett, 2005) have reported that the κ statistic is not always satisfactory for assessing agreement, and recommend that bias and prevalence be taken into account when the magnitude of one or both of their indices is close to one. Bias, prevalence indices, and the prevalence-adjusted bias-adjusted κ (referred to as adjusted κ) were calculated (Byrt et al., 1993). For interpretation purposes, a κ (and adjusted κ) scale was used (poor, <0.40; fair to good, 0.40–0.75; excellent, >0.75) (Fleiss, 1981).

The reliability of the total score of the WAHVAC (range: 0–209) was assessed using intraclass correlation coefficients (ICCs), Bland–Altman plots, and linear regression was used to investigate evidence of proportional bias. To calculate ICC, a two-way random effect model was calculated. For the regression analysis to assess proportional bias, the variables assessed were the difference in score total between the two nurses, and the mean total score for the two nurses. A scatter plot was used to investigate the relationship between the total scores of the interrater reliability as well as the test–retest reliability, and Pearson's product moment correlation coefficient was reported. Preliminary analyses were performed to ensure no violation of the assumptions of normality, linearity and homoscedasticity.

For interpretation purposes, an ICC scale was used (poor, <0; slight, 0.01–0.20; fair, 0.21–0.40; moderate, 0.41–0.60; substantial, 0.61–0.80; almost perfect, 0.81–1.00) (Landis & Koch, 1977). For interpretation purposes, a Pearson r scale was used (poor, 0.00–0.29; fair, 0.30–0.59; moderately strong, 0.60–0.79; very strong, 0.80–1.00) (Chan, 2003).

RESULTS Content validity WAHVAC

Demographic data of the HD nurse experts (n = 8) are presented in Table 1. On the first occasion, the WAHVAC instrument was sent to the experts, the original 35-item instrument was reduced to a 24-item instrument. On the second expert review, the instrument was further reduced to 20 items. The wording of the items was clarified and scores for individual variables were adapted on the advice of the experts. The final instrument reports I-CVI > 0.75 for 19 items with one item at 0.62. The S-CVI was 0.89 and S-CVI UA 0.40 (see Supporting information Appendix 1 and Supporting Information Material 2).

Table 1. Demographic characteristics of content validation experts (N = 8) Mean (SD) Range Age (years) 51.7 (7.9) 42–63 Years as registered nurse 31.8 (7.0) 23–40 Years as haemodialysis nurse 23.9 (7.2) 15–35 n % Employment status Full time 7 87.5 Part time 1 12.5 Sex Female 7 87.5 Male 1 12.5 Highest level of education Postgraduate 3 37.5 Master 4 50.0 PhD 1 12.5 Job title Clinical nurse consultant 2 25.0 Nurse unit manager 2 25.0 Director 1 12.5 Academic 3 37.5 Country of residence Australia Western Australia 2 25.0 Queensland 2 25.0 New South Wales 1 12.5 Victoria 1 12.5 United States of America 2 25.0 Reliability of the WAHVAC Patient and nurse characteristics

The demographic data of the patient and nurse characteristics are presented in Table 2.

Table 2. Patient (n=67) and nurse (n=38) characteristics for Reliability assessment Patient variables Mean (SD) Range Age (years) 67.5 (13.7) 25.9 – 92.9 BMI (kg/m2) 26.6 (5.6) 17.9 – 39.8 Age of access (years) 4.6 (5.3) 0.1 – 32.7 Median IQR 3.0 1.7 – 5.3 n (%) Sex Female 21 (31.3) Male 46 (68.7) Medications Steroids 5 (7.5) Immunosuppressant 2 (3.0) Anticoagulantn 20 (29.9) PAI 24 (36.4) Nurse characteristics Mean (SD) Range Age (years) 47.4 (8.7) 24 – 61 Years as registered nurse 18.8 (9.7) 1 – 40 Years as haemodialysis nurse 14.7 (7.4) 1 – 30 n (%) Sex Female 33 (86.8) Male 5 (13.2) Employment status Full time 13 (34.2) Part time/casual 25 (65.8) Highest level of education RN hospital certificate 3 (7.9) RN diploma 6 (15.8) BScN/BN 11 (28.9) RN post-basic certificate 1 (2.6) Graduate certificate 9 (23.7) Graduate diploma 5 (13.2) Master's degree 3 (7.9) Job title RN 28 (73.7) CN/SDN 10 (26.3) Post-graduate in renal nursing Yes 21 (55.3) No 17 (44.7) Abbreviations: BMI, body mass index; BN, Bachelor of Nursing; BScN, Bachelor of Science in Nursing; CN, clinical nurse; IQR, interquartile range; PAI, platelet aggregation inhibitor; RN, registered nurse; SDN, staff development nurse. Episodes of cannulation

During the 12-week study period, there were 247 episodes of VA cannulation in 67 HD patients, performed by 38 dialysis nurses. The average number of episodes of cannulation per patient was 3.7 (SD = 2.3). Successful cannulation at first attempt occurred in most cannulation episodes (n = 236, 95.5%) with the miscannulation rate small (n = 11, 4.4%). The 11 miscannulations occurred in nine patients; therefore 13.4% of patients had a least one event of miscannulation. The complexity categories were mostly evenly distributed among simple (n = 87, 35.2%), challenging (n = 68, 27.5%) and complex (n = 92, 37.2%) categories. Other characteristics of cannulation are presented in Table 3.

Table 3. Characteristics of cannulation (n = 247) and variables in the WAHVAC instrument n Mean (SD) Range Characteristics of cannulation Confident successful first attempt 242 9.2 (1.3) 5–10 Distance between arterial and venous site (cm) 234 6.3 (3.4) 2–40 Final Online clearance Kt V 246 1.33 (0.17) 0.78–1.85 N n % Complexity categories Simple 247 87 35.2 Challenging 247 68 27.5 Complex 247 92 37.2 Swelling present at cannulation sites 242 5 2.1 Bruising present at cannulation sites 240 26 10.8 Hematoma present at cannulation sites 242 5 2.1 Tourniquet used 245 211 86.1 Ultrasound used 243 13 5.3 Cannulation Area 231 37 16.0 Rope ladder 119 51.5 Both 75 32.5 Arterial needle antegrade insertion 244 186 76.2 Did the allocated nurse cannulate? 247 231 93.5 Did another nurse assist with cannulation? 247 13 5.3 Number of cannulation attempts (artery) 1 247 239 96.8 2 7 2.8 3 1 0.4 Number of cannulation attempts (vein) 1 247 240 97.2 2 6 2.4 3 1 0.4 An existing CVC used 247 1 0.4