Aims

The primary aim of this study is to determine the accuracy of collecting vascular access interventions required to enable and maintain the use of a vascular access for HD by clinical staff as part of routine clinical practice compared to the reference standard of the documented vascular access procedures collated by a vascular access expert during a 6-month follow-up period.

Secondary aims are to assess the feasibility of this data collection process across different clinical settings and countries and to assess the accuracy of collecting details on all vascular access interventions including types of interventions.

We hypothesize that vascular access function can be measured accurately by clinical staff as part of routine clinical practice without requiring advanced vascular access and research expertise, significant time investment or costly equipment.

Study design and setting

VALID is a prospective, multi-center, multinational validation study that will assess the accuracy and feasibility of measuring vascular access function in patients undergoing maintenance HD by clinical staff as part of routine clinical practice in 10 HD centers across 7 countries (Australia, Canada, France, Malaysia, the Netherlands, Switzerland, and the United Kingdom). The study schema is displayed in Fig. 1. Study sites have been purposefully selected to capture a broad range of different HD settings including geographical location (international sites, rural/remote and urban HD units), unit size (small to large HD units), HD modality (in-center-, satellite, home-based HD) and setting (private versus public hospital).

Fig. 1

Study Schema. Abbreviations: HD – Hemodialysis; VA – Vascular Access. The study design aligns with the STARD 2015 guidelines: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies [29]

The study will be conducted according to International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use (ICH) Good Clinical Practice (GCP) guidelines and will be reported according to the Standards for Reporting of Diagnostic Accuracy (STARD) guidelines [29]. The study will be conducted and coordinated by the Australasian Kidney Trials Network (AKTN), led by an International Trial Steering Committee of vascular access experts including two patient partners, study investigators, statisticians, and other researchers (Fig. 2). The VALID study protocol was registered with the ClinicalTrials.gov (NCT03969225) on 31st May 2019. Ethics Approval has been granted for all participating sites (please refer to the Declaration Section for further details). Written consent will be obtained from study participants by the site investigators in line with the site-specific ethics approval document.

Fig. 2

Study Governance Structure. Abbreviations: AKTN – Australasian Kidney Trials Network; UQ – University of Queensland, Australia

Study participants

All patients receiving maintenance HD in the participating units and who are able and willing to provide informed consent (unless a waiver of consent is approved by the local ethics committee), will be invited to take part in the trial. Patients who are expected to require HD for less than 3 months due to anticipated recovery of kidney function or are considered unsuitable to participate by clinical staff will be excluded. The eligibility criteria are deliberately kept broad to reflect routine clinical practice (i.e. not excluding non-English speaking patients) and avoid selection bias within participating units. Pediatric patients are excluded because vascular access outcomes were not considered a core outcome domain for research in pediatric patients with chronic kidney disease, as established by the SONG-Kids initiative [30].

Test methodIndex test

A clinical staff member (Assessor 1) of the participating HD center without special expertise in vascular access research will collect the number, type and date of any vascular access interventions required to enable and maintain the use of the participants’ vascular accesses for HD. Data will be entered in the Research Electronic Data Capture (REDCap), a secure, web-based data base application designed to support data capture for research studies and hosted by the Queensland Clinical Trials and Biostatistics Centre at the University of Queensland [31]. Assessor 1 will be, in most cases, a HD nurse working in the HD unit. Alternatively, Assessor 1 may also be a HD technician, a medical or nursing student or a junior doctor. Data collection for each patient will occur for 6 months or until the patient’s death, transfer to a different kidney replacement therapy (i.e., peritoneal dialysis, kidney transplantation), treatment withdrawal, transfer to a different unit, or kidney recovery with removal of any HD vascular access. Information on vascular access interventions will be acquired as per local practice. Resources may include (but are not limited to) electronic or paper-based medical records, existing databases, or information provided by staff or patients. Assessor 1 may prospectively enter the details on vascular access interventions into REDCap as they occur or retrospectively at 3 and 6 months of follow-up.

Reference standard

The reference standard represents the vascular access procedure notes (i.e., source document) collected by a vascular access expert of the research team, henceforward referred as Assessor 2. The redacted procedure notes will be uploaded into REDCap. Assessor 2 will collect the data independently from Assessor 1. Moreover, Assessor 2 will not be able to access data on vascular access interventions entered by Assessor 1 and vice versa. Assessor 2 may collect these data prospectively or retrospectively at 3 and 6 months of follow-up. This reference standard reflects the accepted research standard of collecting data on vascular access events (i.e., interventions).

Audit assessor role

To ensure completeness and accuracy of data collection by Assessor 2 as the gold standard, a local audit assessor at each site (typically the site principal investigator) will check completeness and verify vascular access procedure notes of 10% of randomly selected participants. In case of VA intervention missingness or inaccuracy for more than 1 participant, the audit assessor will cross-check data accuracy and completeness for all remaining study participants enrolled at the unit.

Maintenance of independence of assessors

In view of the importance of independence of Assessor 1 (Index test) and Assessor 2 (Reference standard), the database structure has been designed to engage role-based access control for assessors. This ensures that assessors of one category (Assessor 1, Assessor 2 or Audit Assessor) can only access the relevant sections in the database for their role, enabling them to only perform functions specified by the study protocol for that role.

Data collectionParticipant demographic and medical history

Table 1 summarizes the data collection timepoints for Assessor 1 and 2, respectively. Baseline data will be collected by the site investigators, typically Assessor 2, including patient demographics (sex, age, ethnicity), comorbidities (diabetes mellitus, cardiovascular diseases [ischemic heart disease, cerebrovascular accidents, peripheral vascular disease], BMI), HD and vascular access details (HD setting [in center, satellite, home HD, in center nocturnal HD, home nocturnal HD], dialysis duration, usual number of dialysis sessions per week, usual number of hours of dialysis per week, vascular access in use, location of AVF/AVG in use [upper arm, lower arm or leg], self-cannulation and cannulation technique [Rope ladder versus button hole]).

Table 1 Data collection schedule

Information about the roles and expertise of Assessors 1 (index test) and Assessors 2 (reference standard) will also be collected at baseline including role/profession, and years of experience in HD, in clinical research and in vascular access monitoring/maintenance.

Vascular access interventions

Table 2 lists all vascular access interventions considered for defining vascular access function and encompasses any interventions/procedures required to enable and maintain the use of the vascular access for HD during the study period. The selection of vascular access interventions was informed by previous research [22, 27, 28] and feedback from patients, caregivers and health professionals with expertise in vascular access [20, 32]. Interventions are considered relevant if they are performed to enable and maintain the function of the vascular access and are regarded as invasive or disruptive to the patient, i.e., potentially painful and requiring a visit to the operating room, radiology suite or special procedure room or are conducted by the bedside. To facilitate data accuracy verification, Assessor 2 will also collect the indication for each of the vascular access interventions (Supplementary Item S1).

Table 2 Vascular access interventions required to enable or maintain the use of a vascular access for hemodialysisFeasibility assessment

Both data collectors, Assessors 1 and 2, will be prompted to complete a feasibility questionnaire addressing all relevant feasibility aspects, as outlined by Prinsen et al. [33] at the end of study (Supplementary Item S2).

Semi-structured interview

At study end, face-to-face or video-call semi-structured interviews will be performed with all Assessors. The interview question guide is outlined in Supplementary Item 3. All interviews to be conducted in English will be performed by the Chief Principal Investigator AV and expected to take approximately 30 minutes. If required, interviews will be conducted in the assessor’s native language by a bilingual member of the SONG vascular access working group [32]. The interview serves to further explore assessors’ perspectives on the feasibility of measuring vascular access function. The interviews will be audio-recorded and transcribed verbatim.

Early withdrawal from the study

Once a participant has been included in the study, the investigator will make every reasonable effort to keep the participant in the study. Reasons that participants may be withdrawn from the study prior to study end include death, kidney transplantation, transfer to peritoneal dialysis, permanent withdrawal from kidney replacement therapy, transfer to another HD unit which is not an active study site, loss to follow-up, withdrawal of participant consent and at the discretion of the treating physician. In case of early termination of participation, the Site Principal Investigator or co-investigator (assessors 1 or 2) will complete the study end form at time of withdrawal of the participant.

Study outcome measuresPrimary outcome measures

The primary outcome will be the accuracy of reporting vascular access function during a 6-months period, defined by the need for any intervention(s) required to enable and maintain the use of a vascular access for HD. The accuracy of reporting vascular access function is defined by the correct identification of each intervention and the correct reporting of the total number of interventions per patient. An intervention reported by Assessor 1 will be considered correctly identified if its intervention date is within +/− 3 days of the date identified by Assessor 2 (the reference standard).

Secondary outcome measures

Accuracy of the reported type of VA intervention will be analyzed as secondary outcomes. Clinical outcomes include the rate of vascular access interventions (n/patient-year), the time to first vascular access intervention, and the type of vascular access interventions. Feasibility outcome measures include the time required for measuring vascular access function, completeness of data collection, Likert score of feasibility questionnaire with data collectors (Assessors 1 and 2), qualitative analysis of semi-structured interview with data collectors, and study feasibility outcomes including recruitment rate, eligibility ratio and enrolment ratio.

Adverse events

The study has no impact on patients’ safety, or the care provided to participants during the study conduct. No adverse events (serious or not) will be collected for this study other than the outcome measure of interest and outcomes captured as reasons for early termination of participant follow-up due to outcomes such as death or treatment withdrawal.

Statistical considerationsSample size calculation

The sample size has been calculated for the sensitivity, i.e. the proportion of interventions correctly identified by Assessor 1 and specificity, i.e. the proportion of patients with no interventions recorded by Assessor 1 that do not have a record of interventions.

Assuming an event rate of 2 vascular access interventions/ patient-year based on recent data from a study collecting a comparable range of vascular access interventions in HD patients [28], the overall expected number of interventions over a 6 month period will be equal to the number of participants, although there will be patients with no intervention and patients with more than one intervention. Assuming a true sensitivity of 90%, 520 records of interventions will provide an approximate 5% width for the 95% confidence interval.

With the rate of 2 vascular access interventions/ patient-year and assuming a Poisson distribution, the expected number of patients without an intervention in the 6 months is 193. This number will allow the calculation of the specificity with a 10% width for the 95% confidence interval, assuming a true specificity of 90%.

The sample size will be inflated to account for 15% potential dropouts (i.e. early study exit). This corresponds to a final sample size of 612 patients.

Accuracy and validity assessment

The unit of analysis is intervention rather than patient because each patient can have multiple interventions. Validity will be measured by the sensitivity and specificity of the data acquisition process. The sensitivity corresponds to the proportion of correctly identified interventions by Assessor 1, among the interventions identified by Assessor 2 (reference standard). The specificity is computed on the patients with no interventions (as measured by Assessor 2) by using the number of patients with no interventions identified by Assessor 1. The confidence intervals will be calculated for both sensitivity and specificity using robust standard errors, through a logistic regression fitted with a generalized estimating equation (GEE) to take into account clustering by center and patient (due to the limitations of the software, each clustering level will be considered in separate models and both confidence intervals will be reported). Additionally, the percentage of patients with the correct number of VA interventions within 6 months, as identified by Assessor 1, will be presented as a measure of accuracy, together with the Cohen’s weighted kappa as the degree of agreement for the number of VA interventions per patient, reported by the two assessors. The 95% confidence intervals for the accuracy and kappa statistics will be computed by bootstrap stratified by center.

Several subgroup analyses will be conducted to investigate variability of these measures according to patients’ and centers’ characteristics. Any tests comparing measures of validity across subgroups will be performed with a logistic model, using GEE to take into account the clustering of the data by patient and center.

Accuracy of rate of vascular access interventions, time to first intervention (allowing for imprecision of +/− 3 days) and type of vascular access will also be analyzed. For the first two continuous outcomes, the accuracy of the data will be reported as the percentage of correct information collected by Assessor 1. Finally, the accuracy of type of intervention collected by Assessor 1 will be reported as the percentage of type of intervention correctly identified and the Cohen’s kappa will be presented as a measure of agreement between the two assessors. All the measures of accuracy will be presented with 95% confidence intervals.

All data will be analyzed using R software. The script for the statistical analysis is provided as Supplementary Item S4.

Feasibility assessment

Time required for measuring vascular access function will be reported as study participant mean with standard deviations or median with interquartile range (IQR) depending on the distribution. Any subgroup comparison will be performed with linear models fitted with GEE with clustering by center.

The feasibility questionnaire will be analyzed quantitatively by mean, median, and percentage of assessors rating feasibility aspects as feasible (4–5 points on Likert scale) and qualitatively by thematic analysis if sufficient comments are provided.

The transcripts of the semi-structured interviews with Assessors 1 and 2 on feasibility aspects of data collection will be imported into HyperRESEARCH (ResearchWare, USA) and analyzed thematically. Thematic analysis involves identifying, examining, coding, comparing and grouping concepts to develop themes that describe the phenomenon being investigated and addresses the research question [34]. Using thematic analysis, the Chief Principal Investigator will code the transcript line-by-line to identify concepts relevant to the assessors’ perspectives on the feasibility of measuring vascular access function as part of routine clinical practice and for research purposes. The preliminary codes will then be reviewed by co-investigators who will independently read the transcripts and discuss any feedback with the chief investigator. This form of investigator triangulation can enhance the analytical framework and ensure that the full range and depth of data are captured in the initial analysis.

Data management and quality assuranceData handling and record retention

The VALID study data will be captured and stored electronically via REDCap. Original consent forms will be stored locally. After closure of the trial, investigators will maintain all study documentation, including consent documents, ethics committee approvals and correspondence, for a minimum of 15 years, or as per local guidelines.

Data sharing

Data sets will be made available by the Central Coordinating Group to researchers within the VALID study for analysis of sub-studies and country specific outcomes after the primary manuscript has been accepted for publication.

For researchers outside the VALID study, individual participant data will be made available upon request to a Data Access Committee, a review board set up to assess proposals based on sound science, benefit-risk balancing and research team expertise. Appropriate data will be made available to approved proposals. This process will be in effect for a period of 2 to 5 years following publication of the main study results. After 5 years, the data will be available in the Sponsor’s data warehouse but without investigator support other than deposited metadata.

Training

All data collectors will receive an induction to using REDCap. An operational user manual for REDCap will also be provided to the participating units. Site Principal Investigators and their co-investigators will also have to meet the following criteria: Adequate time to conduct the study, adequate training and experience to conduct the study, ability to recruit enough participants to conduct the study, and provide evidence of proficiency in the tenets of Good Clinical Practice.

Central monitoring

Utilizing a risk-based monitoring approach, a detailed monitoring plan will outline trial monitoring activities. Monitoring efficiency will be optimized by a system of remote monitoring performed by the Central Coordinating Group of the AKTN. If indicated, and with advance notice, study sites may be visited by a Clinical Monitor. The visits will be an opportunity to provide additional support and training to site staff, ensure the study is conducted according to the protocol, and in line with local regulatory requirements. Source documents from which the data are obtained will be made available by the site for review.

Data quality assessment

The adjudication committee (Audit assessors) will inspect the source data against recorded data collected by Assessor 2 for completeness and accuracy as described under the Audit assessor role.

Ethical consideration and disseminationModification of the protocol

Any modifications to the protocol which may impact on the conduct of the study including changes of study objectives, study design, study population, sample size, study procedures, or significant administrative aspects may require a formal amendment to the protocol. Such amendment will be agreed upon by the global Trial Steering Committee, and approved by the Independent Ethics Committee prior to implementation and notified to the health authorities in accordance with local regulations.

Early termination of study

The study will continue to its planned end unless it is no longer practicable to complete the study, either overall or at an individual site. If such action is taken, the reasons for terminating the trial will be documented in detail. All trial subjects still under follow-up at the time of termination will undergo a final assessment. The Chief Principal Investigator and the Central Coordinating Group must be informed without delay if any investigator has any concerns about continuation of the trial.

Protection of participant confidentiality

Participants’ records and the data generated by the study will be confidential. Any information that may identify a participant will be excluded from data presented in the public arena. Data will be stored in a secure, lockable location, and access to electronic data will be protected through a password protected web interface. The data extracted will be de-identified and a unique study number used. Similarly, data collected on the electronic case report form will be de-identified and a unique subject number will be used.

Dissemination of study outcomes

Study results will be disseminated via publications in peer-reviewed journals, presentations at national and international scientific meetings, and social media as well as State Renal Networks, the SONG initiative database and the AKTN website.