The concept of using an open-source dashboard integrated with control charts for managing recruitment and interventions originated from a PI with extensive experience in clinical research studies. The earlier recruitment challenges are identified, the more likely they can be addressed without falling too far behind to recover. This requires monitoring counts, rates, and trends. Counts must be monitored to ensure accrual is meeting targets. Rates can indicate that an assumption is wrong and that action must be taken. For example, a design of a study may assume that the eligibility rate in a population of patients is higher than it is. In this case, it may be necessary to compensate by adding more sites to increase the size of the population or changing the eligibility criteria to increase the rate. Trends can warn a research team if they are likely to fall behind or catch up on goals.

The primary purpose of the dashboard with control charts is to provide the PIs and other stakeholders with weekly updates on the count of participants who have completed each stage or action within the process, the rates of completion, and trends, both for the current week and cumulatively. This enables immediate action if the counts, rates, or trends indicate that accrual goals may not be reached. Finding for clinical trials may be terminated for failure to reach accrual targets [3].

The PI chose the multi-site study of Delirium and Neuropsychological Recovery among Emergency General Surgery Survivors (DANE study) for prototype development and proof-of-concept testing. In the DANE study, participants progress through stages delineated by various actions, as illustrated in Fig. 1. These actions may be initiated by study staff, such as screening or approaching participants, or by the participants themselves, such as consenting or refusing participation. Attrition may occur in relation to an action, for instance, when it is determined that a participant is ineligible. Attrition may also be discovered during an action, such as when a participant withdraws or cannot be contacted when attempting to schedule an assessment. Additionally, attrition may occur between actions, such as when a participant decides to withdraw from the study.

Stages and actions of the DANE study

The development of DANE dashboard involved re-engineering a successful quality monitoring Excel dashboard with control chart functionality originally created by the PI team for a department within a large hospital, as illustrated in Fig. 2. Dashboard data for the hospital system was segmented by time in the Cerner medical records system and then extracted and imported into Excel for use in the control charts and dashboard. The format of the dashboard and control charts from the quality monitoring software required minimal modification for reuse in DANE dashboard. However, the DANE data came from a REDCap database rather than the Cerner medical records system. The REDCap data required segmentation by time and other processing external to REDCap for use in DANE dashboard. Most of the design and development efforts focused on creating specialized REDCap reports necessary for extracting the required data from REDCap, putting it into a format usable by the control charts and dashboard, and adding logic for time segmentation and calculating the number available for an action by week from the start of recruitment, as depicted in Fig. 3. An update macro was designed and developed to empower the research coordinator to export data from REDCap, import it into Excel, and update the dashboard and control charts with a single click, without requiring assistance from an analyst.

Data flow for large hospital quality monitoring Excel dashboard

Data flow DANE dashboard Excel dashboard

This re-engineering process was facilitated through collaborative development, where minimally viable prototypes of the dashboard user interface were evaluated by stakeholders of the DANE study and other clinical investigators. The DANE study involved a diverse team including the two PIs, project coordinators stationed at each site along with their investigative teams, research coordinators, a data team, nurse care coordinators, surgical faculty serving as site investigators, and graduate student assistants assigned to support the DANE study. Prototypes were refined and modified based on this feedback and re-evaluated.

The software underwent a development phase spanning approximately 4 months, followed by an additional 1-month alpha testing period and 3-month beta testing period. Throughout the development phase, the in-house developer held weekly one-on-one meetings with the PI, whose concept inspired the software. Moreover, the developer actively participated in weekly project team meetings to understand the team’s information requirements and challenges. Occasional meetings were also conducted with the research coordinator and the analyst, who generated reports, to clarify data definitions and with the data team regarding database structure for sourcing the software’s data.

The first month of the project was spent clarifying data definitions, evaluating various software platforms, and dealing with access restrictions. Excel was chosen because of its advantages in development speed, and accessibility. The actual software development began in the second month. At that time, REDCap reports were developed and formatted to load into the dashboard software. In the third month of development, the developer, research coordinator, and PI presented the software’s user interfaces to a group of seasoned researchers, including two PIs on other studies, on a weekly basis to gather formative feedback. Additionally, the PI solicited feedback on the user interfaces from colleagues within the medical school and hospital system where they practiced. As the project progressed into its third month, the PI and developer presented the software’s user interfaces to the entire DANE study group at their weekly meetings. Subsequently, they collaborated on implementing the feedback received, ensuring the software’s refinement in response to user input. During the third and fourth months, the developer focused on writing and testing a Visual Basic macro to seamlessly update the dashboard and control charts from the REDCap database.

The developer conducted alpha testing to assess accuracy in the fifth month of the project. This involved comparing the dashboard and control charts against existing weekly reports extracted from the database using the REDCap API by the data management team. During this phase, the developer identified and rectified several update issues as well as detected various data entry errors and inconsistencies within the database.

Beta testing began at the start of the sixth month when the developer trained the research coordinator on using update macro to update the software for weekly reports. Subsequently, the research coordinator assumed responsibility for generating the weekly reports using the dashboard, comparing them to the report generated by the data management team, and using the dashboard in the weekly report to the research team. The research team, consisting of the two PIs, a third investigator who had been the PI on other studies, the research coordinator, site coordinators, and other experienced staff, evaluated the dashboard and control charts on a weekly basis for 3 months. Only the research coordinator had access to the complete DANE REDCap database and Dashboard package to preserve blinding of the study because the DANE dashboard package contains participant data including how the participants are randomized into groups. The development process is summarized in Fig. 4.

The Delirium and Neuropsychological Recovery among Emergency General Surgery Survivors (DANE) study (NCT05373017, 1R01AG076489-01) is a multi-site, two-arm, single-blinded randomized controlled clinical trial to evaluate the efficacy of the Emergency General Surgery (EGS) Delirium Recovery Model to improve the cognitive, physical, and psychological recovery of EGS delirium survivors over 65. At or near the time of discharge, enrolled patients are be randomized to receive the EGS Delirium Recovery Model or usual care. The hypothesis is that, over 18 months, EGS delirium survivors over 65 who are randomized to the EGS Delirium Recovery Model experience more favorable cognitive, physical, and psychological recovery in comparison to those randomized to usual care.

The EGS Delirium Recovery Model involves development of a recovery care plan and a 12-month interaction period with a care coordinator. The recovery care plan is based on a physical, cognitive, and psychological assessment, a social and community needs assessment for the patient and the informal caregiver if they have one, and a reconciliation of all prescribed and over the counter medications. During the 12-month interaction period, the patients and/or their informal caregivers have a 20-min virtual visit, phone contact, email, fax, or mail with the care coordinator at a minimum of every 2 weeks. During these interactions, the care coordinator will answer any questions generated from previous visits, collect patient and informal caregiver’s feedback, review and reconcile medications and discuss adherence, review specialist and therapist appointment and adherence to recovery care plans, monitor sleep by asking questions about the length and nature of sleep to trigger sleep associated interventions, and facilitate the informal caregiver’s access to appropriate community resources.

The DANE study requires 185 study completers per group to yield a power of 80.6% to detect differences in the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) between the intervention and usual care groups with type I error rate at α = 0.05. Assuming a 30% attrition rate, 264 patients per group must be enrolled into the study with the total randomization target size of 528. The DANE study was about halfway through recruitment at the time of submission of this article.

Data collection for the DANE study is the responsibility of the clinical trial staff at the site under the supervision of the site investigator. The investigator is responsible for ensuring the accuracy, completeness, legibility, and timeliness of the data reported. Clinical data is entered into the University of Wisconsin School of Medicine and Public Health (SMPH) Research Electronic Data Capture (REDCap). SMPH REDCap data are backed up nightly to a secure environment maintained by the SMPH Research Informatics team. The data system includes password protection and internal quality checks to identify data that appear inconsistent, incomplete, or inaccurate.

The DANE dashboard improves security by limiting access to the full REDCap database. Now, only the research coordinator has access to the entire database, reducing the number of people involved. The research coordinator can update the DANE dashboard to generate weekly reports for the team. Previously, a programmer or analyst was required to access the database for these weekly updates.