We adapted time-motion methods previously used in various healthcare settings to track time allocation among staff and clinicians.2,15 Measures of time allocation were mapped within the WorkStudy+ application (Quetech Ltd.), a software for collecting time study data that has been used in other clinical time-motion evaluations.15 Details of our data validation for our virtual observation method are described in the Supplement (Supplement Table 1, Figs. 1 and 2).

Due to the COVID-19 pandemic, we developed a protocol for virtual observations using Microsoft Teams screen share. Participating clinicians were scheduled for half-day, 4-h clinic session observations. During the session, an ad hoc video meeting was created, and the clinician shared their screen and audio with the trained observer.

A total of 23 PCPs from the three regional VA sites were recruited to participate in the observations from January 2020 through September 2021. Three VA sites from one geographic region (called a Veteran Integrated Service Network (VISN)) participated in the study. The VA Puget Sound Health Care System is a large multicenter healthcare system with two main center sites and eight community clinic locations. The Boise VA Medical Center has one main site and five community clinic locations, and the Portland VA Medical Center has two main campuses and 10 community clinic locations. Exclusions include PCPs from contract clinics, clinicians with less than 25% clinical time, and trainees.

PCPs practice as part of a patient-centered medical home model called the Patient Aligned Care Team (PACT). Each PACT consists of a PCP (which can be a physician, physician assistant, or nurse practitioner), registered nurse, health technician or licensed practicing nurse, and an administrative assistant. One full-time, fully staffed PACT manages a panel of 1200 Veterans. PCPs typically conduct visits in clinic half-day blocks of approximately 4 h of back-to-back scheduled clinic visits. Before the COVID-19 pandemic, in-person routine visits were 30 min, in-person new patient visits were 60 min, and phone visits were 15 min. At the time of this study, the VA was implementing video visits, which were allotted 30 min and could be done in place of an in-person visit. Due to the pandemic, there was increased flexibility and a push to hold more visits via video and phone. Given that many traditionally in-person visits were being done via phone calls, phone visits were also allotted 30 min to accommodate the expanded care needs. All encounters during the participating PCP’s observation window were included unless the Veteran or clinician opted out.

Prior to the scheduled observation, clinicians were anonymously surveyed to capture basic demographics including gender, profession, FTE, years in practice, EHR experience, and self-reported EHR time spent outside of clinic hours (Supplemental Table 2). This survey asked clinician to report their EHR time before/after clinic during the workday, after hours on evenings, and after hours on weekends as separate data points. All sections of the survey were optional for the clinician to complete.

One observer captured data on an electronic tablet via the WorkStudy+ application (Quetech Ltd.). This software allows for custom task definitions and recording. Descriptions of the contextual models used for time mapping can be found in Supplemental Table 3. Commonly performed tasks within the EHR (i.e., documentation, chart review, order entry, etc.) were coded and mapped to the software interface (Supplemental Fig. 3). Codes were also created for other computer work or interruptions. Any typing, mouse movement, or clicking within the EHR was captured as EHR work. Therefore, all multitasking during a visit, e.g., speaking with or listening to the patient while typing or clicking within the chart or another application, was captured as the time spent in the EHR or other application. If there was a pause for longer than 3 s in screen activity including mouse movement, typing, or scrolling, the observer would record the end of the last task and start a new task of clinical care. For each instance in which the observer records the start of a new task in the software, a timestamp for the start of the new task and the end of the prior task is captured, allowing for cumulative lengths of time to be recorded for individual tasks. Microsoft Teams screen share meeting function was used to virtually observe the interaction with the EHR and listen to the audio from the clinic session. Except for portions of the visit involving a physical examination, the clinician left their camera on for the clinic session, allowing the observer to see clinician gestures and eye movements, and to see when they might leave the room during a clinic session. For all observations in this study, clinicians were using a single clinic room continuously for patient care, and time spent outside of the room was captured as “Transit” in the software.

The primary outcome of interest was the percentage of time spent on EHR work within a primary care visit defined as the total number of minutes the clinician spent on the EHR divided by the total observation time within a visit. The time spent within a primary care visit was defined as the time spent between the first and last communication of the patient or clinician within the visit. Secondary outcomes included visit-level percentage of time spent on specific EHR tasks: documentation, records review, ordering, reminders, and data entry within a visit. Secondary outcomes also included between-visit work (i.e., work done in the EHR during the 4-h clinician observation, but before or after the patient visit), defined as the percentage of time spent on tasks between patient encounters and within the clinician’s 4-h observed session. These tasks included documentation, records review, ordering, reminders, alerts, data problems, and coding/billing. We also present descriptive statistics of weekly self-reported hours of EHR work done after hours (evenings and weekends) and outside of clinic sessions during the workday.

For primary and secondary visit-level outcomes, we reported observed mean number of minutes across visits for each of the three visit modalities. We calculated average percent of EHR use using a mixed effects model adjusting for visit mode (phone, in-person, video) and a random effect for clinician to account for multiple visits with the same clinician. For between-visit secondary outcomes, we reported observed mean number of minutes across observation sessions. We calculated the average percent of time on EHR tasks between patient visits using a mixed model adjusting for a random effect for clinician to account for multiple sessions observed with the same clinician. We also calculated descriptive statistics of time on EHR by clinician.

This evaluation was conducted as part of ongoing operational quality improvement project and was designated as a non-research activity by the VHA Office of Primary Care. A memorandum of understanding was signed with the regional union for participating sites.