Refining Vendor-Defined Measures to Accurately Quantify EHR Workload Outside Time Scheduled With Patients [Theory]

Abstract

Accurately quantifying clinician time spent on electronic health record (EHR) activities outside the time scheduled with patients is critical for understanding occupational stress associated with ambulatory clinic environments. We make 3 recommendations regarding EHR workload measures that are intended to capture time working in the EHR outside time scheduled with patients, formally defined as work outside of work (WOW): (1) separate all time working in the EHR outside of time scheduled with patients from time working in the EHR during time scheduled with patients, (2) do not exclude any time before or after scheduled time with patients, and (3) encourage the EHR vendor and research communities to develop and standardize validated, vendor-agnostic methods for measuring active EHR use. Attributing all EHR work outside time scheduled with patients to WOW, regardless of when it occurs, will produce an objective and standardized measure better suited for use in efforts to reduce burnout, set policy, and facilitate research.

Key words:INTRODUCTION

The occupational stress clinicians experience from electronic health record (EHR) workload in ambulatory settings is increasing, partly because of the volume and complexity of work demands that occur outside time scheduled with patients. This asynchronous work has increased with immediate release of test results to patient portals,1 the substantial increase in portal messages, and more virtual care.2-4 Several studies have associated so-called after-hours EHR use with higher burnout rates,5-7 making it imperative to understand how EHR workload is quantified and to accurately identify when that work occurs.

There have been myriad attempts using EHR metadata to quantify EHR workload outside time scheduled with patients.7-10 This work, often generically referred to as after-hours EHR work and formally defined by Sinsky and colleagues11 as work outside of work (WOW), is one of several core EHR use measures that reflect multiple dimensions of practice efficiency. The methods used to calculate WOW can be varied and complex, however, requiring substantial analytic expertise, which results in high costs of analysis and difficulty in reproducing and directly comparing results. Many health care organizations therefore rely on vendor-defined measures that capture what clinical tasks are being completed and when that work is being performed.12,13 These measures are automatically computed from EHR metadata based on programmed tracking of clinician EHR activity and are presented in analytic reporting platforms available on demand from some vendors. Theoretically, access to vendor-defined measures and reports could greatly reduce barriers to measuring EHR use; however, the methods used to calculate them can be opaque, may change without warning, and embody assumptions about clinical workflows that make them less illuminative of important and meaningful differences between clinician work patterns.14-17

Here, we make a set of recommendations regarding EHR workload measures that are intended to capture time working in the EHR outside time scheduled with patients (WOW). We also describe how to practically apply these recommendations, focusing on measures available in Epic (Epic Systems Corp) as this vendor is the only one we are aware of that currently provides measures of WOW based on clinicians’ schedules.

CURRENT MEASURES OF OUTSIDE-HOURS EHR WORKLOAD (WOW)

Among EHR vendors, Epic and Cerner (Cerner Corp) hold the majority of the US hospital market share at 32.9% and 24.4%, respectively, while MEDITECH (Medical Information Technology, Inc), CPSI (Computer Programs and Systems, Inc), Veradigm (formerly known as Allscripts; Veradigm LLC), MEDHOST (Medhost, Inc), and other vendors hold the remaining 42.7%.18 Some vendors, notably Epic and Cerner, provide measures that attempt to quantify WOW.12 For example, Epic’s Signal tool measures include Time Outside Scheduled Hours (TOSH) on days with scheduled patient appointments, Time on Unscheduled Days (TUSD), Pajama Time (weekends and weekdays outside 7:00 am to 5:30 pm), and Time Outside of 7:00 am to 7:00 pm on days with scheduled appointments. The first 3 of these measures are reported in Signal only when at least 5 scheduled appointments occur per week. Cerner’s Lights On reporting platform provides a similar measure to Epic’s Pajama Time called Time After Hours (weekends and weekdays outside 6:00 am to 6:00 pm).10

These measures vary in whether and how they reference a clinician’s schedule to determine time spent in the EHR outside time scheduled with patients (ie, TOSH, TUSD) or primarily use clock time without reference to time scheduled with patients (ie, Pajama Time, Time Outside 7:00 am to 7:00 pm, Time After Hours). Even measures that reference time scheduled with patients do not strictly adhere to the patient care schedule, however, as Epic’s TOSH excludes 30 minutes before the first and after the last scheduled appointments in a day as well as any active EHR time during interruptions between scheduled appointments (eg, administrative time, meetings, or teaching).

EHR vendors’ measures also differ in their methods for determining active EHR use regardless of when that work is completed. Epic uses a 5-second inactivity period, whereas Cerner uses a proprietary method based on 45-second periods of inactivity and a combination of clicks, keystrokes, and mouse movements. Both methods likely undercount activities such as note reading, and neither method has been publicly validated. Regardless of the measure used in an attempt to quantify WOW, examining how vendor-defined measures are calculated reveals they undercount EHR workload outside time scheduled with patients when compared with the functional meaning of the WOW8 measure (time spent in the EHR outside of time scheduled with patients, per 8 hours scheduled with patients).11

RECOMMENDATIONS

We offer the following 3 recommendations to better capture time working in the EHR outside time scheduled with patients.

Clearly Separate EHR Time With Respect to Time Scheduled With Patients

To better align with diverse and evolving clinician schedules, we recommend referencing the scheduled appointment time on clinician schedules to allocate all EHR time to 1 of 2 buckets: EHR time during time scheduled with patients and EHR time outside time scheduled with patients (WOW). Although many clinicians work in half-day clinic sessions that are approximately 4 hours in duration, it is not uncommon to have schedules of less than 4 hours in duration. Additionally, it is not uncommon to have scheduled interruptions from continuous patient care for inbox work, paperwork, meetings, teaching, and more. This work between time scheduled with patients may or may not be associated with patient care, and the activities may or may not be associated with work in the EHR. By clearly accounting for and separating WOW from time spent in the EHR during time scheduled with patients, WOW8 benchmarks can be set, creating a framework for health system leaders to make policy decisions related to total clinician workload and overall work hours expectations.

Here we describe how to practically apply this recommendation using Epic’s measures as Cerner does not provide schedule-based EHR workload measures, and we are not aware of other vendors providing measures of EHR use outside scheduled clinical hours. For Epic users, WOW8 is calculated as the sum of Epic’s TOSH and TUSD per 8 hours scheduled with patients, as shown in Figure 1.

Figure 1.Figure 1.Figure 1.

Formula for calculating WOW8.

EHR = electronic health record; WOW = work outside of work; WOW8 = time spent in the EHR outside of time scheduled with patients, per 8 hours scheduled with patients.

Epic’s TOSH excludes EHR work completed between scheduled appointments if patient care is not continuous, however, and as a result, WOW is undercounted. For example, consider a physician who sees patients from 8:00 am to 10:00 am, has an hour-long meeting from 10:00 am to 11:00 am, and resumes patient care for the remainder of the morning from 11:00 am to 12:00 pm. Any time the physician was working in the EHR multitasking during the 1-hour meeting (a common physician practice during virtual meetings) would not be counted in Epic’s current definition of TOSH.

To Calculate WOW, Do Not Exclude Any Time Before or After Scheduled Time With Patients

In addition to excluding all EHR work completed between appointment schedule interruptions, Epic’s TOSH excludes 30 minutes immediately before the first scheduled appointment and 30 minutes immediately after the last scheduled appointment of the day. These 2 factors combined result in distortion and undercounting of EHR workload outside time scheduled with patients. This issue is most pronounced for clinicians with half-day clinic sessions.

Consider 2 physicians who have the same 1.0 full-time equivalent (FTE) breakdown with direct patient care at 0.7 FTE and the remaining 0.3 FTE split among administrative, teaching, and research responsibilities. Figure 2 shows time scheduled with patients as blue blocks, each 4 hours in duration. In this example, 6 patient care sessions, each lasting 4 hours, are required for 0.7 clinical FTE when the health system requires 34 hours of patient contact time per 1.0 clinical FTE (0.7 FTE x 34 hours patient contact time ≈ 24 hours). Physician 1’s schedule includes 10 thirty-minute units in yellow. These 300 minutes (5 hours) are excluded from Epic’s current TOSH definition. In Physician 2’s schedule, patient contact time occurs on 3 weekdays compared with the 5 weekdays in Physician 1’s schedule, and there are 6 thirty-minute units in yellow. These 180 minutes (3 hours) are not currently included in Epic’s TOSH definition. Assuming these 2 physicians log the same total EHR time over the course of the week, one would conclude there has to be some reason (such as Time in Notes or Time in InBasket) for the 2-hour difference in TOSH. These 2 physicians could be phenotypically identical in how they work in the EHR and yet have very different WOW values with Epic’s current TOSH definition. It would be inappropriate to single out the physician who appears to have 2 hours more of TOSH for “enhancement” in work performance when, in reality, this physician could work identically in the EHR—the only difference being when time with patients is scheduled over the course of the week.

Figure 2.Figure 2.Figure 2.

An example of physician schedules having the same work hours (0.7 clinical FTE) but different WOW values using Epic’s current TOSH definition.

FTE = full-time equivalent; TOSH = time outside scheduled hours; WOW = work outside of work.

Regardless of the EHR vendor, we recommend counting all time outside of patient-scheduled hours, including the 30 minutes immediately before and after clinic, as time outside scheduled hours. This approach would more clearly articulate the time clinicians spend in the EHR outside time scheduled with patients and allow more accurate comparisons of WOW across EHR platforms regardless of how clinic time is scheduled. This recommendation, in combination with our first recommendation, results in any time before, between, or after time scheduled with patients being included in a revised version of Epic’s TOSH. It is particularly relevant to academic faculty physicians who have a variety of nonclinical roles and may have partial or fragmented clinic sessions and, more broadly, to any clinicians without whole-day clinic sessions. In addition, this proposed refinement to Epic’s TOSH provides a way to categorize any EHR time outside time scheduled with patients similarly—whether immediately before or after scheduled clinic appointments, over a lunch hour, or late at night—and thereby recognizes the similarity of EHR work outside scheduled time with patients regardless of when clinicians choose to do it depending on their myriad personal and professional circumstances.

Validate Measures of Active EHR Time

At present, there are no standardized and publicly validated methods of quantifying active EHR use by clinicians across vendors and health systems. Whether the EHR is being used during or outside time scheduled with patients, we recognize the value and need to identify when a clinician steps away from the EHR to engage in other clinical or nonclinical tasks while remaining logged into the EHR. We also recognize there are different interruptions (in terms of both frequency and duration) depending on the setting in which the EHR work is completed. We therefore want to reiterate that it is important for the research and vendor communities to develop a standardized and validated vendor-agnostic measure for active EHR time by comparing methods of determining such EHR use from event logs with durations derived from direct observation.9,14,15 Such a measure would be independent of both the 5-second time-out in Epic’s method and the 45-second time-out in Cerner’s proprietary method. We believe each of these vendor-defined specifications underestimates the time a user is meaningfully engaged in EHR-associated work.

An independently developed set of vendor-agnostic specifications would avoid the inherent biases that exist with vendor-defined methods for counting time while accommodating the variety of work patterns that could otherwise dramatically undercount EHR use by categorizing time as inactive even when a clinician is actively engaged with the EHR. Independent measure development, however, is costly, still necessitates vendor involvement for validation, and is not practical for most institutions. For the short term, we realize that most institutions will by necessity use vendor definitions of active EHR time, although they should recognize that these likely underestimate the actual time their clinicians are engaged with the EHR.

CONCLUSIONS

As health systems continue to unravel the many factors that contribute to burnout, it is critical to have a solid understanding of how to quantify EHR workload outside time scheduled with patients and its relationship to declining workplace engagement, perceived value, and wellness. Refining vendor-defined measures to better match the intention behind standardized after-hours EHR workload metrics, including WOW and WOW8, will improve comparative research among health systems regardless of their EHR platforms. To match the measure’s intent, we recommend allocating time in the EHR to time either during or outside time with scheduled patients, and we suggest counting all time before and after time scheduled with patients as time outside scheduled hours. We encourage others to further refine vendor-defined metrics, which can ultimately impact policy change. This effort will result in a set of metrics that allow health system leaders, operations leaders, regulators, and technology vendors to monitor changes in practice and their influence on clinician burnout over time.

Acknowledgments

We wish to thank UW Health’s Technical Support Representative at Epic for helping our team understand the technical details of TOSH and the other measures generated out of Signal.

Footnotes

Conflicts of interest: Dr Sinsky is employed by the American Medical Association. The other authors report no conflicts of interest.

Read or post commentaries in response to this article.

Author contributions: All authors have made substantial contributions to the conception or design of the work; the acquisition, analysis, or interpretation of data for the work; and/or drafting the work or revising it critically for important intellectual content. In addition, all authors gave final approval of the version to be published and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Funding support: This work was supported in part by the American Medical Association’s Practice Transformation Initiative, EHR-Use Metrics Research, which has provided grant funding (grant no. 16114) for authors B.G.A., M.A.M., C.M.S., and J.J.B.

Disclaimer: The opinions expressed in this article are those of the authors and should not be interpreted as American Medical Association policy.

Received for publication August 10, 2022.Revision received December 7, 2022.Accepted for publication December 13, 2022.© 2023 Annals of Family Medicine, Inc.

留言 (0)

沒有登入
gif