Telehealth, or virtual care (VC), encompasses a range of visit types including synchronous modalities such as real time video, telephone or synchronous chats along with asynchronous modalities (ie, messages or chats not in real time) and remote patient monitoring.1 While early forms of VC, like telephone calls with a physician, have been around since the start of the 20th century, improvements in technology, including mobile device use and federal legislation have pushed VC into the 21st century.2–4 Despite the availability of VC options, both patients and providers have identified barriers to VC that have contributed to a slow adoption of VC. These barriers stem from reimbursement models, equipment issues, lack of reliable internet, and state-level policies.5,6 Before the COVID-19 pandemic, coverage of VC services under Medicare was only available to those living in rural areas.7 During the pandemic, the Centers for Medicare & Medicaid Services (CMS) and most private insurance plans expanded coverage for VC appointments, improved reimbursements, decreased or removed co-pays, and/or expanded indications for VC use.7

The onset of the COVID-19 pandemic (March 2020) and the declaration of a national emergency (March 13, 2020) rapidly shifted the delivery of health care in the United States from a mostly in-person model to a hybrid model dominated by VC.8–12 This shift was in part due to the recommendations made by the Centers for Disease Control and Prevention (CDC) encouraging VC use (eg, video and telephone visits) by both patients and providers, when possible, to reduce the spread of COVID-19 and encourage social distancing practices.13 In addition, recommendations were made by CMS to delay nonessential medical visits and procedures to reduce the risk of COVID-19 transmission and decrease the burden on the health care system.14 Consequently, many health care systems saw a decrease in outpatient visits for non–COVID-related issues.

Before the COVID-19 pandemic, 90% of virtual visits (VV) among insured patients were from behavioral health (BH) and primary care visits.15 BH encounters often consist of counseling and psychotherapy and may be more amendable to delivering care virtually. In contrast to BH, many services provided in adult primary care (APC), such as vital sign checks and physical examinations, mostly require in-person visits (IPV). Given the shift to VC caused by the pandemic, little is known about the long-term trends in VC use, especially once restrictions eased and IPV were easier to come by. For several years before the COVID-19 pandemic, the integrated health care delivery systems of Kaiser Permanente (KP) have offered a wide range of VC options including video visits, chats, telephone calls, house calls, and a secure patient portal platform for messaging, appointment requests, prescription refills and laboratory and medical record review. KP is comprised of 8 health care systems that integrate their health plan departments, hospitals and medical groups creating a closed system for both outpatient and inpatient care. When COVID-19 hit the United States, KP was able to leverage their established, mature VC infrastructure to offer virtual appointments to their 12+ million membership.

The overarching goal of this study is to describe the impact of the COVID-19 pandemic on health care utilization within 3 large, independent KP health systems. We focus on 2 departments where VC appointments were utilized before the pandemic, APC and BH. Our objectives were to: (1) describe the impact of the COVID-19 pandemic on visit rates in APC and BH departments; and (2) determine how the rate of VV and IPV changed in APC and BH departments once COVID-19 restrictions eased, and clinics resumed IPV. This study may provide insights into the long-term transition to VC use as the COVID-19 pandemic continues to evolve.

METHODS Setting and Population Studied

This study used data from 3 KP health systems in the Denver and Boulder, Colorado area (KPCO), Atlanta metropolitan-area of Georgia (KPGA), and the Mid-Atlantic States (KPMAS) which encompasses Maryland, Virginia, and the Washington, DC area. These sites represent geographically and racially diverse membership of over 1.6 million adult individuals. Individuals included in the study population were adults age 19 years and older as of the first of the month of each of the 30 months from January 2019 to June 2021.

Study Design

This is a retrospective study using data from health care encounters that occurred in each health care system from January 1, 2019 to June 30, 2021. To describe changes in visit rates pre-COVID-19 and post-COVID-19 onset, we further divided the period into 3 eras of care: (1) pre-COVID=January 1, 2019–March 12, 2020; (2) national COVID-19 shutdown and recovery=March 13, 2020–December 31, 2020; (3) COVID-19 vaccination phase=January 1, 2021–June 30, 2021. Each of the 3 sites maintain a similar local implementation of a common data structure and repository developed under the governance of the Health Care Systems Research Network16 to facilitate multisite collaborations. Demographics, visit characteristics, hospitalizations, diagnoses, procedures, and medication orders are routinely curated from each site’s electronic health record and claims databases and imported into their local virtual data warehouse (VDW). Additional details about the VDW can be found in the Supplemental Digital Content (https://links.lww.com/MLR/C568). Data for this study were extracted from each site’s VDW and electronic health record data using common data definitions and variable formats. Data were aggregated and deidentified and sent to KPGA for analysis. This study was reviewed and approved by the KPGA institutional review board, which served as the institutional review board of record for all 3 sites.

Classification of Visit Model

APC visits consisted of visits, subset into: (1) IPV conducted by providers in adult or family medicine clinics or providers in urgent care clinics, and (2) VV (not in-clinic) conducted through a virtual mode (KPMAS “house calls,” telephone visits, video visits, and synchronized chats). BH visits consisted of visits, subset into: (1) IPV conducted by a BH provider (psychiatrist, clinical psychologist, licensed clinical therapist, or social worker) in a BH clinic, and (2) VV conducted through a virtual mode (telephone visits, video visits, and synchronous chats). KPMAS “house calls” allow for only patient concerns related to specific physical conditions. To provide a more congruent comparison of visit rates, encounters that could only be completed in-person (ie, did not have a virtual analog) were excluded. This included visits such as immunizations, blood pressure checks, eye examinations, or preoperative testing. To reduce the influence of COVID-19 testing and vaccination on health care utilization trends, we excluded visits solely for COVID-19 vaccination or testing.

During the study period modes of VV varied by site and era. Pre-COVID-19, KPGA offered both video visits and telephone visits, KPMAS offered house calls, video visits, and telephone visits, and KPCO offered telephone visits and synchronous chats (Supplemental Digital Content Table 1, https://links.lww.com/MLR/C568). During COVID-19, KPCO added video visits and KPGA added synchronous chats (APC only). Given the sparsity of some VC types, all VV types were collapsed for analysis.

Statistical Analysis

Individual visits were combined to provide weekly summaries of visit counts overall and by department. Weekly rates were further summarized by visit type; VV or IPV. Data were treated as a weekly time series and for each week, we computed total (VV+IPV), VV, and IPV rates as visits per 1000 enrolled adult beneficiaries. To reduce large variations in weekly rates, especially around weeks with holidays, consecutive weekly visit rates were averaged together, reducing the number of data points to 65 observations. Visit rates were plotted over a 30-month period using scatter plots with linear interpolation. Separate time series were created for APC and BH visits for each visit mode and site.

To evaluate trends in visit rates we applied analytic approaches appropriate for time series data including the use autoregressive error models to account for correlated time observations. Lags were included in the model to improve model fit and reduce autocorrelation. Model diagnostics were examined and included partial autocorrelation plots, white noise probability plots, and tests for autocorrelation and heteroscedasticity. To evaluate changes in trends across the 3 eras, segmented regression models were constructed. The parameter estimates obtained from the segmented regression models include 6 terms of interest. These estimates are described further in Supplemental Digital Content Table 2 (https://links.lww.com/MLR/C568). Analyses were conducted using SAS Enterprise Guide, v. 8.2 and statistical significance was assessed at the 0.05 level. Additional details regarding the statistical analyses can be located in the Supplemental Digital Content (https://links.lww.com/MLR/C568).

RESULTS Adult Primary Care Overall Utilization Trends

Figures 1A–C depict the observed weekly encounter rates at each of the 3 sites during the study period. The figures show the overall trend (red line), the IPV rate (green line) and virtual visit rate (blue line). Overall rates of APC visits in the pre-COVID era were stable at each of the 3 sites and trend patterns throughout the study period were similar. The median (25th–75th percentile) weekly visit rate per 1000 enrolled adult members was 44.8 (41.5–47.7), 44.3 (43.7–47.1), and 47.9 (46.2–49.3) at KPCO (Fig. 1A), KPGA (Fig. 1B), and KPMAS (Fig. 1C), respectively. Results from the time series analysis (Table 1) demonstrated a relatively flat trend (b11overall=0) in the weekly visit rate with some seasonal variation noted (higher utilization in first and last quarters of the calendar year). As expected, there was an immediate drop, or level shift, in APC visits coinciding with the national emergency declaration. All sites detected a significant decrease in the weekly visit rate following the shutdown with an estimated level shift (b02overall) ranging from −10.1 to -9.7 visits per 1000 members/week (Table 1). During the second era, median weekly visit rates declined to 39.4 per 1000 members (37.2–42.0), 41.6 (37.8–43.9), and 42.9 (40.4–45.3) at KPCO, KPGA, and KPMAS, respectively. In addition to an immediate level shift, the trend (ie, slope) in the weekly visit rate significantly increased at KPCO and KPGA (b12overall>0) as clinics began to open and IPV were more widely available. Of note, KPMAS did not have a statistically significant change in the pre-COVID slope during the second era. During the third era (January 2021–June 2021), median weekly visit rates in APC at 2 of the 3 sites returned to prepandemic levels. At KPCO (Fig. 1A) and KPGA (Fig. 1B) median weekly visit rates were 44.0 per 1000 members (43.2–44.7) and 44.5 (42.8–45.2). In contrast at KPMAS (Fig. 1C), the median weekly visit rate of 44.9 per 1000 members was still below the median pre-COVID rate. Time series analysis at KPMAS confirmed a slower rate of recovery compared with the 2 other sites (Table 1). Trends in visit rates varied across the 3 sites during the third era. At KPGA, the time series analysis demonstrated a negative trend in the third era corresponding to a significant decrease in the average weekly trend (b13overall<0). In contrast, at KPMAS and KPCO, there was no level shift or trend change during the third era.

FIGURE 1:

(A–C) Weekly encounter rates in adult primary care departments per 1000 enrolled adult members from January 1, 2019, to June 30, 2021, at Kaiser Permanente Colorado (KPCO) (A); Kaiser Permanente Georgia (KPGA) (B); Kaiser Permanente Mid-Atlantic States (KPMAS) (C). The red dashed line is the overall weekly encounter rate. The green solid line is the in-person visit weekly encounter rate. The blue solid line is the virtual visit weekly encounter rate. The dashed black lines define the 3 encounter eras [pre-COVID (era 1), COVID onset and recovery (era 2), and COVID vaccination (era 3)]. The first black dashed line corresponds to the national emergency on March 13, 2020. The second black line corresponds to December 26, 2020 which shortly follows the emergency authorization and distribution of the first 2 COVID-19 vaccines.

TABLE 1 - Model Estimated Trends and Level Changes Before and During COVID-19 Pandemic for Adult Primary Care Encounters Overall In-person Virtual Site Era Parameter Estimate (SE) P Estimate (SE) P Estimate (SE) P KPCO Pre-COVID (era 1) Avg. Enc rate (b01) 45.00 (1.31) <0.001 38.60 (1.19) <0.001 7.03 (0.83) <0.001 Trend (b11) −0.03 (0.07) 0.671 −0.11 (0.06) 0.085 0.06 (0.04) 0.179 Shutdown and recovery (era 2) Δ in Avg. Enc rate (b02) −9.90 (2.09) <0.001 −29.44 (1.57) <0.001 19.74 (1.07) <0.001 Δ Trend from era 1 (b12) 0.50 (0.15) 0.002 1.09 (0.13) <0.001 −0.64 (0.09) <0.001 COVID-19 vaccination phase (era 3) Δ in Avg. Enc rate (b03) 1.33 (2.32) 0.568 −0.62 (1.81) 0.733 3.98 (1.23) <0.001 Δ Trend from era 2 (b13) −0.66 (0.35) 0.066 −0.63 (0.24) 0.012 −0.06 (0.21) 0.783 KPGA Pre-COVID (era 1) Avg. Enc rate (b01) 44.78 (1.40) <0.001 41.58 (1.20) <0.001 3.38 (0.48) <0.001 Trend (b11) 0.02 (0.07) 0.884 −0.02 (0.06) 0.785 0.03 (0.03) 0.303 Shutdown and recovery (era 2) Δ in Avg. Enc rate (b02) −9.74 (1.74) <0.001 −38.08 (1.43) <0.001 27.80 (0.78) <0.001 Δ Trend from era 1 (b12) 0.43 (0.15) 0.006 1.17 (0.14) <0.001 −0.68 (0.05) <0.001 COVID-19 vaccination phase (era 3) Δ in Avg. Enc rate (b03) 2.33 (1.95) 0.238 −4.04 (1.68) 0.019 6.62 (1.05) <0.001 Δ Trend from era 2 (b13) −0.84 (0.28) 0.006 −0.88 (0.24) 0.001 −0.06 (0.11) 0.590 KPMAS Pre-COVID (era 1) Avg. Enc rate (b01) 46.89 (1.40) <0.001 43.36 (1.02) <0.001 3.81 (0.98) <0.001 Trend (b11) 0.08 (0.08) 0.323 −0.07 (0.06) 0.180 0.09 (0.05) 0.078 Shutdown and recovery (era 2) Δ in Avg. Enc rate (b02) −10.14 (1.64) <0.001 −33.93 (1.67) <0.001 28.19 (1.43) <0.001 Δ Trend from era 1 (b12) 0.19 (0.17) 0.268 1.00 (0.11) <0.001 −0.94 (0.11) <0.001 COVID-19 vaccination phase (era 3) Δ in Avg. Enc rate (b03) −0.70 (1.90) 0.715 −5.59 (2.31) 0.019 9.86 (1.74) <0.001 Δ Trend from era 2 (b13) −0.18 (0.27) 0.512 −0.31 (0.31) 0.317 −0.20 (0.22) 0.376

*Outcome was weekly encounter rate per 1000 enrolled adult members.

Avg. indicates average; Enc, encounter; KPCO, Kaiser Permanente Colorado; KPGA, Kaiser Permanente Georgia; KPMAS, Kaiser Permanente Mid-Atlantic States.

In-person and Virtual Visit Trends

Before the national shutdown, weekly VV rates in APC were stagnant with nonsignificant changes in the weekly VV rate. The median weekly VV rate pre-COVID ranged from 3.7 per 1000 members (KPGA) to 4.8 (KPMAS) to 7.8 (KPCO). VV accounted for <20% of all APC visits (Supplemental Digital Content Figs. 1a–c, https://links.lww.com/MLR/C568). Similar stationary trends were observed for IPV at each of the 3 sites (Table 1). After the shutdown at each of the 3 sites, IPV rates immediately and significantly decreased (b02IPV<0) to a rate of <5 IPV per week per 1000 adult members. However, during the recovery period, IPV rates slowly increased (b12IPV>0) with new estimated slopes of 0.98 (KPCO), 1.15 (KPGA), and 0.93 (KPMAS) visits per week per 1000 members (Table 1). During the second era, ∼60% of all visits were virtual (Supplemental Digital Content Figs. 1a–c, https://links.lww.com/MLR/C568); however, as quickly as the VV rate increased, the weekly rate began to decrease into a downward trend (Figs. 1A–C) in mid-2020. This visual pattern was confirmed by the time series model (b12VV<0) resulting in an overall decreasing trend. As clinics reopened, IPV increased and became the dominant visit mode by the third era; however, trends differed by site. At KPGA and KPMAS, the rollout of COVID-19 vaccines (also coinciding with a COVID-19 surge) resulted in an immediate decrease in the weekly IPV rate (b03IPV<0). This effect was not observed at KPCO. In addition, during the third era the IPV weekly visit rate was still increasing but slowed compared with the recovery period (b13IPV<0). All 3 sites observed an immediate increase in the VV rate in January 2021 (b03VV>0); however, the overall trend continued to decline at the same rate in era 3 as it did in era 2 (b13VV=0).

Behavioral Health Overall Utilization Trends

In contrast to APC, trends in weekly visit rates (shown in Figs. 2A–C) for BH visits remained stable or even slightly increased following the onset of the COVID-19 pandemic. Overall weekly visits rates in leading up the COVID-19 shutdown visually appeared similar at each of the 3 sites but were noted to have small trend differences in our models. Results from Table 2 show a small but significant decreasing trend at KPCO (b11overall=−0.02), a stable trend at KPMAS (b11overall=0.006) and small but slightly increasing at KPGA (b11overall=0.04). In the first era, the median (25th–75th percentile) weekly visit rate per 1000 enrolled adult members was 4.8 (4.6–5.8), 8.2 (7.9–9.1), and 5.8 (5.4–6.1) at KPCO, KPGA, and KPMAS, respectively. Unlike the trends observed in APC, there were no significant level shifts or immediate changes in the weekly visit rate resulting from the national emergency (b02overall=0). During the second era, KPCO saw a small, but significant, increase in the weekly trend rate followed by a significant, positive, level change at the start of the third era (b03overall>0). Figure 2A shows an increasing trend in the weekly visit rate which was confirmed by the time series analysis (Table 2). The median visit rate during the second and third eras at KPCO were 5.4 per 1000 members (4.9–5.8) and 5.8 (5.4–6.0). At KPGA, during the second and third eras, no significant changes in trends or level shifts between eras were detected. The median weekly visit rate during the third era was 9.2 (8.9–9.6) and relatively unchanged from the second era [9.3 (8.8–9.6)]. At KPMAS, median weekly visit rates increased during each era from 5.8 to 6.3 to 7.1. The time series analysis detected significant changes in the trend (increased trend rate) during the second era followed by a decreasing trend or slope in the third era (Table 2).

FIGURE 2:

(A–C) Weekly encounter rates in behavioral health departments per 1000 enrolled adult members from January 1, 2019 to June 30, 2021 at Kaiser Permanente Colorado (KPCO) (A); Kaiser Permanente Georgia (KPGA) (B); Kaiser Permanente Mid-Atlantic States (KPMAS) (C). The red dashed line is the overall weekly encounter rate. The green solid line is the in-person visit weekly encounter rate. The blue solid line is the virtual visit weekly encounter rate. The dashed black lines define the 3 encounter eras [pre-COVID (era 1), COVID onset and recovery (era 2), and COVID vaccination (era 3)]. The first black dashed line corresponds to the national emergency on March 13, 2020. The second black line corresponds to December 26, 2020 which shortly follows the emergency authorization and distribution of the first 2 COVID-19 vaccines.

TABLE 2 - Model Estimated Trends and Level Changes Before and During COVID-19 Pandemic for Behavioral Health Encounters Overall In-person Virtual Site Era Parameter Estimate (SE) P Estimate (SE) P Estimate (SE) P KPCO Pre-COVID (era 1) Avg. Enc rate (b01) 4.94 (0.14) <0.001 4.33 (0.09) <0.001 0.64 (0.12) <0.001 Trend (b11) −0.02 (0.008) 0.045 −0.018 (0.005) <0.001 0.00 (0.007) 0.928 Shutdown and recovery (era 2) Δ in Avg. Enc rate (b02) 0.45 (0.26) 0.097 −3.90 (0.13) <0.001 4.43 (0.22) <0.001 Δ Trend from era 1 (b21) 0.05 (0.02) 0.007 0.04 (0.01) 0.474 0.001 (0.01) 0.934 COVID-19 vaccination phase (era 3) Δ in Avg. Enc rate (b03) 0.87 (0.39) 0.029 −0.19 (0.17) 0.268 1.18 (0.34) <0.001 Δ Trend from era 2 (b13) −0.14 (0.05) 0.003 0.01 (0.02) 0.474 −0.16 (0.04) <0.001 KPGA Pre-COVID Avg. Enc rate (b01) 7.63 (0.37) <0.001 5.71 (0.17) <0.001 2.04 (0.27) <0.001 Trend (b11) 0.04 (0.02) 0.025 0.00 (0.01) 0.974 0.03 (0.01) 0.014 Shutdown and recovery Δ in Avg. Enc rate (b02) 0.32 (0.38) 0.402 −5.82 (0.26) <0.001 6.27 (0.30) <0.001 Δ Trend from era 1 (b21) −0.05 (0.03) 0.129 0.03 (0.02) 0.188 −0.07 (0.05) 0.004 Pre-COVID (era 1) Δ in Avg. Enc rate (b03) 0.64 (0.55) 0.251 −0.17 (0.34) 0.618 0.80 (0.42) <0.001 Δ Trend from era 2 (b13) −0.05 (0.06) 0.285 −0.01 (0.04) 0.713 −0.02 (0.05) 0.628 KPMAS Pre-COVID (era 1) Avg. Enc rate (b01) 5.62 (0.16) <0.001 4.99 (0.15) <0.001 0.60 (0.18) 0.001 Trend (b11) 0.006 (0.009) 0.510 −0.04 (0.009) <0.001 0.05 (0.01) <0.001 Shutdown and recovery (era 2) Δ in Avg. Enc rate (b02) −0.10 (0.25) 0.679 −3.63 (0.19) <0.001 3.56 (0.27) <0.001 Δ Trend from era 1 (b21) 0.06 (0.02) 0.002 0.04 (0.01) 0.042 0.02 (0.02) 0.410 COVID-19 vaccination phase (era 3) Δ in Avg. Enc rate (b03) 0.33 (0.33) 0.322 −0.08 (0.22) 0.702 0.50 (0.33) 0.147 Δ Trend from era 2 (b13) −0.11 (0.04) 0.006 0.00 (0.03) 0.994 −0.12 (0.04) 0.004

*Outcome was weekly encounter rate per 1000 enrolled adult members.

Avg. indicates average; Enc, encounter; KPCO, Kaiser Permanente Colorado; KPGA, Kaiser Permanente Georgia; KPMAS, Kaiser Permanente Mid-Atlantic States.

In-person and Virtual Visit Trends

The onset of COVID-19 radically changed how BH departments functioned at KP. Unlike APC departments, BH visits have remained almost entirely virtual (Figs. 2A–C and Supplemental Digital Content Figs. 2a–c, https://links.lww.com/MLR/C568) as in-person services have been limited in this department. As a result, following the national emergency all 3 sites had a significant increase in VV utilization rates (b02VV>0) and a significant decrease in IPV utilization rates (b02IPV<0). During the first era, median weekly IPV rates were 4.1, 5.9, and 4.5 at KPCO, KPGA, and KPMAS, respectively. IPV rates in the second and third eras we nearly 0 at each of the 3 sites. Even in the third era, VV continue to dominate BH visits at each of the 3 sites with median rates of 5.3, 8.9, and 7.0 at KPCO, KPGA, and KPMAS, respectively and accounted for >92% of all BH visits. At KPCO and KPMAS the trend in VV utilization rates continued from the pre-COVID trend but started to significantly decline during the third era (b13VV<0), driving the observed decreasing trend in the overall BH weekly utilization rate.

DISCUSSION

Our study found that the COVID-19 pandemic differentially impacted health care utilization in APC and BH departments within and across our 3 health care systems. Our first goal was to describe visit rates in APC and BH departments before and during the COVID-19 pandemic. In APC, visit rates remained stable in the year preceding the pandemic. At the start of the pandemic, visit rates declined but eventually rebounded to prepandemic levels in 2021. In contrast, BH visit rates remained stable or slightly increased during the pandemic.

Our second goal was to evaluate trends in virtual visit and IPV rates. In both departments, the onset of the pandemic resulted in significant decrease in IPV and increase in VV. In APC, each health care system saw a resumption of IPV in 2021; however, BH visits have remained almost entirely virtual without impacting overall visit rates. The onset of the COVID-19 pandemic and national shutdown significantly decreased the rate of APC visits and led to a large shift in care delivery. Before the pandemic, VV accounted for <20% of APC visits across each of the 3 sites; however, this rate of VV use is higher than other health care organizations. Using cross-sectional data from 2018 to 2020 from the IQVIA National Disease and Therapeutic Index, Alexander and colleagues found that before COVID-19, ∼8% of visits in primary care (including pediatrics) were conducted virtually, while in the second quarter of 2020, VV accounted for 35% of primary care visits. The proportion of APC visits conducted virtually during the second quarter of 2020 was much higher in our study (>50% of APC visits). Our study excluded visits related to COVID-19 and those that could only be conducted only in-person which may have contributed towards our higher observed rate of VV. The higher rate of VV use was not sustained throughout the pandemic and started to decline by mid-2020. By 2021, VV accounted for 35%–45% of visits in APC. Consistent with our findings, a cross-sectional survey of US adults in found a declining rate of VC use such that by April 2021 <30% of adults surveyed used VC in the last month.17

We noted significant fluctuations in VV rates across all 3 sites and both departments at the end of 2020. These changes may be a reflection of end of year trends in ambulatory care visits