The emergence of the COVID-19 pandemic in March 2020 brought about a rapid shift in health care from primarily in-person visits to entirely virtual care. In some health systems, an increased capacity for telehealth was already in progress. Telemental health, or virtual visits (VVs) within behavioral health care, was particularly a capacity that was being developed.1,2 Ongoing behavioral health treatment may be well suited to virtual care, as treatment can involve repeated therapeutic appointments that are longer in duration than other appointments. VVs allow patients to reduce the burden by removing travel time and wait time,3 and the discussion-based format may be minimally disrupted by the virtual mode. Meta-analyses of randomized controlled trials4 and noninferiority studies5 have established that behavioral health treatment through VV is comparable in effectiveness to in-person treatment. Prepandemic, VVs had the potential to provide a viable and effective treatment mode alternative for patients.6

A large-scale and swift transition to virtual care, however, was not without potential challenges as providers and patients unfamiliar with virtual care were required to adjust to new practices and patterns. For integrated health systems that may have pharmacies co-located with clinics, moving quickly to VVs had the potential to disrupt pharmacological treatment patterns because patients must be willing to engage in extra effort to fill prescriptions. Indeed, research on behavioral health virtual care during the pandemic demonstrated that prescription fills for psychotropic medications declined significantly from prepandemic rates.7,8 It is unclear from the existing research, however, if physician orders of the medications were also reduced with the shift to VVs, or just the fulfillment of orders.

The transition from in-person to virtual behavioral health care also required changes in workflows and patterns that had the potential to disrupt other provider practices, including measurement-based care. Behavioral health measurement-based care involves using patient-reported outcome measures, such as symptom screeners, to track symptoms over time and use that feedback dynamically to inform treatment decisions.9 Clinical trials have established that measurement-based care is effective at reducing symptoms,10 and improving remission rates,11 particularly for patients who are not on track for positive outcomes.12 Qualitative research with behavioral health providers found that providers felt it was more difficult to provide measurement-based care during the pandemic shift to virtual care due to excessive time spent administering assessments during VVs.1

Measurement-based care for in-person visits can be achieved through transmitting patient-reported outcome measures to patients in advance of scheduled visits, such as by email or mail, to encourage patients to complete them at home. If patients have not completed them by the time they arrive at a clinic, they can be asked to complete the measures in the waiting room before the visit. Completion during the visit itself would be a last resort. VVs, however, remove the waiting room component of the process and require the patient and provider to develop new patterns to ensure that patient-reported symptom screeners are either completed in advance or administered verbally during the visit. An abrupt shift to VVs has the potential to disrupt the completion of symptom screeners, and therefore the practice of measurement-based care, for providers and patients who do not already have those virtual care patterns in place.

Several factors may determine whether a patient opted to engage in VVs before and during the COVID-19 pandemic. Age, race/ethnicity, comorbidities, and neighborhood deprivation have all been identified as patient characteristics associated with selecting virtual care before the pandemic,13–15 with some shifting in self-selection patterns during the pandemic.13,15,16

The aim of this study is to examine shifts within VV health-care-related practices in behavioral health departments from January 2019 to June 2021 in 3 regions of an integrated health care system. We focus on first follow-up VVs after an incident diagnostic visit for major depression, examining changes in rates of antidepressant medication orders, antidepressant medication order fulfillment, and the completion of symptom screeners in association with VVs.

METHODS Setting

This study took place from January 2019 through June 2021 in 3 regions of Kaiser Permanente’s (KP) integrated health care delivery system. These 3 health systems are geographically and racially/ethnically diverse. Mid-Atlantic States (KPMAS), which serves the Washington DC metro area, Baltimore, and Northern Virginia (more than 750,000 enrollees in Q1 2020); Georgia (KPGA), which serves the Atlanta area (more than 300,000 enrollees in Q1 2020); and Colorado (KPCO), which serves the Denver/Boulder area (more than 575,000 enrollees in Q1 2020). At KP measurement-based care is strongly encouraged as a treatment approach.

All 3 regions transitioned rapidly to a virtual-care-first model in March 2020 at the time of the declaration of a national emergency and associated state-level shutdowns due to the COVID-19 pandemic.17 Although adult primary care gradually returned to a balance of in-person and virtual care encounters once the shutdown periods ended, behavioral health care remained almost entirely virtual, and visit volume remained steady or increased slightly throughout the remainder of the study period,18 providing an opportunity to examine practice variation in medication orders, patient fulfillment of orders, and completion of mental health symptom trackers over time within the behavioral health virtual care setting, as virtual care became the predominant mode of care.

Study Design

This study is a retrospective cohort study, encompassing virtual care in behavioral health during 3 naturally occurring time periods that we selected to reflect the potential for shifts in health-care-related practices: Prepandemic (January 2019–March 2020) represents virtual care as it existed in the 3 KP regions before the pandemic, as an elective option for encounters. Peak pandemic (April 2020–June 2020) represents the abrupt transition to nearly fully virtual care within the 3 KP regions and may be considered a period of adjustment for health care practices. Recovery (July 2020–June 2021) represents a time when medical centers opened again for in-person appointments in the 3 regions, but behavioral health stayed almost entirely virtual and settled into a sustained new pattern of care. Virtual care modes in use in behavioral health in the 3 regions included both telephone and video encounters. Although synchronous chat was also available at KPGA and KPCO, it was not used during this time period for follow-up visits within behavioral health.

The unit of analysis for this study is the virtual health care encounter in behavioral health, specifically the first follow-up behavioral health encounter within 42 days after an incident diagnostic encounter for unipolar major depressive disorder (ICD-10 codes F32 or F33, as either a primary or nonprimary diagnosis). Encounters for depression in adult primary care were excluded due to the study's focus on measurement-based care practices, which are implemented only in the regions’ departments of behavioral health. The 42-day lookback period was selected both to identify the incident encounter, with no encounter with a depression diagnosis for 42 days before the incident encounter and to identify the first follow-up within 42 days after that incident encounter. The 6-week duration of the lookback period was identified by behavioral health clinical experts in the 3 health systems as an upper limit of the lag between an incident diagnostic encounter for major depression and the first follow-up visit. The first follow-up encounter was selected rather than the incident diagnostic encounter because in the prepandemic time period the 3 regions did not allow patients to choose a virtual mode of care for initial diagnostic encounters within behavioral health; at that time, virtual care became an option only at the first follow-up encounter. Patients were at least 19 years old at the time of the first follow-up encounter and enrolled in KP for at least the entire lookback period before the incident diagnostic encounter.

Study Variables Dependent Variables

Order for an antidepressant medication: Antidepressant medications were identified through the generation of a list of pharmaceutical subclasses of antidepressant medications that had been prescribed or were identified by local content experts as an option for treatment, in at least one of the 3 regions during the study period. An antidepressant medication was considered ordered if the order was associated with the encounter in the electronic health record (EHR) or occurred within 5 days after the encounter for the same patient and from a provider in the same department and clinic.

Fulfillment of an antidepressant medication order: Considering only the subset of encounters that included an order for an antidepressant, a medication order was considered fulfilled if the medication was dispensed within 30 days of the order being placed.

Symptom screener completion: Screening was considered completed if a behavioral health screening instrument was entered into the EHR within 3 days before or after the encounter. The 3-day window was specified after a review of data and in consultation with behavioral health clinicians, to capture screeners pushed out to patients and completed through the online patient portal in advance of scheduled encounters, screeners recorded in the EHR by the provider during encounters, or screeners completed in notes and entered into the EHR by the provider after the encounter. Screenings included the Patient Health Questionnaire19—9 items or 2 items; the Generalized Anxiety Disorder—7 items20 the Columbia Suicide Severity Rating Scale;21 or a combination of the screening instruments.

Independent Variables

Time period: The 3 time periods defined for the study represent shifts in virtual patient care, as described above: prepandemic (January 2019–March 2020), peak pandemic (April 2020–June 2020), and recovery (July 2020–June 2021).

Covariates: Patient characteristics included as covariates in the analyses were patient age, sex, race/ethnicity, Charlson Comorbidity Index,22 and Area Deprivation Index.23 These covariates were selected because they have emerged in other research as patient factors related to self-selection into VV use, and changes in who uses VV, before and during the pandemic.13–16

Statistical Analyses

Analyses were conducted separately for each KP region to preserve region-specific patterns that might emerge. We used inverse probability of treatment weighting (IPTW)24 to reduce confounding due to self-selection into VV use for a major depression encounter across the different time periods. IPTW involves calculating propensity scores, or the probability that an individual would be in a particular time-period group, conditional on a set of specified covariates. Propensity scores were calculated using multivariable logistic regression, and those scores were then used to weight the analyses of health care practices by time period. The weights balance the time-period groups on the covariates, controlling for the influence of the covariates on the health care practice outcomes. For IPTW, time periods were weighted in adjacent pairs (prepandemic vs peak pandemic; peak pandemic vs recovery period) because of the a priori analytic focus on transitions into and out of the peak-pandemic period.

For analyses of antidepressant order and symptom screener completion outcomes, two initial logistic regressions were conducted, each with a pair of adjacent time periods (prepandemic vs peak pandemic; peak pandemic vs recovery period) as the dependent variable, and age, sex, race/ethnicity, Charlson Comorbidity Index, and Area Deprivation Index as independent variables. From these analyses, we estimated IPTW to balance the time-period groups for covariates and therefore control for self-selection into the time period. Crosstab analyses of antidepressant order rates by paired time periods, weighted using IPTW, were conducted to produce adjusted order rates by time period. The same crosstab procedure was followed to yield rates of symptom screener completion by time period. χ2 tests of independence were calculated on the weighted data for significance testing.

For analysis of antidepressant medication order fulfillment, the sample was restricted to those VVs that included an antidepressant order. Subsequently, the same process was followed for analyses, including estimating IPTW and applying those weights to crosstab analyses by paired time periods to yield rates of antidepressant order fulfillment by time period. χ2 tests of independence were calculated for significance testing. All analyses were conducted using SAS version 9.4M6 (SAS Institute Inc.). The standard used for statistical significance in all analyses was P <0.05. Confirmation of the effectiveness of the IPTW procedure in balancing the time-period groups on covariates appears in Supplemental Digital Content Table 1A–C (https://links.lww.com/MLR/C584).

RESULTS Sample Characteristics

Eligible health care encounters in behavioral health for major depression were limited to first follow-up after an incident diagnostic encounter for major depression, and further limited to VVs only, resulting in 12,266 encounters at KPMAS, 8238 encounters at KPGA, and 8227 encounters at KPCO. Excluded encounters comprised initial diagnostic encounters, encounters subsequent to the first follow-up, and in-person encounters. See Figure 1 for a flow diagram of included encounters. With the differential number of months in each time period taken into account, the volume of eligible first follow-up encounters, including both VV and in-person, increased slightly between pre and peak-pandemic periods at KPGA and KPMAS and in the recovery period returned to levels similar to prepandemic volume. At KPCO volume decreased slightly during the peak-pandemic period, and in the recovery period returned to levels similar to prepandemic volume. The 3 regions differed in the distributions of patient characteristics, including age, sex, race/ethnicity, number of comorbidities, and area deprivation (see Table 1 for unweighted patient characteristics at the time of the encounters). Due to the differences between the regions in utilization patterns and patient populations, analyses were conducted separately by region to allow for the possibility of different patterns emerging across the health systems.

FIGURE 1:

Flow diagram of inclusion of virtual behavioral health encounters for first follow-up after incident diagnostic encounter for major depression, by health care system region.

TABLE 1 - Unweighted Sample Characteristics by Region at the Time of the First Virtual Follow-up Visit for Major Depression, After an Initial Diagnostic Encounter, January 2019–June 2021 KPMAS KPGA KPCO N=12,266 N=8238 N=8227 Patient characteristics at time of VV n (%) n (%) n (%) Age group at time of visit  19–34 5385 (43.9) 2751 (33.4) 3093 (37.6)  35–49 3028 (24.7) 2609 (31.7) 2159 (26.2)  50–64 2488 (20.3) 2049 (24.9) 1701 (20.7)  ≥65 1365 (11.1) 829 (10.1) 1274 (15.5) Sex  Female 8916 (72.7) 6456 (78.4) 5835 (70.9)  Male 3350 (27.3) 1782 (21.6) 2392 (29.1) Race/ethnicity  Hispanic 1123 (9.2) 286 (3.5) 1155 (14.0)  Non-Hispanic Black 4115 (33.5) 4085 (49.6) 461 (5.6)  Non-Hispanic Asian 748 (6.1) 151 (1.8) 162 (2.0)  Non-Hispanic White 5597 (45.6) 3089 (37.5) 5796 (70.5)  Other known 493 (4.0) 86 (1.0) 369 (4.5)  Unknown 190 (1.5) 541 (6.6) 284 (3.5) CCI score  0 (none) 9203 (75.0) 5788 (70.3) 5746 (69.8)  1 1631 (13.3) 1374 (16.7) 1341 (16.3)  ≥2 1432 (11.7) 1076 (13.1) 1140 (13.9) ADI  Lowest quartile 6896 (56.2) 1959 (23.8) 4648 (56.5)  Lower mid-quartile 3126 (25.5) 2892 (35.1) 3105 (37.7)  Upper mid-quartile 1497 (12.2) 2056 (25.0) 373 (4.5)  Highest quartile 747 (6.1) 1331 (16.2) 101 (1.2)

ADI indicates Area Deprivation Index; CCI, Charlson Comorbidity Index; KPCO, Kaiser Permanente Colorado; KPGA, Kaiser Permanente Georgia; KPMAS, Kaiser Permanente Mid-Atlantic States; VV, virtual visit.

Antidepressant Medication Orders by Period

After IPTW adjustment for covariates, in KPMAS there was a small but statistically significant decrease in antidepressant medication orders between pre and peak-pandemic period VVs (20.3% vs 19.0%, P=0.0235) and a small but significant increase in orders between peak and recovery period VVs (19.1% vs 21.9%, P=0.0053) (Table 2). KPCO followed the same pattern, with a small but statistically significant decrease between pre and peak-pandemic period VVs (17.6% vs 13.0%, P=0.0011) and a small but significant increase between peak and recovery-period VVs (13.1% vs 15.3%, P=0.0380). KPGA’s antidepressant medication orders did not change significantly across time periods.

TABLE 2 - Comparisons of Ancillary Services Rates Associated With the First Virtual Follow-up Visit for Major Depression, After an Initial Diagnostic Encounter, by Time Period Period comparisons*† Prepandemic vs peak pandemic Peak pandemic vs recovery Service type Prepandemic (%) Peak pandemic (%) P Peak pandemic (%) Recovery (%) P KPMAS  Antidepressant medication order 20.3 19.0 0.0235 19.1 21.9 0.0053  Antidepressant medication fulfillment (conditional on order) 79.7 78.2 0.5806 77.6 76.5 0.6147  Behavioral health symptom screener completion 43.2 68.4 <0.0001 68.6 81.0 <0.0001 KPGA  Antidepressant medication order 32.9 29.7 0.0654 29.6 27.3 0.0715  Antidepressant medication fulfillment (conditional on order) 63.0 61.2 0.5745 61.2 61.0 0.9435  Behavioral health symptom screener completion 12.1 19.0 <0.0001 19.3 46.4 <0.0001 KPCO  Antidepressant medication order 17.6 13.0 0.0011 13.1 15.3 0.0380  Antidepressant medication fulfillment (conditional on order) 78.4 70.3 0.0650 70.4 73.1 0.4202  Behavioral health symptom screener completion 20.1 29.9 <0.0001 30.6 35.5 0.0003

*Prepandemic: January 2019–March 2020; Peak pandemic: April 2020–June 2020; Recovery: July 2020–June 2021.

†Percentages for a given period may differ across time-period comparisons due to separate weighting for each comparison.

KPMAS indicates Kaiser Permanente Mid-Atlantic States; KPGA, Kaiser Permanente Georgia; KPCO, Kaiser Permanente Colorado.

Antidepressant Medication Order Fulfillment, Contingent on an Order, by Period

After IPTW adjustment for covariates and contingent upon an antidepressant medication order, none of the 3 regions showed changes in antidepressant medication order fulfillment from pre to peak-pandemic or peak to recovery (Table 2).

Symptom Screener Completion by Period

After IPTW adjustment for covariates, behavioral health symptoms screener completion increased significantly in all 3 regions between pre and peak-pandemic periods and increased again between peak and recovery periods (Table 2). KPMAS increased from 43.2% completion prepandemic to 81.0% completion during the recovery period; KPGA increased from 12.1% completion prepandemic to 46.4% completion during the recovery period, and KPCO increased from 20.1% completion prepandemic to 35.5% completion during the recovery period.

DISCUSSION

Behavioral health treatment for major depression in these 3 integrated health system regions of KP moved quickly from primarily in-person visits to almost entirely VVs in March 2020 and had not returned substantively to in-person care by the end of the study period in June 2021. Despite the abrupt change in the mode of care, our findings demonstrate that in 3 major areas of behavioral health care practice there were no sustained negative effects on care-related practices.

Although antidepressant medication orders declined significantly in 2 of the 3 regions during the peak-pandemic period, the magnitude of the declines was small and rates rebounded during the recovery period. None of the 3 regions experienced a significant change in patient fulfillment of ordered antidepressant medications across the study period. This differs from the larger changes in prescription fills for psychotropic medications seen early in the pandemic in other studies.7,8 The differences in findings may be due to the nature of the diagnoses involved, as our study focused on major depression rather than on broader mental health diagnoses8 or serious mental illness diagnoses such as schizophrenia and bipolar disorder.7 It is also possible that the steps that KP took to make prescription fills more accessible, including existing strategies such as mail-order, and new options such as same-day delivery or curbside pickup at clinics, were sufficient to sustain medication order fulfillment.

Completion of symptom screeners in service of measurement-based care increased significantly in all 3 regions with the onset of the pandemic and continued to increase significantly in the subsequent period. These were substantial and sustained increases, which diverged from the qualitative findings in prior literature that clinicians were having difficulty maintaining measurement-based care at the onset of the pandemic. Even before the pandemic, the 3 KP regions included in this study had the technological capacity to push out electronic versions of symptom screeners through the EHR-based patient portal, but completion rates for VVs were still modest. It is possible that once all care shifted to virtual, and all workflows and care patterns needed to adhere to this mode of care delivery, providers and patients were able to adapt to the new requirements for achieving virtual measurement-based care more successfully.

Limitations

This study has several limitations that restrict generalizability but point to future avenues of research. First, all data were drawn from 3 regions of KP’s integrated health system. Each region was able to invest substantial effort at the onset of the pandemic into training providers who had limited experience with virtual care and ensuring that they had access to resources to carry out VVs. Adjustment to virtual care may have been more challenging in smaller health care systems or provider groups with less resourced or mature information technology or EHR infrastructure.

Second, we limited the visits to the first follow-up after an incident diagnostic encounter, rather than the incident diagnostic encounter itself. This was done because prepandemic incident diagnostic encounters in behavioral health could not be scheduled as VV in the 3 regions. Nonetheless, antidepressant medication orders may have been placed at the incident diagnostic visit rather than at follow-up, thus the data on medication orders and fulfillment may represent a subset of the patients who only received a medication order on follow-up, or who were changing medication on follow-up. It would be valuable for future research to examine whether prescribing patterns and rates for antidepressants at virtual incident diagnostic encounters changed with the move to virtual care.

Finally, we looked only at patterns of behavioral health-care-related practices, not clinical severity, health outcomes, or patient and provider perspectives on the experience. Research should explore whether the clinical severity of depression affected patient engagement in VV during the pandemic. In a recent qualitative study25 psychiatrists reported that the decision to engage in VV with a patient during the peak of the pandemic was driven primarily by patient preference and logistical issues, rather than by elements of the patient’s diagnosis, but quantitative measures of severity could further illuminate the issue. Future research should also examine whether measurement-based care in particular afforded the same positive treatment experiences and outcomes in the pandemic transition to VV that it has for prepandemic in-person care.

CONCLUSION

In many health care settings, behavioral health was increasing the capacity for VVs before the COVID-19 pandemic forced an abrupt transition to fully virtual care. Within these 3 regions of KP, the fact that care has not rebounded to include substantial in-person visits suggests that VVs have proven to be a successful model for providers and patients. Our data demonstrate that a rapid shift to virtual behavioral health care was possible without compromising health-care-related practices. The transition and subsequent adjustment period have instead been marked by improved adherence to measurement-based care practices in VVs, signaling a potential new capacity for virtual health care delivery.

REFERENCES 1. Myers US, Coulon S, Knies K, et al. Lessons learned in implementing VA video connect for evidence-based psychotherapies for anxiety and depression in the Veterans Healthcare Administration. J Technol Behav Sci. 2021;6:320–326. 2. Rosen CS, Morland LA, Glassman LH, et al. Virtual mental health care in the Veterans Health Administration’s immediate response to coronavirus disease-19. Am Psychol. 2021;76:26. 3. Donelan K, Barreto EA, Sossong S, et al. Patient and clinician experiences with telehealth for patient follow-up care. Am J Manag Care. 2019;25:40–44. 4. Giovanetti AK, Punt SE, Nelson EL, et al. Teletherapy versus in-person psychotherapy for depression: a meta-analysis of randomized controlled trials. Telemed J E Health. 2022;28:1077–1089. 5. Drago A, Winding TN, Antypa N. Videoconferencing in psychiatry, a meta-analysis of assessment and treatment. Eur Psychiatry. 2016;36:29–37. 6. Shigekawa E, Fix M, Corbett G, et al. The current state of telehealth evidence: a rapid review. Health Aff. 2018;37:1975–1982. 7. Busch AB, Huskamp HA, Raja P, et al. Disruptions in care for medicare beneficiaries with severe mental illness during the COVID-19 pandemic. JAMA Netw Open. 2022;5:e2145677. 8. Zhang J, Boden M, Trafton J. Mental health treatment and the role of tele-mental health at the Veterans Health Administration during the COVID-19 pandemic. Psychol Serv. 2022;19:375–385. 9. Lewis CC, Boyd M, Puspitasari A, et al. Implementing measurement-based care in behavioral health: a review. JAMA psychiatry. 2019;76:324–335. 10. Duncan BL, Reese RJ, Lengerich AJ, et al. Measurement-based care in integrated health care: a randomized clinical trial. Fam Syst Health. 2021;39:259. 11. Guo T, Xiang YT, Xiao LE, et al. Measurement-based care versus standard care for major depression: a randomized controlled trial with blind raters. Am J Psychiatry. 2015;172:1004–1013. 12. Gondek D, Edbrooke-Childs J, Fink E, et al. Feedback from outcome measures and treatment effectiveness, treatment efficiency, and collaborative practice: a systematic review. Adm Policy Ment Health. 2016;43:325–343. 13. Friedman AB, Gervasi S, Song H, et al. Telemedicine catches on: changes in the utilization of telemedicine services during the COVID-19 pandemic. Am J Manag Care. 2022;28:e1–e6. 14. Poeran J, Cho LD, Wilson L, et al. Pre-existing disparities and potential implications for the rapid expansion of telemedicine in response to the coronavirus disease 2019 pandemic. Med Care. 2021;59:694–698. 15. Whaley CM, Pera MF, Cantor J, et al. Changes in health services use among commercially insured US populations during the COVID-19 pandemic. JAMA Netw Open. 2020;3:e2024984. 16. Ritzwoller DP, Goodrich G, Tavel H, et al. Patient factors associated with use of adult primary care and virtual visits during the COVID-19 pandemic. Med Care. 2023;61(suppl 1):S12–S20. 17. Centers for Medicare and Medicaid Services. COVID-19 Emergency Declaration Blanket Waivers for Health Care Providers. March 13, 2020, updated May 14, 2021. Accessed April 27, 2022. https://www.cms.gov/files/document/summary-covid-19-emergency-declaration-waivers.pdf 18. McCracken CE, Gander JC, McDonald B, et al. Impact of COVID-19 on trends in outpatient clinic utilization: a tale of 2 departments. Med Care. 2023;61:S4–S11. 19. Kroenke K, Spitzer RL, Williams JB. The PHQ‐9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16:606–613. 20. Spitzer RL, Kroenke K, Williams JB, et al. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166:1092–1097. 21. Posner K, Brown GK, Stanley B, et al. The Columbia–Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry. 2011;168:1266–1277. 22. Charlson M, Szatrowski TP, Peterson J, et al. Validation of a combined comorbidity index. J Clin Epidemiol. 1994;47:1245–1251. 23. Knighton AJ, Savitz L, Belnap T, et al. Introduction of an Area Deprivation Index measuring patient socioeconomic status in an integrated health system: implications for population health. EGEMS (Wash DC). 2016;4:1238. 24. Austin PC, Stuart EA. Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies. Stat Med. 2015;34:3661–3679. 25. Uscher-Pines L, Parks AM, Sousa J, et al. Appropriateness of telemedicine versus in-person care: a qualitative exploration of psychiatrists’ decision making. Psychiatr Serv. 2022;73:849–855.