Abstract

Background: Mobile mental health apps are a cost-effective option for managing mental health problems, such as posttraumatic stress disorder (PTSD). The efficacy of mobile health (mHealth) apps depends on engagement with the app, but few studies have examined how users engage with different features of mHealth apps for PTSD.

Objective: This study aims to examine the relationship between app engagement indices and PTSD symptom reduction using data from an unblinded pilot randomized controlled trial of “Renew” (Vertical Design), an exposure-based app for PTSD with and without coaching support. Because exposure is an effective approach for treating PTSD, we expected that engagement with exposure activities would be positively related to symptom reduction, over and above overall app usage.

Methods: Participants were veterans (N=69) with clinically significant PTSD symptoms who were recruited online using Facebook advertisements and invited to use the Renew app as often as they wanted over a 6-week period. Participants completed screening and assessments online but provided informed consent, toured the app, and completed feedback interviews via telephone. We assessed users’ self-reported PTSD symptoms before and after a 6-week intervention period and collected app usage data using a research-instrumented dashboard. To examine overall app engagement, we used data on the total time spent in the app, the number of log-in days, and the number of points that the user gained in the app. To examine engagement with exposure components, we used data on total time spent completing exposure activities (both in vivo and imaginal), the number of in vivo exposure activities completed, and the number of characters written in response to imaginal exposure prompts. We used hierarchical regression analyses to test the effect of engagement indices on change in PTSD symptoms.

Results: Usage varied widely. Participants spent an average of 166.09 (SD 156.52) minutes using Renew, over an average of 14.7 (SD 10.71) mean log-in days. Engagement with the exposure components of the app was positively associated with PTSD symptom reduction (F6,62=2.31; P=.04). Moreover, this relationship remained significant when controlling for overall engagement with the app (ΔF3,62=4.42; P=.007). The number of characters written during imaginal exposure (β=.37; P=.009) and the amount of time spent completing exposure activities (β=.36; P=.03) were significant contributors to the model.

Conclusions: To our knowledge, this is the first study to show a relationship between symptom improvement and engagement with the active therapeutic components of an mHealth app (ie, exposure) for PTSD. This relationship held when controlling for overall app use, which suggests that it was engagement with exposure, specifically, that was associated with symptom change. Future work to identify ways of promoting greater engagement with self-guided exposure may help improve the effectiveness of mHealth apps for PTSD.

Trial Registration: ClinicalTrials.gov NCT04155736; https://clinicaltrials.gov/ct2/show/NCT04155736

JMIR Mhealth Uhealth 2024;12:e49393

doi:10.2196/49393

Keywords
IntroductionOverview

Mobile mental health apps have continued to proliferate in recent years []. As of last year, more than 80 mobile mental health (mHealth) apps aimed at managing posttraumatic stress disorder (PTSD)—a significant public health concern with an estimated 7% lifetime prevalence in the United States of America—are now available for download [,]. Research on mHealth apps for PTSD has also increased and suggests that these tools are generally feasible and acceptable to users []. However, a recent review found that evidence for the efficacy of PTSD mHealth apps was not consistently superior to inactive control conditions [].

The efficacy of an mHealth app for PTSD will depend upon how the user engages with the tool. Only one study has examined the relationship between engagement with a PTSD self-management app and symptom change. This study found that the number of log-ins to the mHealth app PTSD Coach (Vertical Design) was not related to PTSD symptom reduction []. This contrasts with findings from depression mHealth apps showing that number of log-ins [] and number of activities per log-in [] are linked with change in depressive symptoms. Additional research on the relationship between app engagement and PTSD symptom change is clearly needed and could inform user guidance on how to benefit from using PTSD mHealth self-management apps.

Objectives

The current analysis aims to address this gap by exploring the relationship between user engagement indices and symptom reduction in users of a PTSD mHealth self-management app called “Renew” [,]. Based on principles of exposure therapy, Renew uses gamified elements and activities designed to help users target their PTSD symptoms using in vivo and written exposure. Given the evidence supporting exposure as an efficacious therapeutic approach for treating PTSD [], we hypothesized that engagement with the exposure activities in Renew would be uniquely related to symptom reduction, over and above general app use.

MethodsParticipants

Potential participants were recruited through Facebook (Meta) advertisements because we were interested in examining a nationally representative sample of veterans. As of the Veterans Health Administration’s last publicly released report, more than half of veterans do not receive care through the Department of Veterans Affairs (VA) [], so a sample recruited strictly from among VA patients would not necessarily represent veterans as a whole. Potential participants completed an online screening measure through REDCap (Research Electronic Data Capture; Vanderbilt University) [] to assess veteran status, trauma exposure [], and PTSD symptoms. Eligible participants were veterans with clinically significant PTSD symptoms, defined by a score of ≥31 on the Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5) []; access to an Android phone (Renew was only developed for Android OS); and sufficient smartphone literacy and internet connection to successfully download the app with telephone guidance from study staff. There were no exclusion criteria.

Ethical Considerations

All participants gave their informed consent for inclusion before they participated in this trial. The protocol was approved by the Stanford University Institutional Review Board (52829).

Renew

Renew is a research-instrumented mobile app designed to reduce symptoms of PTSD using in vivo exposure and imaginal exposure, developed by Vertical Design. Other components include psychoeducation, behavioral activation through self-care, and assessment. Users gain points for completing activities in the app. Renew was designed to be used both with and without human coaching support, provided through a one-way messaging function. Users downloaded Renew from the Google Play Store and accessed the intervention using an invitation code provided by the study team during app orientation. Readers of this article who possess an Android phone may email the corresponding author to receive a Renew invitation code if they wish to access the intervention. Before this trial, we tested the feasibility and acceptability of Renew in a small pilot study, after which the content of Renew was updated according to user feedback and frozen throughout the duration of the current trial. Miller et al [] and McLean et al [] give more details about Renew, including the details of the feasibility and acceptability pilot study.

MeasuresRenew Usage Data

Participant usage data were collected through a secure research dashboard. User access to the app was restricted to only those users who were provided with an individualized code. Once linked with a given user’s app, this code allowed researchers to track the user’s engagement with the Renew app through the secure research dashboard, including time-stamped button presses, time spent on each screen, number of characters typed into free text boxes, log-ins, and dates of use.

Consistent with previous mHealth app studies that have used multiple objective engagement criteria [], we examined several indices of overall app engagement: total time spent in Renew, number of log-in days, and number of points gained in Renew. To specifically capture engagement with exposure, we examined time spent completing exposure activities (both imaginal and in vivo), the number of in vivo exposure activities completed, and the number of characters written during imaginal exposure. Number of characters was chosen as a proxy for the degree to which users engaged with their worst traumatic memory.

Posttraumatic Stress Symptoms

The PCL-5 [] is a 20-item self-report measure of PTSD symptoms as defined by the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) []. The PCL-5 has demonstrated strong psychometric properties [,] and has been validated for online use []. The Cronbach α in the current sample was 0.84. The PCL-5 was administered to each participant at 3 time points. For participants immediately assigned to one of the active use conditions, the PCL-5 was completed through REDCap before app use as part of the study screening procedures, after the 6-week active use period, and after a 6-week follow-up period. Participants assigned to the delayed use condition completed the PCL-5 before the delayed use period, at the end of the delayed use period before receiving a code to access Renew, and after their 6-week active use period.

Procedures

The Stanford University Institutional Review Board approved the study. Informed consent was collected via telephone call, in which the study staff read a script that included information about Renew and the 3 study conditions to which they could be randomized. Study staff then allowed participants to ask questions about the study before providing consent. Eligible participants who consented to the study were randomly assigned to either 6 weeks of (1) Renew without study staff support, (2) Renew with a study support person as a support team member, or (3) delayed use. Neither participants nor study staff were blinded to conditions. Participants in the delayed use group were invited to use the app for 6 weeks after a 6-week wait period. Immediately before beginning the 6-week active use period, participants received a 30-minute app orientation by phone. During the orientation, a study staff member walked participants through each section of the app, described the features of Renew, and offered participants the chance to ask questions. Following the orientation, participants were then asked to use the app as often as they wanted for the next 6 weeks. Participants could choose to set reminders to use the app. Participants completed postuse measures through the REDCap online survey platform [] at the end of the app-use period. Usage data were collected using a secure research dashboard. All participants who were assigned to the delayed use condition were given an access code for Renew at the end of the 6-week delayed use period. Data about their app usage was collected during a 6-week follow-up period, in addition to pre- and postuse symptom measures. For the current analysis, data were pooled from participants assigned to all 3 conditions, because there were no significant (P=.31) differences in PTSD symptom change during the active use periods across groups []. The majority of data collection took place online or through the online research dashboard that recorded Renew usage data, however, participants did complete postuse feedback interviews with study staff via telephone. Of the 93 participants who enrolled in the parent pilot trial [], 69 participants had completed both pre- and postuse symptom measures and were included in this study.

Data Analysis

Analyses were conducted using SPSS (version 26; IBM Corp) []. Descriptive and correlational analyses were conducted for all engagement variables. A hierarchical linear regression analysis was conducted with the overall app engagement variables entered in step 1 and exposure engagement variables entered in step 2. The dependent variable was change on the PCL-5. All app use data in the analysis came from the participants’ active use period (ie, for participants initially assigned to delayed use, usage data was taken from the active use period that followed the delayed use period).

Results

Participant demographics are reported in . Participants were predominantly middle-aged, women, and White. More than half of participants in this subsample were army veterans (n=36, 52.17%), and the average PCL-5 score was well above the cutoff for clinically significant PTSD symptoms.

Means and SDs for user engagement variables are presented in .

Table 1. Participant demographic characteristics (N=69).VariableValueAge (years), mean (SD)45.28 (9.05)Gender, n (%) Men22 (33.33) Women47 (66.67)Race or ethnicity, n (%) White47 (68.12%) Black or African American13 (18.84) Asian or Asian American2 (2.89) American Indian or Alaska Native4 (5.8) Middle Eastern or North African1 (14.49) Other2 (2.89)Military branch, n (%) Army36 (52.17) Navy13 (18.84) Air force15 (21.74) Marines5 (7.25)Baseline PCL-5 score, mean (SD)55.94 (9.87)

aNo participants identified as transgender, nonbinary, or another gender identity.

bParticipants could select more than one race or ethnicity. Because of this, percentages sum to >100%.

cPCL-5: Posttraumatic Stress Disorder Checklist for DSM-5.

Table 2. Descriptive statistics for app engagement indices.Overall engagement variablesValue, mean (SD)Log-in days14.7 (10.71)Minutes in Renew166.09 (156.52)Number of points gained350.14 (451.42)Exposure engagement variables Number of in vivo exposure activities completed1.84 (4.22) Minutes spent completing exposure activities78.19 (135.35) Number of characters written during imaginal exposure activities5116 (8522.43)

Over the 6-week active use period, participants logged into Renew about one-third of the days. Users spent a little over 2.5 hours total engaging with Renew, not including their initial 30-minute orientation. Users spent over an hour engaging in exposure activities, averaging more than one complete in vivo exposure activity. In response to exposure prompts, participants wrote over 5000 characters on average, which roughly equals 750‐1000 words.

Correlations between app usage variables are presented in . Given the overlap in the engagement variables, there was high intercorrelation between the general and exposure-specific usage variables. Relationships between all variables were significant. The variable “minutes spent completing exposure activities” reflects minutes spent practicing both imaginal and in vivo exposure. Time spent on imaginal and in vivo exposure were combined in order to examine the impact of overall engagement with exposure in addition to the individual effects of engagement with imaginal and in vivo alone.

Table 3. Correlations (Pearson r) among usage variables.VariablesLog-in daysMinutes in RenewNumber of points gainedNumber of in vivo exposure activities completedMinutes spent completing exposure activitiesNumber of characters written during imaginal exposureOverall usage variables Log-in days  r10.740.810.470.410.45  P value—<.001<.001<.001<.001<.001 Minutes in Renew  r0.7410.790.640.650.54  P value<.001—<.001<.001<.001<.001 Number of points gained  r0.810.7910.580.410.46  P value<.001<.001—<.001<.001<.001Exposure engagement variables Number of in vivo exposure activities completed  r0.470.640.5810.580.33  P value<.001<.001<.001—<.001.01 Minutes spent completing exposure activities  r0.410.650.410.5810.38  P value<.001<.001<.001<.001—0 Number of characters written during imaginal exposure  r0.450.540.460.330.381  P value<.001<.001<.001.010—

aNot applicable.

The results of the regression analysis are presented in . The initial model with overall engagement predicting PTSD symptom change was not significant and accounted for 1% of the variance observed in the model. However, the model with the exposure engagement variables added was significant and accounted for an additional 18% of variance in the model. Time spent completing exposure activities (P=.03) and the number of characters written during imaginal exposure (P=.009) were significant contributors to the final model.

Table 4. Hierarchical linear regression model (N=69).Model steps and variablesβRR2ΔR2F test (df)P valueΔF test (df)P valueStep 1: overall engagement—0.10.01—0.18 (3,65)0.9——Log-in days0.01————0.97——Time in Renew0.01————0.64——Number of points gained−0.02————0.53——Step 2: exposure engagement—0.440.190.182.31 (6,62)0.044.42 (3,62)0.01Number of in vivo exposure activities completed−0.03————0.86——Time spent completing exposure activities0.36————0.03——Number of characters written during imaginal exposure0.37————0.01——

aβ: Standardized beta weight.

bSignificance for standardized beta weights of predictors and steps of the model (F test statistic).

cSignificance for change in F test statistic (ΔF test) at second step.

dNot applicable.

DiscussionOverview

We examined the relationship between engagement with imaginal and in vivo exposure and PTSD symptom reduction in an mHealth app, Renew. Few randomized controlled trials have examined mobile app-delivered interventions for PTSD [], and even fewer studies have evaluated the impact of usage factors on symptom change in digital PTSD interventions [,]. As hypothesized, we found that engagement with exposure was associated with greater PTSD symptom reduction over and above overall use of the app in general. Our finding that overall app use was unrelated to PTSD symptom change is consistent with previous research on PTSD Coach [] but inconsistent with meta-analytic findings showing that engagement with mHealth apps in general is associated with symptom change []. To our knowledge, this is the first study to demonstrate a relationship between engagement with the active therapeutic components of an mHealth app and improvement in PTSD symptoms.

Our findings are consistent with the limited research on the relationship between app usage and symptom change in self-management apps for depression and anxiety, which suggests that higher rates of engagement with active intervention components are associated with symptom change. Specifically, Zhang et al [] found that participants’ depressive symptom reduction was associated with greater engagement in self-tracking and moderate engagement with learning and goal setting, but anxiety symptom reduction was unrelated to engagement with specific app components. One other study, Nardi et al [], examined data from a clinical trial of a mindfulness app and also found evidence that engagement with specific components, such as meditation, was linked to improvement in generalized anxiety symptom reduction. To our knowledge, there are no studies of PTSD apps that have examined how engagement with different intervention components relates to symptom change. Even among apps for more common mental health concerns, robust reporting on usage metrics remains scant []. Our analysis of exposure-specific use metrics in this study contributes to a profound gap in the literature on digital mental health interventions. Our finding that exposure may be helpful for reducing PTSD symptoms through a self-guided mHealth app is consistent with the broader literature on efficacious PTSD interventions []. Future work should examine digital interventions that include active cognitive behavioral therapy (CBT) components such as exposure, which are associated with symptom change. Indeed, in a large meta-analysis of 176 randomized controlled trials, effect sizes were larger for self-management apps for anxiety and depression that included CBT components compared to those that did not [].

The number of characters entered during imaginal exposure, which we considered a proxy for the degree to which participants engaged with and recounted their worst traumatic memory, and the time spent in any exposure activity, both significantly contributed to the model predicting PTSD symptom change, whereas the number of exposure activities did not. This may suggest that the depth of engagement and time spent approaching trauma-related memories and situations is important for therapeutic change in the context of mHealth apps for PTSD. This pattern of findings maps to the literature on in-person psychotherapy, where most exposure therapies emphasize the importance of imaginal exposure [].

Limitations

Several study limitations should be noted. Data for this study were drawn from a pilot trial, and replication of the current findings is needed. Participants were veterans with clinically significant symptoms of PTSD; findings may not generalize to other populations diagnosed with PTSD. Data were cross-sectional, which prevents causal interpretations. In addition, many of the usage variables were highly intercorrelated and there is no established gold standard approach for operationalizing app engagement. These limitations notwithstanding, this study provides preliminary evidence of a unique relationship between engagement with exposure and PTSD symptom change among veterans with clinically significant symptoms of PTSD who used a self-guided mHealth app (Renew) for a 6-week period.

The authors would like to thank Vertical Inc and Yuzu Labs Inc for their assistance with app development and participant recruitment, respectively. The views expressed herein are solely those of the authors and do not reflect an endorsement by or the official policy or position of the Department of Veterans Affairs or the US government. This material is the result of work supported with resources and the use of facilities at the VA Palo Alto Health Care System. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

None declared.

Edited by Lorraine Buis; submitted 26.05.23; peer-reviewed by Maha Gasmi, Rebecca Sripada, Yang Pachankis; final revised version received 02.04.24; accepted 22.04.24; published 18.07.24.

© C Adrian Davis, Madeleine Miller, Carmen McLean. Originally published in JMIR mHealth and uHealth (https://mhealth.jmir.org), 18.7.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on https://mhealth.jmir.org/, as well as this copyright and license information must be included.