IntroductionBackground

Digital phenotyping is “the moment-by-moment quantification of the individual level human phenotype in situ using data from personal digital devices” []. Digital phenotyping applies the concept of phenotypes, in other words, the observable characteristics resulting from the genotype and environment, to conceptualize observable patterns in individuals’ digital data. In the last decade, digital phenotyping studies have been able to compare typical and atypical patterns in daily activities to correlate atypical behavior with negative emotions [,]. Behavioral patterns include variations in mobility, frequency of being in various locations, and sleep patterns. In smartphones, user data can be stored, managed, interpreted, and captured in enormous amounts [,,]. This can be done actively or passively. Active data collection requires the user to self-report and complete surveys, whereas passive sensing collects data automatically without user input []. Most studies combine active and passive sensing to more accurately detect and predict behavioral abnormalities. Modern smartphone analytics can be used for the discovery of commonalities and abnormalities in user behavior. The ease of using passive sensing makes it an ideal data gathering method for mental health studies [-] and an ideal technique for assessing mental health [].

Digital phenotyping has been successful in the early detection and prediction of behaviors related to neuropharmacology []; cardiovascular diseases []; diabetes []; and major severe injuries, such as spinal cord injury [], motivating further adoption. Digital phenotyping has also proven useful for the detection of severe mental health issues, such as schizophrenia [,], bipolar disorder [], and suicidal thoughts []. Digital phenotyping has been so successful for specialized, clinical populations that it is increasingly considered for mass market use with nonclinical populations. Digital phenotyping applications and software tools have been used to capture employee information, such as their screen time and clicking patterns []. However, there are not many digital phenotyping studies that have specifically examined the detection or prediction of stress, anxiety, and mild depression.

Individuals with stress, anxiety, and mild depression can develop chronic mental health symptoms that impact their mobility, satisfaction with life, and social interaction [,]. When these symptoms are not detected early, they worsen, and the impact is more significant [-], increasing the need for medication and hospitalization. This makes mild mental health symptoms a valid target for digital phenotyping, as its goal is to enable early detection and, subsequently, early treatment. Smartphones are increasingly ubiquitous [], which makes them an optimal platform for digital phenotyping. We constrained our systematic literature search to the more challenging problem of the detection of mild mental health symptoms using only smartphone sensors and excluded studies that used additional wearable sensors. In general, we believe that additional wearables might increase the effectiveness of digital phenotyping in detecting stress, anxiety, and mild depression. Given the ubiquity of smartphones, we aimed to answer the following question: what is the effectiveness of digital phenotyping using smartphone sensors in detecting stress, anxiety, and mild depression?

Objectives

The objective of this systematic literature review was to better understand the current uses of digital phenotyping and results of using digital phenotyping for the detection and prediction of mild behavioral patterns related to stress, anxiety, and mild depression. The 2 research questions this review sought to answer were as follows:

What is the evidence of the effectiveness of digital phenotyping using smartphones in identifying behavioral patterns related to stress, anxiety, and mild depression?In particular, which smartphone sensors are found to be effective, and what are the associated challenges?

For these research questions, we considered statistically significant associations between sensor patterns and behavioral patterns as evidence of effectiveness.

MethodsType of Studies

This review followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [] (). shows the reviewing process and search results. In the first round of screening studies, 1 author excluded studies that were not relevant to the research questions. Another author reran the queries for confirmation. Studies were included in this review if they were conducted to measure and detect stress, anxiety, or mild depression, even if they included other variables, such as job performance, promotion, or discrimination. We included studies in which data were collected through smartphones with an iOS (Apple Inc) or Android (Google LLC) operating system. Data collected through wearable devices were excluded. We included studies in which the participants were adults aged ≥18 years and were from a nonclinical population. Studies conducted with nonadult participants (eg, teenagers and children) were excluded. Given our research questions, if the studies’ participants had or had had any severe mental health disorder, such as schizophrenia, bipolar disorder, or psychosis, they were not included. We also excluded personality and character measurement and phobia studies. The primary research language was English. The studies included were conducted from September 2010 to September 2023. Peer-reviewed conference articles and journal articles were included. The data we wished to extract were the study aim, data collected, operating system in the smartphone used for data collection, behavioral patterns identified, surveys used for verification, and sample size. A total of 3 authors reviewed the studies independently to extract data and confirm the extracted data. After the first round of data extraction, 1 author re-examined the studies to extract the predictive modeling used. These data are presented in the Results section. We noticed that participants in the included studies fell into 1 of 3 major groups (ie, students, adults, and employees). We refer to the participants of the studies that recruited adults enrolled in universities as “students,” participants of the studies that recruited adults unaffiliated to any particular organization as “adults,” and participants of the studies that recruited adults employed at a particular organization as “employees.”

‎Figure 1. Systematic literature reviewing process and search results with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) diagram. Search Strategy

A total of 3 databases were queried: Web of Science, ACM, and PubMed. PubMed is a medicine-based database, ACM is a technology-based database, and Web of Science is a cross-domain database. The search query was the same for the 3 platforms: “digital phenotyping” OR “passive sensing” AND (stress OR anxiety OR ((mild OR moderate) AND depression)).

ResultsDuration

The study length varied from 10 days [] to 3 years []. One study [] conducted in-depth interviews with students lasting an average of 4.5 hours per person, and another study was a controlled laboratory study []. These 2 studies are not presented in . In the studies conducted with students, a semester or spring or winter term was a common duration. The studies with general nonclinical adult populations were typically longer than those with students.

Table 1. Duration of the reviewed studies (N=38; 2 studies are excluded, as 1 [] is interview based and the other [] is a controlled laboratory study).Study, yearLength of the study (d)Adams et al [], 201410Cai et al [], 201814Boukhechba et al [], 201814Di Matteo et al [], 202114Jacobson et al [], 202016Wen et al [], 202121Melcher et al [], 202328Fukuzawa et al [], 201928Rashid et al [], 202035Zakaria et al [], 201935DaSilva et al [], 201943Nepal et al [], 202060Saha et al [], 201968Morshed et al [], 201970Acikmese et al [], 201970Zakaria et al [], 201981Zakaria et al [], 201981Boukhechba et al [], 201798Tseng et al [], 201698Morshed et al [], 201998Xu et al [], 2019106Chikersal et al [], 2021112Meyerhoff et al [], 2021112Xu et al [], 2019113Rhim et al [], 2020121Wang et al [], 2018121Currey and Torous [], 2022147Di Matteo et al [], 2021153Sefidgar et al [], 2019153Mendu et al [], 2020153Pratap et al [], 2017181Mirjafari et al [], 2019260Currey et al [], 2023336Huckins et al [], 2020458Mack et al [], 2021458Xu et al [], 2023458Nepal et al [], 2022730Servia-Rodríguez et al [], 20171095Number of Participants

The number of participants ranged from a minimum of 7 adults [] to a maximum of 18,000 adults []. Apart from the 3-year longitudinal study with 18,000 participants [], the average number of participants was 129.4 (SD 184.01). We observed a pattern of attrition, where the number of participants who completed the study was lower than the number of the participants recruited. The number of participants reported in this review is the final sample size. For example, one of the studies [] recruited 112 participants, of whom 84 (75%) completed the study. In the study by Pratap et al [], there was a drastic drop in participants, with only 359 (30.42%) of the 1180 enrolled participants completing the study. Another significant drop was seen in the study by Nepal et al [], where 750 participants were interested in the research, whereas only 141 (18.8%) of them completed the study. Some studies were less affected; for example, 86 participants started the study by Rhim et al [], and 78 (91%) completed it.

Publication Years of the Studies

Although the query started with the year 2010, the earliest publication was from 2014 [], extending to articles published as of April 2023 []. Over the years, the interest in detecting and predicting stress, anxiety, and mild depression in the nonclinical population has increased ().

Table 2. Number of reviewed reports (N=36) by year.YearPublication, n (%)20141 (3)20161 (3)20173 (8)20184 (11)201910 (28)20206 (17)20216 (17)20222 (6)20233 (8)Studies With the iOS and Android Operating Systems

The Android operating system was more common than iOS. Among the 40 included studies, only 2 (5%) were compatible with only iOS [,]. A total of 27 (68%) studies were available for both iOS and Android [,,,,,-,-,,-]. A total of 11 (28%) studies were for only Android users [,-,,-,,,]. The reasons identified for the use of the Android operating system were that it has more freedom to capture more modalities, such as keyboard typing and use of apps, and that Android devices enable apps to run more easily in the background [].

Studies With Students

presents the data extracted from the studies that were conducted with student populations. The average length of the studies with students was 158.6 (SD 176.4) days. The average number of participants was 137.3 (SD 152.1). There were significantly more studies with students than studies with employees or general adults. The sample sizes of the studies with students were similar to those of the studies with adults but smaller than those of the studies with employees. In the studies with students, various passive sensors were used, and some were found to be effective for detection, prediction, or both.

Of the 28 studies with students, 23 (82%) used machine learning models for prediction. A total of 12 studies (43%) [,,,,,,,,] used decision tree–based methods, and 9 studies (32%) [,,,-,,] used regression-based methods. A total of 3 (11%) studies conducted in recent years [,,] used deep neural networks because of their enhanced ability to discern underlying patterns in large unstructured data sets. Tree-based models have the best performance when trained with structured data, and the reported studies mostly used tree-based models and structured data. Among the 28 studies, 2 studies [,] conducted in 2023 addressed the generalizability of their proposed detection method and verified its applicability across students from various years, classes, and institutions. Two (7%) studies [,] in used the StudentLife data set []. Each study contributed substantial original analyses including different behavioral patterns and was considered a “study” in this systematic review. Entries with “N/A” in the predictive modeling column indicate that the study did not involve any attempts to predict future occurrences. However, these studies may still contain statistical analyses as part of their research approach. Overall, students who experienced depression, anxiety, and stress visited fewer locations [,,,-] and were more sedentary [,,-]. Depression was also associated with shorter or irregular sleep [,,,,,,] and accrued phone use [,,,,-].

Table 3. Summary of the reviewed studies with student participants.Study, yearAimData collectedOperating systemBehavioral patternsPredictive modelingVerification surveysSample size, nHuckins et al [], 2020Understand how students’ behavioral health and mental health are affected by the COVID-19 pandemicGPS, accelerometer, phone lock and unlock, and light sensor dataiOS (Apple Inc) and Android (Google LLC)At the start of the COVID-19 pandemic, students were more depressed and anxious, used their phones more, visited fewer locations, and spent more time sedentary. Depression and stress were associated with increasing COVID-19–related news coverage.Linear regressors were used to inspect how behavioral changes were affected by COVID-19 news reports.PHQa-4217 studentsMelcher et al [], 2023Understand how behavioral patterns correlate with mental health for students during the COVID-19 pandemicGPS, accelerometer, call log, and phone use dataiOS and AndroidIndividuals with more irregular sleep patterns had worse sleep quality and were experiencing more depression and more stress than those with consistent sleep patterns.N/AbPHQ-9, DASSc, SIASd, GAD-7e, PQf, PSSg, PSQIh, BASISi, SFj-36, SFSk, Flourishing Scale, CGIl, HDRSm, CASn, HAIo, and UCLAp-Loneliness Scale100 studentsJacobson et al [], 2020Predict social anxiety symptom severity and discriminate between depression, negative affect, and positive affectAccelerometer, call log, and SMS text message dataAndroidMeasures of SMS text message and call response time discriminated among depression, negative affect, and positive affect. Accelerometer patterns suggested that persons with low social anxiety walked at a steady pace, whereas persons with high social anxiety walked more quickly with more irregularity.XGBoostq with LOOCVr was used to predict social anxiety symptom severity.SIAS, DASS-21, and PANASs59 studentsDaSilva et al [], 2019Predict stressGPS, accelerometer, phone lock and unlock, microphone, and light sensor dataiOS and AndroidStudents with stress were more likely to spend less time in campus food locations and more time in schoolwork locations. Students with stress traveled less, engaged in fewer conversations, and were in quieter environments during evenings.Penalized generalized estimating equations were used to prune features and fit a marginal regression model to predict stress.MPSMt94 studentsAcikmese and Alptekin [], 2019Predict stress levelAccelerometer, microphone, Bluetooth, light sensor, phone lock and unlock, phone charge, and app use data (GPS and Wi-Fi data were collected but not used)AndroidStudents were successfully categorized as stressed or nonstressed using the measured sensors.LSTMu, CNNv, and CNN-LSTM were used to classify stress, with LSTM yielding the best accuracy.Self-reported stress48 studentsRooksby et al [], 2019Understand students’ perspectives about digital phenotypingGPS, phone lock and unlock, phone charge, battery, microphone, Bluetooth, light sensor, SMS text message, email, app use, call log, camera, and keyboard dataiOS and AndroidNone of the results related sensors to symptoms of depression or anxiety. Students have privacy concerns regarding the use of app use logs, Bluetooth data, call logs, camera data, keyboard data, and microphone data but not regarding the use of battery, or light sensor. Students had privacy concerns with the use of SMS text message content but not with counts of messages.N/APHQ-9, GAD-7, and WEMWBSw15 studentsChikersal et al [], 2021Predict postsemester depressive symptomsGPS, accelerometer, Bluetooth, Wi-Fi, phone use, call log, and microphone dataiOS and AndroidDepression was predicted by participants’ social context in the afternoons and evenings, phone use throughout the day, long periods without exercise, periods of disturbed sleep at night, and time spent outdoors.Trained an ensemble classifier with the outputs from models containing features from 1 sensor, with different setting combinations.BDIx-II138 studentsMorshed et al [], 2019Predict mood instabilityAccelerometer, microphone, Bluetooth, light sensor, Wi-Fi, GPS, phone lock and unlock, and phone charge dataAndroidMood instability was negatively correlated with the duration of sleep, the number of conversations, the amount of activity, and outdoor mobility.Ridge regression with regularization was used to infer mood instability score.EMAsy, PAMz, and PANAS48 studentsZakaria et al [], 2019Detect depression and stressWi-Fi dataiOS and AndroidStudents with severe stress spent significantly less time on campus and were less involved in work-related activities than students with normal stress. Students with severe stress were more involved in these activities at the start of the semester, but the involvement decreased over time.The random forest stress model with domain-specific features achieved the best result, with feature sets changed every 6 days.PSS-4, PHQ-8, and BFIaa62 studentsZakaria et al [], 2019Detect depression and stressWi-Fi dataiOS and AndroidSame patterns as those mentioned earlier.The random forest model that excluded domain-specific features achieved the best result, with feature sets changed every 6 days.PSS-4, PHQ-8, and BFI11 studentsZakaria et al [], 2019Detect depression and stressWi-Fi dataiOS and AndroidSame patterns as those mentioned earlier.The best model is a random forest model with the neuroticism score added as an additional feature, with sensor data sets calculated with a 6-day interval.PSS-4, PHQ-8, and BFI35 studentsWang et al [], 2018Predict depressionLight sensor, GPS, accelerometer, microphone, screen on and off, and phone lock and unlock dataiOS and AndroidStudents who experienced depression had more irregular sleep patterns, used their phones more at study places, spent more time stationary, and visited fewer locations.LASSOab regression was used to predict presurvey and postsurvey PHQ-9 scores.PHQ-4 and PHQ-883 studentsExposito et al [], 2018Detect stressKeyboard 3D touch dataiOSStudents’ typing pressure increased under stress.N/ASelf-reported stress11 studentsRhim et al [], 2020Detect subjective well-being and stressAccelerometer, GPS, screen on and off, app use, and notification dataAndroidLower subjective well-being was associated with more time spent on campus, more time spent stationary, increased phone use in the evenings, and more expenses.Hierarchical regression models were used to predict subjective well-being.COMOSWBac, PHQ, SASad, PPCae, and BFI78 studentsSefidgar et al [], 2019Detect stress, anxiety, and gender discriminationAccelerometer, GPS, phone lock and unlock, screen on and off, and call log dataiOS and AndroidStudents who experienced discrimination became more physically active; their phone use increased in the morning, they had more calls in the evening, and they spent more time in bed on the day of the discrimination.Linear regression was used to predict long-term changes in mental health states; hierarchical linear modeling was used for short-term prediction.UCLA Loneliness Scale, SSSaf, MAASag, ERQah, BRSai, PSS, CES-Daj, STAIak, and self-reported affect and fairness of treatment176 studentsCai et al [], 2018Detect state affect, stress, anxiety, and depressionAccelerometer, GPS, call log, and SMS text message dataiOS and AndroidNegative emotions were related to geographical locations, but this was affected by personal routines and preferences, for example, liking cinema theatres. On Fridays and Saturdays, students reported less negative states.Compared support vector machine, random forest, and XGboost with LOSOCVal and LOOCV to predict negative affect. The best model was support vector machine with LOOCV.SIAS and self-reported affect (EMAs)220 studentsBoukhechba et al [], 2018Predict response rate and latency to EMAGPS, call log, accelerometer, and SMS text message dataAndroidNone of the results related sensors to symptoms of depression or anxiety.Used random forest, support vector machine, and a multilayer perceptron of 1 hidden layer with LOOCV to predict the compliance rate of EMA responses.Self-reported affect (EMAs)65 studentsXu et al [], 2019Detect depressionAccelerometer, battery or charge, Bluetooth, call log, screen, location, and phone lock and unlock dataiOS and AndroidStudents who experienced depression had more disturbed sleep patterns and more phone interactions than students who did not experience depression.AdaBoostam with decision tree–based components achieved the best performance when features were hybrid (contextually filtered + unimodal).BDI-II138 studentsXu et al [], 2019Detect depressionAccelerometer, battery or charge, Bluetooth, call log, screen, location, and phone lock and unlock dataiOS and AndroidSame patterns as those mentioned earlier.AdaBoost with decision tree–based components achieved a similar result to majority-based baseline predictors.BDI-II212 studentsBoukhechba et al [], 2017Predict social anxietyGPS, call log, and SMS text message dataAndroidStudents who experienced high social anxiety may be more likely to buy food so they can eat at home; they tended to visit fewer places and had a narrower range of activities.Decision tree was used to predict SAS.SIAS54 studentsRashid et al [], 2020Predict social anxiety and evaluate the effectiveness of imputation methods in handling missing dataGPS, pedometer, accelerometer, call log, and SMS text message dataiOS and AndroidThe level of social anxiety was predicted, but there were no specific patterns relating sensors to symptoms of social anxiety.Evaluated 7 predictive models: linear regression, decision tree, XBboost, lightGBMan, random forest, MERFao, and CatBoost.SIAS and self-reported dimensions of social anxiety80 studentsMendu et al [], 2020Explore the relationships among private social media messages, personality traits, and symptoms of mental illnessFacebook (Meta Platforms, Inc) private messagesiOS and AndroidStudents who experienced anxiety received responses later, had more night-time communications, talked less about games and sports, and used more plural pronouns.Used random forest classifier to select features and support vector machine with LOOCV to predict each psychological measure binarily.STAI, UCLA Loneliness Scale, and TIPIap103 studentsTseng et al [], 2016Detect stress and its relationship with academic performanceLocation, activity, step count (iOS only), audio, accelerometer (iOS only), device use, charging event, battery, light (Android only), SMS text message (Android only) and call (Android only) data and data about currently running apps (Android only)iOS and AndroidStudents slept less during examination periods and more during breaks; they felt more stressed during the breaks and examination periods; sensor data were able to capture different routines during weekdays, weekends, and breaks.N/APSQI, ESSaq, MCTQar, PROMISas-10, BHMat-20, CD-RISCau, Flourishing Scale, Perceived Stress Scale, BFI, PHQ-8, and UCLA Loneliness Scale22 studentsMack et al [], 2021Understand the association between behavioral and mental health and the COVID-19 pandemicGPS, accelerometer, phone lock and unlock, and light sensor dataiOS and AndroidDuring the COVID-19 pandemic, students experienced more depression and anxiety and increased sedentary time and phone use, whereas sleep and the number of locations visited decreased.N/APHQ-4 and EMAs217 studentsXu et al [], 2023Evaluate the cross–data set generalizability of depression detectionGPS, accelerometer, phone lock and unlock, Bluetooth, Wi-Fi, call log, microphone, gyroscope, and light sensor dataiOS and AndroidIndividuals who experienced depression had shorter sleep duration, had more interrupted sleep, had more frequent phone locks and unlocks, spent more time at home, were more sedentary, had fewer physical activities, visited fewer uncommon places, and had more consistent mobility patterns.A multitask learning model with the 1D-CNNav–based embedding, fully connected layers for reordering and classification.Weekly surveys on self-reported depression symptoms and affect, BDI-II, and PHQ-4534 studentsNepal et al [], 2022Explore the association between students’ COVID-19 concerns and behavioral and mental healthGPS, accelerometer, phone lock and unlock, light sensor, and phone use dataiOS and AndroidHeightened COVID-19 concerns correlated with increased depression, anxiety, and stress. No specific results relating sensors to symptoms of depression, anxiety, or stress were observed.Evaluated different deep learning models in terms of their classification of COVID-19 concerns: CNN, InceptionTime, MCDCNNaw, ResNetax, multilayer perceptron, TWIESNay, LSTM, and FCNNaz; FCNN performed the best, with an AUROCba score of 0.7.Self-reported affect and PHQ-4180 studentsCurrey and Torous [], 2022Predict survey results on mental health from passive sensorsGPS, accelerometer, call, and screen time dataiOSIndividuals at higher risks of psychosis spent less time at home. Individuals who were lonelier had longer sleep duration and fewer calls. Individuals who experienced stress or depression had longer outgoing calls.Logistic regression was used to predict survey scores.PHQ-9, GAD-7, PSS, UCLA Loneliness Scale, PQ-16, and PSQI147 studentsCurrey et al [], 2023Explore the cross–data set generalizability of symptom improvement based on the surveysGPS, accelerometer, and screen time dataiOS and AndroidLogistic regression was able to predict changes in mood across 2 data sets of student participants. No results relating sensors to symptoms of depression or anxiety were observed.Logistic regression was used to predict weekly score improvement from both active and passive features.PHQ-9, GAD-7, PSS, UCLA Loneliness Scale, PSQI, PQ-16, and DWAIbb698 students

aPHQ: Patient Health Questionnaire.

bN/A: not applicable.

cDASS: Depression Anxiety Stress Scales.

dSIAS: Social Interaction Anxiety Scale.

eGAD-7: Generalized Anxiety Disorder Scale-7.

fPQ: Prodromal Questionnaire.

gPSS: Perceived Stress Scale.

hPSQI: Pittsburgh Sleep Quality Index.

iBASIS: Behavior and Symptom Identification Scale.

jSF: Short Form Health Survey.

kSFS: Social Functioning Schedule Scale.

lCGI: Clinical Global Impressions Scale.

mHDRS: Hamilton Depression Rating Scale.

nCAS: Coronavirus Anxiety Scale.

oHAI: Health Anxiety Inventory.

pUCLA: University of California, Los Angeles.

qXGBoost: extreme gradient boosting.

rLOOCV: leave-one-out cross validation.

sPANAS: Positive and Negative Affect Schedule.

tMPSM: Mobile Photographic Stress Meter.

uLSTM: long short-term memory.

vCNN: convolutional neural network.

wWEMWBS: Warwick-Edinburgh Mental Well-Being Scale.

xBDI: Beck Depression Inventory.

yEMA: ecological momentary assessment.

zPAM: Patient Activation Measure.

aaBFI: Big Five Inventory.

abLASSO: least absolute shrinkage and selection operator.

acCOMOSWB: Concise Measure of Subjective Well-Being.

adSAS: Sport Anxiety Scale.

aePPC: Perceived Personal Control.

afSSS: Social Support Scale.

agMAAS: Mindful Attention Awareness Scale.

ahERQ: Emotion Regulation Questionnaire.

aiBRS: Brief Resilience Scale.

ajCES-D: Center for Epidemiological Studies-Depression.

akSTAI: State Trait Anxiety Inventory.

alLOSOCV: leave-one-subject-out cross validation.

amAdaBoost: adaptive boosting.

anLightGBM: light gradient boosting machine.

aoMERF: mixed-effects random forest.

apTIPI: Ten-Item Personality inventory.

aqESS: Epworth Sleepiness Scale.

arMCTQ: Munich Chronotype Questionnaire.

asPROMIS: Patient-Reported Outcomes Measurement Information System.

atBHM: Behavioral Health Measure.

auCD-RISC: Connor-Davidson Resilience Scale.

av1D-CNN: 1-dimensional convolutional neural network.

awMCDCNN: multi-channel deep convolutional neural network.

axResNet: residual network.

ayTWIESN: time warping invariant echo state network.

azFCNN: fully convolutional neural network.

baAUROC: area under the receiver operating characteristic curve.

bbDWAI: Digital Working Alliance Inventory.

Studies With Adults

presents the data extracted from the studies conducted with the general adult population. The average study duration was 201.6 (SD 367) days. Apart from a 3-year longitudinal study with 18,000 participants, the average number of participants was 123.4 (SD 139.8). Of the 8 studies with adults, 2 (25%) [,] were conducted with the same set of participants. A total of 3 (38%) studies used predictive modeling, with regression-based models being the most common [,,], and 1 (12%) study identified gender differences in behavioral patterns []. Overall, the research with adults showed that GPS, accelerometer, ambient audio, and illuminance data related to individuals’ emotional state. Adults with depression were less likely to leave home and were less physically active, whereas adults who were socially anxious were more active and left their home more often but avoided going to places where they needed to socially interact.

Table 4. Summary of the reviewed studies with adult participants.Study, yearAimData collectedOperating systemBehavioral patternsPredictive modelingVerification surveysSample size, nDi Matteo et al [], 2021Understand whether ambient speech correlates with social anxiety, generalized anxiety, and depressive symptomsMicrophone dataAndroidGeneralized anxiety and depression were correlated with reward-related words. Social anxiety was correlated with vision-related words.N/AaLSASb, GAD-7c, PHQd-8, and SDSe86 Canadian adultsDi Matteo et al [], 2021Predict general anxiety disorder, social anxiety disorder, and depressionGPS, microphone, screen on and off, and light sensor dataAndroidDepression and social anxiety were associated with increased screen use. Depression was associated with sleep disturbance and death-related word features.A total of 3 logistic regression models were used to predict social anxiety disorder and generalized anxiety disorder with repeated k-fold cross validation.LSAS, GAD-7, PHQ-8, and SDS84 Canadian adultsWen et al [], 2021Detect impulsive behavior, positivity, and stressCall log, phone lock and unlock, and phone charging dataiOS and AndroidImpulsivity was correlated with increased phone use and screen checking.Used LASSOf regularization to first select features and trained a linear regression model to estimate trait impulsivity scores.BISg-15, UPPSh, PAMi, and self-reported feelings26 adultsFukazawa et al [], 2019Predict anxiety levels and stressLight sensor, gyroscope, accelerometer, and app use dataAndroidAnxiety was higher from Monday to Thursday than on Friday and Saturday. Increased anxiety was associated with decreased mobility. During mild exercise, anxiety was reduced.Used linear classifier by LASSO and XGBoostj to classify the change of anxiety.STAIk20 adultsPratap et al [], 2017Detect depressionGPS, call log, and SMS text message dataiOS and AndroidNone of the results related sensors to symptoms of depression.N/APHQ-2 and PHQ-9359 Hispanic or Latino adultsAdams et al [], 2014Detect stress levelMicrophone dataiOS and AndroidStress can be recognized from pitch, speaking speed, and vocal energy.N/APANASl, PSSm-14, MAASn, and self-reported affect7 adultsMeyerhoff et al [], 2021Detect anxiety and depressionGPS, call log, app use, and SMS text message dataAndroidChanges in the number of locations visited and social activity duration were associated with depression. Time spent at exercise locations was positively correlated with changes in depressive symptoms.N/AGAD-7, PHQ-8, and SPINo282 adultsServia-Rodríguez et al [], 2017Predict moodGPS, Wi-Fi, cell tower, accelerometer, microphone, SMS text message, and call dataAndroidA strong correlation was identified between daily routines and users’ personality, well-being perception, and other psychological variables; the participants who were the most emotionally stable tended to be more active, stayed in more noisy places, and texted less than participants who were unstable.Used stacked RBMsp to classify moods.Big-5 personality test, self-reported mood, and self-reports of locations18,000 adults mainly

aN/A: not applicable.

bLSAS: Liebowitz Social Anxiety Scale.

cGAD-7: Generalized Anxiety Disorder Assessment-7.

dPHQ: Patient Health Questionnaire.

eSDS: Sheehan Disability Scale.

fLASSO: least absolute shrinkage and selection operator.

gBIS: Barratt Impulsiveness Scale.

hUPPS: Impulsive Behavior Scale.

iPAM: Patient Activation Measure.

jXGBoost: extreme gradient boosting.

kSTAI: State Trait Anxiety Inventory.

lPANAS: Positive and Negative Affect Schedule.

mPSS: Perceived Stress Scale.

nMAAS: Mindful Attention Awareness Scale.

oSPIN: Social Phobia Inventory.

pRBM: Restricted Boltzmann Machine.

Studies With Employees

presents the data extracted from the studies that were conducted with employees. Among the 4 studies with employees, 1 (25%) study recruited its own participants [], and the other 3 (75%) studies [-] used the Tesserae data set []. Compared with students and adults, the employee population was the least studied, with the fewest articles. However, the studies with employees had the largest number of participants, with a mean of 427.3 (SD 280.3). All 4 studies used regression-based predictive modeling, and 2 (50%) of them [,] evaluated a variety of models, with logistic regression, support vector machine, and random forest being the most common methods. Detecting and predicting employees’ stress in workplaces were examined in tandem with employees’ work performance. The research goal for these studies was to understand the underlining reasons for lowered work-related productivity. In contrast to the other 2 populations (ie, students and adults), less mobility was seen as positive for employees because less mobility in workplaces was associated with more positivity and higher performance.

Table 5. Summary of the reviewed studies with employee participants.Study, yearAimData collectedOperating systemBehavioral patternsPredictive modelingVerification surveysSample size, nMirjafari et al [], 2019Predict stress and job performanceAccelerometer, GPS, phone lock and unlock, and light sensor dataiOS and AndroidHigher performers unlocked their phone fewer times during evenings, had less physical activity, visited fewer locations on weekday evenings, were more mobile, and visited more locations during weekends.Evaluated logistic regression, support vector machine, random forest, and XGBoosta in terms of employee performance classification; XGBoost was the best model with 5-fold cross validation.ITPb, IRBc, OCBd, and CWBe554 employeesNepal et al [], 2020Detect stress, well-being, and moodGPS, phone lock and unlock, accelerometer, Bluetooth, and phone use dataiOS and AndroidPromoted employees spent more time on their phones during early mornings and late evenings and had more unlocks during the night time than nonpromoted employees. Women’s mobility increased after promotion, whereas men’s mobility decreased.Evaluated logistic regression, support vector machine, Gaussian naive Bayes, random forest, and k-nearest neighbor in terms of their classification between promoted and nonpromoted periods; the best model was logistic regression trained on ROCKETf-based features.CWB, OCB, IRB, and ITP141 employeesSaha et al [], 2019Predict stress and workplace performanceLight sensor, GPS, accelerometer, and phone lock and unlock dataiOS and AndroidStress was higher with increased role ambiguity.Linear regression was used to predict a well-being score.IRB, ITP, and OCB257 employeesMorshed et al [], 2019Predict mood instabilityLight sensor, GPS, accelerometer, and phone lock and unlock dataiOS and AndroidMood instability was negatively correlated with the duration of sleep, the number of conversations, the amount of activity, and outdoor mobility.Ridge regression with regularization was used to infer a mood instability score.EMAsg, PAMh, and PANASi757 employees

aXGBoost: extreme gradient boosting.

bITP: Psychological Type Indicator.

cIRB: in-role behavior.

dOCB: organizational citizenship behavior.

eCWB: counterproductive work behavior.

fROCKET: random convolutional kernel transform.

gEMA: ecological momentary assessment.

hPAM: Patient Activation Measure.

iPANAS: Positive and Negative Affect Schedule.

Passive SensorsOverview

provides an overview of the range of sensors used to detect patterns related to mild mental health symptoms and summarizes the evidence of the effectiveness of the various sensors. The first column lists the sensor, and the second column presents how the data from that sensor are interpreted; in other words, it presents the behavior-related information that the sensor data are intended to represent. The third column indicates which articles found significant associations between the specific sensor and stress, anxiety, or mild depression. The fourth column indicates which articles found no significant associations between the specific sensor and mental health outcomes (ie, explicitly stated so in the articles). In the subsequent sections, we discuss the types of activities detected by the sensors.

Table 6. Sensor summary of the reviewed studies.SensorBehaviorEvidence for effectivenessNo evidenceGPSLocation and physical activity[,,,,-,-,-][,,]MicrophoneVoice recognition, ambient sound, and sleep[,,,,,,,,,,][,,]Light sensorTime spent in darkness and sleep[,,,,,,,,-]