記住我
In recent years, technological advances have made it possible to carry out diverse daily tasks (eg, health management) more quickly, efficiently, and immediately [,]. In this context, the Internet of Things is revolutionizing the health care system, allowing daily users to monitor their health status in real time through, for example, wearable devices []. Thus, the Internet of Things has spurred a digital social transformation resulting from changes in lifestyle patterns through the connection between everyday life and communication networks [].
Activity wristbands have become more popular among consumers because of their usefulness, affordability, and attractive design []. Sales of these device have risen in recent years, with approximately 65.1 million units sold worldwide []. The technology industry has moved to brand these devices as an increasingly rigorous option for measuring biomedical parameters []. Scientific evidence indicates that these wristbands can promote participatory medicine because they encourage people to be active agents in their health management []. Moreover, several studies indicate that these devices can help stakeholders to become more aware of their health status and to improve their healthy lifestyle habits [,].
Activity wristbands can collect a myriad of daily health information in the user’s free-living environments [,]. Although these devices were designed primarily to record physical activity, companies are increasingly focused on developing algorithms that record other variables such as sleep and its patterns [,]. Thus, the use of wristbands not only aims to monitor sleep patterns but also contributes to make the population more aware of sleep relevance [,]. In this sense, it is important to know the details of sleep behavior in the population because sleep is a vital part of daily life and a determinant of health and well-being [-].
Polysomnography (PSG) represents the reference method used to measure sleep patterns. Its objective is the diagnosis of sleep problems by assessing the quality and quantity of sleep [,]. However, PSG is limited to the clinical setting owing to its cost, invasiveness, the need for specialized professionals, and difficulty functioning in the user’s free-living environment [,]. Therefore, clinicians consider actigraphy to be a common alternative to address the drawbacks of PSG [,]. Several studies have compared actigraphy with PSG, showing that the former provides high sensitivity in detecting sleep epochs but low specificity in detecting wakefulness [-]. Actigraphy has poor accessibility, and a lack of feedback to users hinders its daily use in the general population [,]. Nevertheless, it is a validated and accepted tool for detecting certain sleep disorders (SDis) at the clinical and research levels [,].
On the basis of actigraphy, activity wristbands can combine movement signals from an accelerometer and heart rate (HR) variability from sensors to detect sleep-wake cycles [,]. Recently, some sleep researchers have considered these devices to be a potential complement to traditional sleep assessment methods because they can sense long-term variations in the circadian rhythm and sleep quantity and quality [,,]. However, previous evidence indicates that the sleep data recorded by these devices are unreliable because they do not accurately detect sleep stages, being more precise in detecting sleep than wakefulness [,,]. Hence, health professionals have stated that data from these devices can lead to excessive concern among consumers about getting optimal sleep, a phenomenon known as orthosomnia [,]. In addition, the lack of access to the raw data and algorithms used for sleep parameter measurements raises doubts about their use in clinical and research settings [,,].
Given the aforementioned limitations, sleep health entities, such as the American Academy of Sleep Medicine (AASM), consider it necessary to perform validation studies of activity devices to evaluate their performance and reliability against PSG, which is the gold standard [,,]. In this view, it is relevant that validation studies follow some criteria to assess these devices, such as the American National Standards Institute (ANSI) and Consumer Technology Association (CTA) standard []. This standard is based on 2 levels of compliance to test how the devices classify sleep and wake stages and how they classify sleep stages (wake, rapid eye movement [REM] sleep, light sleep, and deep sleep) [].
Few studies have analyzed the sleep data performance of activity wristbands. Most of these studies focused on validation of the Fitbit [-], Jawbone UP [,], and Oura ring [,] devices. In general, compared with PSG, these devices have good accuracy in sleep versus wake differentiation (between 65% and 91%) and high sensitivity (between 96% and 97%) but low specificity (between 42% and 51%) [,-]. These devices overestimate PSG total sleep time (TST) and sleep efficiency (SE), and they underestimate PSG wake after sleep onset (WASO) and sleep onset latency (SOL) [,-,,].
Some of these validated devices can also identify sleep stages, although the Fitbit Charge 2 overestimates PSG N1+N2 stages of non-REM sleep and underestimates N3 stage of non-REM sleep [,], whereas the Oura ring overestimates PSG REM sleep and underestimates N3 sleep []. In addition, the authors of some comparative studies claim that it is essential to know the performance of these devices in people with SDis, considering that most studies have only included healthy populations [,,]. Kahawage et al [] and Moreno-Pino et al [] used the Fitbit Alta HR in a population with obstructive sleep apnea (OSA) and insomnia. Both studies showed similar concordance against PSG, with high sensitivity and low specificity [,].
According to the literature, it is necessary to continue testing the validity of wearable devices [,]. Therefore, this research focused on the Xiaomi Mi Band (Xiaomi Inc), which is one of the most popular activity wristbands on the market owing to its low cost and suitable quality [,]. The Xiaomi Mi Band is not a device that is frequently used in scientific research, unlike other devices, such as those manufactured by Fitbit [,,]. However, recent studies have used the Xiaomi Mi Band as an objective tool to record physical activity and sleep in older populations [-] and work environments [,]. Moreover, previous research compared the sleep data of Xiaomi devices, specifically the Xiaomi Mi Bands 2 and 3, with other devices, concluding that the Xiaomi devices do not correctly identify the TST and WASO periods [,]. Nevertheless, researchers have not comprehensively assessed the Xiaomi Mi Band against PSG and specifically analyzed how it records sleep stages in people with SDis as well as those without SDis. Therefore, this study is the first to address validation of the Xiaomi Mi Band 5 against PSG.
ObjectivesThis study aimed to compare the performance of the Xiaomi Mi Band 5 in measuring the sleep-wake stages compared with PSG performed at a hospital sleep unit. The secondary objectives were (1) to determine the agreement between sleep measures from PSG and the Xiaomi Mi Band 5; (2) to assess the accuracy, specificity, and sensitivity for classifying sleep and wake stages by the Xiaomi Mi Band 5 compared with PSG; and (3) to determine the performance level of the Xiaomi Mi Band 5 for detecting sleep stages (wake, light sleep, deep sleep, and REM sleep) compared with PSG.
This study was carried out from August 4, 2020, to December 10, 2021. Participants were recruited from the sleep unit of a hospital in A Coruña, Spain. These people were attending the sleep unit to participate in a PSG study, intended to detect possible sleep alterations, independent of this project. All participants had access to a study information sheet and provided written informed consent for their participation. The protocol study was registered with the ClinicalTrials.gov Protocol Registration and Results System (NCT04568408) and published in an international journal, where the design and recruitment process of the study are detailed []. The research group maintained the anonymization of all data recorded and obtained from each participant, following and respecting the European (UE 2018/1725) and Spanish (BOE-A-2018-16673) laws on personal data protection at all times.
A total of 58 people participated in the Xiaomi Mi Band 5 validation project. However, data from 13 (22%) of the 58 participants were not used in the sleep analysis owing to different factors, such as Xiaomi device malfunction (10/13, 77%), not meeting the inclusion criteria (2/13, 15%), and not performing a PSG (1/13, 8%). Therefore, the final sample comprised 45 people (n=23, 51%, men and n=22, 49%, women; aged 23-81 [mean 53.24 SD 15.44] years; BMI mean 27.86, SD 4.44 kg/m2). Of these 45 participants, 25 (56%) were diagnosed with SDis after they had undergone PSG. The sleep diagnoses were OSA syndrome (18/25, 72%), insomnia (4/25, 16%), narcoleptic syndrome (1/25, 4%), a combination of hypersomnia and narcoleptic syndrome (1/25, 4%), and a combination of sleep apnea syndrome and hypoventilation syndrome (1/25, 4%). For this reason, the sleep measures were also analyzed in 2 groups according to whether there were SDis. The demographic characteristics of the No SDis (20/45, 44%) and SDis (25/45, 56%) groups are shown in .
Table 1. Sample characteristics (n=45).CharacteristicsNo SDisa group (n=20)SDis group (n=25)Sex, n (%)aSDis: sleep disorders.
bPSQI: Pittsburgh Sleep Quality Index.
ProcedureAll participants slept in the sleep unit for 1 night. On the day of the recording, participants did not drink liquids for 3 hours before PSG, and they attended the sleep unit a few hours earlier to become acquainted with the bedroom. The technical team members of the sleep unit were in charge of supervising, preparing the participants for PSG, and meeting their possible demands. Regarding PSG, the technicians placed the sensors with electrode gel on the participants and connected them to start the test. Participants wore the Xiaomi Mi Band 5 during the recording test, and its location on the wrist was noted by the technicians. During the registration, the technical team members supervised the PSG and the Xiaomi device worn by the participants, making notes and marking alterations that emerged throughout the night for subsequent analysis. The lights-off and lights-on times, temperature, and sound were controlled in the room by the technicians. PSG and Xiaomi data were collected and synchronized simultaneously. The data coincided with the lights-off and lights-on times, providing a record of approximately 8 hours of time in bed (TIB).
PSG AssessmentPSG was performed using the NicoletOne v44 Sleep Diagnostic System (Natus Medical Incorporated). This device uses several recordings that included an electroencephalogram (EEG; 6 leads: FP1/FP2, F3/F4, C3/C4, and O1/O2 referenced by the contralateral mastoid), a submental (P3 and P4) and bilateral anterior tibial (2 electrodes on each leg to assess leg movements) electromyogram (EMG), a bilateral electro-oculogram (EOG), and an electrocardiogram (ECG) []. EEG, EMG, EOG, and ECG signals were sampled at 256 Hz. The EEG and EOG signals were filtered at 0.3 to 35 Hz, the EMG signal was filtered at 10 to 100 Hz, and the ECG signal was filtered at 0.3 to 70 Hz. At the same time, other biomedical parameters such as respiratory movements and efforts (thoracic and abdominal bands), nasal and oral airflow (nasal cannula), arterial oxygen saturation (pulse oximeter), cardiac activity, and body movement band were recorded to provide relevant information for a potential diagnosis of SDis []. PSG parameters were interpreted by the specialized physician of the sleep unit to obtain the cycles and stages of sleep (wake time, N1 sleep, N2 sleep, N3 sleep, and REM sleep) and were scored in 30-second epochs according to the standards of the AASM [].
Xiaomi Mi Band 5The Xiaomi Mi Band 5 includes a 3-axis accelerometer, a 3-axis gyroscope, an HR sensor, and a photoplethysmography sensor to measure some biomedical parameters. This device contains updated software that continuously records daily activity (eg, steps, distance, activity time, and calories), sleep (eg, light sleep, deep sleep, REM sleep, wake, TST, start and end of the sleep period, and naps), and HR []. The device also calculates and classifies the stress level (classified using the terms relaxed, mild, moderate, and high) through HR data. In addition, the wristband requires a series of personal data such as age, sex, weight, height, handedness, and wristband location. The Xiaomi Mi Band connects to its app, the Zepp Life app, via Bluetooth, where the recorded data are transferred and displayed []. The Zepp Life app allows the export of activity data (broken down into total activity, minute-by-minute activity, and activity stage), sleep data (start and end sleep periods, WASO, light sleep, deep sleep, and REM sleep), HR data (broken down into total HR; minimum, maximum, and mean HR; and minute-by-minute HR), sports data, and body data in CSV files []. In this study, some modes of the Xiaomi device were activated, such as automatic HR, sleep assistant, and night mode, to obtain accurate data and to not disturb the participant. Moreover, there were no problems with the charging of the wristband battery or with the battery itself.
Processing the PSG and Xiaomi Mi Band 5 DataThe Xiaomi Mi Band 5 and PSG source data were originally available in different formats. The Xiaomi Mi Band 5 data were collected from the Zepp Life app manually and exported to an Excel sheet (Microsoft Corp) because it did not allow downloading of the raw data from the wristband. By contrast, PSG data consisted of CSV text files that contained the manually scored sleep stages and the corresponding reports with clinical sleep diagnostic parameters, available in Word format (Microsoft Corp), both exported using the NicoletOne software. Hence, to enable performance analysis, data were converted into a common format using the European Data Format + (EDF+) []. For this purpose, a Python script (version 3.10.5; Python Software Foundation) was developed with help of the PyEDFlib library [].
To carry out this process, start time, lights off, lights on, and end of the test markers were set according to the expert annotations available in the corresponding PSG reports. Sleep stages were coded using EDF+ standard texts following the AASM guidelines []. For the Xiaomi Mi Band 5 data, it was necessary to adjust the resolution to standard 30-second epochs because the device originally reported on the basis of 1-minute epochs only. This was achieved by splitting each 1-minute epoch into 2 corresponding epochs of 30 seconds each sharing the same sleep stage. Likewise, the Xiaomi hypnogram information does not consider N1 and N2 stages separately. For this reason, epoch-by-epoch (EBE) performance analysis was carried out using a 4-way classification (wake, light sleep, deep sleep or N3, and REM sleep) by merging the corresponding N1 and N2 labels from the PSG in the corresponding light sleep category. Performance analysis was carried out using information within the TIB periods. It should be noted that the Xiaomi Mi Band 5 only starts reporting after the first sleep period is identified by the device; therefore, it is assumed that the periods before the first sleep period and after the last sleep period detected by the device within the TIB period are scored as wakefulness.
After this process, and using the corresponding EDF+ annotation files, different standard sleep parameters were calculated and compared []. More specifically, TIB (calculated in min), total sleep period duration (TSPD; a measure containing the duration of both sleep and wake cycles; min), TST (min), WASO (min), awakenings (number), SOL (min), SE (percentage), light sleep (min), deep sleep (min), REM sleep (min), and awake (min) were analyzed for both the Xiaomi Mi Band 5 and PSG in this study. Normative values for some of these parameters have been proposed based on the consensus of a panel of experts regarding objective assessment of sleep quality [].
Statistical AnalysisStatistical analysis was performed using R software (version 4.1.2; R Foundation for Statistical Computing). The analysis was carried out with the total sample (n=45). Likewise, data were analyzed with the participants grouped depending on the existence of SDis to determine the differences between PSG and the Xiaomi Mi Band 5.
Summary measures of PSG and the Xiaomi Mi Band 5 equivalents were compared using the paired 2-tailed t test [] or the Mann-Whitney Wilcoxon test []. The choice of the test was based on whether the data were normally distributed, determined by using the Shapiro-Wilk normality test. Normally distributed data were analyzed using the parametric 2-tailed t test, whereas nonnormally distributed data were analyzed using the Mann-Whitney Wilcoxon test. Moreover, the effect size was measured using Cohen d, classified as 0.2 (small effect), 0.5 (moderate effect), and 0.8 (large effect) [].
Furthermore, the Bland-Altman method was used to determine the agreement between PSG and the Xiaomi Mi Band 5 for each sleep parameter. The mean difference (or bias) between the methods, the SD, the 95% CI, and the Bland-Altman 95% limits of agreement (mean observed difference ± 1.96 × SD of observed differences) were calculated. A positive bias indicates that the Xiaomi Mi Band 5 tends to underestimate a variable when compared with the gold standard (PSG). A negative bias indicates that a sleep variable is overestimated [].
EBE analysis was performed according to the 2 levels of compliance as provided in the ANSI and CTA performance evaluation guidelines []. The first level analyzed the performance of the devices in a 2-way classification for detecting sleep-wake stages using a confusion matrix. For this purpose, accuracy (proportion of correctly classified sleep and wake epochs), sensitivity (proportion of epoch segments identified as sleep by the Xiaomi Mi Band 5 of those classified as sleep by the PSG), specificity (proportion of epoch segments identified as wake by the Xiaomi Mi Band 5 of those classified as wake by PSG), and Cohen κ values (agreement corrected by chance between the Xiaomi Mi Band 5 and PSG) were analyzed []. The standard label definitions used to classify Cohen κ values were used: 0 to 0.2 (slight), 0.21 to 0.40 (fair), 0.41 to 0.60 (moderate), 0.61 to 0.80 (substantial), and >0.80 (almost perfect) []. The second level analyzed how the device detected sleep stages using a 4-way classification (wake, REM sleep, light sleep, and deep sleep). For this purpose, accuracy and Cohen κ values were analyzed [].
Ethics ApprovalThe study was approved by the A Coruña-Ferrol research ethics committee (2020/318).
The sleep parameters obtained from PSG and the Xiaomi Mi Band 5 were compared using the 2-tailed t test or the Mann-Whitney Wilcoxon test. provides the sleep outcomes for PSG and the Xiaomi Mi Band 5 in the total sample. Overall, there were no significant differences between the methods in the initial sleep onset (P=.27), TSPD (P=.78), and SOL (P=.29) measures. However, the Xiaomi Mi Band 5 significantly overestimated PSG TST (z=−2.73; P=.009), the percentage of PSG SE (z=−2.59; P=.03), PSG light sleep (t44=−2.49; P=.005), and PSG deep sleep (t44=−2.58; P=.01). In addition, it underestimated PSG WASO (z=2.96; P=.005), PSG awakenings (z=7.72; P=.001), PSG REM sleep (t44=3.59; P=.001), and PSG awake (z=2.61; P=.01).
shows the results of sleep measures for PSG and the Xiaomi Mi Band 5 in the No SDis and SDis groups. There were no significant differences between PSG and the Xiaomi Mi Band 5 for the initial sleep onset (No SDis: P=.37; SDis: P=.52) and TSPD (No SDis: P=.47; SDis: P=.57) variables in either group. However, the Xiaomi Mi Band 5 differed significantly from PSG in the rest of the sleep measures, depending on whether the participants presented or did not present SDis. In the No SDis group, the Xiaomi Mi Band 5 overestimated PSG SOL (z=−1.83; P=.046) and PSG light sleep (t19=−3.23; P=.004), and it underestimated PSG awakenings (z=5.50; P<.001) and PSG REM sleep (t19=2.66; P=.02). By contrast, in the SDis group, it overestimated PSG TST (z=−2.70; P=.007), the percentage of PSG SE (z=−2.19; P=.03), and PSG light sleep (t19=−3.97; P<.001), and it underestimated PSG WASO (z=2.35; P=.02), PSG awakenings (z=5.35; P<.001), PSG REM sleep (t19=2.37; P=.03), and PSG awake time (z=3.13; P=.005).
Table 2. Comparison of polysomnography (PSG) and Xiaomi Mi Band 5 sleep measures in the total sample.aMann-Whitney Wilcoxon test.
bNot available.
cSmall effect.
dTIB: time in bed.
eTSPD: total sleep period duration.
fTST: total sleep time.
gWASO: wake after sleep onset.
hLarge effect.
iSOL: sleep onset latency.
jSE: sleep efficiency.
kREM: rapid eye movement.
lModerate effect.
Table 3. Comparison of polysomnography (PSG) and Xiaomi Mi Band 5 sleep measures in the no sleep disorders (No SDis) and sleep disorders (SDis) groups.aMann-Whitney Wilcoxon test.
bNot available.
cSmall effect.
dTIB: time in bed.
eTSPD: total sleep period duration.
fModerate effect.
gTST: total sleep time.
hWASO: wake after sleep onset.
iLarge effect.
jSOL: sleep onset latency.
kSE: sleep efficiency.
lREM: rapid eye movement.
Bland-Altman Plotsshows Bland-Altman biases, SDs, 95% CIs, and the upper and lower Bland-Altman 95% limits of agreement. presents Bland-Altman plots for the main sleep measures. In the total sample, the Xiaomi Mi Band 5 significantly overestimated PSG TST by 29.54 minutes, PSG SE by 5.82%, PSG light sleep by 29.81 minutes, and PSG deep sleep by 14.64 minutes. By contrast, it underestimated PSG WASO by 31.44 minutes, PSG awakenings by 2.95 epochs, PSG REM sleep by 19.49 minutes, and PSG awake time by 28.36 minutes.
There were also differences between the sleep groups. The Xiaomi Mi Band 5 significantly overestimated PSG light sleep by 42.02 minutes and PSG SOL by 12.28 minutes in the No SDis group. In addition, it underestimated PSG awakenings by 3.25 epochs and PSG REM sleep by 24.05 minutes in this group. However, in the SDis group, the Xiaomi Mi Band 5 overestimated PSG TST by 41.41 minutes, PSG SE by 7.6%, and PSG deep sleep by 28.89 minutes. In addition, it underestimated PSG WASO by 35.03 minutes, PSG awakenings by 2.72 epochs, PSG REM sleep time by 15.83 minutes, and PSG awake time by 41.41 minutes.
On average, the Bland-Altman agreement limits were exceeded every 4 and 2 participants for the total sample, especially in TST, WASO, awakenings, and SE measures. The participants with sleep problems were the ones who mainly exceeded these aggregation limits.
Table 4. Bland-Altman parameters for the comparison between polysomnography and the Xiaomi Mi Band 5 in the total sample as well as the no sleep disorders (No SDis) and sleep disorders (SDis) groups.an=45.
bn=20.
cn=25.
dTST: total sleep time.
eTSPD: total sleep period duration.
fWASO: wake after sleep onset.
gSOL: sleep onset latency.
hSE: sleep efficiency.
iREM: rapid eye movement.
Figure 1. Bland-Altman plots for initial sleep onset, total sleep time (TST), total sleep period duration (TSPD), wake after sleep onset (WASO), awakenings, sleep onset latency (SOL), sleep efficiency (SE), light sleep, deep sleep, rapid eye movement (REM) sleep, and awake time. The PSG-Xiaomi Mi Band 5 differences for sleep parameters (y-axis) are plotted as a function of the PSG-Xiaomi Mi Band 5 means (x-axis) for sleep parameters. Circles represent participants without sleep disorders (No SDis group; n=20), and triangles represent participants with sleep disorders (SDis group; n=25). Zero lines are marked and represent perfect agreement. The dotted lines represent the biases and Bland-Altman 95% limits of agreement (mean observed difference ± 1.96 × SD of observed differences). PSG: polysomnography. EBE Analysis- show the confusion matrices resulting from the 2-way (wake vs sleep) classification of 30-second epochs between PSG and the Xiaomi Mi Band 5. Results are shown separately for the total sample as well as for the No SDis and SDis groups.
Table 5. Confusion matrix for the 2-way (wake vs sleep) epoch-by-epoch classification for the total sample.aPSG: polysomnography.
Table 6. Confusion matrix for the 2-way (wake vs sleep) epoch-by-epoch classification for the no sleep disorders group.
留言 (0)