Self-administered electroencephalography-based sleep assessment: compliance and perceived feasibility in children and adults

Abstract

Background

Sleep is a crucial part of our lives and insufficient sleep has been linked to several health disorders in both children and adults. However, most studies are based on single night laboratory polysomnography, actigraphy, or sleep diaries. The primary aim of this study was to evaluate compliance to and perceived feasibility of the Zmachine insight+ for assessment of habitual sleep parameters in a sample of children and adults for six nights. The secondary aim was to report sleep parameters derived from the Zmachine.

Methods

We analyzed data from 12 families who participated in the SCREENS pilot trial (2018–2019). Children (n=14) and adults (n=19) had to undergo three nights of EEG-based sleep assessment at baseline and follow-up. We assessed compliance to the sleep assessment protocol and summarized perceived feasibility in children and adults. Summary estimates were computed for total sleep time, sleep onset latency, wake after sleep onset, light sleep, deep sleep, and rapid eye movement sleep.

Results

Compliance to the sleep assessment protocol was high with 92.9 and 89.4% of children and adults meeting the a priori specified compliance goal of at least two out of three nights of complete sleep data at both baseline and follow-up. In general, the protocol was perceived as feasible, with low prevalence of sleep disruption and only minor issues, e.g. difficulties with removing sensors. Results on sleep parameters indicate large within group variation.

Conclusions

Our findings support the use of a self-administered EEG-based habitual sleep assessment protocol, including multiple days of measurement, in children and adults.

Trial registration

Cilinicaltrials.gov: NCT03788525 [Secondary outcome measures; Retrospectively registered; 27th December, 2018].

Background

Sleep is an essential part of our lives and important in terms of health and development across the lifespan. Recent evidence suggests that stable habitual sleep duration of around 7.5 h is associated with lower risk of cardiovascular events and all-cause mortality in adults (Wang et al., 2020). Sufficient habitual sleep duration is also important in children and adolescents with evidence showing a relationship between shorter habitual sleep duration and obesity (Ruan et al., 2015). Although several studies highlight the importance of sufficient habitual sleep duration, Matricciani et al. emphasize the need for studies using valid and reliable measures of sleep parameters (Matricciani et al., 2019).

Although laboratory polysomnography is considered the gold standard in objective sleep assessment (Blackwell et al., 2008), some important methodological issues exist when habitual total sleep time is the outcome of interest. One of the main issues is the fact that the polysomnography procedure itself may disrupt an individual’s normal sleep because it is performed by mounting multiple sensors to the head, face, and body of a subject who is then instructed to go to sleep in an unfamiliar laboratory setting (Marcus et al., 2014). Also, important day-to-day variations in sleep duration may be overlooked because it is often too costly to complete polysomnography for more than one night. Sleep diaries and actigraphy may be preferred in the assessment of habitual sleep duration because they capture sleep over several nights and are feasible under free-living conditions. Still, these measures have limitations because sleep diaries are subject to reporting bias and actigraphy capture movement, not sleep.

The Zmachine insight+ model DT-200 (General Sleep Corporation, Cleveland, OH) is a less expensive and less invasive in-home sleep assessment compared to polysomnography. Also, it may overcome some of the limitations of polysomnography in the assessment of habitual sleep parameters. The Zmachine utilizes single channel electroencephalography (EEG) data from two sensors placed at the mastoids (A₁ and A₂) to record the electrical activity of the brain. The EEG-signal is processed into sleep stages by the Zmachine algorithm (Kaplan et al., 2014; Wang et al., 2015). Though, the Zmachine is a promising tool, only one study have examined the feasibility of this device in adolescents (Lunsford-Avery et al., 2020).

Thus, the primary aim of this paper is to evaluate compliance to and perceived feasibility of the Zmachine insight+ for assessment of habitual sleep parameters in children and adults under free living conditions in a pilot trial preceding a definitive randomized controlled trial (the SCREENS trial). The secondary aim is to present baseline, follow-up, and change scores of sleep parameters derived from the Zmachine to inform future studies using similar assessment strategies.

Methods

The SCREENS pilot trial (www.clinicaltrials.gov – NCT03788525) is a two-arm parallel group cluster randomized trial with two intervention groups and no control group (‘SCREENS’ is not an acronym). Data for this trial was collected between October 2018 and March 2019. The overall purpose of the pilot trial was to assess compliance to the included interventions and the included measurement protocol, as well as feasibility of the survey-based recruitment strategy.

The collection of data was reported to the local data protection department SDU RIO (ID: 10.391) in agreement with the rules of the Danish Data Protection Agency.

Participants

Families in the municipality of Middelfart in Denmark were invited to participate if they had at least one child aged 6–10 years residing in the household (n=1686). A digital letter with a survey concerning screen media habits in the family was sent directly to a randomly chosen parent in each household in October 2018. In addition to the survey, a short description of the SCREENS pilot trial was provided on the final page. Here, respondents could note if they were interested in hearing more about the study. Based on the survey responses, families were eligible to participate if the randomly chosen parent’s total screen media use was above the median amount (2.7 h/day) based on all respondents (n=394) because the trial included interventions, whose aims were to reduce family screen media use. We only screened parental screen media use as they answered the survey. All children in the household had to be 4 years or older to avoid potential disturbances of sleep measurement due to an infant or toddler’s pattern of nocturnal awakening.

Eligibility to the trial was assessed further during a phone call. Families had to meet the following inclusion criteria:

• At least one child and one adult in each family had to participate

• All participants had to be able to heavily restrict total leisure screen media use

• Families had to consider their habitual screen media use a problem and be motivated to reduce it for a two-week-period

• Non-participating family members of the household had to respect the conditions which the participants had to follow

Exclusion of participants was based on the following criteria:

• If adults or children resided only part time in the household

• If participants had been diagnosed with a sleep or stress disorder within the last 12 months

• If adults in the household worked night hours

• If family members were not able to do typical everyday physical activities

• If a family member was diagnosed with a neuropsychiatric disorder (e.g. Attention Deficit Hyperactivity Disorder) or autism spectrum disorder

• If family members were already participating in other studies

Eligible families were informed about the content of the trial at a meeting in their home. Three additional meetings were planned with families who agreed to participate. The purpose of these meetings was to set up baseline measurements (baseline day 1), perform randomization procedure (baseline day 8), and collect measurement equipment (experiment day 15). A phone call was also planned to remind participants to start follow-up measurements (experiment day 8) (Fig. 1).

Fig. 1

Meeting and measurement protocol. The figure illustrates the structure of the SCREENS trial in days regarding scheduled meetings, as well as the timing of exposure and outcome measurements. The program is structured such that baseline and follow-up commence on the same day of the week, so the data will be collected on the same days. Note that the only difference between baseline and follow-up is that at baseline a test sleep measurement is included during the first night. The 1st meeting (not shown ) is a mandatory information meeting, prior to the trial. The figure is re-used from an earlier publication (see acknowledgements)

Sample size

We deemed that inclusion of 12 families with at least one child and one adult, with an expected total sample size of more than 30 participants, was sufficient to investigate feasibility of- and compliance to the interventions- and assessment methods we planned to utilize in a future definitive trial. A priori sample size calculation was not considered relevant because hypothesis testing of efficacy was not an aim in the pilot trial.

Interventions

Included families were randomized to one of two screen media restriction interventions; a general restrict or evening restrict group. Those in the general restrict group had to hand over smartphones and tablets and restrict all leisure screen media use for entertainment purposes to a maximum of 3 hours/week/person for 2 weeks. Families randomized to the evening restrict group had to remove all leisure screen media use after 6 PM for 2 weeks. A more thorough description of the components of the intervention can be found elsewhere (www.clinicaltrial.gov (NCT04098913 under “Arms and interventions”)).

The random sequence generation was performed by Odense Patient Explorative Network Randomise (OPEN R). An online platform provided by OPEN R was used to perform randomization in the home of the participants ensuring allocation concealment until the screen media restriction intervention was assigned. The randomization was made in alternating blocks of 2-4 families and was stratified by sibling status (only child/not only child) in the household.

Measurement protocol

Families underwent an extensive measurement protocol at baseline and follow-up spanning eight consecutive days (Fig. 1). This paper will focus on the EEG-based sleep assessments. Details on the remaining components of the measurement protocol can be found elsewhere (Rasmussen et al., 2020).

Sleep assessment

Sleep was objectively measured in the households for three consecutive nights at baseline and follow-up using single channel EEG-based sleep equipment (Zmachine). Participants were instructed to wear the equipment on the first night (only at baseline to include a test night) and the last three nights of the measurement protocol. If a family started baseline measurements on a Wednesday, the sleep assessment nights were Wednesday (test night), Sunday, Monday, and Tuesday). We included the test night at baseline to prevent participants’ sleep from being affected by novelty of wearing the Zmachine.

Participants were instructed to attach three sensors to the back of the head approximately 30 min prior to bedtime which was defined as “when you are lying in bed and you are ready to close your eyes and go to sleep”. One sensor was placed on the neck below the hair line (ground) and one behind each ear on the differential mastoids (signal) (Wang et al., 2015). We custom-made an elastic pocket for the Zmachine, which the participants were instructed to attach to an elastic waist belt at bedtime (Fig. 2 and Additional file 1). We developed the pocket such that the device and its cables could be fixated to the belt and thus eliminate the risk of wire entanglement around the neck during sleep. This solution was mainly developed for children in whom the device had not been tested prior. Participants were instructed to connect the cable to the sensors and the Zmachine just before bedtime.

Fig. 2

Sleep assessment set-up. Image A shows the elastic belt (1), the custom-made elastic pocket (2), the Zmachine (3) and the cable from the Zmachine (4). Image B shows the signal sensors at the differential mastoids (5) and the ground sensor on the middle of the neck (6)

The Zmachine algorithm categorizes the EEG signal on a 30-second epoch basis into five different categories; 1) Wake, 2) Light sleep (Stage N1 & N2), 3) Deep sleep (Stage N3), 4) Rapid eye movement sleep (REM-sleep) and 5) sensor problem (if the sensor connection fails). The Zmachine algorithm develops a unique sleep signature based on EEG signal data (first test night) from each subject accounting for individual differences e.g. age and skull thickness using an unsupervised algorithmic approach (Kaplan et al., 2014). Kaplan et al. found that the Zmachine algorithm has high sensitivity (95.5%) and specificity (92.5%) when compared to polysomnographic technologists in scoring sleep and wake in adults (Kaplan et al., 2014). Wang et al. compared the Z-PLUS algorithm in conjunction with the Z-ALG algorithm to sleep stages scored by polysomnographic technologists and found that it has high sensitivity ranging between 72 to 91% in adults (Kaplan et al., 2014; Wang et al., 2015).

Participants also reported bedtimes and time of awakening each day allowing crude calculation of self-reported total sleep time.

Total sleep time was defined as the sum of time scored as light, deep, and REM sleep. Sleep onset latency was calculated as the time from application of the Zmachine equipment to the first epoch scored as sleep. Wake after sleep onset was calculated as the amount of time scored as wake between the first and the last epoch scored as sleep.

Assessment of feasibility

Compliance

A priori, participants completing at least 2 out of 3 nights at both baseline and follow-up, were defined as compliant (see NCT04098913 at www.clinicaltrial.gov under “Secondary outcome measures”). In more specific terms, participants were only considered compliant if they provided 2 out of 3 nights with complete sleep data from the Zmachine.

• A night with complete sleep data was defined as a night in which less than 10% of the epochs were scored as sensor problems.

Episodes with equipment failure (e.g. due to low battery or disconnection of the cable) where no data was collected was identified by manually looking through the sleep data records in cases where there was a difference of more than 1 hour between self-reported total sleep time and objectively measured total sleep time. Subsequently, a night was excluded if the sleep data record stopped during the night and no data was collected for the rest of the night (a strong indicator of equipment failure).

Perceived feasibility

Adults also completed a questionnaire on behalf of themselves and their children concerning the perceived feasibility of the sleep assessment. The questionnaire was developed by the authors based on experiences with internal testing of the equipment. The questionnaire contained a variety of questions regarding the use of the sleep equipment, e.g. “To which degree were you bothered by the sleep equipment before, during, or following sleep?” (all questions are presented in the results section under Perceived feasibility).

Statistical methods

Baseline characteristics were computed using medians and inter quartile ranges for continuous variables and proportions for categorical variables. Characteristics are presented separately for children and adults; within the two intervention groups and for both groups combined.

Degree of compliance to the sleep protocol was calculated as proportions. Perceived feasibility was reported by calculating proportions in each response category.

A mean based on all nights with complete sleep data was calculated for everyone at baseline and follow-up for all sleep parameters. The means were based on nights with complete sleep data from at least two and maximum three nights for each individual at baseline and follow-up, respectively. Group means and standard deviations were calculated for all sleep parameters at baseline and follow-up. Pearson correlation coefficients were calculated between baseline and follow-up scores. Mean group changes and standard deviations in sleep parameters were calculated by subtracting group means at baseline from group means at follow-up. Not all sleep parameters followed a strict normal distribution; nevertheless, we present means and standard deviations to inform future sample size calculations. Medians and interquartile ranges for all sleep parameters are also given in Additional file 2. We performed supplementary sample size estimations based on the standard deviations of- and the correlation between the baseline and follow-up scores when scores from both groups were pooled (Additional file 3).

All statistical computations were performed in STATA IC 16 software (Statacorp).

Results

Twelve families consisting of 14 children and 19 adults agreed to participate. The intervention groups were similar at baseline regarding gender distribution, age, educational level, and number of children and adults per family (Table 1).

Table 1 Baseline characteristics

No participants dropped out of the study. However, one child did not complete the sleep measurements for more than two nights for unknown reasons. The remaining 32 participants (97.0%) slept with the Zmachine equipment for all 6 nights. Based on data from all 33 participants we acquired a total of 171 nights (86,4%) with complete sleep data (Fig. 3). The proportion of participants with complete sleep data for all six measurement nights was higher among adults compared to children (Table 2). Overall, only 9% of the participants (n=3) had less than four nights of complete sleep data.

Fig. 3

Flow of participants from inclusion to compliant participants. The figure shows the flow of participants from inclusion to compliant participants. The number of nights with complete sleep data is also shown at each stage

Table 2 Proportion of participants with complete sleep data

Feasibility of the Zmachine

Compliance

A total of 30 participants were compliant. Degree of compliance was similar between children and adults. Overall compliance was 5.6% higher in the general restrict group compared to the evening restrict group (Fig. 4).

Fig. 4

Proportion of compliant participants. The figure illustrates the proportion of participants who were compliant to the sleep assessment protocol. Results are shown for all participants and for each intervention group

Sensor problems (< 10%) were present during 3 out of 86 nights at baseline and 10 out of 81 nights at follow-up among compliant participants. These nights were distributed among 3 participants (children=2, adults=1) at baseline and 8 participants (children=4, adults=4) at follow-up.

Perceived feasibility

A total of 16 adults (84.2% of the sample) completed the perceived feasibility questionnaire. The results are presented in Table 3. Approximately one third of the participants were bothered to a medium or high degree before, during or after sleep. Almost one fifth of the participants reported sleeping poorer, but only a few participants reported more awake periods during sleep and having trouble falling asleep. Around one fourth of the participants were bothered by the cables during sleep. A few participants reported being bothered by the equipment when they got up during the night.

Table 3 Perceived feasibility of the Zmachine

Only two participants found it difficult to apply the sensors, but almost one fifth found it challenging to remove the sensors. Also, around one fifth of the participants reported that it was time consuming to apply the Zmachine. Around two fifths of the participants experienced skin irritation where the sensors were applied, and around half of the parents reported that their child were bothered by glue residue. Yet, no parents reported that sleeping with the equipment made their child sad or unhappy.

Sleep parameters

Baseline, follow-up measurements, correlations coefficients, and average two-week change in sleep parameters are presented in Table 4. Sleep parameters were relatively similar between groups at baseline except from total sleep time in children. Standard deviations indicate large within group variation. Change in total sleep time was positive among adults in both intervention groups and children in the general restrict group, while mean change in total sleep time was negative among children in the evening restrict group (for more details see Additional file 4).

Table 4 Sleep parameters

We also conducted sample size calculations for total sleep time applying the observed baseline and follow-up scores, and the correlation between them (Additional file 3). Lastly, we have provided an example of a child’s sleep stages scored by the Zmachine for one night (Additional file 5).

Discussion

Based on data from the children and adults from the SCREENS pilot trial, compliance to the EEG-based sleep assessment protocol was very high with 90.9% of the participants having complete sleep data. Furthermore, perceived feasibility of the Zmachine equipment indicated low prevalence of assessment-related sleep disruption, high participant acceptance, and the use of the equipment was overall deemed feasible. Few participants experienced minor challenges e.g. removing the sensors. Standard deviations indicated relatively large within group variation in sleep parameters.

Feasibility

This study provides novel results on the feasibility of self-administered use of the Zmachine in a free-living context in children and adults. Compliance was high, which is especially interesting considering the already heavy measurement load that participants were under during this study (i.e. accelerometry, heart rate variability, saliva sampling (only in adults), and daily self-report). Also, importantly, the proportion of children with complete sleep data for at least two nights at baseline and follow-up was comparable to that of the adults. We expected more missing data among children relative to adults, due to more frequent displacement of sensors (Beck & Marcus, 2009). Our custom-made elastic pocket may have prevented cable disconnection from occurring.

The participants reported overall that the protocol was feasible, with only minor issues such as skin irritation, glue residue, and being bothered by the cables during sleep. However, the severity of the irritations remains unclear due to the nature of the questions (yes/no), which gives an unnuanced picture of the issues. However, the field researchers (MGBR and JP) did not receive oral or written complaints of any skin lesions or other serious problems with the sleep sensors, only minor issues such as reddening of the skin. Children did not complain noticeably more than adults, except when asked about being bothered by e.g. removing the sensors and glue residue, which we expected. Based on the results of this study we recommend using the Zmachine in combination with an elastic pocket attached to an elastic waist belt in children (Fig. 2).

Lunsford-Avery et al. has also evaluated the feasibility of the Zmachine (without use of a custom-made pocket). The study included 104 adolescents who had to sleep with the Zmachine for 7 days. They found that 87% had at least 3 full nights of sleep EGG recordings. Additionally, they found that 81% of the participants rated the equipment as comfortable to mildly uncomfortable. Also, around 20% of the participants reported sleeping poorer due to the sleep assessment (Lunsford-Avery et al., 2020). These results are in line with the results of our study where 90.9% had complete sleep data for at least 4 out of 6 nights, while around one third reported being bothered to a medium or high degree before, during or after sleep, and 17.2% reported sleeping poorer.

Few studies have evaluated other EEG-based or polysomnographic sleep set-ups. Goodwin et al. investigated parent-reported participant burden of home-based single night unattended polysomnography in 157 children where a trained technologist applied the equipment. The method was found to be feasible with only 6% of the parents reporting that their child slept much worse than usual (Goodwin et al., 2001). Although this was based on a single night assessment, these results were similar to our study where 7.7% of the children reported sleeping poorer due to the Zmachine. A study by Marcus et al. examined the technical feasibility of single night unattended ambulatory polysomnography, where a sleep technologist applied the equipment to the child in their home. The study included 201 school-aged children and found that unattended ambulatory polysomnography is a feasible and well-tolerated method in school-aged children. Marcus et al. also found that 93.5% of the participants had a satisfactory EEG-signal (Marcus et al., 2014). These results are also similar to the results found in children in our study where the proportion of children who had complete data was 92.9%. A study by Mikkelsen et al. investigated the feasibility of sleeping with dry contact ear-EEG in 20 adults in a sleep laboratory (Mikkelsen et al., 2019). They found that 80% of the participants reported the ear-EEG as easy to use, which is slightly lower compared to the results of our study where 93.8% did not find it challenging to apply the sensors. In addition, Mikkelsen et al. found that 20% rated their sleep quality as bad with the ear-EEG, which is comparable to our study.

Based on the similarity between results from Lunsford-Avery et al., Goodwin et al. Marcus et al., Mikkelsen et al. and our study it is worth emphasizing that the participants in Lunsford-Avery et al. and our study applied the equipment themselves for several consecutive nights following instructions whereas it was applied by research personnel on all measurement nights in the other studies. This indicates that the Zmachine is a feasible method for EEG-based evaluation of habitual sleep parameters.

Sleep parameters

We observed large standard deviations in sleep parameters across intervention groups in children and adults. This may to some extent be due to the small sample or the fact that sleep was measured for only three nights at each time point. In some families the protocol included one weeknight and two weekend nights, while in other families it was the opposite depending on what day of the week the assessment protocol was initiated. Also, it is well known that individual sleep parameters may vary widely from day to day (Slavish et al., 2019), which also might explain some of the observed variation in sleep parameters. These considerations should be considered when looking at the estimated sample size examples in Additional file 3. Nevertheless, to our knowledge it is the best available data to qualify a sample size for a future parallel randomized controlled trial investigating the efficacy of a short-term behavioral intervention on habitual sleep parameters derived from the Zmachine in children and adults.

Limitations

This study has some limitations which must be kept in mind when interpreting the results. First, the sample consists of families who accepted an extensive protocol and intervention, which may have caused that the participants are from households that are particularly resourceful. Secondly, reliability and validity of the perceived feasibility questions have not been evaluated and therefore some issues with the use of the Zmachine may have been overlooked or misinterpreted. However, participants had an opportunity to write free text answers, which very few participants chose to do. Thirdly, perceived feasibility data was not available for all participants. Fourthly, no studies have evaluated the Z-ALG and Z-PLUS algorithms in children, thus the accuracy for this group is unclear. However, the Zmachine is programmed to develop a sleep signature based on the first recording, because EEG signal varies with e.g. skull thickness and age (Kaplan et al., 2014). Fifthly, some data loss (8%) was present among compliant participants due to sensor problems. We suggest emphasizing the importance of applying the sensors at least 30 min before bedtime to allow for the sensors to stay in place, and further clarify the instruction to participants that electrode application site should be clean and dry, and that hair should be removed from the application site (i.e. clipped) in some individuals. Sixthly, the small sample is a limitation, which, to some extent, may explain the large variation observed in sleep parameters. Finally, we evaluated compliance to and perceived feasibility of the Zmachine protocol in combination with other physical assessment methods that may have influenced perceived feasibility negatively due to the overall participant burden. Thus, we expect that the compliance and perceived feasibility of the Zmachine is even greater as an isolated sleep assessment tool.

Conclusions

Compliance to self-administered sleep assessment using the Zmachine was high, and perceived feasibility indicated that it is a feasible and tolerated method. Collectively, the findings of this study indicate that the Zmachine may be used to assess habitual sleep parameters using a protocol with three consecutive nights in children and adults.