Sleep prediction using data from oximeter, accelerometer and snoring for portable monitor obstructive sleep apnea diagnosis | Scientific Reports

Scientific Reports volume 14, Article number: 24562 (2024) Cite this article

Metrics details

The aim of this study was to build and validate an artificial neural network (ANN) algorithm to predict sleep using data from a portable monitor (Biologix system) consisting of a high-resolution oximeter with built-in accelerometer plus smartphone application with snoring recording and cloud analysis. A total of 268 patients with suspected obstructive sleep apnea (OSA) were submitted to standard polysomnography (PSG) with simultaneous Biologix (age: \(56\pm 11\) years; body mass index: \(30.9\pm 4.6\)\(\hbox {kg/m}^{2}\), apnea-hypopnea index [AHI]: \(35\pm 30\) events/h). Biologix channels were input features for construction an ANN model to predict sleep. A k-fold cross-validation method (k=10) was applied, ensuring that all sleep studies (N=268; 246,265 epochs) were included in both training and testing across all iterations. The final ANN model, evaluated as the mean performance across all folds, resulted in a sensitivity, specificity and accuracy of 91.5%, 71.0% and 86.1%, respectively, for detecting sleep. As compared to the oxygen desaturation index (ODI) from Biologix without sleep prediction, the bias (mean difference) between PSG-AHI and Biologix-ODI with sleep prediction (Biologix-Sleep-ODI) decreased significantly (3.40 vs. 1.02 events/h, p<0.001). We conclude that sleep prediction by an ANN model using data from oximeter, accelerometer, and snoring is accurate and improves Biologix system OSA diagnostic precision.

Obstructive sleep apnea (OSA) is characterized by repetitive episodes of upper airway obstruction, resulting in sleep fragmentation and oxygen desaturation1. OSA is associated with several health consequences, including poor sleep quality, excessive daytime sleepiness, and increased cardiovascular risk2. Polysomnography (PSG) is considered the gold standard method for OSA diagnosis3. However, PSG is expensive and inconvenient for patients3. Portable monitoring (PM) is a simplified method that has been validated for OSA diagnosis4. In contrast to PSG, PM does not detect sleep. The consequence of this limitation is that the number of respiratory events in PM devices are reported by hour of monitoring rather than hours of sleep. Therefore, the absence of sleep monitoring is a potential source of variability between PSG and PM. Biologix system is a new PM device based on a high-resolution wireless oximeter (\(\hbox {Oxistar}^{\textrm{TM}}\), Biologix Sistemas S.A., Brazil) with built-in accelerometer and a smartphone application (app) that is downloaded to the patient’s smartphone. The app records snoring, and all information is automatically processed in the cloud. Biologix system has been validated for OSA diagnosis against PSG in the sleep laboratory5 and against traditional PM at home6. However, Biologix system does not monitor sleep and therefore reports oxygen desaturation index (ODI) based on hours of monitoring rather than hours of sleep. Therefore, the objective of this study was to build and validate an artificial neural network (ANN) algorithm using data from oximeter, accelerometer and snoring to detect sleep. We also tested the hypothesis that ANN model improves the Biologix system OSA diagnostic precision.

The study included patients recruited in the validation study of the Biologix system against PSG. Full details of the protocol have been published elsewhere5. The local ethics committee (Comissão de Ética para Análise de Projetos de Pesquisa do HCFMUSP - CAPPesq) approved the protocol (SDC 4515/17/015), and all patients gave their informed consent. The study has been performed in accordance with the Declaration of Helsinki. Briefly, we studied patients with suspected OSA referred for overnight-laboratory PSG at the Sleep Laboratory of the Heart Institute (InCor).

PSG included recording of the electroencephalogram (EEG) central (C) and occipital (O) channels referred to the auricular channel (A) (C3/A2, C4/A1, O1/A2, O2/A1), electrooculogram (EOG), submental electromyogram (EMG), left and right anterior tibialis EMG, electrocardiogram, thoraco-abdominal effort, oronasal airflow (thermistor and nasal pressure based airflow measurement), oxygen saturation (\(\hbox {SpO}_2\)) with pulse oximetry, and body position (EMBLA S7000, Embla Systems, USA and Alice 5, Respironics Inc., USA)5. Two certified technicians independently analyzed all PSG studies. Hypopnea was defined as a drop in the peak signal excursion of \(\ge\) 30% from the pre-event baseline nasal pressure signal lasting for at least 10 seconds. Respiratory events were scored according to the American Academy of Sleep Medicine criteria (\(\ge\) 3% reduction in \(\hbox {SpO}_2\) from the pre-event baseline or an event associated with arousal). OSA was classified based on current standards as follows: absence of OSA (AHI < 5 events/hour), mild OSA (5 \(\le\) AHI < 15 events/hour), moderate OSA (15 \(\le\) AHI < 30 events/hour), and severe OSA (AHI \(\ge\) 30 events/hour).

Simultaneously, the patients also wore a high-resolution oximeter (\(\hbox {Oxistar}^{\textrm{TM}}\), Biologix Sistemas S.A., Brazil) with built-in accelerometer linked by Bluetooth to a smartphone app that recorded snoring. The \(\hbox {Oxistar}^{\textrm{TM}}\) firmware captures data at a rate of 100 samples per second, providing beat-to-beat raw \(\hbox {SpO}_2\) measurements with a precision of 0.1%. To smooth the data, a moving average over 4 heartbeats was applied. Oxygen desaturations are calculated providing the ODI. The ODI was calculated as the number of desaturations (\(\ge 3\%\) reduction in \(\hbox {SpO}_2\)) per hour, using either total recording time or total sleep time. The oximeter information was sent to the cloud, and automatically analyzed (Fig. 1). The PSG and Biologix data were time-synchronized.

Biologix system. The wireless oximeter connects via Bluetooth to the smartphone’s Biologix application. The data is sent to the cloud and automatically analyzed by the algorithm.

ANN are algorithms based on the biological structure of the human brain, in which several neurons are connected7,8. These neurons are divided into at least three layers: inputs, a variable number of hidden layers, and outputs. Each of these layers is connected to the next layer by an activation function, a weight associated with its signal, and a bias8. To build and validate our ANN algorithm, we used data derived from the Biologix system including oximeter (\(\hbox {SpO}_2\), heart rate [HR]), with built- in accelerometer (movement), and smartphone app (snoring). Snoring was obtained by recording the audio of the environment performed by the smartphone app and processed by another neural network. This algorithm provides a binary output indicating whether the patient is snoring or not during the audio recording stretches, similar to other approaches found in the literature9. The k-fold cross-validation method (k=10) was used to build and validate the ANN algorithm10,11. The sleep studies were randomly divided into 10 folds, with each fold used for cross-validation to ensure that all studies were both trained and tested across multiple iterations. In each fold, the training and test datasets were employed to optimize the weights and biases, reducing the error between the predicted value by the neural network results. The gold standard for sleep classification was a binary variable (sleep or awake) determined by PSG epochs of 30 seconds. The process starts with a forward pass for initial values of weight and bias and for pre-defined activation functions. Outputs are then calculated, and errors are determined. In the next step, the values of weights and biases are redefined through a process called backpropagation12. Using the newly calculated values, the process is redone. This occurs recursively until a maximum number of iterations8,13. Our model consists of an input layer, one hidden layer and an output layer. The input layer has 97 neurons, the hidden layer has 128 neurons and a ReLU (rectified linear unit)14 activation function. Finally, the output layer has 1 neuron and a sigmoid activation function (Fig. 2). In order to test the accuracy of the accelerometer alone in predicting sleep, another ANN model was built using only the accelerometer channel, with 23 neurons in the input layer, while the other layers remained the same.

Artificial neural network algorithm diagram. The 97 input features were extracted from the channels, processed by the hidden layer with 128 neurons, resulting in a single neuron output that predicted whether the patient was sleep or awake. \(\hbox {SpO}_2\) oxygen saturation, ReLU rectified linear unit.

The first step was the treatment of missing values (less than 1% of the data was missing), which consisted of replacing these values by zero, in the case of the accelerometer, and by the maximum values for the cases of \(\hbox {SpO}_2\) and HR. Subsequently, the features were calculated using epochs of 30-seconds synchronize to PSG epochs. The features used in the model were calculated based on the signals obtained by the Biologix system and included \(\hbox {SpO}_2\), HR, movement detected by the accelerometer, and snoring detected by the smartphone app. The \(\hbox {SpO}_2\) features were: (1) presence or absence of oxygen desaturation, expressed as a binary variable; (2) desaturation range; (3) \(\hbox {SpO}_2\) quartile (75th) as a measure of the tendency of the patient’s \(\hbox {SpO}_2\) values during sleep. The HR signal provided several features: (1) average pulse interval; (2) standard deviation of HR; (3) HR variability (HRV) time domain features (SDNN, RMSSD, PNN50, SD1, SD2); (4) HRV frequency domain features (LF power, HF power, LF/HF ratio). The accelerometer data generated multiple features: (1) variance; (2) root mean square (RMS); (3) skewness; (4) kurtosis. Other variables of interest associated with snoring, obtained by the Biologix app, were also used, which improved the performance of the machine learning model. In addition, some of these features were considered shifted in relation to the current time step for the better composition of the predictive model, totaling 97 inputs for the neural network model summarized in the Table 1. Finally, the data was standardized by removing the mean and scaling to unit variance15,16.

Where u is the mean value, s is the standard deviation, x is the samples and z is the new samples15,16.

Accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), Cohen’s kappa coefficient (\(\kappa\)), F1-score (weighted average between the precision score [PPV] and recall score [sensitivity]), and area under the curve (AUC) were calculated for evaluation of the ANN model. Because several previous studies used only accelerometer data to predict sleep, we used the McNemar’s test17 to compare the ANN performance to predict sleep using only accelerometer data with ANN performance using all Biologix channels (SpO2, HR, accelerometer, and snoring). Overall summary statistics were calculated in terms of means and standard deviations for continuous variables and counts and percentages for categorical variables. Shapiro-Wilk test was used for checking the data normality of the PSG and the Biologix system. Since the data distribution was not normal and they were not independent, the differences were analyzed by the Wilcoxon signed-rank test. In addition, we calculated the sensitivity, specificity, accuracy, PPV, and NPV of the Biologix system, without and with sleep prediction, versus PSG, in the detection of OSA severity. Mc Nemar’s test17 compared the Biologix system performance without and with sleep prediction. Finally, to assess the amount of agreement on OSA diagnosis between PSG-AHI and Biologix-ODI, without and with sleep prediction, Bland-Altman plots were performed. RStudio 2023.06.1 software (R Foundation for Statistical Computing) was used for all statistical analysis. Significance was assessed with a p-value \(<0.05\).

Out of 304 consecutive patients previously evaluated to validate the Biologix system against PSG5, 268 had snoring recordings and were used for this study. The patients had typical characteristics of patients referred for OSA diagnosis, and were predominantly obese middle-aged adults, with comorbidities and with a high proportion of moderate to severe OSA (Table 2). As described in the method section, the sleep studies underwent 10-fold cross-validation. Each fold consisted of a training set of 90% of the patients (approximately 241 patients, corresponding to 221,639 epochs) and a test set of 10% of the patients (approximately 27 patients, corresponding to 24,627 epochs).

The total sleep time determined by PSG and by the Biologix ANN model using all channels was similar (\(353\pm 65\) min vs. \(359\pm 56\) min, respectively, p=0.15), as well as sleep efficiency (\(0.76\pm 0.12\) % vs. \(0.77\pm 0.10\) %, respectively, p=0.15). Table 3 shows the performance metrics of the ANN model using only the accelerometer and the ANN model using all Biologix channels to predict sleep, as assessed by k-fold cross-validation. The performance of the ANN when all Biologix channels were used was significantly higher than when only accelerometer data was used (p\(<0.001\)), as revealed by McNemar’s test. The ANN model using data from all Biologix channels achieved higher AUC values of the receiver operating characteristic (ROC) curve, indicating superior performance in predicting sleep (Fig. 3).

Receiver operating characteristic (ROC) curve. The ROC curve for the ANN using only the accelerometer and the ANN using all Biologix channels (\(\hbox {SpO}_2\), HR, accelerometer, and snoring).

The ability to predict sleep was evaluated for its impact on OSA severity determination. AHI was classified using cut-offs of 5, 15, and 30 events/h. We calculated the specificity, sensitivity, accuracy, PPV, and NPV of the Biologix system, without and with sleep prediction (Table 4). Compared to Biologix-ODI, the specificity to predict OSA as defined as AHI \(\ge\) 5 or 15 events/h increased with sleep prediction, however, the improvement did not reach statistical significance.

Bland-Altman analysis between PSG-AHI versus Biologix-ODI showed that the limits of agreement were similar without and with sleep prediction ([-20,27] vs. [-20,22], respectively), whereas the bias (mean difference) decreased significantly with sleep prediction (3.40 vs. 1.02, p\(<0.001\)) (Fig. 4a and b).

Bland-Altman plots for PSG-AHI and Biologix-ODI. (a) Bland-Altman plot for PSG-AHI and Biologix-ODI without sleep prediction. (b) Bland-Altman plot for PSG-AHI and Biologix-Sleep-ODI with sleep prediction. AHI apnea-hypopnea index, PSG polysomnography, ODI oxygen desaturation index considering total recording time, Sleep ODI oxygen desaturation index considering total sleep time.

In the present study, we built and validated a new algorithm to predict sleep based on the ANN model using data derived from the Biologix system, including a high-resolution oximeter with built-in accelerometer, and snoring detected by the smartphone with the Biologix app.5,6. Firstly, the algorithm exhibited good performance in distinguishing sleep from awake, with a sensitivity of 91.5%, specificity of 71.0%, accuracy of 86.1%, Cohen’s kappa coefficient of 0.60, F1-score of 0.90, and AUC of 0.90. Secondly, the algorithm performance to predict sleep was better using all Biologix channels compared to using only the accelerometer channel. Thirdly, Biologix system with sleep prediction decreased the bias between PSG-AHI and Biologix-ODI.

PM has been validated to detect OSA18,19. However, in contrast to PSG, PM does not discriminate wakefulness from sleep. In this context, several studies have attempted to predict sleep using a binary classification (sleep versus wakefulness) mainly using actigraphy. Overall, actigraphy is recognized as an accurate and sensitive method to detect sleep periods, but with poor specificity to identify wakefulness (ranging from 32 and 61%)20,21,22. A study carried out with 8 commercial sleep tracking devices, showed that the sensitivity to detect sleep, as compared to PSG, was very high (all greater than or equal to 93%). However, the specificity for predicting sleep was variable and generally low, ranging from 18 to 54%23. Alternatively, algorithm models based on data from several channels, either isolated or in combination, including accelerometer, respiratory signals, breathing sounds, and HRV, have been used in an attempt to improve sleep/wakefulness detection by PM22,24,25,26,27,28,29,30,31. For instance, Dafna et al.26 developed and validated an algorithm for detecting sleep periods in patients with OSA based on the analysis of respiratory sounds. Despite the high sensitivity of 92.2%, the specificity to detect wakefulness was low (56.6%). In turn, Montazeri et al.32 reported promising results with a sensitivity of 87.8%, specificity of 71.4%, and accuracy of 82.3% to detect sleep using an algorithm model based on tracheal sound and movement data recorded with a small wearable device attached over the trachea. The reported device is also able to predict the AHI based on tracheal sounds, however, it does not measure \(\hbox {SpO}_2\). In general, the main challenge of all systems that do not measure EEG is to achieve high sensitivity to detect sleep while also maintaining reasonable specificity. To address this challenge, our algorithm combines built-in accelerometer data with other variables, including \(\hbox {SpO}_2\), HR, and snoring. As compared to the ANN algorithm using only the accelerometer, the ANN algorithm using all channels had a better performance in detecting sleep. Specifically, the specificity increased from 65.4% to 71.0%, without compromising sensitivity (91.7% vs. 91.5%, respectively), resulting in an increase of the accuracy (84.5% vs. 86.1%). In addition, the AUC increased from 0.88 to 0.90, indicating a greater ability to discriminate between sleep and wakefulness when all Biologix channels were used.

Compared to PSG-AHI, Biologix-ODI with sleep prediction (Biologix-Sleep-ODI) improved the performance of Biologix-ODI.The Bland-Altman plots (Fig. 4) showed a significant decrease in the bias between PSG-AHI and Biologix-ODI when sleep prediction was taken into account (Biologix-Sleep-ODI). As compared to Biologix-ODI, the specificity to predict OSA as defined as AHI \(\ge\) 5 or 15 events/h increased with sleep prediction (Biologix-Sleep-ODI), however, the improvement did not reach statistical significance (Table 4).

Despite the strengths, our study has limitations. Firstly, the number of patients was relatively small. On the other hand, because sleep prediction was based on 30-second epoch, the study used a large data set to build an algorithm to predict sleep. Secondly, the study was carried out using data from patients with suspected OSA, so the accuracy of our ANN model to predict sleep in healthy subjects may be different. Another caveat is that in contrast to wrist actigraphy, Biologix built-in actigraphy is placed on the finger. Although we have no reason to believe that there is a substantial difference between wrist and finger movements, we acknowledge that this comparison was not performed.

In conclusion, we showed that an algorithm based on ANN using all Biologix channels, including \(\hbox {SpO}_2\), HR, accelerometer, and snoring is able to detect sleep with a good accuracy. Sleep prediction resulted in a reduction in the bias between PSG-AHI and Biologix-Sleep-ODI.

The data that support the findings of this study are available from Biologix Sistemas S.A., but restrictions apply to the availability of these data, which were used under license for the current study and are not publicly available. Data are, however, available from the authors upon reasonable request and with the permission of Biologix Sistemas S.A.

Sleep-related breathing disorders in adults. Recommendations for syndrome definition and measurement techniques in clinical research. Sleep 22, 667–689 (1999).

Colten, H. R. & Altevogt, B. M. Sleep Disorders and Sleep Deprivation: An Unmet Public Health Problem (Institute of Medicine, 2006).

Google Scholar

Rundo, J. V. & Downey, R. Chapter 25 - polysomnography. In Levin, K. H. & Chauvel, P. (eds.) Clinical Neurophysiology: Basis and Technical Aspects. In Handbook of Clinical Neurology, vol. 160, 381–392 (Elsevier, 2019).

Practice Committee of the American Sleep Disorders Association. Practice parameters for the use of portable recording in the assessment of obstructive sleep apnea. Sleep 17, 372–377 (1994).

Google Scholar

Do Lago Pinheiro, G. et al. Validation of an overnight wireless high-resolution oximeter plus cloud-based algorithm for the diagnosis of obstructive sleep apnea. Clinics 75, e2414 (2020).

Article Google Scholar

Hasan, R. et al. Validation of an overnight wireless high-resolution oximeter for the diagnosis of obstructive sleep apnea at home. Sci. Rep. 12, 15136 (2022).

Article ADS PubMed PubMed Central Google Scholar

Johnson, K. Supervised Learning for Sequence Prediction Using Keras Sequential Models. Master’s thesis, California State University, Northridge (2018).

Poole, D. L. & Mackworth, A. K. Artificial Intelligence: Foundations of Computational Agents (Cambridge University Press, 2010).

Book Google Scholar

Camacho, M. et al. Smartphone apps for snoring. J. Laryngol. Otol. 129, 974–979 (2015).

Article PubMed Google Scholar

Poldrack, R. A., Huckins, G. & Varoquaux, G. Establishment of best practices for evidence for prediction: a review. JAMA Psychiat. 77, 534–540 (2020).

Article Google Scholar

Mencar, C. et al. Application of machine learning to predict obstructive sleep apnea syndrome severity. Health Inform. J. 26, 298–317 (2020).

Article Google Scholar

Leung, H. & Haykin, S. The complex backpropagation algorithm. IEEE Trans. Signal Process. 39, 2101–2104 (1991).

Article ADS Google Scholar

Prechelt, L. Early stopping-but when? In Neural Networks: Tricks of the trade 55–69 (Springer, 2002).

Google Scholar

Rasamoelina, A. D., Adjailia, F. & Sinčák, P. A review of activation function for artificial neural network. In 2020 IEEE 18th World Symposium on Applied Machine Intelligence and Informatics (SAMI), 281–286 (IEEE, 2020).

Pedregosa, F. et al. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).

MathSciNet Google Scholar

Buitinck, L. et al. API design for machine learning software: experiences from the scikit-learn project. In ECML PKDD Workshop: Languages for Data Mining and Machine Learning, 108–122 (2013).

Kim, S. & Lee, W. Does Mcnemar’s test compare the sensitivities and specificities of two diagnostic tests?. Stat. Methods Med. Res. 26, 142–154 (2017).

Article MathSciNet PubMed Google Scholar

Khor, Y. H. et al. Portable evaluation of obstructive sleep apnea in adults: A systematic review. Sleep Med. Rev. 101743 (2023).

Collop, N. A. et al. Obstructive sleep apnea devices for out-of-center (ooc) testing: technology evaluation. J. Clin. Sleep Med. 7, 531–548 (2011).