CoChat
Sign Up
Cited by 151EEG and Brain-Computer InterfacesSleep and related disordersECG Monitoring and Analysis

Automatic Human Sleep Stage Scoring Using Deep Neural Networks

Open access
Malafeev, Alexander, Laptev, Dmitry, Bauer, Stefan, Omlin, Ximena, Wierzbicka, Aleksandra, Wichniak, Adam, Jernajczyk, Wojciech, Riener, Robert +2 more·2018·Frontiers in Neuroscience
DOI 10.3389/fnins.2018.00781PMC PMC6232272
Open this paper in CoChat — free accountOpen PDF

Abstract

The classification of sleep stages is the first and an important step in the quantitative analysis of polysomnographic recordings. Sleep stage scoring relies heavily on visual pattern recognition by a human expert and is time consuming and subjective. Thus, there is a need for automatic classification. In this work we developed machine learning algorithms for sleep classification: random forest (RF) classification based on features and artificial neural networks (ANNs) working both with features and raw data. We tested our methods in healthy subjects and in patients. Most algorithms yielded good results comparable to human interrater agreement. Our study revealed that deep neural networks (DNNs) working with raw data performed better than feature-based methods. We also demonstrated that taking the local temporal structure of sleep into account a priori is important. Our results demonstrate the utility of neural network architectures for the classification of sleep.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/fnins.2018.00781

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • AI provides an inexpensive and objective alternative to manual sleep stage scoring by translating collected sensor data into pre-defined knowledge (e.g., class label).

    “This is achieved by translating collected sensor data into pre-defined knowledge (e.g., class label), providing an inexpensive and objective alternative to manual sleep stage scoring95.”

    From The future of sleep health: a data-driven revolution in sleep science and medicine (2020) · cites this paper as [95]

    The Introduction (Problem Statement; page unavailable) says expert visual scoring is time-consuming and subjective, while the Methods (Machine Learning: Classification; page unavailable) describes classifying recorded signals into sleep stages. This supports automated classification as an alternative to manual scoring, but the paper does not establish that it is inexpensive. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 75

  1. Abadi M. Barham P. Chen J. Chen Z. Davis A. Dean J. (2016). “TensorFlow: a system for large-scale machine learning,” in Proceeding OSDI’16 Proceedings of the 12th USENIX conference on Operating Systems Design and Implementation, Savannah, GA, 265– 283.
  2. Abdel-Hamid O. Mohamed A.-R. Jiang H. Deng L. Penn G. Yu D. (2014). Convolutional neural networks for speech recognition. IEEE/ACM Trans. Audio Speech Lang. Proc. 22 1533– 1545.
  3. Agarwal R. Gotman J. (2001). Computer-assisted sleep staging. IEEE Trans. Biomed. Eng. 48 1412– 1423. 11759922
  4. Al-Rfou R. Alain G. Almahairi A. Angermueller C. Bahdanau D. Ballas N. (2016). Theano: a python framework for fast computation of mathematical expressions. arXiv:1605.02688 [Preprint]. · arXiv
  5. Anderer P. Gruber G. Parapatics S. Woertz M. Miazhynskaia T. Klösch G. (2005). An E-health solution for automatic sleep classification according to Rechtschaffen and Kales: validation study of the Somnolyzer 24 × 7 utilizing the Siesta database. Neuropsychobiology 51 115– 133. 10.1159/000085205 15838184 · DOI
  6. Babadi B. Brown E. N. (2014). A review of multitaper spectral analysis. IEEE Trans. Biomed. Eng. 61 1555– 1564. 10.1109/TBME.2014.2311996 24759284 · DOI
  7. Bishop C. (2016). Pattern Recognition and Machine Learning. New York, NY: Springer-Verlag.
  8. Breiman L. (2001). Random forests. Mach. Learn. 45 5– 32. 10.1023/A:1010933404324 · DOI · Paper page
  9. Breiman L. Friedman J. Stone C. J. Olshen R. A. (1984). Classification and Regression Trees. Boca Raton, FL: CRC press.
  10. Cecotti H. Graeser A. (2008). “Convolutional neural network with embedded Fourier transform for EEG classification,” in Pattern Recognition, 2008. ICPR 2008. 19th International Conference on: IEEE, Tampa, FL, 1– 4.
  11. Chambon S. Galtier M. N. Arnal P. J. Wainrib G. Gramfort A. (2018). A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series. IEEE Trans. Neural Syst. Rehabil. Eng. 26 758– 769. 10.1109/TNSRE.2018.2813138 29641380 · DOI
  12. Cohen J. (1960). A coefficient of agreement for nominal scales. Edu. Psychosoc. Measure. 20 37– 46. 10.1177/001316446002000104 · DOI · Paper page
  13. Danker-Hopfe H. Anderer P. Zeitlhofer J. Boeck M. Dorn H. Gruber G. (2009). Interrater reliability for sleep scoring according to the Rechtschaffen & Kales and the new AASM standard. J. Sleep Res. 18 74– 84. 10.1111/j.1365-2869.2008.00700.x 19250176 · DOI · Paper page
  14. Danker-Hopfe H. Kunz D. Gruber G. Klösch G. Lorenzo J. L. Himanen S.-L. (2004). Interrater reliability between scorers from eight European sleep laboratories in subjects with different sleep disorders. J. Sleep Res. 13 63– 69. 10.1046/j.1365-2869.2003.00375.x 14996037 · DOI
  15. Davidson P. Jones R. Peiris M. (2006). “Detecting behavioral microsleeps using EEG and LSTM recurrent neural networks,” in Engineering in Medicine and Biology Society, 2005. IEEE-EMBS 2005. 27th Annual International Conference of the: IEEE, Shanghai, 5754– 5757. 17281565
  16. De Boer P.-T. Kroese D. P. Mannor S. Rubinstein R. Y. (2005). A tutorial on the cross-entropy method. Ann. Operat. Res. 134 19– 67. 10.1007/s10479-005-5724-z · DOI
  17. Dice L. R. (1945). Measures of the amount of ecologic association between species. Ecology 26 297– 302. 10.2307/1932409 · DOI
  18. Doroshenkov L. Konyshev V. Selishchev S. (2007). Classification of human sleep stages based on EEG processing using hidden Markov models. Biomed. Eng. 41 25– 28. 10.1007/s10527-007-0006-5 17419342 · DOI
  19. Dos Santos C. Gatti M. (2014). “Deep convolutional neural networks for sentiment analysis of short texts,” in Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, Santa Fe, NM, 69– 78.
  20. Farley B. Clark W. (1954). Simulation of self-organizing systems by digital computer. Trans. IRE Prof. Group Inform. Theory 4 76– 84. 10.1109/TIT.1954.1057468 · DOI
  21. Fell J. Röschke J. Mann K. Schäffner C. (1996). Discrimination of sleep stages: a comparison between spectral and nonlinear EEG measures. Electroencephalogr. Clin. Neurophysiol. 98 401– 410. 10.1016/0013-4694(96)95636-9 8647043 · DOI
  22. Fukushima K. Miyake S. (1982). “Neocognitron: a self-organizing neural network model for a mechanism of visual pattern recognition,” in Competition and Cooperation in Neural Nets, (Berlin: Springer), 267– 285. 10.1007/978-3-642-46466-9_18 · DOI · Paper page
  23. Gaillard J. Tissot R. (1973). Principles of automatic analysis of sleep records with a hybrid system. Comput. Biomed. Res. 6 1– 13. 10.1016/0010-4809(73)90059-1 4348642 · DOI
  24. Gath I. Geva A. B. (1989). Unsupervised optimal fuzzy clustering. IEEE Trans. Pattern Anal. Mach. Intell. 11 773– 780. 10.1109/34.192473 · DOI
  25. Gevins A. S. Rémond A. (1987). Methods of Analysis of Brain Electrical and Magnetic Signals. Amsterdam: Elsevier Science Limited.
  26. Goodfellow I. Bengio Y. Courville A. Bengio Y. (2016). Deep Learning. Cambridge: MIT press.
  27. Graves A. Mohamed A.-R. Hinton G. (2013). “Speech recognition with deep recurrent neural networks,” in Acoustics, speech and signal processing (ICASSP), 2013 IEEE International Conference on: IEEE, Vancouver, BC, 6645– 6649. 10.1109/ICASSP.2013.6638947 · DOI · Paper page
  28. Grube G. Flexer A. Dorffner G. (2002). Unsupervised continuous sleep analysis. Methods Find. Exp. Clin. Pharmacol. 24 51– 56. 12575468
  29. Ho T. K. (1995). “Random decision forests,” in Document Analysis and Recognition, 1995., Proceedings of the Third International Conference on: IEEE, Montreal, QC, 278– 282.
  30. Hochreiter S. Schmidhuber J. (1997). Long short-term memory. Neural Comput. 9 1735– 1780. 10.1162/neco.1997.9.8.1735 9377276 · DOI · Paper page
  31. Hunt E. B. Marin J. Stone P. J. (1966). Experiments in Induction. Oxford: Academic Press.
  32. Ioffe S. Szegedy C. (2015). Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167 [Preprint]. · arXiv
  33. Itil T. Shapiro D. Fink M. Kassebaum D. (1969). Digital computer classifications of EEG sleep stages. Electroencephalogr. Clin. Neurophysiol. 27 76– 83. 10.1016/0013-4694(69)90112-6 4182894 · DOI
  34. Ivakhnenko A. G. Lapa V. G. E. (1967). Cybernetics and Forecasting Techniques. New York, NY: American Elsevier Pub. Co.
  35. Jouvet M. Michel F. Mounier D. (1960). Analyse électroencépholographique comparée du sommeil physiologique chez le chat et chez l’homme. Revue Neurologique 103 189D– 205D. 13790851
  36. Klosh G. Kemp B. Penzel T. Schlogl A. Rappelsberger P. Trenker E. (2001). The SIESTA project polygraphic and clinical database. IEEE Eng. Med. Biol. Mag. 20 51– 57. 10.1109/51.932725 11446210 · DOI
  37. Längkvist M. Karlsson L. Loutfi A. (2012). Sleep stage classification using unsupervised feature learning. Adv. Artif. Neural Syst. 2012: 5 10.1155/2012/107046 · DOI
  38. Laptev D. Buhmann J. M. (2014). “Convolutional decision trees for feature learning and segmentation,” in German Conference on Pattern Recognition, (Cham: Springer), 95– 106. 10.1007/978-3-319-11752-2_8 · DOI
  39. Laptev D. Buhmann J. M. (2015). “Transformation-invariant convolutional jungles,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Denver, CO, 3043– 3051. 10.1109/CVPR.2015.7298923 · DOI
  40. Larsen L. Walter D. (1970). On automatic methods of sleep staging by EEG spectra. Electroencephalogr. Clin. Neurophysiol. 28 459– 467. 10.1016/0013-4694(70)90271-3 4192812 · DOI
  41. Libourel P.-A. Corneyllie A. Luppi P.-H. Chouvet G. Gervasoni D. (2015). Unsupervised online classifier in sleep scoring for sleep deprivation studies. Sleep 38 815– 828. 10.5665/sleep.4682 25325478 PMC4402670 · DOI
  42. Louis R. P. Lee J. Stephenson R. (2004). Design and validation of a computer-based sleep-scoring algorithm. J. Neurosci. Methods 133 71– 80. 10.1016/j.jneumeth.2003.09.025 14757347 · DOI
  43. Malafeev A. Omlin X. Wierzbicka A. Wichniak A. Jernajczyk W. Riener R. (2018). Automatic artefact detection in single-channel sleep EEG recordings. J. Sleep Res. 10.1111/jsr.12679 [Epub ahead of print]. 29516562 · DOI
  44. Martin W. Johnson L. Viglione S. Naitoh P. Joseph R. Moses J. (1972). Pattern recognition of EEG-EOG as a technique for all-night sleep stage scoring. Electroencephalogr. Clin. Neurophysiol. 32 417– 427. 10.1016/0013-4694(72)90009-0 4111497 · DOI
  45. Mayers A. G. Baldwin D. S. (2005). Antidepressants and their effect on sleep. Hum. Psychopharmacol. Clin. Exp. 20 533– 559. 10.1002/hup.726 16229049 · DOI
  46. Mccarley R. W. (2007). Neurobiology of REM and NREM sleep. Sleep Med. 8 302– 330. 10.1016/j.sleep.2007.03.005 17468046 · DOI
  47. Mchugh M. L. (2012). Interrater reliability: the kappa statistic. Biochem. Med. 22 276– 282. 10.11613/BM.2012.031 PMC3900052 23092060 · DOI · Paper page
  48. Mirowski P. W. Lecun Y. Madhavan D. Kuzniecky R. (2008). “Comparing SVM and convolutional networks for epileptic seizure prediction from intracranial EEG,” in Machine Learning for Signal Processing, 2008. MLSP 2008. IEEE Workshop on: IEEE, Cancún, 244– 249.
  49. Mitchell T. M. (1997). Machine Learning. Boston, MA: McGraw-Hill.
  50. Morgan J. N. Sonquist J. A. (1963). Problems in the analysis of survey data, and a proposal. J. Am. Stat. Assoc. 58 415– 434. 10.1080/01621459.1963.10500855 · DOI
  51. Omlin X. Crivelli F. Näf M. Heinicke L. Skorucak J. Malafeev A. (2018). The effect of a slowly rocking bed on sleep. Sci. Rep. 8: 2156. 10.1038/s41598-018-19880-3 29391413 PMC5794757 · DOI
  52. Pan S.-T. Kuo C.-E. Zeng J.-H. Liang S.-F. (2012). A transition-constrained discrete hidden Markov model for automatic sleep staging. Biomed. Eng. Online 11: 52. 10.1186/1475-925X-11-52 22908930 PMC3462123 · DOI
  53. Pardey J. Roberts S. Tarassenko L. Stradling J. (1996). A new approach to the analysis of the human sleep/wakefulness continuum. J. Sleep Res. 5 201– 210. 10.1111/j.1365-2869.1996.00201.x 9065871 · DOI
  54. Park H.-J. Oh J.-S. Jeong D.-U. Park K.-S. (2000). Automated sleep stage scoring using hybrid rule-and case-based reasoning. Comput. Biomed. Res. 33 330– 349. 10.1006/cbmr.2000.1549 11017725 · DOI
  55. Penzel T. Zhang X. Fietze I. (2013). Inter-scorer reliability between sleep centers can teach us what to improve in the scoring rules. J. Clin. Sleep Med. 9 89– 91. 10.5664/jcsm.2352 23319911 PMC3525995 · DOI
  56. Phan H. Andreotti F. Cooray N. Chén O. Y. De Vos M. (2018). Joint classification and prediction CNN framework for automatic sleep stage classification. arXiv: 1805.06546 [Preprint]. PMC6487915 30346277 · arXiv
  57. Powers D. (2014). What the F-measure doesn’t measure: features, flaws, fallacies and fixes. arXiv: 1503.06410 [Preprint]. · arXiv
  58. Rochester N. Holland J. Haibt L. Duda W. (1956). Tests on a cell assembly theory of the action of the brain, using a large digital computer. IRE Trans. Inform. Theory 2 80– 93. 10.1109/TIT.1956.1056810 · DOI
  59. Rosenberg R. S. Van Hout S. (2013). The american academy of sleep medicine inter-scorer reliability program: sleep stage scoring. J. Clin. Sleep Med. 9: 81. 10.5664/jcsm.2350 23319910 PMC3525994 · DOI · Paper page
  60. Safavian S. R. Landgrebe D. (1991). A survey of decision tree classifier methodology. IEEE Trans. Syst. Man Cybernet. 21 660– 674. 10.1109/21.97458 · DOI
  61. Schaltenbrand N. Lengelle R. Macher J.-P. (1993). Neural network model: application to automatic analysis of human sleep. Comput. Biomed. Res. 26 157– 171. 10.1006/cbmr.1993.1010 8477587 · DOI
  62. Sharpley A. Williamson D. Attenburrow M. Pearson G. Sargent P. Cowen P. (1996). The effects of paroxetine and nefazodone on sleep: a placebo controlled trial. Psychopharmacology 126 50– 54. 10.1007/BF02246410 8853216 · DOI
  63. Smith J. R. Karacan I. (1971). EEG sleep stage scoring by an automatic hybrid system. Electroencephalogr. Clin. Neurophysiol. 31 231– 237. 10.1016/0013-4694(71)90092-7 4105870 · DOI
  64. Sors A. Bonnet S. Mirek S. Vercueil L. Payen J.-F. (2018). A convolutional neural network for sleep stage scoring from raw single-channel EEG. Biomed. Signal Proc. Control 42 107– 114. 10.1109/TNSRE.2017.2721116 28678710 · DOI · Paper page
  65. Stanus E. Lacroix B. Kerkhofs M. Mendlewicz J. (1987). Automated sleep scoring: a comparative reliability study of two algorithms. Electroencephalogr. Clin. Neurophysiol. 66 448– 456. 10.1016/0013-4694(87)90214-8 2435525 · DOI
  66. Stratonovich R. L. E. (1960). Conditional markov processes. Theory Probabil. Appl. 5 156– 178. 10.1137/1105015 · DOI · Paper page
  67. Sun H. Jia J. Goparaju B. Huang G.-B. Sourina O. Bianchi M. T. (2017). Large-scale automated sleep staging. Sleep 40 zsx139. 10.1093/sleep/zsx139 29029305 PMC6251659 · DOI
  68. Sunagawa G. A. Séi H. Shimba S. Urade Y. Ueda H. R. (2013). FASTER: an unsupervised fully automated sleep staging method for mice. Genes Cells 18 502– 518. 10.1111/gtc.12053 23621645 PMC3712478 · DOI
  69. Supratak A. Dong H. Wu C. Guo Y. (2017). DeepSleepNet: a model for automatic sleep stage scoring based on raw single-channel EEG. IEEE Trans. Neural. Syst. Rehabil. Eng. 25 1998– 2008. 10.1109/TNSRE.2017.2721116 28678710 · DOI · Paper page
  70. Tinguely G. Landolt H. Cajochen C. (2014). Sleep habits, sleep quality and sleep medicine use of the Swiss population result. Ther. Umsch. 71 637– 646. 10.1024/0040-5930/a000604 25377287 · DOI
  71. Tsinalis O. Matthews P. M. Guo Y. Zafeiriou S. (2016). Automatic sleep stage scoring with single-channel EEG using convolutional neural networks. arXiv: 1610.01683 [Preprint]. · arXiv
  72. Von Rotz R. Kometer M. Dornbierer D. Gertsch J. Gachet M. S. Vollenweider F. X. (2017). Neuronal oscillations and synchronicity associated with gamma-hydroxybutyrate during resting-state in healthy male volunteers. Psychopharmacology 234 1957– 1968. 10.1007/s00213-017-4603-z 28429067 · DOI
  73. Welch P. (1967). The use of fast Fourier transform for the estimation of power spectra: a method based on time averaging over short, modified periodograms. IEEE Trans. Audio Electroacoust. 15 70– 73. 10.1109/TAU.1967.1161901 · DOI
  74. Younes M. Kuna S. T. Pack A. I. Walsh J. K. Kushida C. A. Staley B. (2018). Reliability of the American academy of sleep medicine rules for assessing sleep depth in clinical practice. J. Clin. Sleep Med. 14 205– 213. 10.5664/jcsm.6934 29351821 PMC5786839 · DOI
  75. Younes M. Raneri J. Hanly P. (2016). Staging sleep in polysomnograms: analysis of inter-scorer variability. J. Clin. Sleep Med. 12: 885. 10.5664/jcsm.5894 27070243 PMC4877322 · DOI

Cited by

Sorted by citations

Authors
Malafeev, A., Laptev, D., Bauer, S., Omlin, X., Wierzbicka, A., Wichniak, A., Jernajczyk, W., Riener, R., Buhmann, J., & Achermann, P.
Affiliations
  • University of Zurich (CH)
  • ETH Zurich (CH)
  • Institute of Psychiatry and Neurology (PL)
Corresponding author
Achermann, Peter
ORCID
Venue
Frontiers in Neuroscience
Publisher
Frontiers Media SA
Published
2018-11-06
Volume
12
Pages
781-781
ISSN
1662-453X (ISSN-L)
DOI
10.3389/fnins.2018.00781
PMC
PMC6232272
OpenAlex
W2900178629
Cited by
151
References
7
Citations by year
2026: 9 · 2025: 7 · 2024: 19 · 2023: 25 · 2022: 20 · 2021: 33 · 2020: 25 · 2019: 13
FWCI
10.57
Type
journal article
Language
en
Open access
Licence
Locations
  • Frontiers in Neuroscience (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Repository for Publications and Research Data (ETH Zurich) (publishedVersion, cc-by)
  • Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • MPG.PuRe (Max Planck Society) (submittedVersion)
  • Zurich Open Repository and Archive (University of Zurich) (submittedVersion, cc-by)
  • ETH Zürich Research Collection (cc-by)
  • Zurich Open Repository and Archive (University of Zurich)
Funders
Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Sleep and related disorders
  • Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › ECG Monitoring and Analysis
Keywords
sleep stage classification, automated sleep scoring, polysomnography, deep neural network, random forest, feature-based classification, interobserver agreement

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review