The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
An open-source, high-performance tool for automated sleep staging
Open accessAbstract
The clinical and societal measurement of human sleep has increased exponentially in recent years. However, unlike other fields of medical analysis that have become highly automated, basic and clinical sleep research still relies on human visual scoring. Such human-based evaluations are time-consuming, tedious, and can be prone to subjective bias. Here, we describe a novel algorithm trained and validated on +30,000 hr of polysomnographic sleep recordings across heterogeneous populations around the world. This tool offers high sleep-staging accuracy that matches human scoring accuracy and interscorer agreement no matter the population kind. The software is designed to be especially easy to use, computationally low-demanding, open source, and free. Our hope is that this software facilitates the broad adoption of an industry-standard automated sleep staging software package.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,887
Welch, P. (1967) · IEEE Transactions on Audio and Electroacoustics · doi:10.1109/tau.1967.1161901 · Cited by 12,277
From local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551 · cited 1×
“Features importance was measured on the full training set using the Shapley Additive Explanation algorithm (SHAP; Lundberg et al., 2020 ).”
MethodChicco, D., & Jurman, G. (2020) · BMC Genomics · doi:10.1186/s12864-019-6413-7 · Cited by 6,143 · cited 1×
“The latter is thought to be the most robust and informative score for classification as it naturally takes into account imbalance between sleep stages ( Chicco and Jurman, 2020 ).”
BackgroundBerry, R. B., Budhiraja, R., Gottlieb, D. J., Gozal, D., Iber, C., Kapur, V. K., Marcus, C. L., Mehra, R., Parthasarathy, S., Quan, S. F., et al. (2012) · Journal of Clinical Sleep Medicine · doi:10.5664/jcsm.2172 · Cited by 5,302
Permutation Entropy: A Natural Complexity Measure for Time Series
Bandt, C., & Pompe, B. (2002) · Physical Review Letters · doi:10.1103/physrevlett.88.174102 · Cited by 4,867
Ohayon, M. M., Carskadon, M. A., Guilleminault, C., & Vitiello, M. V. (2004) · Sleep · doi:10.1093/sleep/27.7.1255 · Cited by 3,349
Sleep duration and all-cause mortality: a systematic review and meta-analysis of prospective studies
Cappuccio, F. P., D'Elia, L., Strazzullo, P., & Miller, M. A. (2010) · Sleep · doi:10.1093/sleep/33.5.585 · Cited by 2,224
Approach to an irregular time series on the basis of the fractal theory
Higuchi, T. (1988) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(88)90081-4 · Cited by 2,160
MNE software for processing MEG and EEG data
Gramfort, A., Luessi, M., Larson, E., Engemann, D. A., Strohmeier, D., Brodbeck, C., Parkkonen, L., & Hämäläinen, M. S. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.10.027 · Cited by 1,951
EEG analysis based on time domain properties
Hjorth, B. (1970) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(70)90143-4 · Cited by 1,504
The Sleep-Immune Crosstalk in Health and Disease
Besedovsky, L., Lange, T., & Haack, M. (2019) · Physiological Reviews · doi:10.1152/physrev.00010.2018 · Cited by 1,479 · cited 1×
“Adequate sleep supports a panoply of physiological body functions, including immune, metabolic, and cardiovascular systems ( Besedovsky et al., 2019 ; Cappuccio and Miller, 2017 ; Harding et al., 2020 ).”
BackgroundThe visual scoring of sleep in adults
Silber, M. H., Ancoli-Israel, S., Bonnet, M. H., Chokroverty, S., Grigg-Damberger, M. M., Hirshkowitz, M., Kapen, S., Keenan, S. A., Kryger, M. H., Penzel, T., et al. (2007) · Journal of Clinical Sleep Medicine · doi:10.5664/jcsm.26814 · Cited by 1,064
Sleep and Alzheimer disease pathology--a bidirectional relationship
Ju, Y.-E. S., Lucey, B. P., & Holtzman, D. M. (2014) · Nature Reviews Neurology · doi:10.1038/nrneurol.2013.269 · Cited by 996 · cited 1×
“…strategy to reduce the risk of cardio-vascular and metabolic disease, all-cause mortality risk, and more recently, the accumulation of Alzheimer’s disease pathology within the brain ( Cappuccio et al., 2010 ; Leary et al., 2020Cappuccio and Miller, 2017 ; Ju et al., 2014 ; Winer et al., 2020 ).”
BackgroundThe Role of Sleep in Cognition and Emotion
Walker, M. P. (2009) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2009.04416.x · Cited by 993
Self-citationThe National Sleep Research Resource: towards a sleep data commons
Zhang, G.-Q., Cui, L., Mueller, R., Tao, S., Kim, M., Rueschman, M., Mariani, S., Mobley, D., & Redline, S. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy064 · Cited by 917
Staresina, B. P., Bergmann, T. O., Bonnefond, M., van der Meij, R., Jensen, O., Deuker, L., Elger, C. E., Axmacher, N., & Fell, J. (2015) · Nature Neuroscience · doi:10.1038/nn.4119 · Cited by 831 · cited 1×
“…more complex and novel analysis methods that we and others have developed, including event-locked cross-frequency coupling ( Helfrich et al., 2018 ; Staresina et al., 2015 ) and decomposition of the sleep power spectrum into aperiodic and oscillatory components ( Lendner et al., 2020 ; Wen and…”
MethodCurrent Applications and Future Impact of Machine Learning in Radiology
Choy, G., Khalilzadeh, O., Michalski, M., Do, S., Samir, A. E., Pianykh, O. S., Geis, J. R., Pandharipande, P. V., Brink, J. A., & Dreyer, K. J. (2018) · Radiology · doi:10.1148/radiol.2018171820 · Cited by 800
Phan, H., Andreotti, F., Cooray, N., Chen, O. Y., & De Vos, M. (2019) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2019.2896659 · Cited by 623
Old brains come uncoupled in sleep: slow wave-spindle synchrony, brain atrophy, and forgetting
Helfrich, R. F., Mander, B. A., Jagust, W. J., Knight, R. T., & Walker, M. P. (2018) · Neuron · doi:10.1016/j.neuron.2017.11.020 · Cited by 497
Self-citationCarrier, J., Land, S., Buysse, D. J., Kupfer, D. J., & Monk, T. H. (2001) · Psychophysiology · doi:10.1111/1469-8986.3820232 · Cited by 494
A comparison of waveform fractal dimension algorithms
Esteller, R., Vachtsevanos, G., Echauz, J., & Litt, B. (2001) · IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications · doi:10.1109/81.904882 · Cited by 470
The American Academy of Sleep Medicine Inter-scorer Reliability Program: Sleep Stage Scoring
Rosenberg, R. S., & Van Hout, S. (2013) · Journal of Clinical Sleep Medicine · doi:10.5664/jcsm.2350 · Cited by 456
A convolutional neural network for sleep stage scoring from raw single-channel EEG
Sors, A., Bonnet, S., Mirek, S., Vercueil, L., & Payen, J.-F. (2018) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2017.12.001 · Cited by 424
Separating Fractal and Oscillatory Components in the Power Spectrum of Neurophysiological Signal
Wen, H., & Liu, Z. (2016) · Brain Topography · doi:10.1007/s10548-015-0448-0 · Cited by 419
Neural network analysis of sleep stages enables efficient diagnosis of narcolepsy
Stephansen, J. B., Olesen, A. N., Olsen, M., Ambati, A., Leary, E. B., Moore, H. E., Carrillo, O., Lin, L., Han, F., Yan, H., et al. (2018) · Nature Communications · doi:10.1038/s41467-018-07229-3 · Cited by 413
An electrophysiological marker of arousal level in humans
Lendner, J. D., Helfrich, R. F., Mander, B. A., Romundstad, L., Lin, J. J., Walker, M. P., Larsson, P. G., & Knight, R. T. (2020) · eLife · doi:10.7554/elife.55092 · Cited by 410
Self-citationU-Sleep: resilient high-frequency sleep staging
Perslev, M., Darkner, S., Kempfner, L., Nikolic, M., Jennum, P. J., & Igel, C. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00440-5 · Cited by 362
A Novel Algorithm for the Automatic Detection of Sleep Apnea From Single-Lead ECG
Varon, C., Caicedo, A., Testelmans, D., Buyse, B., & Van Huffel, S. (2015) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2015.2422378 · Cited by 347
Lajnef, T., Chaibi, S., Ruby, P., Aguera, P.-E., Eichenlaub, J.-B., Samet, M., Kachouri, A., & Jerbi, K. (2015) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2015.01.022 · Cited by 318
Sleep and Cardio-Metabolic Disease
Cappuccio, F. P., & Miller, M. A. (2017) · Current Cardiology Reports · doi:10.1007/s11886-017-0916-0 · Cited by 313
Scaling Up Scientific Discovery in Sleep Medicine: The National Sleep Research Resource
Dean, D. A., Goldberger, A. L., Mueller, R., Kim, M., Rueschman, M., Mobley, D., Sahoo, S. S., Jayapandian, C. P., Cui, L., Morrical, M. G., et al. (2016) · Sleep · doi:10.5665/sleep.5774 · Cited by 309
Automated sleep scoring: A review of the latest approaches
Fiorillo, L., Puiatti, A., Papandrea, M., Ratti, P.-L., Favaro, P., Roth, C., Bargiotas, P., Bassetti, C. L., & Faraci, F. D. (2019) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2019.07.007 · Cited by 284
The electroencephalographic fingerprint of sleep is genetically determined: A twin study
De Gennaro, L., Marzano, C., Fratello, F., Moroni, F., Pellicciari, M. C., Ferlazzo, F., Costa, S., Couyoumdjian, A., Curcio, G., Sforza, E., et al. (2008) · Annals of Neurology · doi:10.1002/ana.21434 · Cited by 268
Interobserver Agreement Among Sleep Scorers From Different Centers in a Large Dataset
Norman, R. G., Pal, I., Stewart, C., Walsleben, J. A., & Rapoport, D. M. (2000) · Sleep · doi:10.1093/sleep/23.7.1e · Cited by 246
Sleep Loss and the Socio-Emotional Brain
Ben Simon, E., Vallat, R., Barnes, C. M., & Walker, M. P. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.02.003 · Cited by 243
Self-citationKolmogorov complexity of finite sequences and recognition of different preictal EEG patterns
Petrosian, A. (2002) · Proceedings Eighth IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms.1995.465426 · Cited by 234
Performance of an Automated Polysomnography Scoring System Versus Computer-Assisted Manual Scoring
Malhotra, A., Younes, M., Kuna, S. T., Benca, R., Kushida, C. A., Walsh, J., Hanlon, A., Staley, B., Pack, A. I., & Pien, G. W. (2013) · Sleep · doi:10.5665/sleep.2548 · Cited by 222
Trait-like individual differences in the human sleep electroencephalogram
Buckelmüller, J., Landolt, H.-P., Stassen, H. H., & Achermann, P. (2006) · Neuroscience · doi:10.1016/j.neuroscience.2005.11.005 · Cited by 214
Sleep Disturbance Forecasts β-Amyloid Accumulation across Subsequent Years
Winer, J. R., Mander, B. A., Kumar, S., Reed, M., Baker, S. L., Jagust, W. J., & Walker, M. P. (2020) · Current Biology · doi:10.1016/j.cub.2020.08.017 · Cited by 203
Self-citationDreem Open Datasets: Multi-Scored Sleep Datasets to Compare Human and Automated Sleep Staging
Guillot, A., Sauvet, F., During, E., & Thorey, V. (2020) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2020.3011181 · Cited by 175
An end-to-end framework for real-time automatic sleep stage classification
Patanaik, A., Ong, J. L., Gooley, J. J., Ancoli-Israel, S., & Chee, M. W. L. (2018) · Sleep · doi:10.1093/sleep/zsy041 · Cited by 162
Menghini, L., Cellini, N., Goldstone, A., Baker, F. C., & de Zambotti, M. (2020) · SLEEP · doi:10.1093/sleep/zsaa170 · Cited by 154
Automatic Human Sleep Stage Scoring Using Deep Neural Networks
Malafeev, A., Laptev, D., Bauer, S., Omlin, X., Wierzbicka, A., Wichniak, A., Jernajczyk, W., Riener, R., Buhmann, J., & Achermann, P. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00781 · Cited by 151
Harding, E. C., Franks, N. P., & Wisden, W. (2019) · Current Opinion in Physiology · doi:10.1016/j.cophys.2019.11.008 · Cited by 127
Large-Scale Automated Sleep Staging
Sun, H., Jia, J., Goparaju, B., Huang, G.-B., Sourina, O., Bianchi, M. T., & Westover, M. B. (2017) · SLEEP · doi:10.1093/sleep/zsx139 · Cited by 124
Automatic sleep scoring: A search for an optimal combination of measures
Krakovská, A., & Mezeiová, K. (2011) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2011.06.004 · Cited by 120
Association of Rapid Eye Movement Sleep With Mortality in Middle-aged and Older Adults
Leary, E. B., Watson, K. T., Ancoli‐Israel, S., Redline, S. S., Yaffe, K. S., Ravelo, L. A., Peppard, P. E., Zou, J., Goodman, S. N., Mignot, E. J.-M., et al. (2020) · JAMA Neurology · doi:10.1001/jamaneurol.2020.2108 · Cited by 119
Sleep Tracking, Wearable Technology, and Opportunities for Research and Clinical Care
Shelgikar, A. V., Anderson, P. F., & Stephens, M. R. (2016) · Chest · doi:10.1016/j.chest.2016.04.016 · Cited by 104
Understanding the interplay of sleep and aging: Methodological challenges
Muehlroth, B. E., & Werkle‐Bergner, M. (2020) · Psychophysiology · doi:10.1111/psyp.13523 · Cited by 94
Complex-valued unsupervised convolutional neural networks for sleep stage classification
Zhang, J., & Wu, Y. X. (2018) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2018.07.015 · Cited by 66
Koley, B. L., & Dey, D. (2013) · Measurement · doi:10.1016/j.measurement.2013.03.016 · Cited by 54
Cited by
Pupil size reveals arousal level fluctuations in human sleep
Carro-Domínguez, M., Huwiler, S., Oberlin, S., Oesch, T. L., Badii, G., Lüthi, A., Wenderoth, N., Meissner, S. N., & Lustenberger, C. (2025) · Nature Communications · doi:10.1038/s41467-025-57289-5 · Cited by 41
Nicolas, J., King, B. R., Levesque, D., Lazzouni, L., Coffey, E., Swinnen, S., Doyon, J., Carrier, J., & Albouy, G. (2022) · eLife · doi:10.7554/elife.73930 · Cited by 28
Metadata
- Authors
- Vallat, R., & Walker, M. P.
- Venue
- eLife
- Publisher
- eLife Sciences Publications, Ltd
- Published
- 2021-10-14
- Volume
- 10
- ISSN
- 2050-084X (ISSN-L)
- OpenAlex
- W3206812283
- Cited by
- 363
- References
- 53
- FWCI
- 21.23
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.