Probabilistic Harmonization and Annotation of Single-cell Transcriptomics Data with Deep Generative Models
Open accessPreprint — published as 10.15252/msb.20209620 · Paper page
The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Abstract
As single-cell transcriptomics becomes a mainstream technology, the natural next step is to integrate the accumulating data in order to achieve a common ontology of cell types and states. However, owing to various nuisance factors of variation, it is not straightforward how to compare gene expression levels across data sets and how to automatically assign cell type labels in a new data set based on existing annotations. In this manuscript, we demonstrate that our previously developed method, scVI, provides an effective and fully probabilistic approach for joint representation and analysis of cohorts of single-cell RNA-seq data sets, while accounting for uncertainty caused by biological and measurement noise. We also introduce single-cell ANnotation using Variational Inference (scANVI), a semi-supervised variant of scVI designed to leverage any available cell state annotations — for instance when only one data set in a cohort is annotated, or when only a few cells in a single data set can be labeled using marker genes. We demonstrate that scVI and scANVI compare favorably to the existing methods for data integration and cell state annotation in terms of accuracy, scalability, and adaptability to challenging settings such as a hierarchical structure of cell state labels. We further show that different from existing methods, scVI and scANVI represent the integrated datasets with a single generative model that can be directly used for any probabilistic decision making task, using differential expression as our case study. scVI and scANVI are available as open source software and can be readily used to facilitate cell state annotation and help ensure consistency and reproducibility across studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Massively parallel digital transcriptional profiling of single cells
Zheng, G. X. Y., Terry, J. M., Belgrader, P., Ryvkin, P., Bent, Z. W., Wilson, R., Ziraldo, S. B., Wheeler, T. D., McDermott, G. P., Zhu, J., et al. (2017) · Nature Communications · doi:10.1038/ncomms14049 · Cited by 7,353
Single-cell RNA-seq highlights intratumoral heterogeneity in primary glioblastoma
Patel, A. P., Tirosh, I., Trombetta, J. J., Shalek, A. K., Gillespie, S. M., Wakimoto, H., Cahill, D. P., Nahed, B. V., Curry, W. T., Martuza, R. L., et al. (2014) · Science · doi:10.1126/science.1254257 · Cited by 5,063
StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation
Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., & Choo, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00916 · Cited by 4,163
Full-length RNA-seq from single cells using Smart-seq2
Picelli, S., Faridani, O. R., Björklund, Å. K., Winberg, G., Sagasser, S., & Sandberg, R. (2014) · Nature Protocols · doi:10.1038/nprot.2014.006 · Cited by 4,137
(2012) · MPG.PuRe (Max Planck Society) · doi:10.5555/2188385.2188410 · Cited by 3,603
Droplet barcoding for single-cell transcriptomics applied to embryonic stem cells
Klein, A. M., Mazutis, L., Akartuna, I., Tallapragada, N., Veres, A., Li, V., Peshkin, L., Weitz, D. A., & Kirschner, M. W. (2015) · Cell · doi:10.1016/j.cell.2015.04.044 · Cited by 3,356
CyCADA: Cycle-Consistent Adversarial Domain Adaptation
Hoffman, J., Tzeng, E., Park, T., Zhu, J.-Y., Isola, P., Saenko, K., Efros, A. A., & Darrell, T. (2017) · arXiv · doi:10.48550/arxiv.1711.03213 · Cited by 2,656
Molecular Architecture of the Mouse Nervous System
Zeisel, A., Hochgerner, H., Lönnerberg, P., Johnsson, A., Memic, F., van der Zwan, J., Häring, M., Braun, E., Borm, L. E., La Manno, G., et al. (2018) · Cell · doi:10.1016/j.cell.2018.06.021 · Cited by 2,552
Deep generative modeling for single-cell transcriptomics
Lopez, R., Regier, J., Cole, M. B., Jordan, M. I., & Yosef, N. (2018) · Nature Methods · doi:10.1038/s41592-018-0229-2 · Cited by 2,530
Self-citationBatch effects in single-cell RNA-sequencing data are corrected by matching mutual nearest neighbors
Haghverdi, L., Lun, A. T. L., Morgan, M. D., & Marioni, J. C. (2018) · Nature Biotechnology · doi:10.1038/nbt.4091 · Cited by 2,416
Data-Driven Phenotypic Dissection of AML Reveals Progenitor-like Cells that Correlate with Prognosis
Levine, J. H., Simonds, E. F., Bendall, S. C., Davis, K. L., Amir, E.-A. D., Tadmor, M. D., Litvin, O., Fienberg, H. G., Jager, A., Zunder, E. R., et al. (2015) · Cell · doi:10.1016/j.cell.2015.05.047 · Cited by 2,303
Regev, A., Teichmann, S. A., Lander, E. S., Amit, I., Benoist, C., Birney, E., Bodenmiller, B., Campbell, P., Carninci, P., Clatworthy, M., et al. (2017) · eLife · doi:10.7554/elife.27041 · Cited by 2,232
Self-citationTransfer Feature Learning with Joint Distribution Adaptation
Long, M., Wang, J., Ding, G., Sun, J., & Yu, P. S. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.274 · Cited by 2,071
Return of Frustratingly Easy Domain Adaptation
Sun, B., Feng, J., & Saenko, K. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10306 · Cited by 2,004
The Immunological Genome Project: networks of gene expression in immune cells
The Immunological Genome Project Consortium, Heng, T. S. P., Painter, M. W., Elpek, K., Lukacs-Kornek, V., Mauermann, N., Turley, S. J., Koller, D., Kim, F. S., Wagers, A. J., et al. (2008) · Nature Immunology · doi:10.1038/ni1008-1091 · Cited by 1,864
Massively parallel single-cell RNA-seq for marker-free decomposition of tissues into cell types
Jaitin, D. A., Kenigsberg, E., Keren-Shaul, H., Elefant, N., Paul, F., Zaretsky, I., Mildner, A., Cohen, N., Jung, S., Tanay, A., et al. (2014) · Science · doi:10.1126/science.1247651 · Cited by 1,795
Baron, M., Veres, A., Wolock, S. L., Faust, A. L., Gaujoux, R., Vetere, A., Ryu, J. H., Wagner, B. K., Shen-Orr, S. S., Klein, A. M., et al. (2016) · Cell Systems · doi:10.1016/j.cels.2016.08.011 · Cited by 1,508
Comparative Analysis of Single-Cell RNA Sequencing Methods
Ziegenhain, C., Vieth, B., Parekh, S., Reinius, B., Guillaumet-Adkins, A., Smets, M., Leonhardt, H., Heyn, H., Hellmann, I., & Enard, W. (2017) · Molecular Cell · doi:10.1016/j.molcel.2017.01.023 · Cited by 1,494
Single-Cell Multi-omic Integration Compares and Contrasts Features of Brain Cell Identity
Welch, J. D., Kozareva, V., Ferreira, A., Vanderburg, C., Martin, C., & Macosko, E. Z. (2019) · Cell · doi:10.1016/j.cell.2019.05.006 · Cited by 1,193
CEL-seq2: sensitive highly-multiplexed single-cell RNA-Seq
Hashimshony, T., Senderovich, N., Avital, G., Klochendler, A., de Leeuw, Y., Anavy, L., Gennert, D., Li, S., Livak, K. J., Rozenblatt-Rosen, O., et al. (2016) · Genome Biology · doi:10.1186/s13059-016-0938-8 · Cited by 1,125
Single-cell RNA-seq denoising using a deep count autoencoder
Eraslan, G., Simon, L. M., Mircea, M., Mueller, N. S., & Theis, F. J. (2019) · Nature Communications · doi:10.1038/s41467-018-07931-2 · Cited by 1,084
Scaling single-cell genomics from phenomenology to mechanism
Tanay, A., & Regev, A. (2017) · Nature · doi:10.1038/nature21350 · Cited by 792
Bias, robustness and scalability in single-cell differential expression analysis
Soneson, C., & Robinson, M. D. (2018) · Nature Methods · doi:10.1038/nmeth.4612 · Cited by 770
Lineage tracing on transcriptional landscapes links state to fate during differentiation
Weinreb, C., Rodriguez-Fraticelli, A., Camargo, F. D., & Klein, A. M. (2020) · Science · doi:10.1126/science.aaw3381 · Cited by 759
A general and flexible method for signal extraction from single-cell RNA-seq data
Risso, D., Perraudeau, F., Gribkova, S., Dudoit, S., & Vert, J.-P. (2018) · Nature Communications · doi:10.1038/s41467-017-02554-5 · Cited by 718
Revealing the vectors of cellular identity with single-cell genomics
Wagner, A., Regev, A., & Yosef, N. (2016) · Nature Biotechnology · doi:10.1038/nbt.3711 · Cited by 632
Self-citationDeep generative models of genetic variation capture the effects of mutations
Riesselman, A. J., Ingraham, J. B., & Marks, D. S. (2018) · Nature Methods · doi:10.1038/s41592-018-0138-4 · Cited by 601
Hochgerner, H., Zeisel, A., Lönnerberg, P., & Linnarsson, S. (2018) · Nature Neuroscience · doi:10.1038/s41593-017-0056-2 · Cited by 482
Normalizing single-cell RNA sequencing data: challenges and opportunities
Vallejos, C. A., Risso, D., Scialdone, A., Dudoit, S., & Marioni, J. C. (2017) · Nature Methods · doi:10.1038/nmeth.4292 · Cited by 436
Population snapshots predict early haematopoietic and erythroid hierarchies
Tusi, B. K., Wolock, S. L., Weinreb, C., Hwang, Y., Hidalgo, D., Zilionis, R., Waisman, A., Huh, J. R., Klein, A. M., & Socolovsky, M. (2018) · Nature · doi:10.1038/nature25741 · Cited by 384
Droplet scRNA-seq is not zero-inflated
Svensson, V. (2019) · doi:10.1101/582064 · Cited by 356
Interpretable dimensionality reduction of single cell transcriptome data with deep generative models
Ding, J., Condon, A., & Shah, S. P. (2018) · Nature Communications · doi:10.1038/s41467-018-04368-5 · Cited by 337
Performance Assessment and Selection of Normalization Procedures for Single-Cell RNA-Seq
Cole, M. B., Risso, D., Wagner, A., DeTomaso, D., Ngai, J., Purdom, E., Dudoit, S., & Yosef, N. (2019) · Cell Systems · doi:10.1016/j.cels.2019.03.010 · Cited by 173
Self-citationpowsimR: power analysis for bulk and single cell RNA-seq experiments
Vieth, B., Ziegenhain, C., Parekh, S., Enard, W., & Hellmann, I. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx435 · Cited by 155
Lopez, R., Nazaret, A., Langevin, M., Samaran, J., Regier, J., Jordan, M. I., & Yosef, N. (2019) · arXiv · doi:10.48550/arxiv.1905.02269 · Cited by 105
Self-citationFastProject: a tool for low-dimensional analysis of single-cell RNA-Seq data
DeTomaso, D., & Yosef, N. (2016) · BMC Bioinformatics · doi:10.1186/s12859-016-1176-5 · Cited by 58
Self-citation
Research works printed but not linked: 59
- 2017, Highly parallel genome-wide expression profiling of individual cells using nanoliter droplets, Cell, 161, 1202
- 2017, Dynamics of lineage commitment revealed by single-cell transcriptomics of differentiating embryonic stem cells, Nature Communications, 8, 1096, 10.1038/s41467-017-01076-4 · DOI · Paper page
- 10.1016/j.cell.2015.11.009 · DOI · Paper page
- 2017, Single cells make big data: New challenges and opportunities in transcriptomics, Current Opinion in Systems Biology, 4, 85, 10.1016/j.coisb.2017.07.004 · DOI · Paper page
- S. R. Quake , T. Wyss-Coray , S. Darmanis , T. M. Consortium et al., “Single-cell transcriptomic characterization of 20 organs and tissues from individual mice creates a tabula muris,” bioRxiv, p. 237446, 2018.
- 2018, Boosting the power of single-cell analysis, Nature Biotechnology, 36, 408, 10.1038/nbt.4131 · DOI
- 10.1038/nbt.4096 · DOI · Paper page
- 10.1093/biostatistics/kxj037 · DOI · Paper page
- 10.1093/nar/gkv007 · DOI · Paper page
- 2019, Efficient integration of heterogeneous single-cell transcriptomes using scanorama, Nature biotechnology, 37, 685, 10.1038/s41587-019-0113-3 · DOI · Paper page
- T. Stuart , A. Butler , P. Hoffman , C. Hafemeister , E. Papalexi , W. M. Mauck III , Y. Hao , M. Stoeckius , P. Smibert , and R. Satija , “Comprehensive integration of single-cell data,” Cell, 2019. · DOI
- M. Amodio , K. Srinivasan , D. van Dijk , H. Mohsen , K. Yim , R. Muhle , K. R. Moon , S. Kaech , R. Sowell , R. Montgomery , J. Noonan , G. Wolf , and S. Krishnaswamy , “Exploring Single-Cell Data with Multitasking Deep Neural Networks,” bioRxiv, 2017. [Online]. Available: https://www.biorxiv.org/content/early/2017/12/19/237065 · DOI · Paper page
- 10.1093/bioinformatics/btx196 · DOI · Paper page
- D. DeTomaso , M. Jones , M. Subramaniam , T. Ashuach , C. J. Ye , and N. Yosef , “Functional interpretation of single-cell similarity maps,” bioRxiv, 2018. [Online]. Available: https://www.biorxiv.org/content/early/2018/08/29/403055 · DOI
- 2017, Simultaneous epitope and transcriptome measurement in single cells, Nature methods, 14, 865, 10.1038/nmeth.4380 · DOI · Paper page
- F. Wagner and I. Yanai , “Moana: A robust and scalable cell type classification framework for single-cell RNA-Seq data,” bioRxiv, 2018. [Online]. Available: https://www.biorxiv.org/content/early/2018/10/30/456129 · DOI · Paper page
- 2018, scmap: projection of single-cell RNA-seq data across data sets, Nature Methods, 15, 359, 10.1038/nmeth.4644 · doi:10.1038/nmeth.4644
- D. P. Kingma and M. Welling , “Auto-encoding variational Bayes,” in The International Conference on Learning Representations, 2014.
- C. Louizos , K. Swersky , Y. Li , M. Welling , and R. Zemel , “The Variational Fair Autoencoder,” in International Conference on Learning Representations, 2016.
- 10.1093/biostatistics/kxv027 · DOI · Paper page
- 2014, Semi-supervised Learning with Deep Generative Models, Advances in Neural Information Processing Systems, 27, 3581
- L. Maaløe , C. K. Sønderby , S. K. Sønderby , and O. Winther , “Auxiliary deep generative models,” in International Conference on Machine Learning, 2016, pp. 1445–1453.
- 2016, Domain-adversarial training of neural networks, The Journal of Machine Learning Research, 17, 2096
- T. Tommasi and B. Caputo , “Frustratingly easy nbnn domain adaptation,” in IEEE International Conference on Computer Vision, 2013. · DOI
- 10.1073/pnas.1602306113 · DOI · Paper page
- J.- Y. Zhu , T. Park , P. Isola , and A. A. Efros , “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2223–2232. · DOI · Paper page
- M. Amodio and S. Krishnaswamy , “Magan: Aligning biological manifolds,” in International Conference on Machine Learning, 2018, pp. 215–223.
- 2018, Umap: Uniform manifold approximation and projection, The Journal of Open Source Software, 3, 861, 10.21105/joss.00861 · DOI · Paper page
- 10.1016/j.cell.2015.11.013 · DOI · Paper page
- 10.1152/physiolgenomics.00247.2006 · DOI · Paper page
- 2013, Defining cell-type specificity at the transcriptional level in human disease, Genome research, 697, gr
- 2017, Expression atlas: gene and protein expression across multiple studies and organisms, Nucleic acids research, 46, D246
- M. Ester , H.-P. Kriegel , J. Sander , and X. Xu , “A density-based algorithm for discovering clusters a density-based algorithm for discovering clusters in large spatial databases with noise,” in Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, 1996, pp. 226–231.
- X. Zhang , C. Xu , and N. Yosef , “SymSim: simulating multi-faceted variability in single cell RNA sequencing,” Nature Communications, 2019. · DOI
- 10.1038/ni.2067 · DOI · Paper page
- 10.1172/JCI24176 · DOI · Paper page
- 10.1093/bioinformatics/btp616 · DOI · Paper page
- 2018, Vasc: Dimension reduction and visualization of single-cell rna-seq data by deep variational autoencoder, Genomics, proteomics – bioinformatics, 16, 320, 10.1016/j.gpb.2018.08.003 · DOI · Paper page
- C. H. Grønbech , M. F. Vording , P. N. Timshel , C. K. Sønderby , T. H. Pers , and O. Winther , “scVAE: Variational auto-encoders for single-cell gene expression data,” bioRxiv, 2018. [Online]. Available: https://www.biorxiv.org/content/early/2018/05/16/318295 · DOI
- 2019, Parameter tuning is a key part of dimensionality reduction via deep variational autoencoders for single cell rna transcriptomics, Pacific Symposium on Biocomputing, 24, 362.
- 2003, Latent dirichlet alocation, Journal of Machine Learning Research, 3, 993
- D. A. duVerle , S. Yotsukura , S. Nomura , H. Aburatani , and K. Tsuda , “CellTree: an R/bioconductor package to infer the hierarchical structure of cell populations from single-cell RNA-seq data,” BMC Bioinformatics, Sep 2016. · DOI · Paper page
- D. Schafflick , C. X. Antelope , M. Hartlehnert , M. Cole , T. Lautwein , A. Schulte-Mecklenbeck , J. Wolbert , M. O. Heming , S. G. Meuth , T. Kuhlmann et al., “Integrated single cell analysis of blood and cerebrospinal fluid leukocytes in multiple sclerosis,” bioRxiv, p. 403527, 2019.
- J. Goldberger and E. Ben-Reuven , “Training deep neural-networks using a noise adaptation layer,” in International Conference in Learning Representations, 2017.
- 2018, Hierarchical multi-label classification networks, Proceedings of the 35th International Conference on Machine Learning, 80, 5075
- Y. W. Teh , D. Newman , and M. Welling , “A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation,” in Advances in Neural Information Processing Systems 19, 2007. · DOI · Paper page
- C. K. Sønderby , T. Raiko , L. Maaløe , S. K. Sønderby , and O. Winther , “Ladder variational autoencoders,” in Advances in Neural Information Processing Systems, 2016.
- S. Ioffe and C. Szegedy , “Batch normalization: accelerating deep network training by reducing internal covariate shift,” in International Conference on Machine Learning, 2015.
- 2018, On p-values and Bayes factors, Annual Review of Statistics and Its Application, 5, 393, 10.1146/annurev-statistics-031017-100307 · DOI · Paper page
- 10.2307/2291091 · doi:10.2307/2291091 · Paper page
- T. S. Andrews and M. Hemberg , “Dropout-based feature selection for scRNASeq,” bioRxiv, 2018. [Online]. Available: https://www.biorxiv.org/content/early/2018/05/17/065094
- 2016, A single-cell transcriptome atlas of the human pancreas, Cell systems, 3, 385, 10.1016/j.cels.2016.09.002 · DOI · Paper page
- I. Goodfellow , J. Pouget-Abadie , M. Mirza , B. Xu , D. Warde-Farley , S. Ozair , A. Courville , and Y. Bengio , “Generative adversarial nets,” in Advances in neural information processing systems, 2014, pp. 2672–2680.
- 2015, Learning Structured Output Representation using Deep Conditional Generative Models, Advances in Neural Information Processing Systems, 28, 3483
- E. Becht , L. McInnes , J. Healy , C.-A. Dutertre , I. W. H. Kwok , L. G. Ng , F. Ginhoux , and E. W. Newell , “Dimensionality reduction for visualizing single-cell data using umap,” Nature Biotechnology, vol. 37, Dec 2018. · DOI · Paper page
- A. Rasmus , M. Berglund , M. Honkala , H. Valpola , and T. Raiko , “Semi-supervised learning with ladder networks,” in Advances in neural information processing systems, 2015, pp. 3546–3554.
- R. Lopez , J. Regier , M. I. Jordan , and N. Yosef , “Information Constraints on Auto-Encoding Variational Bayes,” Advances in Neural Information Processing Systems, 2018.
- M. Langevin , E. Mehlman , J. Regier , R. Lopez , M. I. Jordan , and N. Yosef , “A deep generative model for semi-supervised classification with noisy labels,” arXiv, vol. abs/1809.05957, 2018. [Online]. Available: http://arxiv.org/abs/1809.05957 · arXiv
- 2017, The brain initiative cell census consortium: lessons learned toward generating a comprehensive brain cell atlas, Neuron, 96, 542, 10.1016/j.neuron.2017.10.007 · DOI · Paper page
Cited by
Dictionary learning for integrative, multimodal and scalable single-cell analysis
Hao, Y., Stuart, T., Kowalski, M. H., Choudhary, S., Hoffman, P., Hartman, A., Srivastava, A., Molla, G., Madad, S., Fernandez-Granda, C., et al. (2024) · Nature Biotechnology · doi:10.1038/s41587-023-01767-y · Cited by 5,631
Benchmarking atlas-level data integration in single-cell genomics
Luecken, M. D., Büttner, M., Chaichoompu, K., Danese, A., Interlandi, M., Mueller, M. F., Strobl, D. C., Zappia, L., Dugas, M., Colomé-Tatché, M., et al. (2022) · Nature Methods · doi:10.1038/s41592-021-01336-8 · Cited by 1,438
A Python library for probabilistic analysis of single-cell omics data
Gayoso, A., Lopez, R., Xing, G., Boyeau, P., Valiollah Pour Amiri, V., Hong, J., Wu, K., Jayasuriya, M., Mehlman, E., Langevin, M., et al. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-01206-w · Cited by 789
Self-citationLarge-scale foundation model on single-cell transcriptomics
Hao, M., Gong, J., Zeng, X., Liu, C., Guo, Y., Cheng, X., Wang, T., Ma, J., Zhang, X., & Song, L. (2024) · Nature Methods · doi:10.1038/s41592-024-02305-7 · Cited by 628
Multi-omics single-cell data integration and regulatory inference with graph-linked embedding
Cao, Z.-J., & Gao, G. (2022) · Nature Biotechnology · doi:10.1038/s41587-022-01284-4 · Cited by 616
Metadata
- Authors
- Xu, C., Lopez, R., Mehlman, E., Regier, J., Jordan, M. I., & Yosef, N.
- Venue
- bioRxiv (Cold Spring Harbor Laboratory)
- Publisher
- openRxiv
- Published
- 2019-01-29
- OpenAlex
- W2911551350
- Cited by
- 24
- References
- 37
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.