Performance of machine learning algorithms for glioma segmentation of brain MRI: a systematic literature review and meta-analysis
Open accessThe DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
Objectives: Different machine learning algorithms (MLAs) for automated segmentation of gliomas have been reported in the literature. Automated segmentation of different tumor characteristics can be of added value for the diagnostic work-up and treatment planning. The purpose of this study was to provide an overview and meta-analysis of different MLA methods. Methods: A systematic literature review and meta-analysis was performed on the eligible studies describing the segmentation of gliomas. Meta-analysis of the performance was conducted on the reported dice similarity coefficient (DSC) score of both the aggregated results as two subgroups (i.e., high-grade and low-grade gliomas). This study was registered in PROSPERO prior to initiation (CRD42020191033). Results: After the literature search (n = 734), 42 studies were included in the systematic literature review. Ten studies were eligible for inclusion in the meta-analysis. Overall, the MLAs from the included studies showed an overall DSC score of 0.84 (95% CI: 0.82–0.86). In addition, a DSC score of 0.83 (95% CI: 0.80–0.87) and 0.82 (95% CI: 0.78–0.87) was observed for the automated glioma segmentation of the high-grade and low-grade gliomas, respectively. However, heterogeneity was considerably high between included studies, and publication bias was observed. Conclusion: MLAs facilitating automated segmentation of gliomas show good accuracy, which is promising for future implementation in neuroradiology. However, before actual implementation, a few hurdles are yet to be overcome. It is crucial that quality guidelines are followed when reporting on MLAs, which includes validation on an external test set. Key Points: • MLAs from the included studies showed an overall DSC score of 0.84 (95% CI: 0.82–0.86), indicating a good performance. • MLA performance was comparable when comparing the segmentation results of the high-grade gliomas and the low-grade gliomas. • For future studies using MLAs, it is crucial that quality guidelines are followed when reporting on MLAs, which includes validation on an external test set.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
Çiçek, Ö., Abdulkadir, A., Lienkamp, S. S., Brox, T., & Ronneberger, O. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_49 · Cited by 7,945 · cited 1×
“Within the BraTS challenges, the MLAs which yielded the most accurate results included different 2D and 3D convolutional neural networks (CNNs) [13 – 17], including 3D U-Nets [18, 19].”
MethodThe multimodal brain tumor image segmentation benchmark (BRATS)
Menze, B. H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al. (2015) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2014.2377694 · Cited by 6,781
Radiomics: extracting more information from medical images using advanced feature analysis
Lambin, P., Rios-Velazquez, E., Leijenaar, R., Carvalho, S., van Stiphout, R. G. P. M., Granton, P., Zegers, C. M. L., Gillies, R., Boellard, R., Dekker, A., et al. (2012) · European Journal of Cancer · doi:10.1016/j.ejca.2011.11.036 · Cited by 6,084
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147 · cited 1×
“Data extracted from the included studies comprised the following: The quality of the included studies was not formally assessed, as a formal quality assessment is a well-known challenge in this area of research [23 – 25].”
MethodCollins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · BMJ · doi:10.1136/bmj.g7594 · Cited by 4,388
Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation
Kamnitsas, K., Ledig, C., Newcombe, V. F. J., Simpson, J. P., Kane, A. D., Menon, D. K., Rueckert, D., & Glocker, B. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.10.004 · Cited by 3,498 · cited 4×
“In 36 studies, the ground truth (i.e., segmentations) was derived from the BraTS dataset [13, 17, 33 – 36, 38 – 45, 47 – 49, 52 – 55, 57 – 72].”
BackgroundMethodInfluentialBrain tumor segmentation with Deep Neural Networks
Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.-M., & Larochelle, H. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.05.004 · Cited by 3,278
Brain Tumor Segmentation with Deep Neural Networks
Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.-M., & Larochelle, H. (2015) · arXiv · doi:10.48550/arxiv.1505.03540 · Cited by 3,278
Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images
Pereira, S., Pinto, A., Alves, V., & Silva, C. A. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2538465 · Cited by 2,735 · cited 3×
“In 36 studies, the ground truth (i.e., segmentations) was derived from the BraTS dataset [13, 17, 33 – 36, 38 – 45, 47 – 49, 52 – 55, 57 – 72].”
MethodRadiomics: the process and the challenges
Kumar, V., Gu, Y., Basu, S., Berglund, A., Eschrich, S. A., Schabath, M. B., Forster, K., Aerts, H. J. W. L., Dekker, A., Fenstermacher, D., et al. (2012) · Magnetic Resonance Imaging · doi:10.1016/j.mri.2012.06.010 · Cited by 2,258 · cited 1×
“More specifically, automatic segmentations form the basis of further sophisticated analyses to clarify meaningful and reliable associations between neuroimaging features and survival rate [79, 80].”
BackgroundA survey of MRI-based medical image analysis for brain tumor studies
Bauer, S., Wiest, R., Nolte, L.-P., & Reyes, M. (2013) · Physics in Medicine and Biology · doi:10.1088/0031-9155/58/13/r97 · Cited by 1,036
Reporting of artificial intelligence prediction models
Collins, G. S., & Moons, K. G. M. (2019) · The Lancet · doi:10.1016/s0140-6736(19)30037-6 · Cited by 701
State of the art survey on MRI brain tumor segmentation
Gordillo, N., Montseny, E., & Sobrevilla, P. (2013) · Magnetic Resonance Imaging · doi:10.1016/j.mri.2013.05.002 · Cited by 697 · cited 1×
“Glioma segmentation requires the distinguishing of tumor tissue from healthy surrounding tissues by the radiologist [7] and the segmented region of interest or volume of interest can be used to compute feature-based radiomics and quantifiable measurements [8, 9].”
MethodDecision Forests for Tissue-Specific Segmentation of High-Grade Gliomas in Multi-channel MR
Zikic, D., Glocker, B., Konukoglu, E., Criminisi, A., Demiralp, C., Shotton, J., Thomas, O. M., Das, T., Jena, R., & Price, S. J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33454-2_46 · Cited by 276
Mazzara, G. P., Velthuizen, R. P., Pearlman, J. L., Greenberg, H. M., & Wagner, H. (2004) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2004.01.026 · Cited by 216 · cited 1×
“Accurate segmentation of gliomas on clinical magnetic resonance imaging (MRI) scans plays an important role in the quantification and objectivation of diagnosis, treatment decision, and prognosis [2 – 4].”
BackgroundDvořák, P., & Menze, B. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-42016-5_6 · Cited by 119
Brain Tumour Segmentation based on Extremely Randomized Forest with high-level features
Pinto, A., Pereira, S., Correia, H., Oliveira, J., Rasteiro, D. M. L. D., & Silva, C. A. (2015) · 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2015.7319032 · Cited by 108
Research works printed but not linked: 64
- Perry A, Wesseling P (2016) Histologic classification of gliomas handbook of clinical neurology. Elsevier, pp 71–95 · DOI · Paper page
- Yamahara T, Numa Y, Oishi T et al (2010) Morphological and flow cytometric analysis of cell infiltration in glioblastoma: a comparison of autopsy brain and neuroimaging. Brain Tumor Pathol 27:81–87 · DOI
- Johnson DR, Guerin JB, Giannini C, Morris JM, Eckel LJ, Kaufmann TJ (2017) 2016 Updates to the WHO brain tumor classification system: what the radiologist needs to know. Radiographics 37:2164–2180 · DOI · Paper page
- Larsen J, Wharton SB, McKevitt F et al (2017) ‘Low grade glioma’: an update for radiologists. Br J Radiol 90:20160600 · DOI
- Velthuizen RP, Hall LO, Clarke LP (1999) Feature extraction for MRI segmentation. J Neuroimaging 9:85–90 · doi:10.1111/jon19999285
- Ditmer A, Zhang B, Shujaat T et al (2018) Diagnostic accuracy of MRI texture analysis for grading gliomas. J Neurooncol 140:583–589 · DOI
- Egger J, Kapur T, Fedorov A et al (2013) GBM volumetry using the 3D Slicer medical image computing platform. Sci Rep 3:1364. https://doi.org/10.1038/srep01364 · DOI · Paper page
- Chaddad A, Kucharczyk MJ, Daniel P et al (2019) Radiomics in glioblastoma: current status and challenges facing clinical implementation. Front Oncol 9:374 · DOI · Paper page
- Havaei M, Dutil F, Pal C, Larochelle H, Jodoin P-M (2015) A convolutional neural network approach to brain tumor segmentation BrainLes 2015. Springer, pp 195–208 · DOI
- Randhawa RS, Modi A, Jain P, Warier P (2016) Improving boundary classification for brain tumor segmentation and longitudinal disease progression. Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, 2016 10154:65–74 · DOI
- Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. 2015 Ieee Conference on Computer Vision and Pattern Recognition (Cvpr). https://doi.org/10.1109/cvpr.2015.7298965:3431-3440 · doi:10.1109/cvpr.2015.7298965:3431-3440
- Sakai K, Yamada K (2019) Machine learning studies on major brain diseases: 5-year trends of 2014-2018. Jpn J Radiol 37:34–72 · DOI
- Lotan E, Jain R, Razavian N, Fatterpekar GM, Lui YW (2018) State of the art: machine learning applications in glioma imaging. AJR Am J Roentgenol 212:26–37 · DOI
- Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group (2010) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg 8:336–341 · DOI · Paper page
- Liu XX, Faes L, Kale AU et al (2019) A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digital Health 1:E271–E297 · DOI · Paper page
- Yeghiazaryan V, Voiculescu I (2018) Family of boundary overlap metrics for the evaluation of medical image segmentation. J Med Imaging (Bellingham) 5:015006
- Taha AA, Hanbury A (2015) Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Med Imaging 15:29 · DOI · Paper page
- Viechtbauer W (2010) Conducting Meta-Analyses in R with the metafor Package. Journal of Statistical Software, 36(3):1–48. https://doi.org/10.18637/jss.v036.i03 · DOI · Paper page
- Higgins JP, Thompson SG, Deeks JJ, Altman DG (2003) Measuring inconsistency in meta-analyses. BMJ 327:557–560 · DOI · Paper page
- Amirmoezzi Y, Salehi S, Parsaei H, Kazemi K, Torabi Jahromi A (2019) A knowledge-based system for brain tumor segmentation using only 3D FLAIR images. Australas Phys Eng Sci Med 42:529–540 · DOI
- Banerjee S, Mitra S (2020) Novel volumetric sub-region segmentation in brain tumors. Front Comput Neurosci 14:3 · DOI
- Bonte S, Goethals I, Van Holen R (2018) Machine learning based brain tumour segmentation on limited data using local texture and abnormality. Comput Biol Med 98:39–47 · DOI
- Choi Y, Nam Y, Lee YS et al (2020) IDH1 mutation prediction using MR-based radiomics in glioblastoma: comparison between manual and fully automated deep learning-based approach of tumor segmentation. Eur J Radiol 128:109031 · DOI
- Cui S, Mao L, Jiang J, Liu C, Xiong S (2018) Automatic semantic segmentation of brain gliomas from MRI images using a deep cascaded neural network. J Healthc Eng 2018:4940593
- Hasan SMK, Linte CA (2018) A modified U-Net convolutional network featuring a nearest-neighbor re-sampling-based elastic-transformation for brain tissue characterization and segmentation. Proc IEEE West N Y Image Signal Process Workshop 2018
- Havaei M, Larochelle H, Poulin P, Jodoin PM (2016) Within-brain classification for brain tumor segmentation. Int J Comput Assist Radiol Surg 11:777–788 · DOI
- Hussain S, Anwar SM, Majid M (2017) Brain tumor segmentation using cascaded deep convolutional neural network. Annu Int Conf IEEE Eng Med Biol Soc 2017:1998–2001 · DOI
- Iqbal S, Ghani Khan MU, Saba T et al (2019) Deep learning model integrating features and novel classifiers fusion for brain tumor segmentation. Microsc Res Tech 82:1302–1315 · DOI
- Iqbal S, Ghani MU, Saba T, Rehman A (2018) Brain tumor segmentation in multi-spectral MRI using convolutional neural networks (CNN). Microsc Res Tech 81:419–427 · DOI · Paper page
- Jiang J, Wu Y, Huang M, Yang W, Chen W, Feng Q (2013) 3D brain tumor segmentation in multimodal MR images based on learning population- and patient-specific feature sets. Comput Med Imaging Graph 37:512–521 · DOI
- Kao PY, Shailja S, Jiang J et al (2019) Improving patch-based convolutional neural networks for MRI brain tumor segmentation by leveraging location information. Front Neurosci 13:1449 · DOI
- Li Z, Wang Y, Yu J et al (2017) Low-grade glioma segmentation based on CNN with fully connected CRF. J Healthc Eng 2017:9283480
- Liu Y, Shi X, Xia Y et al (2018) Multi-scale V-Net: a deep learning framework for brain tumor segmentation in multiparametric MRI. Med Phys 45(6):e568 · DOI
- Meng Z, Fan Z, Zhao Z, Su F (2018) ENS-Unet: end-to-end noise suppression U-Net for brain tumor segmentation. Annu Int Conf IEEE Eng Med Biol Soc 2018:5886–5889 · DOI
- Naser MA, Deen MJ (2020) Brain tumor segmentation and grading of lower-grade glioma using deep learning in MRI images. Comput Biol Med 121:103758 · DOI · Paper page
- Perkuhn M, Stavrinou P, Thiele F et al (2018) Clinical evaluation of a multiparametric deep learning model for glioblastoma segmentation using heterogeneous magnetic resonance imaging data from clinical routine. Invest Radiol 53:647–654 · DOI · Paper page
- Razzak MI, Imran M, Xu G (2019) Efficient brain tumor segmentation with multiscale two-pathway-group conventional neural networks. IEEE J Biomed Health Inform 23:1911–1919 · DOI · Paper page
- Savareh BA, Emami H, Hajiabadi M, Azimi SM, Ghafoori M (2019) Wavelet-enhanced convolutional neural network: a new idea in a deep learning paradigm. Biomed Tech (Berl) 64:195–205 · DOI
- Soltaninejad M, Yang G, Lambrou T et al (2018) Supervised learning based multimodal MRI brain tumour segmentation using texture features from supervoxels. Comput Methods Programs Biomed 157:69–84 · DOI · Paper page
- Sun J, Chen W, Peng S, Liu B (2019) DRRNet: dense residual refine networks for automatic brain tumor segmentation. J Med Syst 43:221 · DOI
- Wang F, Niu J, Fan W, Cao Q (2018) Brain tumor medical image segmentation based on CRF 3D-CNN introduction. Basic Clin Paharmacol Toxicol 124(Supplement 2):12
- Wu S, Li H, Quang D, Guan Y (2020) Three-plane-assembled deep learning segmentation of gliomas. Radiol Artif Intell 2:e190011 · DOI
- Wu Y, Zhao Z, Wu W, Lin Y, Wang M (2019) Automatic glioma segmentation based on adaptive superpixel. BMC Med Imaging 19:73 · DOI
- Yang C, Guo X, Wang T et al (2019) Automatic brain tumor segmentation method based on modified convolutional neural network. Annu Int Conf IEEE Eng Med Biol Soc 2019:998–1001 · DOI
- Yang T, Song J, Li L (2019) A deep learning model integrating SK-TPCNN and random forests for brain tumor segmentation in MRI. Biocybern Biomed Eng 39(3):613–623. https://doi.org/10.1016/j.bbe.2019.06.003 · DOI
- Yang T, Song J, Li L, Tang Q (2020) Improving brain tumor segmentation on MRI based on the deep U-net and residual units. J Xray Sci Technol 28:95–110
- Zhao L, Wu W, Corso JJ (2013) Semi-automatic brain tumor segmentation by constrained MRFs using structural trajectories. Med Image Comput Comput Assist Interv 16:567–575
- Zhou Z, He Z, Shi M, Du J, Chen D (2020) 3D dense connectivity network with atrous convolutional feature pyramid for brain tumor segmentation in magnetic resonance imaging of human heads. Comput Biol Med 121:103766 · DOI
- Zhuge Y, Krauze AV, Ning H et al (2017) Brain tumor segmentation using holistically nested neural networks in MRI images. Med Phys 44:5234–5243 · DOI · Paper page
- Dong H, Yang G, Liu F, Mo Y, Guo Y (2017) Automatic brain tumor detection and segmentation using U-Net based fully convolutional networks. In: Valdés Hernández M, González-Castro V (eds) Medical image understanding and analysis. Springer International Publishing, Cham, pp 506–517 · DOI · Paper page
- Lyksborg M, Puonti O, Agn M, Larsen R (2015) An ensemble of 2D convolutional neural networks for tumor segmentation. In: Paulsen RR, Pedersen KS (eds) Image Analysis. Springer International Publishing, Cham, pp 201–211 · DOI
- Tustison NJ, Shrinidhi KL, Wintermark M et al (2015) Optimal symmetric multimodal templates and concatenated random forests for supervised brain tumor segmentation (simplified) with ANTsR. Neuroinformatics 13:209–225 · DOI · Paper page
- Usman K, Rajpoot K (2017) Brain tumor classification from multi-modality MRI using wavelets and machine learning. Pattern Anal Applic 20:871–881 · DOI · Paper page
- Xue Y, Xu T, Zhang H, Long LR, Huang XL (2018) SegAN: adversarial network with multi-scale L (1) loss for medical image segmentation. Neuroinformatics 16:383–392 · DOI · Paper page
- Durmo F, Lätt J, Rydelius A et al (2018) Brain tumor characterization using multibiometric evaluation of MRI. Tomography 4:14–25 · DOI
- De Baene W, Rutten GJM, Sitskoorn MM (2017) The temporal pattern of a lesion modulates the functional network topology of remote brain regions. Neural Plast 2017:3530723 · DOI
- Jiang Z, Ding C, Liu M, Tao D (2020) Two-stage cascaded U-Net: 1st place solution to BraTS challenge 2019 segmentation task. Springer International Publishing, Cham, pp 231–241
- Zhao Y-X, Zhang Y-M, Liu C-L (2020) Bag of tricks for 3D MRI brain tumor segmentation. Springer International Publishing, Cham, pp 210–220
- McKinley R, Rebsamen M, Meier R, Wiest R (2020) Triplanar ensemble of 3D-to-2D CNNs with label-uncertainty for brain tumor segmentation. Springer International Publishing, Cham, pp 379–387
- Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts H (2018) Artificial intelligence in radiology. Nat Rev Cancer 18:500–510 · DOI · Paper page
- Herrmann E, Ermis E, Meier R et al (2019) Fully automated segmentation of the brain resection cavity for radiation target volume definition in glioblastoma patients. Strahlenther Onkol 195:586–586
- Meier R, Knecht U, Loosli T et al (2016) Clinical evaluation of a fully-automatic Segmentation method for longitudinal brain tumor volumetry. Sci Rep 6:23376. https://doi.org/10.1038/srep23376 · DOI · Paper page
- Zeng K, Bakas S, Sotiras A et al (2016) Segmentation of gliomas in pre-operative and post-operative multimodal magnetic resonance imaging volumes based on a hybrid generative-discriminative framework. Brainlesion 10154:184–194
- Tang F, Liang S, Zhong T et al (2020) Postoperative glioma segmentation in CT image using deep feature fusion model guided by multi-sequence MRIs. Eur Radiol 30:823–832 · DOI
Cited by
Artificial intelligence and machine learning in cancer imaging
Koh, D.-M., Papanikolaou, N., Bick, U., Illing, R., Kahn, C. E., Kalpathi-Cramer, J., Matos, C., Martí-Bonmatí, L., Miles, A., Mun, S. K., et al. (2022) · Communications Medicine · doi:10.1038/s43856-022-00199-0 · Cited by 311
AI in imaging: the regulatory landscape
Hill, D. L. G. (2024) · British Journal of Radiology · doi:10.1093/bjr/tqae002 · Cited by 52
“14 Unlike the fi rst category, this approach involves detecting whether an abnormality is present in the image, rather than measuring a structure that is known to be present.”
BackgroundWang, T.-W., Shiao, Y.-C., Hong, J.-S., Lee, W.-K., Hsu, M.-S., Cheng, H.-M., Yang, H.-C., Lee, C.-C., Pan, H.-C., You, W. C., et al. (2024) · Mayo Clinic Proceedings: Digital Health · doi:10.1016/j.mcpdig.2024.01.002 · Cited by 23
Prabhudesai, S., Wang, N. C., Ahluwalia, V., Huan, X., Bapuraj, J. R., Banovic, N., & Rao, A. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.740353 · Cited by 13
Metadata
- Authors
- van Kempen, E. J., Post, M., Mannil, M., Witkam, R. L., ter Laan, M., Patel, A., Meijer, F. J. A., & Henssen, D.
- Venue
- European Radiology
- Published
- 2021-05-21
- Volume
- 31
- Issue
- 12
- Pages
- 9638-9653
- ISSN
- 0938-7994 (ISSN-L)
- 1432-1084
- OpenAlex
- W3161829336
- Cited by
- 85
- References
- 18
- FWCI
- 2.17
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Licence
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.