The DOI matches its Crossref record; 117 of the 117 references it cites were found in a registry or an index.
A systematic review of learning computational thinking through Scratch in K-9
Abstract
As computational thinking (CT) is being embraced by educational systems worldwide, researchers and teachers have posed important questions such as “what to teach” and “what can be learned.” These questions are universally crucial to the learning of all subjects. Nevertheless, there is no up-to-date, systematic overview of CT education for K-9 students that attempt to provide answers to these crucial questions. Thus, this systematic review presents a synthesis of 55 empirical studies, providing evidence of the development of computational thinking through programming in Scratch, one of the most popular visual block programming languages in schools. The purpose of this review is to systematically examine the CT skills that can be obtained through Scratch in K-9 based on empirical evidence. This systematic review has adopted Brennan and Resnick's (2012) framework as the basis for defining and identifying the expected CT skills in K-9. The major findings entail what computational thinking skills students in K-9 can learn through Scratch in relation to the framework mentioned above, taking the progression of learning into account. Additional CT skills that are not captured by the framework were identified including input/output, reading, interpreting and communicating code, using multimodal media, predictive thinking, and human-computer interaction. These additional CT skills are not currently presented in Brennan and Resnick's (2012) framework and can be considered as possible supplements to their framework. Furthermore, the paper discusses the difficulties regarding assessment and the progression of the identified skills, as well as problems with study designs. Finally, the paper sets out suggestions for future studies based on the current research gaps.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The qualitative content analysis process
Elo, S., & Kyngäs, H. (2008) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2007.04569.x · Cited by 22,617
Wing, J. M. (2006) · Communications of the ACM · doi:10.1145/1118178.1118215 · Cited by 8,102
(1980) · Medical Entomology and Zoology · Cited by 7,534
(2004) · Cited by 5,198
Computational Thinking in K–12: A Review of the State of the Field
Grover, S., & Pea, R. (2013) · Educational Researcher · doi:10.3102/0013189x12463051 · Cited by 2,513
Mindstorms: Children, Computers, and Powerful Ideas
Malraison, P. J., & Papert, S. (1981) · The Two-Year College Mathematics Journal · doi:10.2307/3027082 · Cited by 1,824
Barr, V., & Stephenson, C. (2011) · ACM Inroads · doi:10.1145/1929887.1929905 · Cited by 1,692
Defining Computational Thinking for Mathematics and Science Classrooms
Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L., & Wilensky, U. (2016) · Journal of Science Education and Technology · doi:10.1007/s10956-015-9581-5 · Cited by 1,609
Lye, S. Y., & Koh, J. H. L. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.09.012 · Cited by 1,421
The Scratch Programming Language and Environment
Maloney, J., Resnick, M., Rusk, N., Silverman, B., & Eastmond, E. (2010) · ACM Transactions on Computing Education · doi:10.1145/1868358.1868363 · Cited by 1,410
Román-González, M., Pérez-González, J.-C., & Jiménez-Fernández, C. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2016.08.047 · Cited by 859
A study of the difficulties of novice programmers
Lahtinen, E., Ala-Mutka, K., & Järvinen, H.-M. (2005) · Proceedings of the 10th annual SIGCSE conference on Innovation and technology in computer science education · doi:10.1145/1067445.1067453 · Cited by 827
Computational thinking in compulsory education: Towards an agenda for research and practice
Voogt, J., Fisser, P., Good, J., Mishra, P., & Yadav, A. (2015) · Education and Information Technologies · doi:10.1007/s10639-015-9412-6 · Cited by 641
Atmatzidou, S., & Demetriadis, S. (2016) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2015.10.008 · Cited by 630
Facts Are More Important Than Novelty: Replication in the Education Sciences
Makel, M. C., & Plucker, J. A. (2014) · Educational Researcher · doi:10.3102/0013189x14545513 · Cited by 609
Computation and Computational Thinking
Aho, A. V. (2012) · The Computer Journal · doi:10.1093/comjnl/bxs074 · Cited by 601
Sáez-López, J.-M., Román-González, M., & Vázquez-Cano, E. (2016) · Computers & Education · doi:10.1016/j.compedu.2016.03.003 · Cited by 530
Computational Thinking: The Developing Definition
Selby, C. C. (2013) · ePrints Soton (University of Southampton) · Cited by 506
A study of the difficulties of novice programmers
Lahtinen, E., Ala-Mutka, K., & Järvinen, H.-M. (2005) · ACM SIGCSE Bulletin · doi:10.1145/1151954.1067453 · Cited by 446
Assessing elementary students’ computational thinking in everyday reasoning and robotics programming
Chen, G., Shen, J., Barth-Cohen, L., Jiang, S., Huang, X., & Eltoukhy, M. (2017) · Computers & Education · doi:10.1016/j.compedu.2017.03.001 · Cited by 420
A K-6 computational thinking curriculum framework: implications for teacher knowledge
Angeli, C., Voogt, J. M., Fluck, A. E., Webb, M., Cox, M., Malyn‐Smith, J., & Zagami, J. (2016) · UvA-DARE (University of Amsterdam) · Cited by 416
Designing for deeper learning in a blended computer science course for middle school students
Grover, S., Pea, R., & Cooper, S. (2015) · Computer Science Education · doi:10.1080/08993408.2015.1033142 · Cited by 410
A Framework for Computational Thinking Based on a Systematic Research Review
Kalelıoğlu, F., Gülbahar, Y., & Kukul, V. (2016) · Cited by 385
Computer science unplugged: school students doing real computing without computers
Bell, T., Alexander, J., Freeman, I., & Grimley, M. (2009) · Swinburne figshare (Swinburne University of Technology) · Cited by 383
Learning computer science concepts with Scratch
Meerbaum-Salant, O., Armoni, M., & Ben-Ari, M. (. (2013) · Computer Science Education · doi:10.1080/08993408.2013.832022 · Cited by 356
KALELIOĞLU, F., & GÜLBAHAR, Y. (2014) · Informatics in Education · doi:10.15388/infedu.2014.03 · Cited by 352
Computational Thinking in K-9 Education
Mannila, L., Dagiene, V., Demo, B., Grgurina, N., Mirolo, C., Rolandsson, L., & Settle, A. (2014) · Proceedings of the Working Group Reports of the 2014 on Innovation & Technology in Computer Science Education Conference · doi:10.1145/2713609.2713610 · Cited by 314
Nouri, J., Zhang, L., Mannila, L., & Norén, E. (2020) · Education Inquiry · doi:10.1080/20004508.2019.1627844 · Cited by 303
Self-citationCoding as a Playground: Programming and Computational Thinking in the Early Childhood Classroom
Bers, M. U. (2017) · Cited by 294
Designing ScratchJr: support for early childhood learning through computer programming
Flannery, L. P., Silverman, B., Kazakoff, E. R., Bers, M. U., Bontá, P., & Resnick, M. (2013) · Proceedings of the 12th International Conference on Interaction Design and Children · doi:10.1145/2485760.2485785 · Cited by 255
Grover, S., & Basu, S. (2017) · Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education · doi:10.1145/3017680.3017723 · Cited by 249
Heintz, F., Mannila, L., & Farnqvist, T. (2016) · 2016 IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie.2016.7757410 · Cited by 221
Modeling the learning progressions of computational thinking of primary grade students
Seiter, L., & Foreman, B. (2013) · Proceedings of the ninth annual international ACM conference on International computing education research · doi:10.1145/2493394.2493403 · Cited by 217
Chao, P.-Y. (2016) · Computers & Education · doi:10.1016/j.compedu.2016.01.010 · Cited by 205
Habits of programming in scratch
Meerbaum-Salant, O., Armoni, M., & Ben-Ari, M. (2011) · Proceedings of the 16th annual joint conference on Innovation and technology in computer science education · doi:10.1145/1999747.1999796 · Cited by 199
Comparing Textual and Block Interfaces in a Novice Programming Environment
Price, T. W., & Barnes, T. (2015) · Proceedings of the eleventh annual International Conference on International Computing Education Research · doi:10.1145/2787622.2787712 · Cited by 196
An Exploration of Three-Dimensional Integrated Assessment for Computational Thinking
Zhong, B., Wang, Q., Chen, J., & Li, Y. (2016) · Journal of Educational Computing Research · doi:10.1177/0735633115608444 · Cited by 193
How programming environment shapes perception, learning and goals: logo vs. scratch
Lewis, C. M. (2010) · Proceedings of the 41st ACM technical symposium on Computer science education · doi:10.1145/1734263.1734383 · Cited by 191
Improving the Computational Thinking Pedagogical Capabilities of School Teachers
Macquarie University, Bower, M., Wood, L., Macquarie University, Lai, J., Macquarie University, Howe, C., Macquarie ICT Innovations Centre, Lister, R., University of Technology Sydney, et al. (2017) · Australian Journal of Teacher Education · doi:10.14221/ajte.2017v42n3.4 · Cited by 190
Berland, M., & Wilensky, U. (2015) · Journal of Science Education and Technology · doi:10.1007/s10956-015-9552-x · Cited by 184
Bridging Primary Programming and Mathematics: Some Findings of Design Research in England
Benton, L., Hoyles, C., Kalas, I., & Noss, R. (2017) · Digital Experiences in Mathematics Education · doi:10.1007/s40751-017-0028-x · Cited by 183
Alice, Greenfoot, and Scratch -- A Discussion
Utting, I., Cooper, S., Kölling, M., Maloney, J., & Resnick, M. (2010) · ACM Transactions on Computing Education · doi:10.1145/1868358.1868364 · Cited by 173
Proceeding of the 44th ACM technical symposium on Computer science education
(2013) · doi:10.1145/2445196 · Cited by 166
Assessing computational learning in K-12
Grover, S., Cooper, S., & Pea, R. D. (2014) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/2591708.2591713 · Cited by 164
Falloon, G. (2016) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12155 · Cited by 160
Measurement in action: an activity-theoretical perspective on producer–user interaction
Hasu, M., & Engeström, Y. (2000) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.2000.0375 · Cited by 156
Strawhacker, A. L., Lee, M., & Bers, M. U. (2017) · International Journal of Technology and Design Education · doi:10.1007/s10798-017-9400-9 · Cited by 150
Comparing loops misconceptions in block-based and text-based programming languages at the K-12 level
Mladenović, M., Boljat, I., & Žanko, Ž. (2017) · Education and Information Technologies · doi:10.1007/s10639-017-9673-3 · Cited by 149
Developing Mathematical Thinking with Scratch: An Experiment with 6th Grade Students
Calao, L. A., Moreno-León, J., Correa, H. E., & Robles, G. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24258-3_2 · Cited by 136
Can computational talent be detected? Predictive validity of the Computational Thinking Test
Román-González, M., Pérez-González, J., Moreno-León, J., & Robles, G. (2018) · Int. J. Child Comput. Interact. · doi:10.1016/j.ijcci.2018.06.004 · Cited by 136
Dr. Scratch: Automatic Analysis of Scratch Projects to Assess and Foster Computational Thinking
Moreno-León, J., Robles, G., & Román-González, . (2015) · DOAJ (DOAJ: Directory of Open Access Journals) · Cited by 133
Constructing the ScratchJr programming language in the early childhood classroom
Portelance, D. J., Strawhacker, A. L., & Bers, M. U. (2015) · International Journal of Technology and Design Education · doi:10.1007/s10798-015-9325-0 · Cited by 124
Burke, Q., & Kafai, Y. B. (2012) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/2157136.2157264 · Cited by 121
A framework of curriculum design for computational thinking development in K-12 education
Kong, S. C. (2016) · Journal of Computers in Education · doi:10.1007/s40692-016-0076-z · Cited by 119
Baytak, A., & Land, S. M. (2011) · Educational Technology Research and Development · doi:10.1007/s11423-010-9184-z · Cited by 118
Hermans, F., & Aivaloglou, E. (2017) · Workshop in Primary and Secondary Computing Education · doi:10.1145/3137065.3137072 · Cited by 116
Proceedings of the 41st ACM technical symposium on Computer science education
Lewandowski, G., Wolfman, S. A., Cortina, T. J., Walker, E., & Musicant, D. R. (2010) · doi:10.1145/1734263 · Cited by 116
Introducing teachers to computational thinking using unplugged storytelling
Curzon, P., McOwan, P. W., Plant, N., & Meagher, L. R. (2014) · Workshop in Primary and Secondary Computing Education · doi:10.1145/2670757.2670767 · Cited by 112
The Role of Statistical Significance Testing In Educational Research
McLean, J. E., & Ernest, J. M. (1998) · Cited by 105
Understanding problem solving behavior of 6–8 graders in a debugging game
Liu, Z., Zhi, R., Hicks, A. G., & Barnes, T. M. (2017) · Computer Science Education · doi:10.1080/08993408.2017.1308651 · Cited by 104
Basawapatna, A. R., Repenning, A., Koh, K. H., & Nickerson, H. (2013) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/2493394.2493404 · Cited by 103
Design for Teaching and Learning in a Networked World
Gráinne, C., Tomaz, K., Rensing, C., et al. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24258-3 · Cited by 101
Assessment of computer science learning in a scratch-based outreach program
Franklin, D., Conrad, P., Boe, B., Nilsen, K., Hill, C., Len, M., Dreschler, G., Aldana, G., Almeida-Tanaka, P., Kiefer, B., et al. (2013) · Technical Symposium on Computer Science Education · doi:10.1145/2445196.2445304 · Cited by 99
2014 IEEE Frontiers in Education Conference (FIE) Proceedings
(2014) · doi:10.1109/fie21181.2014 · Cited by 89
Evaluating Scratch to Introduce Younger Schoolchildren to Programming.
Wilson, A., & Moffatt, D. C. (2010) · ResearchOnline · Cited by 85
Wohl, B., Porter, B., & Clinch, S. (2015) · Workshop in Primary and Secondary Computing Education · doi:10.1145/2818314.2818340 · Cited by 84
Scratch vs. Karel: impact on learning outcomes and motivation
Ruf, A., Mühling, A., & Hubwieser, P. (2014) · Workshop in Primary and Secondary Computing Education · doi:10.1145/2670757.2670772 · Cited by 83
The Markings of a New Pencil: Introducing Programming-as-Writing in the Middle School Classroom
Burke, Q. (2012) · Journal of Media Literacy Education · doi:10.23860/jmle-4-2-3 · Cited by 82
(2017) · Educational Technology & Society · Cited by 78
Frontiers in Education Conference
(2008) · IEEE Transactions on Education · doi:10.1109/te.2008.917748 · Cited by 78
Rose, S. P., Habgood, M. P. J., & Jay, T. (2017) · The Electronic Journal of e-Learning · doi:10.34190/ejel.15.4.2368 · Cited by 75
Using SOLO to Classify the Programming Responses of Primary Grade Students
Seiter, L. M. (2015) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/2676723.2677244 · Cited by 72
Repenning, A., Basawapatna, A. R., & Escherle, N. A. (2016) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2016.7739688 · Cited by 71
Make, wear, play: remix designs of wearable controllers for scratch games by middle school youth
Vasudevan, V., Kafai, Y. B., & Yang, L. (2015) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/2771839.2771911 · Cited by 69
Franklin, D. M., Hill, C., Dwyer, H. A., Hansen, A. K., Iveland, A., & Harlow, D. B. (2016) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/2839509.2844569 · Cited by 67
Smart Education and e-Learning 2017
Lněnička, M., Máchová, R., Komárková, J., & Pásler, M. (2017) · Smart innovation, systems and technologies · doi:10.1007/978-3-319-59451-4 · Cited by 67
Effect of design-based learning on improving computational thinking
Jun, S., Han, S., & Kim, S. (2016) · Behaviour & Information Technology · doi:10.1080/0144929x.2016.1188415 · Cited by 66
Gender Differences in Scratch Programs of Primary School Children
Funke, A., & Geldreich, K. (2017) · Workshop in Primary and Secondary Computing Education · doi:10.1145/3137065.3137067 · Cited by 61
Learning Abstraction in Computer Science
Statter, D., & Armoni, M. (2017) · Workshop in Primary and Secondary Computing Education · doi:10.1145/3137065.3137081 · Cited by 56
Fields, D. A., Vasudevan, V., & Kafai, Y. B. (2015) · Interactive Learning Environments · doi:10.1080/10494820.2015.1065892 · Cited by 55
Wong, G. K. W., Cheung, H.-Y., Ching, E. C. C., & Huen, J. (2015) · IEEE International Conference on Teaching, Assessment, and Learning for Engineering · doi:10.1109/tale.2015.7386007 · Cited by 55
Educational robots and computational thinking
Catlin, D., & Woollard, J. (2014) · ePrints Soton (University of Southampton) · Cited by 52
Frontiers in Education Conference - FIE 2001
(2001) · doi:10.1109/fie.2001.963826 · Cited by 51
Hubbard, G. T. (2012) · Journal of Media Literacy Education · doi:10.23860/jmle-4-2-6 · Cited by 49
Proceedings of the 2014 conference on Innovation & technology in computer science education
Cajander, Å., Daniels, M., Clear, T., & Pears, A. (2014) · Cited by 47
Digital ink for cognitive assessment of computational thinking
Ambrosio, A. P., Xavier, C., & Georges, F. J. (2014) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie.2014.7044237 · Cited by 43
Using scaffolded examples to teach computational thinking concepts
Webb, H. C., & Rosson, M. B. (2013) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/2445196.2445227 · Cited by 43
Use of cutting edge educational tools for an initial programming course
Giordano, D., & Maiorana, F. (2014) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon.2014.6826147 · Cited by 42
Kafai, Y. B., & Vasudevan, V. (2015) · Workshop in Primary and Secondary Computing Education · doi:10.1145/2818314.2818334 · Cited by 42
cs4fn and computational thinking unplugged
Curzon, P. (2013) · Workshop in Primary and Secondary Computing Education · doi:10.1145/2532748.2611263 · Cited by 38
Promoting computational thinking with programming
Selby, C. C. (2012) · Workshop in Primary and Secondary Computing Education · doi:10.1145/2481449.2481466 · Cited by 36
Assessing Computational Thinking in Students' Game Designs
Hoover, A. K., Barnes, J., Fatehi, B., Moreno-León, J., Puttick, G. M., Tucker‐Raymond, E., & Harteveld, C. (2016) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/2968120.2987750 · Cited by 36
Kafai, Y. B., & Vasudevan, V. (2015) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/2771839.2771853 · Cited by 31
Proceedings of the Workshop in Primary and Secondary Computing Education
(2015) · doi:10.1145/2818314 · Cited by 31
Rößling, G., Naps, T., & Spannagel, C. (2011) · doi:10.1145/1999747 · Cited by 31
Ke, F.-F., & Im, T.-M. (2013) · International Journal of Technology and Design Education · doi:10.1007/s10798-013-9248-6 · Cited by 28
Stories from the scratch community: connecting with ideas, interests, and people
Brennan, K. A., & Resnick, M. (2013) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/2445196.2445336 · Cited by 27
Davis, R., Kafai, Y. B., Vasudevan, V., & Lee, E. (2013) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/2485760.2485846 · Cited by 26
An HCI Approach to Computing in the Real World
Yardi, S., Krolikowski, P., Marshall, T., & Bruckman, A. S. (2008) · Journal on Educational Resources in Computing · doi:10.1145/1404935.1404938 · Cited by 26
Sáez-López, J.-M., & Sevillano-García, M. (2017) · Culture and Education · doi:10.1080/11356405.2017.1305075 · Cited by 23
Integrating computational thinking into english dialogue learning through graphical programming tool
Weng, X., & Wong, G. K. W. (2017) · IEEE International Conference on Teaching, Assessment, and Learning for Engineering · doi:10.1109/tale.2017.8252356 · Cited by 19
2015 IEEE Global Engineering Education Conference (EDUCON)
(2015) · doi:10.1109/educon35741.2015 · Cited by 19
Proceedings of the 9th Workshop in Primary and Secondary Computing Education
Schulte, C., Germany, W. '. W. I. P. A. S. C. E. 2. V. E., Association for Computing Machinery-Digital Library, ACM Special Interest Group on Computer Science Education, & SIGs, A. (2014) · doi:10.1145/2670757 · Cited by 18
The Use of Robotics in Introductory Programming for Elementary Students
Athanasiou, L., Topali, P., & Mikropoulos, T. A. (2017) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-55553-9_14 · Cited by 14
Evaluating algorithmic thinking ability of primary schoolchildren who learn computer programming
Tsukamoto, H., Oomori, Y., Nagumo, H., Takemura, Y., Monden, A., & Matsumoto, K. (2017) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie.2017.8190609 · Cited by 10
Teaching of social software and web 2.0
Chiazzese, G., & Chifari, A. (2012) · Italian Journal of Educational Technology · doi:10.17471/2499-4324/199 · Cited by 9
Segredo, E., Miranda, G., León, C., & Santos, A. (2016) · Smart innovation, systems and technologies · doi:10.1007/978-3-319-39690-3_21 · Cited by 8
Cited by
Assessing computational thinking: A systematic review of empirical studies
Tang, X., Yin, Y., Lin, Q., Hadad, R., & Zhai, X. (2020) · Computers & Education · doi:10.1016/j.compedu.2019.103798 · Cited by 647
du Plooy, E., Casteleijn, D., & Franzsen, D. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e39630 · Cited by 310
Tikva, C., & Tambouris, E. (2021) · Computers & Education · doi:10.1016/j.compedu.2020.104083 · Cited by 296
A systematic review of integrating computational thinking in early childhood education
Su, J., & Yang, W. (2023) · Computers and Education Open · doi:10.1016/j.caeo.2023.100122 · Cited by 174
From Gaming to Computational Thinking: An Adaptive Educational Computer Game-Based Learning Approach
Hooshyar, D., Pedaste, M., Yang, Y., Malva, L., Hwang, G.-J., Wang, M., Lim, H., & Delev, D. (2021) · Journal of Educational Computing Research · doi:10.1177/0735633120965919 · Cited by 132
Metadata
- Authors
- Zhang, L., & Nouri, J.
- Venue
- Computers & Education
- Publisher
- Elsevier BV
- Published
- 2019-11
- Volume
- 141
- Pages
- 103607
- ISSN
- 0360-1315 (ISSN-L)
- OpenAlex
- W2952562653
- Cited by
- 449
- References
- 117
- FWCI
- 11.02
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.