The DOI matches its Crossref record; 99 of the 99 references it cites were found in a registry or an index.
Learning Analytics for Learning Design: A Systematic Literature Review of Analytics-Driven Design to Enhance Learning
Open accessAbstract
As the fields of learning analytics and learning design mature, the convergence and synergies between the two are becoming an important area for research. This paper intends to summarize the main outcomes of a systematic review of empirical evidence on learning analytics for learning design. Moreover, this paper presents an overview of what and how learning analytics have been used to inform learning design decisions and in what contexts. The search was performed in seven academic databases, resulting in 43 papers included in the main analysis. The results from the review depict the ongoing design patterns and learning phenomena that emerged from the synergy that learning analytics and learning design impose on the current status of learning technologies. Finally, this review stresses that future research should consider developing a framework on how to capture and systematize learning design data grounded in learning analytics and learning theory, and document what learning design choices made by educators influence subsequent learning activities and performances over time.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Situated Cognition and the Culture of Learning
Brown, J. S., Collins, A., & Duguid, P. (1989) · Educational Researcher · doi:10.3102/0013189x018001032 · Cited by 13,645 · cited 1×
“[105], how to design learning activities differently over time [46], [86], and expertise reversal effects [106].”
BackgroundDunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013) · Psychological Science in the Public Interest · doi:10.1177/1529100612453266 · Cited by 3,138
Empirical studies of agile software development: A systematic review
Dybå, T., & Dingsøyr, T. (2008) · Information and Software Technology · doi:10.1016/j.infsof.2008.01.006 · Cited by 2,668 · cited 4×
“These criteria were informed by CASP and adapted for the purpose of this study following the Quality Assessment form used in a systematic review study about empirical studies of agile software development [52].”
BackgroundMethodInfluentialSocial Science Research: Principles, Methods and Practices
Bhattacherjee, A., Toleman, M., Rowling, S., Frederiks, A., & Andersen, N. (2019) · doi:10.26192/q7w89 · Cited by 2,031
Penetrating the Fog: Analytics in Learning and Education.
Siemens, G., & Long, P. (2011) · Adelaide Research & Scholarship (AR&S) (University of Adelaide) · Cited by 1,752
Preliminary guidelines for empirical research in software engineering
Kitchenham, B. A., Pfleeger, S. L., Pickard, L. M., Jones, P. W., Hoaglin, D. C., El Emam, K., & Rosenberg, J. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2002.1027796 · Cited by 1,510
Learning analytics: drivers, developments and challenges
Ferguson, R. (2012) · International Journal of Technology Enhanced Learning · doi:10.1504/ijtel.2012.051816 · Cited by 1,354
Learning Analytics: The Emergence of a Discipline
Siemens, G. (2013) · American Behavioral Scientist · doi:10.1177/0002764213498851 · Cited by 1,284
Expertise Reversal Effect and Its Implications for Learner-Tailored Instruction
Kalyuga, S. (2007) · Educational Psychology Review · doi:10.1007/s10648-007-9054-3 · Cited by 1,056 · cited 1×
“from shortterm interventions to continuous monitoring) [47] but also whether and when certain scaffolds can gradually be removed to avoid cognitive load [105], how to design learning activities differently over time [86], [46], and expertise reversal effects [106].”
BackgroundLet’s not forget: Learning analytics are about learning
Gašević, D., Dawson, S., & Siemens, G. (2015) · TechTrends · doi:10.1007/s11528-014-0822-x · Cited by 1,049
Course signals at Purdue: using learning analytics to increase student success
Arnold, K. E., & Pistilli, M. D. (2012) · Proceedings of the 2nd International Conference on Learning Analytics and Knowledge · doi:10.1145/2330601.2330666 · Cited by 987
Deconstructing disengagement: analyzing learner subpopulations in massive open online courses
Kizilcec, R. F., Piech, C., & Schneider, E. (2013) · Proceedings of the Third International Conference on Learning Analytics and Knowledge · doi:10.1145/2460296.2460330 · Cited by 984
Academic studing and the development of personal skill: A self-regulatory perspective
Zimmerman, B. J. (1998) · Educational Psychologist · doi:10.1080/00461520.1998.9653292 · Cited by 843
(2012) · DSpace (Open University in the Netherlands) · Cited by 754
A reference model for learning analytics
Chatti, M. A., Dyckhoff, A. L., Schroeder, U., & Thüs, H. (2012) · International Journal of Technology Enhanced Learning · doi:10.1504/ijtel.2012.051815 · Cited by 702
Shum, S. B., & Ferguson, R. (2012) · OPUS - Open Publications of UTS Scholars (University of Technology Sydney) · Cited by 699
Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2016) · The Internet and Higher Education · doi:10.1016/j.iheduc.2015.10.002 · Cited by 683
Using mapping studies as the basis for further research – A participant-observer case study
Kitchenham, B. A., Budgen, D., & Pearl Brereton, O. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2010.12.011 · Cited by 621
A history of instructional design and technology: Part II: A history of instructional design
Reiser, R. A. (2001) · Educational Technology Research and Development · doi:10.1007/bf02504928 · Cited by 607
(2014) · Educational Technology & Society · Cited by 598
Strong concepts: Intermediate-level knowledge in interaction design research
Höök, K., & Löwgren, J. (2012) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2362364.2362371 · Cited by 540
Academic Analytics: A New Tool for a New Era.
Campbell, J. P., DeBlois, P. B., & Oblinger, D. G. (2007) · Educational Review · Cited by 539
Designing for Learning in an Open World
Conole, G. (2013) · Springer eBooks · doi:10.1007/978-1-4419-8517-0 · Cited by 536
Social learning analytics: five approaches
Ferguson, R., & Buckingham Shum, S. (2012) · Proceedings of the 2nd International Conference on Learning Analytics and Knowledge · doi:10.1145/2330601.2330616 · Cited by 523
Informing Pedagogical Action: Aligning Learning Analytics With Learning Design
Lockyer, L., Heathcote, E., & Dawson, S. (2013) · American Behavioral Scientist · doi:10.1177/0002764213479367 · Cited by 471
The learning analytics cycle: closing the loop effectively
Clow, D. (2012) · Proceedings of the 2nd International Conference on Learning Analytics and Knowledge · doi:10.1145/2330601.2330636 · Cited by 405
Design for classroom orchestration
Dillenbourg, P. (2013) · Computers & Education · doi:10.1016/j.compedu.2013.04.013 · Cited by 386
Greenhalgh, P., & Greenhalgh, T. (1999) · Cited by 342
Rienties, B., & Toetenel, L. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.02.074 · Cited by 312
Model Driven Architecture and Ontology Development
(2006) · Springer eBooks · doi:10.1007/3-540-32182-9 · Cited by 305
A multivariate approach to predicting student outcomes in web-enabled blended learning courses
Zacharis, N. Z. (2015) · The Internet and Higher Education · doi:10.1016/j.iheduc.2015.05.002 · Cited by 302
Guest Editorial-Learning and Knowledge Analytics
Siemens, G., & Gašević, D. (2012) · HAL · Cited by 293
Designing pedagogical interventions to support student use of learning analytics
Wise, A. F. (2014) · Proceedings of the Fourth International Conference on Learning Analytics And Knowledge · doi:10.1145/2567574.2567588 · Cited by 252
Ma, J., Han, X., Yang, J., & Cheng, J. (2015) · The Internet and Higher Education · doi:10.1016/j.iheduc.2014.09.005 · Cited by 246
Why Theory Matters More than Ever in the Age of Big Data
Wise, A. F., & Shaffer, D. W. (2015) · Journal of Learning Analytics · doi:10.18608/jla.2015.22.2 · Cited by 241
Understanding MOOC students: motivations and behaviours indicative of MOOC completion
Pursel, B. K., Zhang, L., Jablokow, K. W., Choi, G. W., & Velegol, D. (2016) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12131 · Cited by 222
Pardo, A., Han, F., & Ellis, R. A. (2017) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2016.2639508 · Cited by 216
Mor, Y., Ferguson, R., & Wasson, B. (2015) · British Journal of Educational Technology · doi:10.1111/bjet.12273 · Cited by 192
Informing learning design with learning analytics to improve teacher inquiry
Persico, D., & Pozzi, F. (2014) · British Journal of Educational Technology · doi:10.1111/bjet.12207 · Cited by 176
Orchestrating technology enhanced learning: a literature review and a conceptual framework
Prieto, L. P., Dlab, M. H., Gutié, I., rrez, N. A., Abdulwahed, M., & Balid, W. (2011) · International Journal of Technology Enhanced Learning · doi:10.1504/ijtel.2011.045449 · Cited by 171
Piecing the learning analytics puzzle: a consolidated model of a field of research and practice
Gašević, D., Kovanović, V., & Joksimovíc, S. (2017) · Learning Research and Practice · doi:10.1080/23735082.2017.1286142 · Cited by 171
Technology Enhanced Learning as a Tool for Pedagogical Innovation
LAURILLARD, D. (2008) · Journal of Philosophy of Education · doi:10.1111/j.1467-9752.2008.00658.x · Cited by 168
Learning at distance: Effects of interaction traces on academic achievement
Joksimović, S., Gašević, D., Loughin, T. M., Kovanović, V., & Hatala, M. (2015) · Computers & Education · doi:10.1016/j.compedu.2015.07.002 · Cited by 167
Dataset-Driven Research to Support Learning and Knowledge Analytics
Verbert, K., Manouselis, N., Drachsler, H., & Duval, E. (2012) · DSpace (Open University in the Netherlands) · Cited by 165
A conceptual framework linking learning design with learning analytics
Bakharia, A., Corrin, L., de Barba, P., Kennedy, G., Gašević, D., Mulder, R., Williams, D., Dawson, S., & Lockyer, L. (2016) · Proceedings of the Sixth International Conference on Learning Analytics & Knowledge - LAK '16 · doi:10.1145/2883851.2883944 · Cited by 164
Ruipérez‐Valiente, J. A., Muñoz‐Merino, P. J., Leony, D., & Kloos, C. D. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.07.002 · Cited by 163
Prieto, L. P., Sharma, K. K., Kidziński, Ł., Rodríguez‐Triana, M. J., & Dillenbourg, P. (2018) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12232 · Cited by 162
How Software Technologies Can Improve Research on Learning and Bolster School Reform
Winne, P. H. (2006) · Educational Psychologist · doi:10.1207/s15326985ep4101_3 · Cited by 158
Rodríguez‐Triana, M. J., Martínez‐Monés, A., Asensio-Pérez, J. I., & Dimitriadis, Y. A. (2014) · British Journal of Educational Technology · doi:10.1111/bjet.12198 · Cited by 150
Nguyen, Q., Rienties, B., Toetenel, L., Ferguson, R., & Whitelock, D. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.03.028 · Cited by 147 · cited 6×
“Furthermore, findings show that learning analytics mostly have been used to explore the physical, digital, and human elements in the learning ecosystem [42], [81], for post-course reflection and recommendation of resources [67], for prediction [46], [66], [82], [83], as a tool [71], [77], [84], to detect weaknesses an…”
BackgroundResultInfluentialFoundations of dynamic learning analytics: Using university student data to increase retention
de Freitas, S., Gibson, D., Plessis, C. D., Halloran, P., Williams, E. J., Ambrose, M., Dunwell, I., & Arnab, S. (2014) · British Journal of Educational Technology · doi:10.1111/bjet.12212 · Cited by 145
Learning design: reflections upon the current landscape
Mor, Y., & Craft, B. (2012) · Research in Learning Technology · doi:10.3402/rlt.v20i0.19196 · Cited by 143
In medias res: reframing design for learning
Goodyear, P., & Dimitriadis, Y. (2013) · Research in Learning Technology · doi:10.3402/rlt.v21i0.19909 · Cited by 134
Using Learning Analytics to Assess Student Learning in Online Courses
Martin, F., & Ndoye, A. (2016) · Journal of University Teaching and Learning Practice · doi:10.53761/1.13.3.7 · Cited by 134
Li, N., Marsh, V., & Rienties, B. (2016) · Decision Sciences Journal of Innovative Education · doi:10.1111/dsji.12096 · Cited by 132
Learning Analytics for Online Discussions: Embedded and Extracted Approaches
Wise, A. F., Zhao, Y., & Hausknecht, S. N. (2014) · Journal of Learning Analytics · doi:10.18608/jla.2014.12.4 · Cited by 131
Adaptive web‐based learning: accommodating individual differences through system's adaptation
Magoulas, G. D., Papanikolaou, K. A., & Grigoriadou, M. (2003) · British Journal of Educational Technology · doi:10.1111/1467-8535.00347 · Cited by 119
Prieto, L. P., Rodríguez‐Triana, M. J., Martínez‐Maldonado, R., Dimitriadis, Y. A., & Gašević, D. (2018) · Australasian Journal of Educational Technology · doi:10.14742/ajet.4314 · Cited by 114
"Scaling up" learning design: impact of learning design activities on LMS behavior and performance
Rienties, B., Toetenel, L., & Bryan, A. (2015) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2723576.2723600 · Cited by 107
Connecting learning analytics with learning research: the role of design-based research
Reimann, P. (2016) · Learning Research and Practice · doi:10.1080/23735082.2016.1210198 · Cited by 101
Joksimovíc, S., Manataki, A., Gašević, D., Dawson, S., Kovanović, V., & de Kereki, I. F. (2016) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2883851.2883928 · Cited by 97
Supporting action research with learning analytics
Dyckhoff, A. L., Lukarov, V., Muslim, A., Chatti, M. A., & Schroeder, U. (2013) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2460296.2460340 · Cited by 93
Wise, A. F., Vytasek, J., Hausknecht, S. N., & Zhao, Y. (2016) · Online Learning · doi:10.24059/olj.v20i2.783 · Cited by 91
Supporting teachers in data‐informed educational design
McKenney, S. E., & Mor, Y. (2015) · British Journal of Educational Technology · doi:10.1111/bjet.12262 · Cited by 90
Rienties, B., Nguyen, Q., Holmes, W., & Reedy, K. (2017) · Interaction design & architecture(s)/ID&A Interaction design & architecture(s) · doi:10.55612/s-5002-033-007 · Cited by 90
AMOEBA: Designing for collaboration in computer science classrooms through live learning analytics
Berland, M., Davis, D., & Smith, C. P. (2015) · International Journal of Computer-Supported Collaborative Learning · doi:10.1007/s11412-015-9217-z · Cited by 77
Leveraging multimodal learning analytics to differentiate student learning strategies
Worsley, M. B., & Blikstein, P. (2015) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2723576.2723624 · Cited by 69
Designing for student-facing learning analytics
Kitto, K., Lupton, M., Davis, K., & Waters, Z. (2017) · Australasian Journal of Educational Technology · doi:10.14742/ajet.3607 · Cited by 69
Application of Learning Analytics in educational videogames
Serrano-Laguna, Á., Torrente, J., Pablo Moreno-ger, A., & Fernández‐Manjón, B. (2014) · Entertainment Computing · doi:10.1016/j.entcom.2014.02.003 · Cited by 68
Effects of instructional conditions and experience on the adoption of a learning tool
Gašević, D., Mirriahi, N., Dawson, S., & Joksimovíc, S. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.10.026 · Cited by 68
How was the activity? A visualization support for a case of location‐based learning design
Melero, J., Hernández‐Leo, D., Sun, J., Santos, P., & Blat, J. (2014) · British Journal of Educational Technology · doi:10.1111/bjet.12238 · Cited by 67
Video annotation and analytics in CourseMapper
Chatti, M. A., Marinov, M., Sabov, O., Laksono, R., Sofyan, Z., Yousef, A. M. F., & Schroeder, U. (2016) · Smart Learning Environments · doi:10.1186/s40561-016-0035-1 · Cited by 64
Haya, P. A., Daems, O., Malzahn, N., Castellanos, J., & Hoppe, H. U. (2015) · British Journal of Educational Technology · doi:10.1111/bjet.12264 · Cited by 57
Towards general models of effective science inquiry in virtual performance assessments
Baker, R., Clarke-Midura, J., & Ocumpaugh, J. L. (2016) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12128 · Cited by 57
A method for teacher inquiry in cross‐curricular projects: Lessons from a case study
Avramides, K., Hunter, J., Oliver, M., & Luckin, R. (2014) · British Journal of Educational Technology · doi:10.1111/bjet.12233 · Cited by 55
Using learning analytics to explore help-seeking learner profiles in MOOCs
Corrin, L., Barba, P. D., & Bakharia, A. (2017) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3027385.3027448 · Cited by 55
Martínez‐Maldonado, R., Yacef, K., & Kay, J. (2015) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2015.05.001 · Cited by 53
Processing and Visualizing Data in Complex Learning Environments
Thompson, K., Ashe, D., Carvalho, L., Goodyear, P., Kelly, N., & Parisio, M. L. (2013) · American Behavioral Scientist · doi:10.1177/0002764213479368 · Cited by 53
Orchestration in learning technology research: evaluation of a conceptual framework
Prieto, L. P., Dimitriadis, Y. A., Asensio-Pérez, J. I., & Looi, C. (2015) · Research in Learning Technology · doi:10.3402/rlt.v23.28019 · Cited by 51
Koh, E., Shibani, A., Tan, J. P.-L., & Hong, H. T. S. (2016) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2883851.2883914 · Cited by 50
Papamitsiou, Z. K., & Economides, A. A. (2016) · Springer eBooks · doi:10.1007/978-3-319-17727-4_15-1 · Cited by 47
Nguyen, Q., Rienties, B., & Toetenel, L. (2017) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3027385.3027409 · Cited by 42
Florián-Gaviria, B., Glahn, C., & Fabregat, R. (2013) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2013.18 · Cited by 41
Combining observational and experiential data to inform the redesign of learning activities
Pardo, A., Ellis, R. A., & Calvo, R. A. (2015) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2723576.2723625 · Cited by 38
Martínez‐Maldonado, R., Schneider, B., Charleer, S., Shum, S. B., Klerkx, J., & Duval, E. (2016) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2883851.2883873 · Cited by 36
E-Learning Systems: Intelligent Techniques for Personalization
Klanja-Milievi, A., Vesin, B., Ivanovi, M., Budimac, Z., & Jain, L. C. (2016) · Cited by 33
Mixing and Matching Learning Design and Learning Analytics
Nguyen, Q., Rienties, B., & Toetenel, L. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-58515-4_24 · Cited by 33
Constructing a taxonomy of media attributes for research purposes
Clark, R. E. (1975) · AV communication review · doi:10.1007/bf02768378 · Cited by 32
Forward‐oriented designing for learning as a means to achieve educational quality
Ghislandi, P., & Raffaghelli, J. E. (2015) · British Journal of Educational Technology · doi:10.1111/bjet.12257 · Cited by 32
Supporting orchestration of CSCL scenarios in web-based Distributed Learning Environments
Prieto, L. P., Asensio-Pérez, J. I., Cristóbal, J., Abellán, I. M. J., Dimitriadis, Y. A., & Gómez‐Sánchez, E. (2014) · Computers & Education · doi:10.1016/j.compedu.2013.12.008 · Cited by 31
A Novel Group Engagement Score for Virtual Learning Environments
Castellanos, J., Haya, P. A., & Urquiza‐Fuentes, J. (2016) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2016.2582164 · Cited by 29
Supporting orchestration of blended CSCL scenarios in distributed learning environments
Prieto, L. P. (2012) · UVaDOC UVaDOC University of Valladolid Documentary Repository (University of Valladolid) · doi:10.35376/10324/1794 · Cited by 23
Where learning analytics meets learning design
Lockyer, L., & Dawson, S. (2012) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2330601.2330609 · Cited by 23
Pantazos, K., & Vatrapu, R. (2016) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2016.14 · Cited by 18
Advancing MOOC and SPOC development via a learner decision journey analytic framework
Lei, C., Hou, X., Kwok, T. T.-O., Chan, T. S. F., Lee, J. J., Oh, E., Gonda, D. E., Yeung, Y.-C. A., & Lai, C. (2015) · IEEE International Conference on Teaching, Assessment, and Learning for Engineering · doi:10.1109/tale.2015.7386034 · Cited by 13
Tempelaar, D. T., Rienties, B., & Giesbers, B. (2016) · Communications in computer and information science · doi:10.1007/978-3-319-29585-5_15 · Cited by 12
Thompson, K. (2013) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2522848.2533792 · Cited by 9
Teachers' learning analytics tools in a social media enhanced learning environment
Tervakari, A.-M., Kuosa, K., Koro, J., Paukkeri, J., & Kailanto, M. (2014) · International Conference on Interactive Collaborative Learning · doi:10.1109/icl.2014.7017798 · Cited by 6
Cited by
The promise and challenges of generative AI in education
Giannakos, M., Azevedo, R., Brusilovsky, P., Cukurova, M., Dimitriadis, Y., Hernandez-Leo, D., Järvelä, S., Mavrikis, M., & Rienties, B. (2025) · Behaviour & Information Technology · doi:10.1080/0144929x.2024.2394886 · Cited by 543
Self-citationLeading teachers' perspective on teacher-AI collaboration in education
Kim, J. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-12109-5 · Cited by 376
Towards hybrid human‐AI learning technologies
Molenaar, I. (2022) · European Journal of Education · doi:10.1111/ejed.12527 · Cited by 348
Raj, N. S., & Renumol, V. G. (2022) · Journal of Computers in Education · doi:10.1007/s40692-021-00199-4 · Cited by 253
Banihashem, S. K., Noroozi, O., van Ginkel, S., Macfadyen, L. P., & Biemans, H. J. A. (2022) · Educational Research Review · doi:10.1016/j.edurev.2022.100489 · Cited by 246
Metadata
- Authors
- Mangaroska, K., & Giannakos, M.
- Venue
- IEEE Transactions on Learning Technologies
- Published
- 2019-10-01
- Volume
- 12
- Issue
- 4
- Pages
- 516-534
- ISSN
- 1939-1382 (ISSN-L)
- 2372-0050
- OpenAlex
- W2890544186
- Cited by
- 378
- References
- 99
- FWCI
- 5.64
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.