Avoiding Methodological Biases in Meta-Analysis: Use of Online Versus Offline Individual Participant Data (IPD) in Educational Psychology
Open accessAbstract
Abstract. Individual participant data (IPD) meta-analysis is the gold standard of meta-analyses. This paper points out several advantages of IPD meta-analysis over classical meta-analysis, such as avoiding aggregation bias (e.g., ecological fallacy or Simpson’s paradox) and shows how its two main disadvantages (time and cost) can be overcome through Internet-based research. Ideally, we recommend carrying out IPD meta-analyses that consider online versus offline data gathering processes and examine data quality. Through a comprehensive literature search, we investigated whether IPD meta-analyses published in the field of educational psychology already follow these recommendations; this was not the case. For this reason, the paper demonstrates characteristics of ideal meta-analysis on teachers’ judgment accuracy and links it to recent meta-analyses on that topic. The recommendations are important for meta-analysis researchers and for readers and reviewers of meta-analyses. Our paper is also relevant to current discussions within the psychological community on study replication.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Potentially relevant moderators using summary data were not examined due to potential aggregation bias.
“Potentially relevant moderators using summary data (e.g.duration/hours caregiving) were not examined due to potential aggregation bias 34 .”
From Anxiety in Informal Dementia Carers: A Meta-Analysis of Prevalence (2020) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116 · see citing works
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · see citing works
Visible learning: a synthesis of over 800 meta‐analyses relating to achievement
Carter, M. (2009) · Educational Psychology · doi:10.1080/01443410903415150 · Cited by 7,696 · see citing works
(2008) · Cited by 6,939 · see citing works
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542 · see citing works
Primary, Secondary, and Meta-Analysis of Research
GLASS, G. V. (1976) · Educational Researcher · doi:10.3102/0013189x005010003 · Cited by 4,890 · see citing works
Socioeconomic Status and Academic Achievement: A Meta-Analytic Review of Research
Sirin, S. R. (2005) · Review of Educational Research · doi:10.3102/00346543075003417 · Cited by 4,687 · cited 1× · see citing works
“…the inappropriate aggregation level within classical meta-analysis approaches to draw conclusions about relationships between constructs at the individual level may result in misleading conclusions within educational psychology (see Hanushek, Rivkin, & Taylor, 1996; Moffitt, 1996; Sirin, 2005).”
ResultEcological Correlations and the Behavior of Individuals
Robinson, W. S. (1950) · American Sociological Review · doi:10.2307/2087176 · Cited by 4,547 · see citing works
Outlier and influence diagnostics for meta-analysis
Viechtbauer, W., & Cheung, M. W.-L. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.11 · Cited by 2,949 · see citing works
Quantitative Synthesis in Systematic Reviews
Lau, J., Ioannidis, J. P. A., & Schmid, C. H. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-9-199711010-00008 · Cited by 2,517 · see citing works
The interpretation of interaction in contingency tables
Simpson, E. H. (1951) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1951.tb00088.x · Cited by 2,301 · cited 2× · see citing works
“The Prevention of Simpson’s Paradox The IPD approach avoids a second type of ecological fallacy aggregation bias, namely, Simpson’s paradox (Simpson, 1951), which occurs when the heterogeneity in the population is underestimated.”
MethodStewart, L. A., Clarke, M., Rovers, M., Riley, R. D., Simmonds, M., Stewart, G., & Tierney, J. F. (2015) · JAMA · doi:10.1001/jama.2015.3656 · Cited by 2,134 · see citing works
Lipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033 · see citing works
Meta-Analysis: Recent Developments in Quantitative Methods for Literature Reviews
Rosenthal, R., & DiMatteo, M. R. (2001) · Annual Review of Psychology · doi:10.1146/annurev.psych.52.1.59 · Cited by 1,780 · see citing works
Integrative data analysis: The simultaneous analysis of multiple data sets
Curran, P. J., & Hussong, A. M. (2009) · Psychological Methods · doi:10.1037/a0015914 · Cited by 867 · see citing works
Meta-analysis of the literature or of individual patient data: is there a difference?
Stewart, L. A., & Parmar, M. (1993) · The Lancet · doi:10.1016/0140-6736(93)93004-k · Cited by 771 · see citing works
Accuracy of teachers' judgments of students' academic achievement: A meta-analysis
Südkamp, A., Kaiser, J., & Möller, J. (2012) · Journal of Educational Psychology · doi:10.1037/a0027627 · Cited by 696 · see citing works
Bickel, P. J., Hammel, E. A., & O'Connell, J. W. (1975) · Science · doi:10.1126/science.187.4175.398 · Cited by 659 · see citing works
Teacher-Based Judgments of Academic Achievement: A Review of Literature
Hoge, R. D., & Coladarci, T. (1989) · Review of Educational Research · doi:10.3102/00346543059003297 · Cited by 588 · see citing works
Stewart, L. A., & Tierney, J. F. (2002) · Evaluation & the Health Professions · doi:10.1177/0163278702025001006 · Cited by 581 · see citing works
CHALMERS, I. (1993) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1993.tb26345.x · Cited by 561 · see citing works
Simmonds, M. C., Higgins, J. P. T., Stewart, L. A., Tierney, J. F., Clarke, M. J., & Thompson, S. G. (2005) · Clinical Trials · doi:10.1191/1740774505cn087oa · Cited by 557 · see citing works
Standards for Internet-Based Experimenting
Reips, U. (2002) · Experimental Psychology (formerly Zeitschrift für Experimentelle Psychologie) · doi:10.1026//1618-3169.49.4.243 · Cited by 557 · see citing works
Self-citationBerlin, J. A., Santanna, J., Schmid, C. H., Szczech, L. A. M., & Feldman, H. I. (2002) · Statistics in Medicine · doi:10.1002/sim.1023 · Cited by 468 · see citing works
Aggregation and the Estimated Effects of School Resources
Hanushek, E. A., Rivkin, S. G., & Taylor, L. L. (1996) · National Bureau of Economic Research · doi:10.3386/w5548 · Cited by 411 · see citing works
Experimental evidence of dietary factors and hormone-dependent cancers.
Carroll, K. K. (1975) · PubMed · Cited by 379 · see citing works
Tierney, J. F., Vale, C. L., Riley, R. D., Smith, C. T., Stewart, L. A., Clarke, M. J., & Rovers, M. M. (2015) · PLoS Medicine · doi:10.1371/journal.pmed.1001855 · Cited by 361 · see citing works
Determinants of linear judgment: A meta-analysis of lens model studies.
Karelaia, N., & Hogarth, R. M. (2008) · Psychological Bulletin · doi:10.1037/0033-2909.134.3.404 · Cited by 358 · see citing works
Get real in individual participant data (IPD) meta‐analysis: a review of the methodology
Debray, T. P. A., Moons, K. G. M., van Valkenhoef, G., Efthimiou, O., Hummel, N., Groenwold, R. H. H., Reitsma, J. B., & on behalf of the GetReal methods review group (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1160 · Cited by 348 · see citing works
Cooper, H., & Patall, E. A. (2009) · Psychological Methods · doi:10.1037/a0015565 · Cited by 339 · see citing works
Social desirability is the same in offline, online, and paper surveys: A meta-analysis
Dodou, D., & de Winter, J. C. F. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.04.005 · Cited by 325 · see citing works
Aggregation and the Estimated Effects of School Resources
Hanushek, E. A., Rivkin, S. G., & Taylor, L. L. (1996) · The Review of Economics and Statistics · doi:10.2307/2109949 · Cited by 322 · see citing works
Dimensions of Internet science
Reips, U., & Bošnjak, M. (2001) · Cited by 319 · see citing works
Self-citationThe essential brunswik: Beginnings, explications, applications.
Brunswik, E., Hammond, K. R., & Stewart, T. R. (2001) · Cited by 292 · see citing works
A typology of ecological fallacies
Alker, H. (1969) · Cited by 255 · see citing works
The strengths and limitations of meta-analyses based on aggregate data
Lyman, G. H., & Kuderer, N. M. (2005) · BMC Medical Research Methodology · doi:10.1186/1471-2288-5-14 · Cited by 250 · see citing works
Schmid, C. H., Stark, P. C., Berlin, J. A., Landais, P., & Lau, J. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2003.12.001 · Cited by 245 · see citing works
Batinic, B., Reips, U., & Bošnjak, M. (2002) · Cited by 242 · see citing works
Self-citationMeta‐research: The art of getting it wrong
Ioannidis, J. P. A. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.19 · Cited by 231 · see citing works
Publication bias as a threat to the validity of meta-analytic results
Rothstein, H. R. (2008) · Journal of Experimental Criminology · doi:10.1007/s11292-007-9046-9 · Cited by 216 · see citing works
Riley, R. D., Simmonds, M. C., & Look, M. P. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.09.009 · Cited by 195 · see citing works
Accounting for heterogeneity via random-effects models and moderator analyses in meta-analysis.
Viechtbauer, W. (2007) · Zeitschrift für Psychologie / Journal of Psychology · doi:10.1027/0044-3409.215.2.104 · Cited by 193 · see citing works
(2001) · doi:10.1093/oso/9780195130133.001.0001 · Cited by 191 · see citing works
Stöber, J. (1999) · Diagnostica · doi:10.1026//0012-1924.45.4.173 · Cited by 184 · see citing works
Meta‐analysis of individual‐ and aggregate‐level data
Sutton, A. J., Kendrick, D., & Coupland, C. A. C. (2007) · Statistics in Medicine · doi:10.1002/sim.2916 · Cited by 162 · see citing works
Schmidt, F. L., & Oh, I.-S. (2016) · Archives of Scientific Psychology · doi:10.1037/arc0000029 · Cited by 145 · see citing works
Overviews in Education Research
Polanin, J. R., Maynard, B. R., & Dell, N. A. (2016) · Review of Educational Research · doi:10.3102/0034654316631117 · Cited by 132 · see citing works
A decade of individual participant data meta-analyses: A review of current practice
Simmonds, M. C., Stewart, G. B., & Stewart, L. A. (2015) · Contemporary Clinical Trials · doi:10.1016/j.cct.2015.06.012 · Cited by 131 · see citing works
Commentary: Meta-analysis of Individual Participants' Data in Genetic Epidemiology
Ioannidis, J. P. A. (2002) · American Journal of Epidemiology · doi:10.1093/aje/kwf031 · Cited by 117 · see citing works
Claxton, S. E., Bishop, H. K. D., & van Dulmen, M. H. M. (2015) · Archives of Sexual Behavior · doi:10.1007/s10508-014-0392-1 · Cited by 72 · see citing works
Televantou, I., Marsh, H. W., Kyriakides, L. E., Nagengast, B., Fletcher, J., & Malmberg, L. (2015) · School Effectiveness and School Improvement · doi:10.1080/09243453.2013.871302 · Cited by 66 · see citing works
Mangan, M. A., & Reips, U. (2007) · Behavior Research Methods · doi:10.3758/bf03193152 · Cited by 64 · see citing works
Self-citationReips, U. (2006) · American Psychological Association eBooks · doi:10.1037/11383-006 · Cited by 47 · see citing works
Self-citationTelevantou, I., Marsh, H. W., Kyriakides, L. E., Nagengast, B., Fletcher, J., & Malmberg, L. (2018) · Routledge eBooks · doi:10.4324/9781315231037-5 · Cited by 38 · see citing works
How Internet-Mediated Research Changes Science
Reips, U. (2008) · Cambridge University Press eBooks · doi:10.1017/cbo9780511813740.013 · Cited by 33 · see citing works
Self-citationGlass, G. V. (2016) · Educational Researcher · doi:10.3102/0013189x16639026 · Cited by 32 · see citing works
Combining individual participant and aggregated data in a meta‐analysis with correlational studies
Pigott, T., Williams, R. T., & Polanin, J. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1051 · Cited by 27 · see citing works
A Critical Meta-Analysis of Lens Model Studies in Human Judgment and Decision-Making
Kaufmann, E., Reips, U., & Wittmann, W. W. (2013) · PLoS ONE · doi:10.1371/journal.pone.0083528 · Cited by 27 · see citing works
Self-citationA Meta-Analysis of Judgment Achievement as Defined by the Lens Model Equation
Kaufmann, E., & Athanasou, J. A. (2009) · Swiss Journal of Psychology · doi:10.1024/1421-0185.68.2.99 · Cited by 25 · see citing works
Self-citationJudgment of Factors Influencing Interest: An Australian Study
Athanasou, J. A., & Cooksey, R. (2001) · Journal of Vocational Education Research · doi:10.5328/jver26.1.77 · Cited by 20 · see citing works
Moffitt, R. (1996) · The Review of Economics and Statistics · doi:10.2307/2109947 · Cited by 14 · see citing works
Introduction to the special issue on the origins of modern meta‐analysis
Shadish, W. R. (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1148 · Cited by 13 · see citing works
Albarracín, D. (2015) · Psychological Bulletin · doi:10.1037/bul0000007 · Cited by 13 · see citing works
Ein national gefördertes Onlinelabor als Infrastruktur für die psychologische Forschung
Bruder, M., Göritz, A. S., Reips, U., & Gebhard, R. K. (2014) · Psychologische Rundschau · doi:10.1026/0033-3042/a000198 · Cited by 8 · see citing works
Self-citationStatistical Harmonization Methods in Individual Participants Data Meta-Analysis are Highly Needed
van den Heuvel, E. R. (2016) · Biometrics & Biostatistics International Journal · doi:10.15406/bbij.2016.03.00064 · Cited by 8 · see citing works
Flagged first, then by citations
1 of the 66 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- Albarracín, D. (2015). Editorial. Psychological Bulletin, 141, 1-5.
- Bernard, R. M. (2014). Things I have learned about meta-analysis since 1990: Reducing bias in search of "The Big Picture". Canadian Journal of Learning and Instruction, 40, 17.
- Dyssegaard, C. B., Egeberg, J. H., Sommersel, H. B., Steenberg, K., & Vestergaard, S. (2015). A systematic review of the impact of multiple language teaching, prior language experience and acquisition order on student's language proficiency in primary and secondary school. Copenhagen, Denmark: Danish Clearinghouse for Educational Research, Department of Education, Aarhus University.
- Moffitt, R. (1996). Symposium on school quality and educational outcomes. Review of Economics and Statistics, 78, 559-561.
- Reips, U.-D., Buchanan, T., Krantz, J. H., & McGraw, K. (2015). Methodological challenges in the use of the Internet for scientific research: Ten solutions and recommendations. Studia Psychologica. Advance online publication.
- Viechtbauer, W. (2007). Random-effects models and moderator analyses in meta-analysis. Zeitschrift für Psychologie/Journal of Psychology, 215, 104-121.
Cited by
Sorted by citations
Theobald, E. J., Hill, M. J., Tran, E., Agrawal, S., Arroyo, E. N., Behling, S., Chambwe, N., Cintrón, D. L., Cooper, J. D., Dunster, G., et al. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1916903117 · Cited by 1,387 · see citing works
Smell and Taste Dysfunction in Patients With COVID-19: A Systematic Review and Meta-analysis
Agyeman, A. A., Chin, K. L., Landersdorfer, C. B., Liew, D., & Ofori‐Asenso, R. (2020) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2020.05.030 · Cited by 452 · see citing works
Data-based decision-making for school improvement: Research insights and gaps
Schildkamp, K. (2019) · Educational Research · doi:10.1080/00131881.2019.1625716 · Cited by 347 · see citing works
Steel, P. D. G., Beugelsdijk, S., & Aguinis, H. (2021) · Journal of International Business Studies · doi:10.1057/s41267-020-00385-z · Cited by 5 · see citing works
The Structure of the Rosenberg Self-Esteem Scale
Gnambs, T., Scharl, A., & Schroeders, U. (2018) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000317 · Cited by 123 · see citing works
Anxiety in Informal Dementia Carers: A Meta-Analysis of Prevalence
Kaddour, L., & Kishita, N. (2020) · Journal of Geriatric Psychiatry and Neurology · doi:10.1177/0891988719868313 · Cited by 98 · see citing works
How Do Teachers Make Sense of Data in the Context of High-Stakes Decision Making?
Vanlommel, K., & Schildkamp, K. (2019) · American Educational Research Journal · doi:10.3102/0002831218803891 · Cited by 79 · see citing works
A Meta-Analysis of Test Scores in Proctored and Unproctored Ability Assessments
Steger, D., Schroeders, U., & Gnambs, T. (2018) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000494 · Cited by 58 · see citing works
Yang, Y., Macleod, M. R., Pan, J., Lagisz, M., & Nakagawa, S. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.105016 · Cited by 49 · see citing works
Wolff, F., & Møller, J. (2022) · Educational Psychology Review · doi:10.1007/s10648-022-09686-1 · Cited by 31 · see citing works
Task-specific algorithm advice acceptance: A review and directions for future research
Kaufmann, E., Chacon, A., Kausel, E. E., Herrera, N., & Reyes, T. H. (2023) · Data and Information Management · doi:10.1016/j.dim.2023.100040 · Cited by 28 · see citing works
Self-citationHow accurately do teachers’ judge students? Re-analysis of Hoge and Coladarci (1989) meta-analysis
Kaufmann, E. (2020) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2020.101902 · Cited by 26 · see citing works
Self-citationScoring German Alternate Uses Items Applying Large Language Models
Saretzki, J., Knopf, T., Forthmann, B., Goecke, B., Jaggy, A., Benedek, M., & Weiss, S. (2025) · Journal of Intelligence · doi:10.3390/jintelligence13060064 · Cited by 11 · see citing works
Schumann, S., Kaufmann, E., Eberle, F., Jüttler, A., & Ackermann, N. (2017) · Empirical research in vocational education and training · doi:10.1186/s40461-017-0056-1 · Cited by 10 · see citing works
Self-citationErdfelder, E., & Bošnjak, M. (2016) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000249 · Cited by 9 · see citing works
Lens model studies: Revealing teachers’ judgements for teacher education
Kaufmann, E. (2022) · Journal of Education for Teaching International Research and Pedagogy · doi:10.1080/02607476.2022.2061336 · Cited by 7 · see citing works
Self-citationWang, Z., Murray, T., Xiao, M., Lin, L.-F., Alemayehu, D., & Chu, H. (2023) · Statistics in Medicine · doi:10.1002/sim.9902 · Cited by 5 · see citing works
Meta-analysis in a digitalized world: A step-by-step primer
Kaufmann, E., & Reips, U. (2024) · Behavior Research Methods · doi:10.3758/s13428-024-02374-8 · Cited by 7 · see citing works
Self-citationTeachers’ judgment accuracy: A replication check by psychometric meta-analysis
Kaufmann, E. (2024) · PLoS ONE · doi:10.1371/journal.pone.0307594 · Cited by 4 · see citing works
Self-citation“My Goal Is to Lose 2.923 kg!”—Efficacy of Precise Versus Round Goals for Body Weight Reduction
Frech, M.-L., Friese, M., & Loschelder, D. D. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.793962 · Cited by 4 · see citing works
Hotspots in Psychology – 2018 Edition
Bošnjak, M., & Erdfelder, E. (2018) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000323 · Cited by 4 · see citing works
Campos, D., Cheung, M. W. ‐., & Scherer, R. (2021) · doi:10.31234/osf.io/r9p62 · Cited by 4 · see citing works
A Meta-Analysis of Test Scores in Proctored and Unproctored Ability AssessmentsPreprint
Steger, D., Schroeders, U., & Gnambs, T. (2018) · doi:10.31234/osf.io/wxj4y · Cited by 4 · see citing works
An Individual Participant Data Meta-Analysis Method Primer for Psychology
Fritz, T. M., & Göritz, A. S. (2026) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000614 · Cited by 2 · see citing works
Klose, M., Steger, D., Fick, J., & Artelt, C. (2022) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000484 · Cited by 2 · see citing works
Where’s the data to support educators’ data use for instructional practice?
Barnes, N., Brighton, C. M., Fives, H., Meyers, C. V., & Moon, T. R. (2022) · Theory Into Practice · doi:10.1080/00405841.2022.2096379 · Cited by 2 · see citing works
Dekkers, R., Carey, L., & Langhorne, P. (2022) · Springer eBooks · doi:10.1007/978-3-030-90025-0_7 · Cited by 2 · see citing works
Data Aggregation: A Proposed Psychometric IPD Meta-Analysis
Kaufmann, E. (2018) · Open Journal of Statistics · doi:10.4236/ojs.2018.81004 · Cited by 1 · see citing works
Self-citationSimpson's Paradox in Psychology
Mangalam, M. (2022) · doi:10.1093/obo/9780199828340-0301 · Cited by 1 · see citing works
Chow, J. C., & Martinez, R. G. (2026) · Educational Psychology Review · doi:10.1007/s10648-026-10152-5
- Authors
- Kaufmann, E., Reips, U.-D., & Maag Merki, K.
- Affiliations
- University of Zurich (CH)
- University of Konstanz (DE)
- Institute of Education, University of Zurich, Switzerland
- Venue
- Zeitschrift für Psychologie
- Publisher
- Hogrefe Publishing Group
- Published
- 2016-07
- Volume
- 224
- Issue
- 3
- Pages
- 157-167
- ISSN
- 0044-3409 (ISSN-L)
- 2190-8370
- 2151-2604
- OpenAlex
- W2560101504
- Cited by
- 31
- References
- 66
- Citations by year
- 2026: 2 · 2025: 1 · 2024: 2 · 2023: 2 · 2022: 8 · 2021: 2 · 2020: 3 · 2019: 3 · 2018: 6 · 2017: 1 · 2016: 1
- FWCI
- 3.43
- Type
- journal article
- Language
- en
- Open access
- green
- nbn-resolving.de
- Locations
- Zeitschrift für Psychologie (publishedVersion)
- KOPS (University of Konstanz) (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Education › Educational Research and Methods
- Keywords
- individual patient data meta-analysis, aggregation bias, ecological fallacy, Simpson's paradox, online data collection, data quality, replication study
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
