Many Labs 3: Evaluating participant pool quality across the academic semester via replication
Open accessAbstract
Abstract The university participant pool is a key resource for behavioral research, and data quality is believed to vary over the course of the academic semester. This crowdsourced project examined time of semester variation in 10 known effects, 10 individual differences, and 3 data quality indicators over the course of the academic semester in 20 participant pools (N = 2696) and with an online sample (N = 737). Weak time of semester effects were observed on data quality indicators, participant sex, and a few individual differences—conscientiousness, mood, and stress. However, there was little evidence for time of semester qualifying experimental or correlational effects. The generality of this evidence is unknown because only a subset of the tested effects demonstrated evidence for the original result in the whole sample. Mean characteristics of pool samples change slightly during the semester, but these data suggest that those changes are mostly irrelevant for detecting effects.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Attention paid to positive, novel, and clean findings may contribute to publication bias, selective reporting, and questionable research practices that reduce the credibility of the published literature.
“This may contribute to publication bias, selective reporting, and questionable research practices that reduce the credibility of the published literature 18-25 .”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [25]
Many Labs projects yielded variable rates of replication success.
“Using similar criteria, the”
From Many Labs 4: Failure to Replicate Mortality Salience Effect With and Without Original Author Involvement (2022) · cites this paper as Ebersole et al., 2016
Cites
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139 · see citing works
A Global Measure of Perceived Stress
Cohen, S., Kamarck, T., & Mermelstein, R. (1983) · Journal of Health and Social Behavior · doi:10.2307/2136404 · Cited by 32,807 · see citing works
Studies of interference in serial verbal reactions
Stroop, J. R. (1935) · Journal of Experimental Psychology · doi:10.1037/h0054651 · Cited by 18,622 · see citing works
Availability: A heuristic for judging frequency and probability
Tversky, A., & Kahneman, D. (1973) · Cognitive Psychology · doi:10.1016/0010-0285(73)90033-9 · Cited by 9,984 · cited 2× · see citing works
“& Kahneman, 1973, study 3) Tversky and Kahneman (1973) examined whether undergraduates recruited separately from a participant pool would overestimate the frequency of easier-to-imagine words relative to harder-to-imagine words.”
BackgroundEstimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · see citing works
A very brief measure of the Big-Five personality domains
Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003) · Journal of Research in Personality · doi:10.1016/s0092-6566(03)00046-1 · Cited by 8,307 · see citing works
Understanding and using the Implicit Association Test: I. An improved scoring algorithm.
Greenwald, A. G., Nosek, B. A., & Banaji, M. R. (2003) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.85.2.197 · Cited by 5,815 · see citing works
Self-citationHalf a century of research on the Stroop effect: An integrative review.
MacLeod, C. M. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.109.2.163 · Cited by 5,500 · see citing works
Cacioppo, J. T., & Petty, R. E. (1982) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.42.1.116 · Cited by 5,359 · see citing works
Schwarz, N., & Clore, G. L. (1983) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.45.3.513 · Cited by 4,952 · see citing works
A Systematic Method for Clinical Description and Classification of Personality Variants: A Proposal
Cloninger, C. R. (1987) · Archives of General Psychiatry · doi:10.1001/archpsyc.1987.01800180093014 · Cited by 3,678 · see citing works
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493 · see citing works
Robins, R. W., Hendin, H. M., & Trzesniewski, K. H. (2001) · Personality and Social Psychology Bulletin · doi:10.1177/0146167201272002 · Cited by 3,041 · see citing works
The Efficient Assessment of Need for Cognition
Cacioppo, J. T., Petty, R. E., & Feng Kao, C. (1984) · Journal of Personality Assessment · doi:10.1207/s15327752jpa4803_13 · Cited by 2,913 · see citing works
Robertson, I. H., Manly, T., Andrade, J., Baddeley, B., & Yiend, J. (1997) · Neuropsychologia · doi:10.1016/s0028-3932(97)00015-8 · Cited by 2,064 · see citing works
Handbook of theories of social psychology
(2012) · Choice Reviews Online · doi:10.5860/choice.49-5135 · Cited by 1,924 · see citing works
The Work Preference Inventory: Assessing intrinsic and extrinsic motivational orientations.
Amabile, T. M., Hemphill, S. A., Hennessey, B. A., & Tighe, E. M. (1994) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.66.5.950 · Cited by 1,892 · see citing works
Metaphoric structuring: understanding time through spatial metaphors
Boroditsky, L. (2000) · Cognition · doi:10.1016/s0010-0277(99)00073-6 · Cited by 1,849 · see citing works
Kane, M. J., & Engle, R. W. (2003) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.132.1.47 · Cited by 1,725 · see citing works
Power and Perspectives Not Taken
Galinsky, A. D., Magee, J. C., Inesi, M. E., & Gruenfeld, D. H. (2006) · Psychological Science · doi:10.1111/j.1467-9280.2006.01824.x · Cited by 1,286 · see citing works
Investigating Variation in Replicability: A “Many Labs” Replication Project
Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., et al. (2014) · Social Psychology · doi:10.1027/1864-9335/a000178 · Cited by 1,208 · see citing works
Self-citationMoral credentials and the expression of prejudice.
Monin, B., & Miller, D. T. (2001) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.81.1.33 · Cited by 1,086 · see citing works
Many Labs 2: Investigating variation in replicability across samples and settings
Klein, R. A., Vianello, M., Hasselman, F., Adams, B. G., Adams, R. B., Alper, S., Aveyard, M., Axt, J. R., Babalola, M. T., Bahník, Š., et al. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918810225 · Cited by 944 · see citing works
Self-citationEffects of need for cognition on message evaluation, recall, and persuasion.
Cacioppo, J., Petty, R., & Morris, K. J. (1983) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.45.4.805 · Cited by 899 · see citing works
When do we need procedural fairness? The role of trust in authority.
van den Bos, K., Wilke, H., & Lind, E. A. (1998) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.75.6.1449 · Cited by 521 · see citing works
De Fruyt, F., Van De Wiele, L., & Van Heeringen, C. (2000) · Personality and Individual Differences · doi:10.1016/s0191-8869(99)00204-4 · Cited by 455 · see citing works
Weight as an Embodiment of Importance
Jostmann, N. B., Lakens, D., & Schubert, T. W. (2009) · Psychological Science · doi:10.1111/j.1467-9280.2009.02426.x · Cited by 404 · see citing works
Effect Sizes for Research: Univariate and Multivariate Applications
Grissom, R. J., & Kim, J. J. (2005) · Cited by 376 · see citing works
Inzlicht, M., & Gutsell, J. N. (2007) · Psychological Science · doi:10.1111/j.1467-9280.2007.02004.x · Cited by 351 · see citing works
Ross, M., & Wilson, A. E. (2002) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.82.5.792 · Cited by 348 · see citing works
The Elaboration Likelihood Model
Petty, R. E., & Briñol, P. (2012) · Handbook of Theories of Social Psychology: Volume 1 · doi:10.4135/9781446249215.n12 · Cited by 324 · see citing works
Priming, Replication, and the Hardest Science
Cesario, J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691613513470 · Cited by 320 · see citing works
Aging and the Stroop effect: A meta-analysis.
Verhaeghen, P., & Meersman, L. D. (1998) · Psychology and Aging · doi:10.1037//0882-7974.13.1.120 · Cited by 293 · see citing works
Aging and the Stroop effect: A meta-analysis.
Verhaeghen, P., & Meersman, L. D. (1998) · Psychology and Aging · doi:10.1037/0882-7974.13.1.120 · Cited by 269 · see citing works
Testing the latent factor structure and construct validity of the Ten-Item Personality Inventory
Ehrhart, M. G., Ehrhart, K. H., Roesch, S. C., Chung‐Herrera, B. G., Nadler, K., & Bradshaw, K. (2009) · Personality and Individual Differences · doi:10.1016/j.paid.2009.07.012 · Cited by 255 · see citing works
Aspinwall, L. G., & Richter, L. (1999) · Motivation and Emotion · doi:10.1023/a:1021367331817 · Cited by 232 · see citing works
The elaboration likelihood model
Teeny, J. D., Briñol, P., & Petty, R. E. (2012) · doi:10.4324/9781315727448.ch21 · Cited by 204 · see citing works
Need for cognition and external information search effort
Verplanken, B., Hazenberg, P. T., & Palenéwen, G. R. (1992) · Journal of Research in Personality · doi:10.1016/0092-6566(92)90049-a · Cited by 192 · see citing works
Sommer, K. L., & Baumeister, R. F. (2002) · Personality and Social Psychology Bulletin · doi:10.1177/01467202028007006 · Cited by 180 · see citing works
Gnambs, T. (2014) · Journal of Research in Personality · doi:10.1016/j.jrp.2014.06.003 · Cited by 170 · see citing works
Replication of “Experiencing Physical Warmth Promotes Interpersonal Warmth” by
Lynott, D., Corker, K. S., Wortman, J., Connell, L., Donnellan, M. B., Lucas, R. E., & O’Brien, K. (2014) · Social Psychology · doi:10.1027/1864-9335/a000187 · Cited by 147 · see citing works
Self-citationConsciousness, brain, neuroplasticity
Askenasy, J., & Lehmann, J. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00142 · Cited by 110 · see citing works
Hom, H. L. (1987) · Personality and Social Psychology Bulletin · doi:10.1177/0146167287132006 · Cited by 94 · see citing works
Many Labs 2: Investigating Variation in Replicability Across Sample and SettingPreprint
Klein, R. A., Vianello, M., Hasselman, F., Adams, B. G., Adams, R. B., Alper, S., Aveyard, M. E., Axt, J., Babalola, M. T., Bahník, Š., et al. (2018) · doi:10.31234/osf.io/9654g · Cited by 81 · see citing works
Self-citationVerplanken, B. (1991) · Personality and Social Psychology Bulletin · doi:10.1177/014616729101700211 · Cited by 76 · see citing works
Convergent and Discriminant Validity of the Five Factor Form
Rojas, S. L., & Widiger, T. A. (2013) · Assessment · doi:10.1177/1073191113517260 · Cited by 66 · see citing works
Faulty assumptions: A comment on Blanton, Jaccard, Gonzales, and Christie (2006).
Nosek, B. A., & Sriram, N. (2007) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2006.10.018 · Cited by 58 · see citing works
Self-citationAviv, A., Zelenski, J. M., Rallo, L., et al. (2002) · Personality and Individual Differences · doi:10.1016/s0191-8869(01)00199-4 · Cited by 44 · see citing works
Nicholls, M. E. R., Loveless, K. M., Thomas, N. A., Loetscher, T., & Churches, O. F. (2014) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2014.925481 · Cited by 44 · see citing works
Szymków, A., Chandler, J., IJzerman, H., Parzuchowski, M., & Wojciszke, B. (2013) · Social Psychology · doi:10.1027/1864-9335/a000147 · Cited by 42 · see citing works
Are within-subjects designs transparent?
Lambdin, C., & Shaffer, V. A. (2009) · Judgment and Decision Making · doi:10.1017/s1930297500001133 · Cited by 33 · see citing works
Timing and selection effects within a psychology subject pool: Personality and sex matter
Witt, E. A., Donnellan, M. B., & Orlando, M. J. (2010) · Personality and Individual Differences · doi:10.1016/j.paid.2010.10.019 · Cited by 30 · see citing works
Correlational biases in mean response latency differences
Sriram, N., Greenwald, A. G., & Nosek, B. A. (2009) · Statistical Methodology · doi:10.1016/j.stamet.2009.10.004 · Cited by 28 · see citing works
Self-citationPoint-of-Time Effects Across the Semester: Is There a Sampling Bias?
Wang, A. Y., & Jentsch, F. G. (1998) · The Journal of Psychology · doi:10.1080/00223989809599160 · Cited by 20 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397 · see citing works
Self-citationThe reliability paradox: Why robust cognitive tasks do not produce reliable individual differences
Hedge, C., Powell, G., & Sumner, P. (2018) · Behavior Research Methods · doi:10.3758/s13428-017-0935-1 · Cited by 1,457 · see citing works
Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Nave, G., Nosek, B. A., Pfeiffer, T., et al. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0399-z · Cited by 1,521 · see citing works
Self-citationMany Labs 2: Investigating variation in replicability across samples and settings
Klein, R. A., Vianello, M., Hasselman, F., Adams, B. G., Adams, R. B., Alper, S., Aveyard, M., Axt, J. R., Babalola, M. T., Bahník, Š., et al. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918810225 · Cited by 944 · see citing works
Self-citationReplicability, Robustness, and Reproducibility in Psychological Science
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., et al. (2022) · Annual Review of Psychology · doi:10.1146/annurev-psych-020821-114157 · Cited by 770 · see citing works
Self-citationViolating the normality assumption may be the lesser of two evils
Knief, U., & Forstmeier, W. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01587-5 · Cited by 654 · see citing works
“In a growing body of literature, researchers express their concerns about irreproducible results (Open Science Collaboration 2015; Ebersole et al. 2016; Camerer et al. 2018; Silberzahn et al. 2018) and it has been argued that the inappropriate use of statistics is a leading cause of irreproducible…”
BackgroundNelson, L. D., Simmons, J., & Simonsohn, U. (2018) · Annual Review of Psychology · doi:10.1146/annurev-psych-122216-011836 · Cited by 548 · see citing works
Investigating the replicability of preclinical cancer biology
Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021) · eLife · doi:10.7554/elife.71601 · Cited by 393 · see citing works
“Large- scale replication studies in the social and behavioral sciences provide evidence of replicability challenges (Camerer et al., 2016; Camerer et al., 2018; Ebersole et al., 2016; Ebersole et al., 2020; Klein et al., 2014; Klein et al., 2018; Open Science Collaboration, 2015).”
BackgroundSelf-citationThe Very Efficient Assessment of Need for Cognition: Developing a Six-Item Version
Lins de Holanda Coelho, G., H. P. Hanel, P., & J. Wolf, L. (2020) · Assessment · doi:10.1177/1073191118793208 · Cited by 358 · see citing works
Are Social Media Ruining Our Lives? A Review of Meta-Analytic Evidence
Appel, M., Marker, C., & Gnambs, T. (2020) · Review of General Psychology · doi:10.1177/1089268019880891 · Cited by 290 · see citing works
Soderberg, C. K., Errington, T. M., Schiavone, S. R., Bottesini, J., Thorn, F. S., Vazire, S., Esterling, K. M., & Nosek, B. A. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01142-4 · Cited by 289 · see citing works
Self-citationNonreplicable publications are cited more than replicable ones
Serra-Garcia, M., & Gneezy, U. (2021) · Science Advances · doi:10.1126/sciadv.abd1705 · Cited by 177 · see citing works
Crowdsourcing hypothesis tests: Making transparent how design choices shape research results.
Landy, J. F., Jia, M. (., Ding, I. L., Viganola, D., Tierney, W., Dreber, A., Johannesson, M., Pfeiffer, T., Ebersole, C. R., Gronau, Q. F., et al. (2020) · Psychological Bulletin · doi:10.1037/bul0000220 · Cited by 168 · see citing works
Self-citationThe Replication Crisis Has Led to Positive Structural, Procedural, and Community Changes
Korbmacher, M., Azevedo, F., Pennington, C. R., Hartmann, H., Pownall, M., Schmidt, K., Elsherif, M., Breznau, N., Robertson, O., Kalandadze, T., et al. (2023) · Communications Psychology · doi:10.1038/s44271-023-00003-2 · Cited by 137 · see citing works
99% impossible: A valid, or falsifiable, internal meta-analysis.
Vosgerau, J., Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2019) · Journal of Experimental Psychology: General · doi:10.1037/xge0000663 · Cited by 98 · see citing works
Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability
Ebersole, C. R., Mathur, M. B., Baranski, E., Bart-Plange, D.-J., Buttrick, N. R., Chartier, C. R., Corker, K. S., Corley, M., Hartshorne, J. K., IJzerman, H., et al. (2020) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245920958687 · Cited by 105 · see citing works
Self-citationKlein, R. A., Cook, C. L., Ebersole, C. R., Vitiello, C., Nosek, B. A., Hilgard, J., Ahn, P. H., Brady, A. J., Chartier, C. R., Christopherson, C. D., et al. (2022) · Collabra Psychology · doi:10.1525/collabra.35271 · Cited by 193 · see citing works
Self-citationPollet, T. V., & Saxton, T. K. (2019) · Evolutionary Psychological Science · doi:10.1007/s40806-019-00192-2 · Cited by 89 · see citing works
Evaluating Registered Reports: A Naturalistic Comparative Study of Article Impact
Hopun, L. T., Singleton Thorn, F., Nosek, B. A., & Errington, T. M. (2017) · doi:10.31219/osf.io/5y8w7 · Cited by 29 · see citing works
Self-citationDoes expertise matter in replication? An examination of the reproducibility project: Psychology
Bench, S. W., Rivera, G. N., Schlegel, R. J., Hicks, J. A., & Lench, H. C. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2016.07.003 · Cited by 24 · see citing works
Self-citationDetection of data fabrication using statistical tools
Hartgerink, C. H. J., Voelkel, J. G., Wicherts, J. M., & van Assen, M. A. L. M. (2019) · doi:10.31234/osf.io/jkws4 · Cited by 16 · see citing works
Psychologists update their beliefs about effect sizes after replication studies
McDiarmid, A. D., Tullett, A. M., Whitt, C. M., Vazire, S., Smaldino, P. E., & Stephens, J. E. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01220-7 · Cited by 16 · see citing works
- Authors
- Ebersole, C. R., Atherton, O. E., Belanger, A. L., Skulborstad, H. M., Allen, J. M., Banks, J. B., Baranski, E., Bernstein, M. J., Bonfiglio, D. B. V., Boucher, L., Brown, E. R., Budiman, N. I., Cairo, A. H., Capaldi, C. A., Chartier, C. R., Chung, J. M., Cicero, D. C., Coleman, J. A., Conway, J. G., Davis, W. E., Devos, T., Fletcher, M. M., German, K., Grahe, J. E., Hermann, A. D., Hicks, J. A., Honeycutt, N., Humphrey, B., Janus, M., Johnson, D. J., Joy-Gaba, J. A., Juzeler, H., Keres, A., Kinney, D., Kirshenbaum, J., Klein, R. A., Lucas, R. E., Lustgraaf, C. J. N., Martin, D., Menon, M., Metzger, M., Moloney, J. M., Morse, P. J., Prislin, R., Razza, T., Re, D. E., Rule, N. O., Sacco, D. F., Sauerberger, K., Shrider, E., Shultz, M., Siemsen, C., Sobocko, K., Weylin Sternglanz, R., Summerville, A., Tskhay, K. O., van Allen, Z., Vaughn, L. A., Walker, R. J., Weinberg, A., Wilson, J. P., Wirth, J. H., Wortman, J., & Nosek, B. A.
- Affiliations
- University of Virginia (US)
- University of California, Davis (US)
- Miami University (US)
- Montana State University (US)
- Nova Southeastern University (US)
- University of California, Riverside (US)
- Pennsylvania State University (US)
- Ashland University (US)
- University of North Florida (US)
- Virginia Commonwealth University (US)
- Carleton University (CA)
- University of Hawaiʻi at Mānoa (US)
- University of Florida (US)
- Texas A&M University (US)
- Corresponding author
- Ebersole, Charles R.
- ORCID
- Ebersole, Charles R.
- Atherton, Olivia E.
- Allen, Jill M.
- Banks, Jonathan B.
- Baranski, Erica
- Bernstein, Michael J.
- Boucher, Leanne
- Cairo, Athena H.
- Capaldi, Colin A.
- Chartier, Christopher R.
- Chung, Joanne M.
- Cicero, David C.
- Coleman, Jennifer A.
- Devos, Thierry
- Grahe, Jon E.
- Hicks, Joshua A.
- Honeycutt, Nathan
- Johnson, David J.
- Joy-Gaba, Jennifer A.
- Klein, Richard A.
- Lucas, Richard E.
- Lustgraaf, Christopher J.N.
- Martin, Daniel
- Moloney, Jaclyn M.
- Morse, Patrick J.
- Re, Daniel E.
- Rule, Nicholas O.
- Sacco, Donald F.
- Sobocko, Karin
- Summerville, Amy
- Vaughn, Leigh Ann
- Weinberg, Ashley
- Wilson, John Paul
- Wirth, James H.
- Nosek, Brian A.
- Venue
- Journal of Experimental Social Psychology
- Publisher
- Elsevier BV
- Published
- 2016-11
- Volume
- 67
- Pages
- 68-82
- ISSN
- 0022-1031 (ISSN-L)
- OpenAlex
- W2520456228
- Semantic Scholar
- dfad0bb590673bd0e06aee182baa2d091052e2ab
- Cited by
- 468
- References
- 54
- Citations by year
- 2026: 19 · 2025: 38 · 2024: 43 · 2023: 51 · 2022: 47 · 2021: 65 · 2020: 55 · 2019: 41 · 2018: 50 · 2017: 31 · 2016: 18 · 2015: 8 · 2014: 1
- FWCI
- 62.25
- Type
- journal article
- Language
- en
- Open access
- green
- escholarship.org
- Licence
- Locations
- Journal of Experimental Social Psychology (publishedVersion)
- Research portal (Tilburg University) (publishedVersion)
- Aquila Digital Community (University of Southern Mississippi) (submittedVersion)
- Digital Collections - Ithaca College Library (Ithaca College) (submittedVersion, public-domain)
- University of North Florida Digital Commons (University of North Florida) (submittedVersion)
- NSUWorks (Nova Southeastern University) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- replication, data quality metrics, individual differences, conscientiousness, mood, stress, online sample
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
