The DOI matches its Crossref record; 86 of the 86 references it cites were found in a registry or an index.
Navigating the garden of forking paths for data exclusions in fear conditioning research
Open accessPublished version of a preprint
Abstract
In this report, we illustrate the considerable impact of researcher degrees of freedom with respect to exclusion of participants in paradigms with a learning element. We illustrate this empirically through case examples from human fear conditioning research, in which the exclusion of 'non-learners' and 'non-responders' is common - despite a lack of consensus on how to define these groups. We illustrate the substantial heterogeneity in exclusion criteria identified in a systematic literature search and highlight the potential problems and pitfalls of different definitions through case examples based on re-analyses of existing data sets. On the basis of these studies, we propose a consensus on evidence-based rather than idiosyncratic criteria, including clear guidelines on reporting details. Taken together, we illustrate how flexibility in data collection and analysis can be avoided, which will benefit the robustness and replicability of research findings and can be expected to be applicable to other fields of research that involve a learning element.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“While there is substantial heterogeneity in the exclusion criteria used for SCR nonresponse and nonlearning in the field (74), for the current meta-analysis, SCR exclusion criteria were relatively similar across studies.”
From I Doubt It Is Safe: A Meta-analysis of Self-reported Intolerance of Uncertainty and Threat Extinction Training (2021) · cites this paper as [74]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
(1970) · Cited by 24,983
Wang, G., Ma, R., Qiao, G., Wada, K., Aizawa, Y., & Satoh, T. (2015) · Integrative Medicine International · doi:10.1159/000368408 · Cited by 18,762
ggplot2: Elegant Graphics for Data Analysis
Wickham, H. (2009) · doi:10.1007/978-0-387-98141-3 · Cited by 14,447
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
Effects of an expanding-spaced vs massed exposure schedule on fear reduction and return of fear
Rowe, M. K., & Craske, M. G. (1998) · Behaviour Research and Therapy · doi:10.1016/s0005-7967(97)10016-x · Cited by 1,782
Flagged first, then by citations
Research works printed but not linked: 2
- Pavlov, 1927, Conditioned Reflexes
- Sjouwerman, 2018, Individual differences in fear learning: specificity to trait-anxiety beyond other measures of negative affect and mediation via amygdala activation, 10.1101/233528 · DOI
Cited by
Sorted by citations
Items 101–164 of 164
Shner‐Livne, G., Barak, N., Shitrit, I., Abend, R., & Shechner, T. (2025) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-025-02745-2 · Cited by 4
Fear learning sculpts functional brain connectivity at rest beyond the traditional fear network
Fraenz, C., Metzen, D., Merz, C. J., Selpien, H., Friedrich, P., Ocklenburg, S., Axmacher, N., & Genç, E. (2025) · Behavioural Brain Research · doi:10.1016/j.bbr.2025.115764 · Cited by 6
Berg, M., Suchotzki, K., Zimmermann, J., Merz, C. J., Szota, K., Brandt, H., Hartwigsen, G., Lincoln, T. M., Mokros, A., Gade, M., et al. (2025) · European Psychologist · doi:10.1027/1016-9040/a000555 · Cited by 4
Variability in threat processing is related to endogenous butyrate levels in healthy men
Dalile, B., Oudenhove, L. V., Verbeke, K., & Vervliet, B. (2025) · Biological Psychology · doi:10.1016/j.biopsycho.2025.109097 · Cited by 3
Di, T., Liu, J., Liu, W., Guo, Y., Zheng, E., Yu, Z., Shi, L., Jia, T., Lü, L., Sun, Y., et al. (2025) · BMC Medicine · doi:10.1186/s12916-025-04411-5 · Cited by 3
Increased generalisation in trait anxiety is driven by aversive value transfer
Verra, L., Spitzer, B., Schuck, N. W., & Zika, O. (2026) · Communications Psychology · doi:10.1038/s44271-026-00415-w · Cited by 3
Utilizing rewards to dampen fear and its recovery
Brown, E. C., Artigas, S. O., Elsner, S., Liu, L., & Park, S. Q. (2025) · Scientific Reports · doi:10.1038/s41598-025-99758-3 · Cited by 3
Skin conductance levels and responses in Asian and White participants during fear conditioning✰
Gold, A. K., Kredlow, M. A., Orr, S. P., Hartley, C. A., & Otto, M. W. (2022) · Physiology & Behavior · doi:10.1016/j.physbeh.2022.113802 · Cited by 3
Pavlovian Conditioning and Extinction Methods for Studying the Neurobiology of Fear Learning in PTSD
Miller, D. B., Rassaby, M. M., Wen, Z., & Milad, M. R. (2023) · Neuromethods · doi:10.1007/978-1-0716-3218-5_5 · Cited by 3
Generalized expectancy of threat in threatening compared to safe contexts
Aslanidou, A., Andreatta, M., Wong, A. H. K., & Wieser, M. J. (2024) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsae097 · Cited by 3
Affective and computational determinants of threat extinction biases
Stussi, Y. (2025) · Behaviour Research and Therapy · doi:10.1016/j.brat.2025.104804 · Cited by 5
Undeger, I., Visser, R. M., Becker, N., de Boer, L., Golkar, A., & Olsson, A. (2021) · Royal Society Open Science · doi:10.1098/rsos.202116 · Cited by 3
Fraenz, C., Metzen, D., Merz, C. J., Selpien, H., Friedrich, P., Ocklenburg, S., Axmacher, N., & Genç, E. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.05.26.115840 · Cited by 3
Morriss, J., Wake, S. J., elizabeth, C., & van Reekum, C. M. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/nrqv6 · Cited by 7
Velasco, E. R., Nabás, J. F., Torrents‐Rodas, D., Arias, B., Torrúbia, R., Fullana, M. A., & Andero, R. (2024) · Biological Psychology · doi:10.1016/j.biopsycho.2024.108981 · Cited by 2
Eales, K., Lipp, O. V., Anderson, J., Jarvis, M. A., & Ney, L. J. (2025) · Behaviour Research and Therapy · doi:10.1016/j.brat.2025.104828 · Cited by 4
Fornix subdivisions and spatial learning: a diffusion MRI study
Hodgetts, C. J., Postans, M., Williams, A. N., Graham, K. S., & Lawrence, A. D. (2025) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2025.109350 · Cited by 2
Rule-based generalization of threat without similarity
Marstaller, L., Al-Jiboury, R., Kemp, A. H., & Dymond, S. (2021) · Biological Psychology · doi:10.1016/j.biopsycho.2021.108042 · Cited by 2
Lipp, O. V., Luck, C. C., & Waters, A. M. (2021) · Psychophysiology · doi:10.1111/psyp.13982 · Cited by 3
Klingelhöfer-Jens, M., Reiser, F., Brandt, N. D., Schümann, D., Gamer, M., Kalisch, R., Sommer, T., & Lonsdorf, T. B. (2025) · Journal of Personality Assessment · doi:10.1080/00223891.2025.2513936 · Cited by 3
Self-compassion writing facilitates fear extinction
Mei, Y., Juntong, L. I. U., Honghong, L. I. U., Yang, F., Luo, X., & Lei, Y. (2023) · Acta Psychologica Sinica · doi:10.3724/sp.j.1041.2023.01317 · Cited by 2
Verbal manipulations of learning expectancy do not enhance reconsolidation
Marinos, J., Simioni, O., & Ashbaugh, A. R. (2022) · PLoS ONE · doi:10.1371/journal.pone.0241943 · Cited by 2
Koppold, A., Kastrinogiannis, A., Kuhn, M., & Lonsdorf, T. B. (2022) · doi:10.31234/osf.io/wjeyg · Cited by 2
The role of consciousness in threat extinction learning
Lam, C. L. M., Barry, T. J., Yiend, J., & Lee, T. M. C. (2023) · Consciousness and Cognition · doi:10.1016/j.concog.2023.103599 · Cited by 1
Genetic and Environmental Influences on Fear Learning and Generalization
Sheerin, C. M., Moore, A. A., Sawyers, C. K., Kirkpatrick, R. M., Hettema, J. M., & Roberson‐Nay, R. (2025) · Psychophysiology · doi:10.1111/psyp.70052 · Cited by 1
No support for an effect of a daytime nap on the consolidation or generalization of fear learning
Davidson, P., Garcia, S. M., Denis, D., Kensinger, E. A., & Pace-Schott, E. F. (2025) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2025.109241 · Cited by 1
Viselli, L., Salfi, F., Naccarato, F., Arnone, B., Corigliano, D., Amicucci, G., Festucci, F., Colombo, C., Cellini, N., Tempesta, D., et al. (2025) · SLEEP · doi:10.1093/sleep/zsaf409 · Cited by 1
Matsubara, Y., Tamura, H., Minami, T., & Nakauchi, S. (2026) · International Journal of Social Robotics · doi:10.1007/s12369-025-01341-3 · Cited by 1
Zhang, D., Zou, Z., Zhang, B. Y., Feng, B., Fei, Z., Zhang, M., Liu, Y., & Zheng, X. (2026) · Behaviour Research and Therapy · doi:10.1016/j.brat.2026.105019 · Cited by 1
Andres, E., Chuan-Peng, H., Gerlicher, A. M. V., Tüscher, O., & Kalisch, R. (2026) · Molecular Psychiatry · doi:10.1038/s41380-026-03565-9 · Cited by 2
Zuccolo, P. F., & Hunziker, M. H. L. (2021) · Psychophysiology · doi:10.1111/psyp.13971 · Cited by 1
Klingelhöfer-Jens, M., Hutterer, K., Schiele, M. A., Leehr, E. J., Schümann, D., Rosenkranz, K., Böhnlein, J., Repple, J., Deckert, J., Domschke, K., et al. (2025) · eLife · doi:10.1101/2023.09.26.559474 · Cited by 3
A Roadmap for Future Interactions Between Research on Personality and Learning
Houwer, J. D., Perugini, M., Boddez, Y., & Sava, F. A. (2023) · Collabra Psychology · doi:10.1525/collabra.88334 · Cited by 1
Yin, X., Germanos, O., & Dietis, N. (2025) · Behavioural Processes · doi:10.1016/j.beproc.2025.105176 · Cited by 1
Development of a novel startle response task in Duchenne muscular dystrophy
Maresh, K. E., Papageorgiou, A., Ridout, D. A., Harrison, N. A., Mandy, W., Skuse, D., & Francesco Muntoni, A. (2022) · PLoS ONE · doi:10.1371/journal.pone.0264091 · Cited by 1
Conditioned fear reactions are associated with gray matter density but not cortical microstructurePreprint
Fraenz, C., Metzen, D., Merz, C. J., Selpien, H., Axmacher, N., & Genç, E. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.12.18.423236 · Cited by 1
Enhancing Precision in Human NeurosciencePreprint
Nebe, S., Reutter, M., Baker, D. H., Bölte, J., Domes, G., Gamer, M., Gärtner, A., Gießing, C., Mann, C., Hilger, K., et al. (2023) · doi:10.31234/osf.io/m8c4k · Cited by 1
Klingelhöfer-Jens, M., Hutterer, K., Schiele, M. A., Leehr, E. J., Schümann, D., Rosenkranz, K., Böhnlein, J., Repple, J., Deckert, J., Domschke, K., et al. (2024) · eLife · doi:10.7554/elife.91425.2 · Cited by 1
Jambazova, A. A., van Rijsbergen, N. J., & Scheepers, C. (2020) · doi:10.31234/osf.io/p6z5u
Meanings of Quality: Semantic-Information Distance and Total Quality Management
Steier, F., Zhao, B., Woodhams, W., & Gallup, S. P. (1993)
Lonsdorf, T. B., & Sjouwerman, R. (2021) · doi:10.31234/osf.io/65njk · Cited by 1
Rule-based generalization of threat without similarityPreprint
Marstaller, L., Al-Jiboury, R., Kemp, A. H., & Dymond, S. (2020) · doi:10.31219/osf.io/kfv3j
Meier, M., Lonsdorf, T. B., LUPIEN, S. J., Tobias Stalder, A., Laufer, S., Sicorello, M., Linz, R., & Puhlmann, L. M. C. (2022) · doi:10.31234/osf.io/d2zvc
Jiang, X. (2021) · Louisiana State University · doi:10.31390/gradschool_dissertations.5532
Preregistration of Studies with Existing Data
Mertens, G., & Krypotos, A.-M. (2022) · Integrity of Scientific Research · doi:10.1007/978-3-030-99680-2_36
Zika, O., Appel, J. E., Klinge, C., Shkreli, L., Browning, M. D., Wiech, K., & Reinecke, A. (2023) · doi:10.31234/osf.io/e5wdv
Klingelhöfer-Jens, M., Hutterer, K., Schiele, M. A., et al. (2023) · eLife · doi:10.7554/elife.91425.1.sa0
Klingelhöfer-Jens, M., Hutterer, K., Schiele, M. A., Leehr, E. J., Schümann, D., Rosenkranz, K., Böhnlein, J., Repple, J., Deckert, J., Domschke, K., et al. (2023) · eLife · doi:10.7554/elife.91425.1
Disentangling safety: A summation test between absence and termination of threatPreprint
Andreatta, M. (2024) · doi:10.22541/au.171315629.91304686/v1
Puberty, sex, and fear extinction retention: A neuroimaging study in youth
Bhargava, S., Zundel, C. G., Shampine, M. M., Ely, S. L., Carpenter, C., Losiowski, J., Chanamolu, S., Jande, J., Tamimi, R., Sharma, K., et al. (2025) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2025.101595 · Cited by 1
Matsubara, Y., Tamura, H., Minami, T., & Nakauchi, S. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2025.12.23.696107
Imagery-based fear conditioning enhances the late positive potential
Panitz, C., & Mueller, E. M. (2026) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-025-01385-y · Cited by 2
How Temporal Predictability of Threat and Action Preparation Affect Defensive Freezing Responses
Koppold, A., Ehlers, M. R., Kastrinogiannis, A., Klaassen, F. H., Roelofs, K., & Lonsdorf, T. B. (2026) · Psychophysiology · doi:10.1111/psyp.70278 · Cited by 2
[These authors contributed equally: Nicolas W. Schuck, Ondrej Zika.]
Verra, L., Spitzer, B., Schuck, N. W., & Zika, O. (2026) · MPG.PuRe (Max Planck Society)
Feng, B., Ai, X., Huang, F., Chen, W., & Zheng, X. (2026) · Psychophysiology · doi:10.1111/psyp.70344
Chhabra, H., Ma, Y., Genç, E., Nitsche, M. A., & Yavari, F. (2026) · Translational Psychiatry · doi:10.1038/s41398-026-04190-4
Li, Y., Feng, W., & Zhou, X. (2026) · Language Cognition and Neuroscience · doi:10.1080/23273798.2026.2705939
Unterschemmann, S., Panitz, C., Bierwirth, P., Petereit, P., Comteße, H., Sperl, M. F. J., Hermann, C., & Mueller, E. M. (2026) · Behaviour Research and Therapy · doi:10.1016/j.brat.2026.105127
Vandormael, M., Goossens, L., Cort, K. D., Brouwer, J., van der Velde, T., de Vos, J. H., Schuhmann, T., & Schruers, K. R. J. (2026) · Behaviour Research and Therapy · doi:10.1016/j.brat.2026.105136
Davidson, P., Daffre, C., Oliver, K. I., Seo, J., Moore, K. N., Gazecki, S., Lasko, N. B., Orr, S. P., Milad, M. R., & Pace-Schott, E. F. (2026) · Behaviour Research and Therapy · doi:10.1016/j.brat.2026.105118
Ventura‐Bort, C., Klingelhöfer-Jens, M., Mertens, G., Wong, A. H. K., Xia, Y., Andres, E., Kuhn, M., Bouwman, V., Engelhard, I. M., Lonsdorf, T. B., et al. (2026) · Research Square · doi:10.21203/rs.3.rs-10905008/v1
Whalen, M., Beech, A., Paterson, M., Riser, M., & Kredlow, M. A. (2026) · Behaviour Research and Therapy · doi:10.1016/j.brat.2026.105170
Distinct Visual Pathways of Threat Retrieval in Fear‐Conditioned Faces
Weidner, E. M., Götze, M., Taday, A., & Kissler, J. M. (2026) · Psychophysiology · doi:10.1111/psyp.70429
Metadata
- Authors
- Lonsdorf, T. B., Klingelhöfer-Jens, M., Andreatta, M., Beckers, T., Chalkia, A., Gerlicher, A., Jentsch, V. L., Meir Drexler, S., Mertens, G., Richter, J., Sjouwerman, R., Wendt, J., & Merz, C. J.
- Affiliations
- University Medical Center Hamburg-Eppendorf (DE)
- University of Würzburg (DE)
- KU Leuven (BE)
- University of Amsterdam (NL)
- Ruhr University Bochum (DE)
- Institute of Cognitive Neuroscience, Department of Cognitive Psychology, Ruhr University Bochum, Bochum, Germany
- Department of Psychology, Utrecht University, Utrecht, Netherlands
- Universität Greifswald (DE)
- University of Potsdam (DE)
- ORCID
- Venue
- eLife
- Publisher
- eLife Sciences Publications, Ltd
- Published
- 2019-12-16
- Volume
- 8
- ISSN
- 2050-084X (ISSN-L)
- PMC
- PMC6989118
- OpenAlex
- W2995231114
- Cited by
- 164
- References
- 85
- Citations by year
- 2026: 16 · 2025: 23 · 2024: 15 · 2023: 27 · 2022: 26 · 2021: 33 · 2020: 21 · 2019: 1
- FWCI
- 2.56
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- eLife (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UvA-DARE (University of Amsterdam) (submittedVersion, cc-by)
- Lirias (submittedVersion, other-oa)
- RePub (Erasmus University Rotterdam) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- publish.UP (University of Potsdam) (submittedVersion)
- PUB – Publications at Bielefeld University (Bielefeld University) (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Funders
- Deutsche Forschungsgemeinschaft (LO 1980/2-1, LO 1980/1-1, Project B07 44541416, 316803389 - SFB1280, WE 5873/1-1, WE 5873/5-1)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- researcher degrees of freedom, data exclusion, fear conditioning, non-responders, exclusion criteria, learning paradigms, reporting guidelines, replicability, robustness of results
- MeSH terms
- Adult, Conditioning, Psychological, Fear, Female, Humans, Male
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review