The DOI matches its Crossref record; 48 of the 50 references it cites were found in a registry or an index.
Lack of group-to-individual generalizability is a threat to human subjects research
Open accessAbstract
Significance: The current study quantified the degree to which group data are able to describe individual participants. We utilized intensive repeated-measures data—data that have been collected many times, across many individuals—to compare the distributions of bivariate correlations calculated within subjects vs. those calculated between subjects. Because the vast majority of social and medical science research aggregates across subjects, we aimed to assess how closely such aggregations reflect their constituent individuals. We provide evidence that conclusions drawn from aggregated data may be worryingly imprecise. Specifically, the variance in individuals is up to four times larger than in groups. These data call for a focus on idiography and open science that may substantially alter best-practice guidelines in the medical and behavioral sciences.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Given that between-person differences do not necessarily generalize to within-person changes (Fisher et al., 2018), it is important to confirm previous between-person results regarding the structure of aversive reactivity and its associations with symptom change and extend these results to withinperson processes.”
From Within-person changes in aversive reactivity predict session-to-session reductions in anxiety and depression in the unified protocol. (2024) · cites this paper as Fisher et al., 2018
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“In doing this, SCEDs provide evidence of the effectiveness of interventions, drawing on substantially fewer participants than nomothetic approaches (such as RCTs) [51,52,53,54,55,56,57,58].”
From Brief and Telehealth Acceptance and Commitment Therapy (ACT) Interventions for Stress in Inflammatory Bowel Disease (IBD): A Series of Single Case Experimental Design (SCED) Studies (2022) · cites this paper as [52]
Experimental — accuracy not yet measuredIn the citing paper
“in clinical trials, individual outcomes within tested groups are aggregated but outcomes between individuals are rarely the same (e.g., the variance in individuals can be up to four times greater than in groups [12,13]).”
From Hasty generalizations and generics in medical research: A systematic review (2024) · cites this paper as [13]
In the citing paper
“However, it has been recognised that the inherent clinical heterogeneity associated with mental disorders may necessitate a greater focus on inter-individual differences to ensure generalisability to unseen cases (39, 43, 44).”
From Personalised depression forecasting using mobile sensor data and ecological momentary assessment (2022) · cites this paper as [44]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Diagnostic and Statistical Manual of Mental Disorders
American Psychiatric Association (2013) · doi:10.1176/appi.books.9780890425596 · Cited by 116,369
Development and validation of brief measures of positive and negative affect: The PANAS scales.
Watson, D., Clark, L. A., & Tellegen, A. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.54.6.1063 · Cited by 32,433
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
Ecological Correlations and the Behavior of Individuals
Robinson, W. S. (1950) · American Sociological Review · doi:10.2307/2087176 · Cited by 4,547
(2001) · doi:10.1017/cbo9780511546396 · Cited by 4,482
TESTING FOR SERIAL CORRELATION IN LEAST SQUARES REGRESSION. II
Durbin, J., & Watson, G. S. (1951) · Biometrika · doi:10.1093/biomet/38.1-2.159 · Cited by 3,636
Analysis of serial measurements in medical research.
Matthews, J. N. S., Altman, D. G., Campbell, M. J., & Royston, P. J. (1990) · BMJ · doi:10.1136/bmj.300.6719.230 · Cited by 3,137
Variance component model to account for sample structure in genome-wide association studies
Kang, H. M., Sul, J. H., Service, S. K., Zaitlen, N. A., Kong, S.-Y., Freimer, N. B., Sabatti, C., & Eskin, E. (2010) · Nature Genetics · doi:10.1038/ng.548 · Cited by 2,817
Taking Time Seriously: Time-Series-Cross-Section Analysis with a Binary Dependent Variable
Beck, N. L., Katz, J. N., & Tucker, R. P. (1998) · American Journal of Political Science · doi:10.2307/2991857 · Cited by 2,443
The interpretation of interaction in contingency tables
Simpson, E. H. (1951) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1951.tb00088.x · Cited by 2,301 · cited 1×
“Simpson’s paradox (12) is a statistical effect in which trends in subgroups differ from (or are even inverse to) the aggregate trend when the groups are combined.”
BackgroundHandbook of research methods for studying daily life
Mehl, M. R., Conner, T. S., & Csíkszentmihályi, M. (2012) · Choice Reviews Online · doi:10.5860/choice.50-1159 · Cited by 2,121
Molenaar, P. C. M. (2004) · Measurement: Interdisciplinary Research & Perspective · doi:10.1207/s15366359mea0204_1 · Cited by 1,943
TESTING FOR SERIAL CORRELATION IN LEAST SQUARES REGRESSION. I
Durbin, J., & Watson, G. S. (1950) · Biometrika · doi:10.1093/biomet/37.3-4.409 · Cited by 1,775
Rethinking the mood and anxiety disorders: A quantitative hierarchical model for DSM-V.
Watson, D. (2005) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.114.4.522 · Cited by 1,332 · cited 1×
“Similarly, the strong, positive covariation between depressed mood and worry has been so consistently replicated that some have argued that their respective clinical diagnoses—GAD and MDD—should belong in the same class of mental disorders (38).”
BackgroundIs psychology suffering from a replication crisis? What does “failure to replicate” really mean?
Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015) · American Psychologist · doi:10.1037/a0039400 · Cited by 1,102
The new person-specific paradigm in psychology
Molenaar, P. C. M., & Campbell, C. G. (2009) · Current Directions in Psychological Science · doi:10.1111/j.1467-8721.2009.01619.x · Cited by 1,088
Testing for Serial Correlation in Least Squares Regression: I
Durbin, J., & Watson, G. S. (1950) · Biometrika · doi:10.2307/2332391 · Cited by 956
Snijders, T. A. B. (2011) · International Encyclopedia of Statistical Science · doi:10.1007/978-3-642-04898-2_387 · Cited by 552
Simpson's paradox in psychological science: a practical guide
Kievit, R. A., Frankenhuis, W. E., Waldorp, L. J., & Borsboom, D. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00513 · Cited by 542
Kessler, R. C., Gruber, M., Hettema, J. M., Hwang, I., Sampson, N., & Yonkers, K. A. (2008) · Psychological Medicine · doi:10.1017/s0033291707002012 · Cited by 493
Geschwind, N., Peeters, F., Drukker, M., van Os, J., & Wichers, M. (2011) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0024595 · Cited by 433
Toward an idiothetic psychology of personality.
Lamiell, J. T. (1981) · American Psychologist · doi:10.1037/0003-066x.36.3.276 · Cited by 406
Exploring the idiographic dynamics of mood and anxiety via network analysis
Fisher, A. J., Reeves, J. W., Lawyer, G., Medaglia, J. D., & Rubel, J. A. (2017) · Journal of Abnormal Psychology · doi:10.1037/abn0000311 · Cited by 385
Self-citationMultilevel Models: Methods and Substance
DiPrete, T. A., & Forristal, J. D. (1994) · Annual Review of Sociology · doi:10.1146/annurev.so.20.080194.001555 · Cited by 361
Bridging the Nomothetic and Idiographic Approaches to the Analysis of Clinical Data
Beltz, A. M., Wright, A. G. C., Sprague, B. N., & Molenaar, P. C. M. (2016) · Assessment · doi:10.1177/1073191116648209 · Cited by 293
Variance Component Models with Binary Response: Interviewer Variability
Anderson, D. A., & Aitkin, M. A. (1985) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1985.tb01346.x · Cited by 279
Between the General and the Unique
Salvatore, S., & Valsiner, J. (2010) · Theory & Psychology · doi:10.1177/0959354310381156 · Cited by 277
Toward a dynamic model of psychological assessment: Implications for personalized care.
Fisher, A. J. (2015) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000026 · Cited by 271
Self-citationMolenaar, P. C. M. (2008) · Developmental Psychobiology · doi:10.1002/dev.20262 · Cited by 237
Kim, J., Zhu, W., Chang, L., Bentler, P. M., & Ernst, T. (2007) · Human Brain Mapping · doi:10.1002/hbm.20259 · Cited by 236
Hamaker, E. L., Dolan, C. V., & Molenaar, P. C. M. (2005) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr4002_3 · Cited by 219
Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions
Harris, R. F. (2017) · Cited by 180
Quine, W. V. O. (1969) · Springer eBooks · doi:10.1007/978-94-017-1466-2_2 · Cited by 174
Stability and fluctuation of personality disorder features in daily life.
Wright, A. G. C., & Simms, L. J. (2016) · Journal of Abnormal Psychology · doi:10.1037/abn0000169 · Cited by 145
HowNutsAreTheDutch (HoeGekIsNL): A crowdsourcing study of mental symptoms and strengths
van der Krieke, L., Jeronimus, B. F., Blaauw, F. J., Wanders, R. B. K., Emerencia, A. C., Schenk, H., de Vos, S., Snippe, E., Wichers, M., Wigman, J. T. W., et al. (2015) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1495 · Cited by 119
Self-citationPanic Disorder, Phobias, and Generalized Anxiety Disorder
Craske, M. G., & Waters, A. M. (2005) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.1.102803.143857 · Cited by 106
Cross-study validation for the assessment of prediction algorithms
Bernau, C., Riester, M., Boulesteix, A., Parmigiani, G., Huttenhower, C., Waldron, L., & Trippa, L. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu279 · Cited by 94
Snippe, E., Viechtbauer, W., Geschwind, N., Klippel, A., de Jonge, P., & Wichers, M. (2017) · Scientific Reports · doi:10.1038/srep46523 · Cited by 91
Effects of Antidepressant Treatment on the Quality of Daily Life
Barge‐Schaapveld, D. Q. C. M., & Nicolson, N. A. (2002) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v63n0603 · Cited by 87
Daily Interpersonal and Affective Dynamics in Personality Disorder
Wright, A. G. C., Hopwood, C. J., & Simms, L. J. (2015) · Journal of Personality Disorders · doi:10.1521/pedi.2015.29.4.503 · Cited by 80
Castro‐Schilo, L., & Ferrer, E. E. (2013) · Multivariate Behavioral Research · doi:10.1080/00273171.2012.736042 · Cited by 42
The challenge of non-ergodicity in network neuroscience
Medaglia, J. D., Ramanathan, D. M., Venkatesan, U. M., & Hillary, F. G. (2011) · Network Computation in Neural Systems · doi:10.3109/09638237.2011.639604 · Cited by 35
Self-citationConsilience Among the Great Branches of Learning 1
Wilson, E. O. (2018) · Routledge eBooks · doi:10.4324/9781351306928-5 · Cited by 32
Making massive computational experiments painless
Monajemi, H., Donoho, D. L., & Stodden, V. (2016) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2016.7840870 · Cited by 31
Fisher, A. J., & Woodward, S. H. (2013) · Psychophysiology · doi:10.1111/psyp.12148 · Cited by 27
Self-citationCARRIER, M. J. (2012) · PUB – Publications at Bielefeld University (Bielefeld University) · Cited by 21
Bioinformatic Analysis of Epidemiological and Pathological Data
Tyekucheva, S., & Parmigiani, G. (2016) · Springer eBooks · doi:10.1007/978-3-319-35153-7_8 · Cited by 2
Hama, A. T. (2017) · Mankind Quarterly · doi:10.46469/mq.2017.58.2.13 · Cited by 1
Research works printed but not linked: 3
- L Boltzmann, On the relation between the second law of the mechanical theory of heat and the probability calculus with respect to the theorems on thermal equilibrium. Kais Akad Wiss Wien Math Natumiss Classe 76, 373–435 (1877).
- PCM Molenaar, Rejoinder to Rogosa’s commentary on “A manifesto on psychology as idiographic science.”. Measurement 3, 116–119 (2005).
- WW Wei, Time series analysis. The Oxford Handbook of Quantitative Methods in Psychology (Oxford Univ Press, Oxford) Vol 2 (2006).
Cited by
Sorted by citations
Artificial Intelligence and Personalized Medicine
Schork, N. J. (2019) · Cancer treatment and research · doi:10.1007/978-3-030-16391-4_11 · Cited by 531
Network Analysis of Psychopathology: Controversies and Challenges
McNally, R. J. (2020) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-081219-092850 · Cited by 504
Psychometric Network Models from Time-Series and Panel Data
Epskamp, S. (2020) · Psychometrika · doi:10.1007/s11336-020-09697-3 · Cited by 365
Lack of theory building and testing impedes progress in the factor and network literature
Fried, E. I. (2020) · Psychological Inquiry · doi:10.1080/1047840x.2020.1853461 · Cited by 317
Nelson, B. W., & Allen, N. B. (2019) · JMIR mHealth and uHealth · doi:10.2196/10828 · Cited by 310
Metadata
- Authors
- Fisher, A. J., Medaglia, J. D., & Jeronimus, B. F.
- Affiliations
- University of California, Berkeley (US)
- Drexel University (US)
- University of Groningen (NL)
- Corresponding author
- Fisher, Aaron J.
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2018-06-18
- Volume
- 115
- Issue
- 27
- Pages
- E6106-E6115
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC6142277
- OpenAlex
- W2808831456
- Cited by
- 1,099
- References
- 50
- Citations by year
- 2026: 126 · 2025: 159 · 2024: 132 · 2023: 141 · 2022: 139 · 2021: 137 · 2020: 135 · 2019: 106 · 2018: 21
- FWCI
- 82.78
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, other-oa)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, other-oa)
- Funders
- HHS | NIH | National Institute of Mental Health (1-DP5- OD021352)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Keywords
- within-person variability, interindividual variability, aggregation bias, individual differences
- MeSH terms
- Anxiety Disorders, Bipolar Disorder, Female, Humans, Male, Psychotherapy, Group
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
