The DOI matches its Crossref record; 48 of the 50 references it cites were found in a registry or an index.
Lack of group-to-individual generalizability is a threat to human subjects research
Open accessAbstract
Significance: The current study quantified the degree to which group data are able to describe individual participants. We utilized intensive repeated-measures data—data that have been collected many times, across many individuals—to compare the distributions of bivariate correlations calculated within subjects vs. those calculated between subjects. Because the vast majority of social and medical science research aggregates across subjects, we aimed to assess how closely such aggregations reflect their constituent individuals. We provide evidence that conclusions drawn from aggregated data may be worryingly imprecise. Specifically, the variance in individuals is up to four times larger than in groups. These data call for a focus on idiography and open science that may substantially alter best-practice guidelines in the medical and behavioral sciences.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Given that between-person differences do not necessarily generalize to within-person changes (Fisher et al., 2018), it is important to confirm previous between-person results regarding the structure of aversive reactivity and its associations with symptom change and extend these results to withinperson processes.”
From Within-person changes in aversive reactivity predict session-to-session reductions in anxiety and depression in the unified protocol. (2024) · cites this paper as Fisher et al., 2018
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“In doing this, SCEDs provide evidence of the effectiveness of interventions, drawing on substantially fewer participants than nomothetic approaches (such as RCTs) [51,52,53,54,55,56,57,58].”
From Brief and Telehealth Acceptance and Commitment Therapy (ACT) Interventions for Stress in Inflammatory Bowel Disease (IBD): A Series of Single Case Experimental Design (SCED) Studies (2022) · cites this paper as [52]
Experimental — accuracy not yet measuredIn the citing paper
“in clinical trials, individual outcomes within tested groups are aggregated but outcomes between individuals are rarely the same (e.g., the variance in individuals can be up to four times greater than in groups [12,13]).”
From Hasty generalizations and generics in medical research: A systematic review (2024) · cites this paper as [13]
In the citing paper
“However, it has been recognised that the inherent clinical heterogeneity associated with mental disorders may necessitate a greater focus on inter-individual differences to ensure generalisability to unseen cases (39, 43, 44).”
From Personalised depression forecasting using mobile sensor data and ecological momentary assessment (2022) · cites this paper as [44]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Diagnostic and Statistical Manual of Mental Disorders
American Psychiatric Association (2013) · doi:10.1176/appi.books.9780890425596 · Cited by 116,369
Development and validation of brief measures of positive and negative affect: The PANAS scales.
Watson, D., Clark, L. A., & Tellegen, A. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.54.6.1063 · Cited by 32,433
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
Ecological Correlations and the Behavior of Individuals
Robinson, W. S. (1950) · American Sociological Review · doi:10.2307/2087176 · Cited by 4,547
(2001) · doi:10.1017/cbo9780511546396 · Cited by 4,482
TESTING FOR SERIAL CORRELATION IN LEAST SQUARES REGRESSION. II
Durbin, J., & Watson, G. S. (1951) · Biometrika · doi:10.1093/biomet/38.1-2.159 · Cited by 3,636
Analysis of serial measurements in medical research.
Matthews, J. N. S., Altman, D. G., Campbell, M. J., & Royston, P. J. (1990) · BMJ · doi:10.1136/bmj.300.6719.230 · Cited by 3,137
Variance component model to account for sample structure in genome-wide association studies
Kang, H. M., Sul, J. H., Service, S. K., Zaitlen, N. A., Kong, S.-Y., Freimer, N. B., Sabatti, C., & Eskin, E. (2010) · Nature Genetics · doi:10.1038/ng.548 · Cited by 2,817
Taking Time Seriously: Time-Series-Cross-Section Analysis with a Binary Dependent Variable
Beck, N. L., Katz, J. N., & Tucker, R. P. (1998) · American Journal of Political Science · doi:10.2307/2991857 · Cited by 2,443
The interpretation of interaction in contingency tables
Simpson, E. H. (1951) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1951.tb00088.x · Cited by 2,301 · cited 1×
“Simpson’s paradox (12) is a statistical effect in which trends in subgroups differ from (or are even inverse to) the aggregate trend when the groups are combined.”
Background
Research works printed but not linked: 3
- L Boltzmann, On the relation between the second law of the mechanical theory of heat and the probability calculus with respect to the theorems on thermal equilibrium. Kais Akad Wiss Wien Math Natumiss Classe 76, 373–435 (1877).
- PCM Molenaar, Rejoinder to Rogosa’s commentary on “A manifesto on psychology as idiographic science.”. Measurement 3, 116–119 (2005).
- WW Wei, Time series analysis. The Oxford Handbook of Quantitative Methods in Psychology (Oxford Univ Press, Oxford) Vol 2 (2006).
Cited by
Sorted by citations
Artificial Intelligence and Personalized Medicine
Schork, N. J. (2019) · Cancer treatment and research · doi:10.1007/978-3-030-16391-4_11 · Cited by 531
Network Analysis of Psychopathology: Controversies and Challenges
McNally, R. J. (2020) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-081219-092850 · Cited by 504
Psychometric Network Models from Time-Series and Panel Data
Epskamp, S. (2020) · Psychometrika · doi:10.1007/s11336-020-09697-3 · Cited by 365
Lack of theory building and testing impedes progress in the factor and network literature
Fried, E. I. (2020) · Psychological Inquiry · doi:10.1080/1047840x.2020.1853461 · Cited by 317
Nelson, B. W., & Allen, N. B. (2019) · JMIR mHealth and uHealth · doi:10.2196/10828 · Cited by 310
Metadata
- Authors
- Fisher, A. J., Medaglia, J. D., & Jeronimus, B. F.
- Affiliations
- University of California, Berkeley (US)
- Drexel University (US)
- University of Groningen (NL)
- Corresponding author
- Fisher, Aaron J.
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2018-06-18
- Volume
- 115
- Issue
- 27
- Pages
- E6106-E6115
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC6142277
- OpenAlex
- W2808831456
- Cited by
- 1,099
- References
- 50
- Citations by year
- 2026: 126 · 2025: 159 · 2024: 132 · 2023: 141 · 2022: 139 · 2021: 137 · 2020: 135 · 2019: 106 · 2018: 21
- FWCI
- 82.78
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, other-oa)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, other-oa)
- Funders
- HHS | NIH | National Institute of Mental Health (1-DP5- OD021352)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Keywords
- within-person variability, interindividual variability, aggregation bias, individual differences
- MeSH terms
- Anxiety Disorders, Bipolar Disorder, Female, Humans, Male, Psychotherapy, Group
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
