The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Detecting Neuroimaging Biomarkers for Psychiatric Disorders: Sample Size Matters
Open accessAbstract
In a recent review, it was suggested that much larger cohorts are needed to prove the diagnostic value of neuroimaging biomarkers in psychiatry. While within a sample, an increase of diagnostic accuracy of schizophrenia (SZ) with number of subjects (N) has been shown, the relationship between N and accuracy is completely different between studies. Using data from a recent meta-analysis of machine learning (ML) in imaging SZ, we found that while low-N studies can reach 90% and higher accuracy, above N/2 = 50 the maximum accuracy achieved steadily drops to below 70% for N/2 > 150. We investigate the role N plays in the wide variability in accuracy results in SZ studies (63-97%). We hypothesize that the underlying cause of the decrease in accuracy with increasing N is sample heterogeneity. While smaller studies more easily include a homogeneous group of subjects (strict inclusion criteria are easily met; subjects live close to study site), larger studies inevitably need to relax the criteria/recruit from large geographic areas. A SZ prediction model based on a heterogeneous group of patients with presumably a heterogeneous pattern of structural or functional brain changes will not be able to capture the whole variety of changes, thus being limited to patterns shared by most patients. In addition to heterogeneity (sample size), we investigate other factors influencing accuracy and introduce a ML effect size. We derive a simple model of how the different factors, such as sample heterogeneity and study setup determine this ML effect size, and explain the variation in prediction accuracies found from the literature, both in cross-validation and independent sample testing. From this, we argue that smaller-N studies may reach high prediction accuracy at the cost of lower generalizability to other samples. Higher-N studies, on the other hand, will have more generalization power, but at the cost of lower accuracy. In conclusion, when comparing results from different ML studies, the sample sizes should be taken into account. To assess the generalizability of the models, validation (by direct application) of the prediction models should be tested in independent samples. The prediction of more complex measures such as outcome, which are expected to have an underlying pattern of more subtle brain abnormalities (lower effect size), will require large samples.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regier, D. A., Narrow, W. E., Clarke, D. E., Kraemer, H. C., Kuramoto, S. J., Kuhl, E. A., & Kupfer, D. J. (2013) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2012.12070999 · Cited by 1,163 · cited 1×
“(22), for a study on reliability of DSM-5 diagnoses, using interclass kappa (23)].”
High interrater reliability for the Structured Clinical Interview for DSM-III-R Axis I (SCID-I)
Skre, I., Onstad, S., Torgersen, S., & Kringlen, E. (1991) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.1991.tb03123.x · Cited by 353
Cited by
Machine Learning Approaches for Clinical Psychology and Psychiatry
Dwyer, D. B., Falkai, P., & Koutsouleris, N. (2018) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-032816-045037 · Cited by 966
Ten simple rules for predictive modeling of individual differences in neuroimaging
Scheinost, D., Noble, S., Horien, C., Greene, A. S., Lake, E. M., Salehi, M., Gao, S., Shen, X., O'Connor, D., Barron, D. S., et al. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.02.057 · Cited by 381
Han, L. K. M., Dinga, R., Hahn, T., Ching, C. R. K., Eyler, L. T., Aftanas, L., Aghajani, M., Aleman, A., Baune, B. T., Berger, K., et al. (2021) · Molecular Psychiatry · doi:10.1038/s41380-020-0754-0 · Cited by 334
Balki, I., Amirabadi, A., Levman, J., Martel, A. L., Emersic, Z., Meden, B., Garcia-Pedrero, A., Ramirez, S. C., Kong, D., Moody, A. R., et al. (2019) · Canadian Association of Radiologists Journal · doi:10.1016/j.carj.2019.06.002 · Cited by 299
Meehan, A. J., Lewis, S. J., Fazel, S., Fusar-Poli, P., Steyerberg, E. W., Stahl, D., & Danese, A. (2022) · Molecular Psychiatry · doi:10.1038/s41380-022-01528-4 · Cited by 219
Metadata
- Authors
- Schnack, H. G., & Kahn, R. S.
- Venue
- Frontiers in Psychiatry
- Publisher
- Frontiers Media SA
- Published
- 2016-03-31
- Volume
- 7
- Pages
- 50-50
- ISSN
- 1664-0640 (ISSN-L)
- OpenAlex
- W2322983841
- Cited by
- 229
- References
- 2
- FWCI
- 16.30
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Detecting Neuroimaging Biomarkers for Psychiatric Disorders: Sample Size Matters. https://app.cochat.ai/papers/10.3389/fpsyt.2016.00050
- The paper
- Schnack, H. G., & Kahn, R. S. (2016). Detecting Neuroimaging Biomarkers for Psychiatric Disorders: Sample Size Matters. Frontiers in Psychiatry. https://doi.org/10.3389/fpsyt.2016.00050
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.