Multisite reliability of cognitive BOLD data
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
Investigators perform multi-site functional magnetic resonance imaging studies to increase statistical power, to enhance generalizability, and to improve the likelihood of sampling relevant subgroups. Yet undesired site variation in imaging methods could off-set these potential advantages. We used variance components analysis to investigate sources of variation in the blood oxygen level dependent (BOLD) signal across four 3T magnets in voxelwise and region of interest (ROI) analyses. Eighteen participants traveled to four magnet sites to complete eight runs of a working memory task involving emotional or neutral distraction. Person variance was more than 10 times larger than site variance for five of six ROIs studied. Person-by-site interactions, however, contributed sizable unwanted variance to the total. Averaging over runs increased between-site reliability, with many voxels showing good to excellent between-site reliability when eight runs were averaged and regions of interest showing fair to good reliability. Between-site reliability depended on the specific functional contrast analyzed in addition to the number of runs averaged. Although median effect size was correlated with between-site reliability, dissociations were observed for many voxels. Brain regions where the pooled effect size was large but between-site reliability was poor were associated with reduced individual differences. Brain regions where the pooled effect size was small but between-site reliability was excellent were associated with a balance of participants who displayed consistently positive or consistently negative BOLD responses. Although between-site reliability of BOLD data can be good to excellent, acquiring highly reliable data requires robust activation paradigms, ongoing quality assurance, and careful experimental control.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods
Jack, C. R., Bernstein, M. A., Fox, N. C., Thompson, P., Alexander, G., Harvey, D., Borowski, B., Britson, P. J., L. Whitwell, J., Ward, C., et al. (2008) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.21049 · Cited by 4,347
Susceptibility weighted imaging (SWI)
Haacke, E. M., Xu, Y., Cheng, Y. N., & Reichenbach, J. R. (2004) · Magnetic Resonance in Medicine · doi:10.1002/mrm.20198 · Cited by 1,582
Brennan, R. L. (2001) · doi:10.1007/978-1-4757-3456-0 · Cited by 1,121
Measuring fMRI reliability with the intra-class correlation coefficient
Caceres, A., Hall, D. L., Zelaya, F. O., Williams, S. C. R., & Mehta, M. A. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.12.035 · Cited by 255
Mumford, J. A., & Nichols, T. E. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2007.07.061 · Cited by 252 · cited 3×
“Because run averaging improved between-site reliability, the results of the present study have implications for study design, especially when considering the tradeoffs between the duration of a task and the number of subjects to be studied given a fixed budget (Mumford and Nichols, 2008 ).”
InfluentialCostafreda, S. G., Khanna, A., Mourao-Miranda, J., & Fu, C. H. Y. (2009) · NeuroReport · doi:10.1097/wnr.0b013e3283294159 · Cited by 129
Cited by
Yu, M., Linn, K. A., Cook, P. A., Phillips, M. L., McInnis, M., Fava, M., Trivedi, M. H., Weissman, M. M., Shinohara, R. T., & Sheline, Y. I. (2018) · Human Brain Mapping · doi:10.1002/hbm.24241 · Cited by 455
Needle, A. R., Lepley, A. S., & Grooms, D. R. (2017) · Sports Medicine · doi:10.1007/s40279-016-0666-y · Cited by 297
Interpreting and Utilising Intersubject Variability in Brain Function
Seghier, M. L., & Price, C. J. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.03.003 · Cited by 291
Potential Reporting Bias in fMRI Studies of the Brain
David, S. P., Ware, J. J., Chu, I. M., Loftus, P. D., Fusar-Poli, P., Radua, J., Munafò, M. R., & Ioannidis, J. P. A. (2013) · PLoS ONE · doi:10.1371/journal.pone.0070104 · Cited by 130
Thome, J., Ehlis, A.-C., Fallgatter, A. J., Krauel, K., Lange, K. W., Riederer, P., Romanos, M., Taurines, R., Tucha, O., Uzbekov, M., et al. (2012) · The World Journal of Biological Psychiatry · doi:10.3109/15622975.2012.690535 · Cited by 130
“So far there are very few studies that have focused on the reliability of functional imaging measures (Brown et al. 2011).”
Background
Metadata
- Authors
- Brown, G. G., Mathalon, D. H., Stern, H., Ford, J., Mueller, B., Greve, D. N., McCarthy, G., Voyvodic, J., Glover, G., Diaz, M., Yetter, E., Ozyurt, I. B., Jorgensen, K. W., Wible, C. G., Turner, J. A., Thompson, W. K., & Potkin, S. G.
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2011-02
- Volume
- 54
- Issue
- 3
- Pages
- 2163-2175
- ISSN
- 1053-8119 (ISSN-L)
- OpenAlex
- W2012586744
- Cited by
- 82
- References
- 7
- FWCI
- 4.58
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Multisite reliability of cognitive BOLD data. https://app.cochat.ai/papers/10.1016/j.neuroimage.2010.09.076
- The paper
- Brown, G. G., Mathalon, D. H., Stern, H., Ford, J., Mueller, B., Greve, D. N., McCarthy, G., Voyvodic, J., Glover, G., Diaz, M., Yetter, E., Ozyurt, I. B., Jorgensen, K. W., Wible, C. G., Turner, J. A., Thompson, W. K., & Potkin, S. G. (2011). Multisite reliability of cognitive BOLD data. NeuroImage. https://doi.org/10.1016/j.neuroimage.2010.09.076
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.