Implementation considerations for multisite clinical trials with cognitive neuroscience tasks
Open accessAbstract
Multisite clinical trials aimed at cognitive enhancement across various neuropsychiatric conditions have employed standard neuropsychological tests as outcome measures. While these tests have enjoyed wide clinical use and have proven reliable and predictive of functional disability, a number of implementation challenges have arisen when these tests are used in clinical trials. These issues are likely to be magnified in future studies when cognitive neuroscience (CN) procedures are explored in these trials, because in their current forms CN procedures are less standardized and more difficult to teach and monitor. For multisite trials, we anticipate that the most challenging issues will include assuring tester competence, monitoring tester performance, specific challenges with complex assessment methods, and having resources available for adequate monitoring of data quality. Suggestions for overcoming these implementation challenges are offered.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The low power of completed trials increases the likelihood of a type II statistical error of concluding that a drug with true efficacy did not have beneficial effect.
“Our analysis shows that only one in 17 completed trials with available results had sufficient power to detect a medium (d = 0.5) effect size, thus increasing the likelihood of a type II statistical error of concluding that a drug with true efficacy did not have beneficial effect.54,55”
From Clinical trials of potential cognitive-enhancing drugs in schizophrenia: what have we learned so far? (2013) · cites this paper as [55]
Only one in 17 completed trials with available results had sufficient power to detect a medium (d = 0.5) effect size.
“Our analysis shows that only one in 17 completed trials with available results had sufficient power to detect a medium (d = 0.5) effect size, thus increasing the likelihood of a type II statistical error of concluding that a drug with true efficacy did not have beneficial effect.54,55”
From Clinical trials of potential cognitive-enhancing drugs in schizophrenia: what have we learned so far? (2013) · cites this paper as [55]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Keefe, R. (2004) · Schizophrenia Research · doi:10.1016/j.schres.2003.09.011 · Cited by 1,573 · see citing works
Dickinson, D., Ramsey, M. E., & Gold, J. M. (2007) · Archives of General Psychiatry · doi:10.1001/archpsyc.64.5.532 · Cited by 863 · see citing works
Keefe, R. S. E., Silva, S. G., Perkins, D. O., & Lieberman, J. A. (1999) · Schizophrenia Bulletin · doi:10.1093/oxfordjournals.schbul.a033374 · Cited by 690 · see citing works
Studies of Cognitive Change in Patients With Schizophrenia Following Novel Antipsychotic Treatment
Harvey, P. D., & Keefe, R. S. E. (2001) · American Journal of Psychiatry · doi:10.1176/appi.ajp.158.2.176 · Cited by 617 · see citing works
Baseline Neurocognitive Deficits in the CATIE Schizophrenia Trial
Keefe, R. S. E., Bilder, R. M., Harvey, P. D., Davis, S. M., Palmer, B. W., Gold, J. M., Meltzer, H. Y., Green, M. F., Miller, D. D., Canive, J. M., et al. (2006) · Neuropsychopharmacology · doi:10.1038/sj.npp.1301072 · Cited by 472 · see citing works
Cornblatt, B. A., Lenzenweger, M. F., & Erlenmeyer-Kimling, L. (1989) · Psychiatry Research · doi:10.1016/0165-1781(89)90188-1 · Cited by 399 · see citing works
Keefe, R. S. E., Seidman, L. J., Christensen, B. K., Hamer, R. M., Sharma, T., Sitskoorn, M. M., Lewine, R. R. J., Yurgelun-Todd, D. A., Gur, R. C., Tohen, M., et al. (2004) · American Journal of Psychiatry · doi:10.1176/appi.ajp.161.6.985 · Cited by 314 · see citing works
Self-citationHandbook for clinical memory assessment of older adults.
(1986) · American Psychological Association eBooks · doi:10.1037/10057-000 · Cited by 274 · see citing works
Harvey, P. D., Rabinowitz, J., Eerdekens, M., & Davidson, M. (2005) · American Journal of Psychiatry · doi:10.1176/appi.ajp.162.10.1888 · Cited by 161 · see citing works
Self-citationEffects of rivastigmine on cognitive function in patients with traumatic brain injury
Silver, J., Koumaras, B., Chen, M., Mirski, D. F., Potkin, S. G., Reyes, P. F., Warden, D. L., Harvey, P. D., Arciniegas, D. B., Katz, D. I., et al. (2006) · Neurology · doi:10.1212/01.wnl.0000234062.98062.e9 · Cited by 125 · see citing works
NIMH-MATRICS survey on assessment of neurocognition in schizophrenia
Kern, R. S., Green, M. F., Nuechterlein, K. H., & Deng, B.-H. (2004) · Schizophrenia Research · doi:10.1016/j.schres.2004.09.004 · Cited by 124 · see citing works
Dickinson, D., Ragland, J. D., Calkins, M. E., Gold, J. M., & Gur, R. C. (2006) · Schizophrenia Research · doi:10.1016/j.schres.2006.03.003 · Cited by 105 · see citing works
Cognitive impairment as a target for pharmacological treatment in schizophrenia
Davidson, M., & Keefe, R. S. E. (1995) · Schizophrenia Research · doi:10.1016/0920-9964(95)00037-m · Cited by 59 · see citing works
Hill, S. K., Sweeney, J. A., Hamer, R. M., Keefe, R. S. E., Perkins, D. O., Gu, H., McEvoy, J. P., & Lieberman, J. A. (2008) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617708080570 · Cited by 50 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Cognitive Impairment in Schizophrenia
Keefe, R. S. E., & Harvey, P. D. (2012) · Handbook of experimental pharmacology · doi:10.1007/978-3-642-25758-2_2 · Cited by 497 · see citing works
Assessing cognitive function in clinical trials of schizophrenia
Barnett, J. H., Robbins, T. W., Leeson, V. C., Sahakian, B. J., Joyce, E. M., & Blackwell, A. D. (2010) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2010.01.012 · Cited by 173 · see citing works
Keefe, R. S. E., Buchanan, R. W., Marder, S. R., Schooler, N. R., Dugar, A., Zivkov, M., & Stewart, M. (2013) · Schizophrenia Bulletin · doi:10.1093/schbul/sbr153 · Cited by 138 · see citing works
Barch, D. M., Carter, C. S., Arnsten, A. F. T., Buchanan, R. W., Cohen, J. D., Geyer, M. A., Green, M. F., Krystal, J. H., Nuechterlein, K. H., Robbins, T. W., et al. (2008) · Schizophrenia Bulletin · doi:10.1093/schbul/sbn163 · Cited by 133 · see citing works
CNTRICS Final Task Selection: Working Memory
Barch, D. M., Berman, M. G., Engle, R. F., Jones, J. H., Jonides, J. J., MacDonald, A. W., Nee, D. E., Redick, T. S., & Sponheim, S. R. (2008) · Schizophrenia Bulletin · doi:10.1093/schbul/sbn153 · Cited by 121 · see citing works
McIntosh, B. J., Zhang, X. Y., Kosten, T., Tan, S. P., Xiu, M. H., Rakofsky, J., & Harvey, P. D. (2011) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2011.01.012 · Cited by 58 · see citing works
Wu, L., Caprihan, A., Bustillo, J., Mayer, A. R., & Calhoun, V. D. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2018.06.024 · Cited by 57 · see citing works
van der Lee, J. H., Morton, J., Adams, H., Clarke, L. A., Ebbink, B. J., Escolar, M. L., Giugliani, R., Harmatz, P., Hogan, M. J., Jones, S. A., et al. (2017) · Molecular Genetics and Metabolism · doi:10.1016/j.ymgme.2017.05.004 · Cited by 47 · see citing works
A comparative study of the MATRICS and IntegNeuro cognitive assessment batteries
Silverstein, S. M., Jaeger, J., Donovan-Lepore, A.-M., WILKNISS, S. M., Savitz, A. J., Malinovsky, I., Hawthorne, D., Raines, S. R., Carson, S. J., Marcello, S. C., et al. (2010) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/13803391003596496 · Cited by 40 · see citing works
McLaughlin, P. M., Sunderland, K. M., Beaton, D., Binns, M. A., Kwan, D. T., Levine, B., Orange, J. B., Peltsch, A. J., Roberts, A., Strother, S. C., et al. (2020) · Assessment · doi:10.1177/1073191120913933 · Cited by 34 · see citing works
Memory-Prediction Errors and Their Consequences in Schizophrenia
Kraus, M., Keefe, R. S. E., & Krishnan, R. K. R. (2009) · Neuropsychology Review · doi:10.1007/s11065-009-9106-1 · Cited by 32 · see citing works
An Update on Treatment of Cognitive Impairment Associated with Schizophrenia
Horan, W. P., Catalano, L. T., & Green, M. F. (2022) · Current topics in behavioral neurosciences · doi:10.1007/7854_2022_382 · Cited by 31 · see citing works
Bosia, M., Buonocore, M., Bechi, M., Santarelli, L., Spangaro, M., Cocchi, F., Guglielmino, C., Bianchi, L., Bringheli, S., Bosinelli, F., et al. (2018) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2018.00647 · Cited by 30 · see citing works
Cognitive rehabilitation for patients with schizophrenia in Korea.
Lee, W., & Lee, W.-K. (2017) · Asian Journal of Psychiatry · doi:10.1016/j.ajp.2016.10.010 · Cited by 12 · see citing works
Buckow, K., Quade, M., Rienhoff, O., & Nussbeck, S. Y. (2014) · Neuroscience Research · doi:10.1016/j.neures.2014.08.005 · Cited by 9 · see citing works
Möller, H. ‐., Schaub, A., & Riedel, M. (2011) · Springer eBooks · doi:10.1007/978-3-642-03637-8_52 · Cited by 5 · see citing works
Schizophrenia and Nicotine - A Bittersweet Psychiatric Story
Ghosal, S., Halder, A., Keshri, S., Ravindran, N. P., & Joshi, A. (2024) · Annals of Indian Psychiatry · doi:10.4103/aip.aip_144_23 · Cited by 2 · see citing works
Keefe, R. S. E. (2010) · Cambridge University Press eBooks · doi:10.1017/cbo9780511712265.007 · Cited by 2 · see citing works
Ransing, R. S., Khairkar, P. H., Mishra, K. K., & Sakekar, G. S. (2017) · Journal of Neuropsychiatry · doi:10.1176/appi.neuropsych.16090163 · Cited by 2 · see citing works
Vogel, A. P. (2010) · Minerva Access (University of Melbourne) · Cited by 1 · see citing works
Assessment and treatment of cognitive impairment and related features
Harvey, P. D. (2013) · Cambridge University Press eBooks · doi:10.1017/cbo9781139003872.014
Oberleitner, R., Reischl, U., Gazieva, K. G., Nazneen, N., Suri, J. S., & Smith, C. J. (2017) · ScholarWorks (Boise State University)
Oberleitner, R., Reischl, U., Gazieva, K. G., Nazneen, N., Suri, J. S., & Smith, C. J. (2016) · CRC Press eBooks · doi:10.1201/9781315371375-18
Maximizing fidelity of neuropsychology assessments in fully remote studies
Fishman, K. N., McLaughlin, P. M., Tan, B., Troyer, A. K., Orange, J. B., Roberts, A., Kwan, D., Levine, B., Rashkovan, N., & Swartz, R. H. (2026) · Applied Neuropsychology Adult · doi:10.1080/23279095.2026.2637842
- Authors
- Keefe, R. S. E., & Harvey, P. D.
- Affiliations
- Duke University (US)
- Emory University (US)
- Venue
- Schizophrenia Bulletin
- Publisher
- Oxford University Press (OUP)
- Published
- 2007-11-21
- Volume
- 34
- Issue
- 4
- Pages
- 656-663
- ISSN
- 0586-7614 (ISSN-L)
- 1745-1701
- PMC
- PMC2632463
- OpenAlex
- W2112526545
- Cited by
- 24
- References
- 14
- Citations by year
- 2026: 1 · 2024: 1 · 2022: 1 · 2020: 1 · 2018: 2 · 2017: 3 · 2016: 2 · 2014: 1 · 2013: 1 · 2012: 1
- FWCI
- 3.54
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Schizophrenia Bulletin (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Bristol-Myers Squibb
- Eli Lilly and Company
- Pfizer
- Sanofi
- Dainippon Sumitomo Pharma
- H. Lundbeck A/S
- National Institute of Mental Health
- Topics
- Social Sciences › Psychology › Applied Psychology › Psychological Testing and Assessment
- Health Sciences › Medicine › Psychiatry and Mental health › Schizophrenia research and treatment
- Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
- Keywords
- cognitive enhancement
- MeSH terms
- Clinical Trials as Topic, Cognition Disorders, Humans, Neuropsychological Tests, Neurosciences, Psychometrics, Research, Research Design, Research Personnel, Schizophrenia, Schizophrenic Psychology, Task Performance and Analysis, Multicenter Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
