Video-Based Remote Administration of Cognitive Assessments and Interventions: a Comparison with In-Lab Administration
Open accessAbstract
While remote data collection is not a new concept, the quality and psychometric properties of data collected remotely often remain unclear. Most remote data collection is done via online survey tools or web-conferencing applications (i.e., Skype or Zoom) and largely involves questionnaires, interviews, or other self-report data. Little research has been done on the collection of cognitive assessments and interventions via web-conferencing that requires multiple sessions with or without the assistance of an experimenter. The present paper discusses limitations and challenges of studies administered remotely, and outlines methods used to overcome such challenges while effectively collecting cognitive performance data remotely via Zoom. We further discuss relative recruitment, retention rates, compliance, and performance findings between in-lab and remotely administered cognitive assessment and intervention studies, as well as limitations to remote data collection. We found that while it was necessary to recruit more participants in remote studies to reach enrollment goals, compliance and performance were largely comparable between in-lab and remotely administered studies, illustrating the opportunities of conducting this type of experimental research remotely with adequate fidelity.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A recent comparison of lab and remote videoconferencing cognitive assessment and training studies revealed similar compliance and performance rates.
“a recent comparison of cognitive assessment and training studies conducted in the lab and remotely via videoconferencing revealed similar compliance and performance rates78.”
From Near transfer to an unrelated N-back task mediates the effect of N-back working memory training on matrix reasoning (2022) · cites this paper as [78]
The Data Quality/Participant Performance section reports very similar training progress in the lab and remotely. But the Participant Compliance section reports that remote participants took significantly longer to complete training and were less compliant with the assigned schedule. The claim is supported for performance, not for similar compliance rates; page numbers were not available in our copy. (LLM, conf=0.92)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2004) · American Psychologist · doi:10.1037/0003-066x.59.2.93 · Cited by 2,932 · see citing works
McCambridge, J., Witton, J., & Elbourne, D. R. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.08.015 · Cited by 2,748 · see citing works
Comparing Telephone and Face-to-Face Qualitative Interviewing: a Research Note
Sturges, J. E., & Hanrahan, K. J. (2004) · Qualitative Research · doi:10.1177/1468794104041110 · Cited by 1,577 · see citing works
Germine, L., Nakayama, K., Duchaine, B. C., Chabris, C. F., Chatterjee, G., & Wilmer, J. B. (2012) · Psychonomic Bulletin & Review · doi:10.3758/s13423-012-0296-9 · Cited by 669 · see citing works
Online Data Collection: Strategies for Research
Granello, D. H., & Wheaton, J. E. (2004) · Journal of Counseling & Development · doi:10.1002/j.1556-6678.2004.tb00325.x · Cited by 547 · see citing works
Telemedicine in the Time of Coronavirus
Calton, B. A., Abedini, N. C., & Fratkin, M. D. (2020) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2020.03.019 · Cited by 464 · see citing works
Weigold, A., Weigold, I. K., & Russell, E. J. (2013) · Psychological Methods · doi:10.1037/a0031607 · Cited by 333 · see citing works
Comparing online and lab methods in a problem-solving experiment
Dandurand, F., Shultz, T. R., & Onishi, K. H. (2008) · Behavior Research Methods · doi:10.3758/brm.40.2.428 · Cited by 215 · see citing works
Online webcam-based eye tracking in cognitive science: A first look
Semmelmann, K., & Weigelt, S. (2017) · Behavior Research Methods · doi:10.3758/s13428-017-0913-7 · Cited by 198 · see citing works
Big Five traits as predictors of perceived stressfulness of the COVID-19 pandemic
Zacher, H., & Rudolph, C. W. (2021) · Personality and Individual Differences · doi:10.1016/j.paid.2021.110694 · Cited by 100 · see citing works
Buchanan, T., Ali, T., Heffernan, T. M., Ling, J., Parrott, A. C., Rodgers, J., & Scholey, A. B. (2005) · Behavior Research Methods · doi:10.3758/bf03206409 · Cited by 98 · see citing works
Comparing Response Distributions of Offline and Online
Schillewaert, N., & Meulemeester, P. (2005) · International Journal of Market Research · doi:10.1177/147078530504700203 · Cited by 94 · see citing works
Internet Administration of Paper-and-Pencil Questionnaires Used in Couple Research
Brock, R. L., Barry, R. A., Lawrence, E. E., Dey, J., & Rolffs, J. L. (2010) · Assessment · doi:10.1177/1073191110382850 · Cited by 72 · see citing works
de Larrea-Mancera, E. S. L., Stavropoulos, T., Hoover, E. C., et al. (2020) · The Journal of the Acoustical Society of America · doi:10.1121/10.0002108 · Cited by 52 · see citing works
Voice-only Skype for use in researching sensitive topics: a research note
Sipes, J., Roberts, L., & Mullan, B. (2019) · Qualitative Research in Psychology · doi:10.1080/14780887.2019.1577518 · Cited by 51 · see citing works
The Problem with Online Data Collection: Predicting Invalid Responding in Undergraduate Samples
Al-Salom, P., & Miller, C. J. (2017) · Current Psychology · doi:10.1007/s12144-017-9674-9 · Cited by 48 · see citing works
Ramani, G. B., Daubert, E. N., Lin, G. C., Kamarsu, S., Wodzinski, A., & Jaeggi, S. M. (2020) · Developmental Science · doi:10.1111/desc.12908 · Cited by 48 · see citing works
Self-citationGroup judgment processes and outcomes in video-conferencing versus face-to-face groups
Credé, M., & Sniezek, J. A. (2003) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2003.07.001 · Cited by 40 · see citing works
Gough, N., Brkan, L., Subramaniam, P., Chiuccariello, L., Petrillo, A., Mulsant, B. H., Bowie, C. R., & Rajji, T. K. (2020) · PLoS ONE · doi:10.1371/journal.pone.0223029 · Cited by 37 · see citing works
Using Skype to facilitate team-based qualitative research, including the process of data analysis
Quartiroli, A., Knight, S. M., Etzel, E. F., & Monaghan, M. M. (2017) · International Journal of Social Research Methodology · doi:10.1080/13645579.2016.1275371 · Cited by 24 · see citing works
UCancellation: A new mobile measure of selective attention and concentration
Pahor, A., Mester, R. E., Carrillo, A. A., Ghil, E., Reimer, J. F., Jaeggi, S. M., & Seitz, A. R. (2022) · Behavior Research Methods · doi:10.3758/s13428-021-01765-5 · Cited by 23 · see citing works
Self-citationBeginning of the Pandemic: COVID-19-Elicited Anxiety as a Predictor of Working Memory Performance
Fellman, D., Ritakallio, L., Waris, O., Jylkkä, J., & Laine, M. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.576466 · Cited by 21 · see citing works
Solís-Cordero, K., Lerner, R., Marinho, P., de Camargo, P., Takey, S., & Fujimori, E. (2021) · International Journal of Social Research Methodology · doi:10.1080/13645579.2021.1933067 · Cited by 6 · see citing works
de Larrea-Mancera, E. S. L., Stavropoulos, T., Carrillo, A. A., et al. (2021) · doi:10.31234/osf.io/9u68p · Cited by 5 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Pahor, A., Seitz, A. R., & Jaeggi, S. M. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01384-w · Cited by 34 · see citing works
Self-citation
- Authors
- Collins, C. L., Pina, A., Carrillo, A., Ghil, E., Smith-Peirce, R. N., Gomez, M., Okolo, P., Chen, Y., Pahor, A., Jaeggi, S. M., & Seitz, A. R.
- Affiliations
- University of California, Riverside (US)
- University of California, Irvine (US)
- Corresponding author
- Collins, Cindy L.
- Venue
- Journal of Cognitive Enhancement
- Published
- 2022-03-03
- Volume
- 6
- Issue
- 3
- Pages
- 316-326
- ISSN
- 2509-3290 (ISSN-L)
- 2509-3304
- PMC
- PMC8891736
- OpenAlex
- W4214807063
- Cited by
- 15
- References
- 24
- Citations by year
- 2026: 1 · 2025: 5 · 2024: 4 · 2023: 2 · 2022: 3
- FWCI
- 3.09
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Journal of Cognitive Enhancement (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Funders
- National Institute on Aging (K02AG054665)
- National Institute of Mental Health (R01MH111742)
- National Institute on Deafness and Other Communication Disorders (DC015051)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Abilities and Testing
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Keywords
- remote cognitive assessment, cognitive intervention, videoconferencing, Zoom, participant recruitment, participant retention, compliance, cognitive performance, remote data acquisition, psychometric properties
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
