The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Reliability and Validity of Ambulatory Cognitive Assessments
Open accessAbstract
Mobile technologies are increasingly used to measure cognitive function outside of traditional clinic and laboratory settings. Although ambulatory assessments of cognitive function conducted in people’s natural environments offer potential advantages over traditional assessment approaches, the psychometrics of cognitive assessment procedures have been understudied. We evaluated the reliability and construct validity of ambulatory assessments of working memory and perceptual speed administered via smartphones as part of an ecological momentary assessment protocol in a diverse adult sample ( N = 219). Results indicated excellent between-person reliability (≥0.97) for average scores, and evidence of reliable within-person variability across measurement occasions (0.41-0.53). The ambulatory tasks also exhibited construct validity, as evidence by their loadings on working memory and perceptual speed factors defined by the in-lab assessments. Our findings demonstrate that averaging across brief cognitive assessments made in uncontrolled naturalistic settings provide measurements that are comparable in reliability to assessments made in controlled laboratory environments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ecological momentary assessment
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.3.022806.091415 · Cited by 6,454
A capacity theory of comprehension: individual differences in working memory
Just, M. A., & Carpenter, P. A. (1992) · Psychological Review · doi:10.1037/0033-295x.99.1.122 · Cited by 4,153 · cited 1×
“…ability, which reflects the capacity to maintain information in active memory while simultaneously performing distracting or interfering activities (Just & Carpenter, 1992), is also predictive of developmental, aging, and realworld outcomes (Alloway & Alloway, 2010; SchneiderGarces et al., 2010).”
BackgroundWorking memory, short-term memory, and general fluid intelligence: A latent-variable approach.
Engle, R. W., Tuholski, S. W., Laughlin, J. E., & Conway, A. R. A. (1999) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.128.3.309 · Cited by 3,078
An integrated process model of stereotype threat effects on performance
Schmader, T., Johns, M., & Forbes, C. (2008) · Psychological Review · doi:10.1037/0033-295x.115.2.336 · Cited by 1,576
The Raven's Progressive Matrices: Change and Stability over Culture and Time
Raven, J. (2000) · Cognitive Psychology · doi:10.1006/cogp.1999.0735 · Cited by 1,560 · cited 2×
“Items in Raven’s Progressive Matrices tests consisted of a series of geometrical shapes and participants were asked to guess what the next shape should be among multiple options.”
MethodHeron, K. E., & Smyth, J. M. (2010) · British Journal of Health Psychology · doi:10.1348/135910709x466063 · Cited by 1,438
Self-citationUnsworth, N., & Engle, R. W. (2007) · Psychological Review · doi:10.1037/0033-295x.114.1.104 · Cited by 1,307
Investigating the predictive roles of working memory and IQ in academic attainment
Alloway, T. P., & Alloway, R. G. (2010) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2009.11.003 · Cited by 1,290
Miyake, A., Friedman, N. P., Rettinger, D. A., Shah, P., & Hegarty, M. (2001) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.130.4.621 · Cited by 1,139
Aging and measures of processing speed
Salthouse, T. A. (2000) · Biological Psychology · doi:10.1016/s0301-0511(00)00052-1 · Cited by 1,114
Conway, A. R. A., Cowan, N., Bunting, M. F., Therriault, D. J., & Minkoff, S. R. B. (2002) · Intelligence · doi:10.1016/s0160-2896(01)00096-4 · Cited by 1,108
Verhaeghen, P., & Salthouse, T. A. (1997) · Psychological Bulletin · doi:10.1037//0033-2909.122.3.231 · Cited by 1,078
Visual Processing Impairment and Risk of Motor Vehicle Crash Among Older Adults
Owsley, C. (1998) · JAMA · doi:10.1001/jama.279.14.1083 · Cited by 858
Cranford, J. A., Shrout, P. E., Iida, M., Rafaeli, E., Yip, T., & Bolger, N. (2006) · Personality and Social Psychology Bulletin · doi:10.1177/0146167206287721 · Cited by 856
Organismic achievement and environmental probability.
Brunswik, E. (1943) · Psychological Review · doi:10.1037/h0060889 · Cited by 706
The Handbook of Life‐Span Development
(2010) · Wiley eBooks · doi:10.1002/9780470880166 · Cited by 618
The wear-and-tear of daily stressors on mental health
Charles, S. T., Piazza, J. R., Mogle, J., Sliwinski, M. J., & Almeida, D. M. (2013) · Psychological Science · doi:10.1177/0956797612462222 · Cited by 459
Ecological momentary assessment research in behavioral medicine
Smyth, J. M., & Stone, A. A. (2003) · Journal of Happiness Studies · doi:10.1023/a:1023657221954 · Cited by 439
Self-citationNeuropsychological profiles of adolescents with ADHD: effects of reading difficulties and gender
Rucklidge, J. J., & Tannock, R. (2002) · Journal of Child Psychology and Psychiatry · doi:10.1111/1469-7610.00227 · Cited by 426 · cited 1×
“…compare different types of stimuli (e.g., numbers, symbols, patterns), is important for understanding child and adolescent cognitive development (Rucklidge & Tannock, 2002; Stenneken et al., 2011), cognitive aging (Salthouse, 2000), and is also predictive of real-world outcomes, such as job…”
BackgroundRam, N., & Gerstorf, D. (2009) · Psychology and Aging · doi:10.1037/a0017915 · Cited by 415
Span, CRUNCH, and Beyond: Working Memory Capacity and the Aging Brain
Schneider-Garces, N. J., Gordon, B. A., Brumback-Peltz, C. R., Shin, E., Lee, Y., Sutton, B. P., Maclin, E. L., Gratton, G., & Fabiani, M. (2010) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2009.21230 · Cited by 394
Unsworth, N., & Spillers, G. J. (2010) · Journal of Memory and Language · doi:10.1016/j.jml.2010.02.001 · Cited by 369
Neural underpinnings of within-person variability in cognitive functioning.
MacDonald, S. W. S., Li, S.-C., & Bäckman, L. (2009) · Psychology and Aging · doi:10.1037/a0017798 · Cited by 368
Hultsch, D. F., MacDonald, S. W. S., Hunter, M. A., Levy-Bencheton, J., & Strauss, E. (2000) · Neuropsychology · doi:10.1037//0894-4105.14.4.588 · Cited by 362
Measurement‐Burst Designs for Social Health Research
Sliwinski, M. J. (2008) · Social and Personality Psychology Compass · doi:10.1111/j.1751-9004.2007.00043.x · Cited by 333
Self-citationSPOONER, D., & PACHANA, N. (2006) · Archives of Clinical Neuropsychology · doi:10.1016/j.acn.2006.04.004 · Cited by 323
Ambulatory Assessment - Monitoring Behavior in Daily Life Settings
Fahrenberg, J., Myrtek, M., Pawlik, K., & Perrez, M. (2007) · European Journal of Psychological Assessment · doi:10.1027/1015-5759.23.4.206 · Cited by 317
Intraindividual coupling of daily stress and cognition.
Sliwinski, M. J., Smyth, J. M., Hofer, S. M., & Stawski, R. S. (2006) · Psychology and Aging · doi:10.1037/0882-7974.21.3.545 · Cited by 306
Self-citationHultsch, D. F., MacDonald, S. W. S., Hunter, M. A., Levy-Bencheton, J., & Strauss, E. (2000) · Neuropsychology · doi:10.1037/0894-4105.14.4.588 · Cited by 272
The reliability and stability of verbal working memory measures
Waters, G. S., & Caplan, D. (2003) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195534 · Cited by 257
The warp and the woof of the developmental fabric.
Nesselroade, J. R. (1991) · Cited by 254
Sliwinski, M. J., Almeida, D. M., Smyth, J., & Stawski, R. S. (2009) · Psychology and Aging · doi:10.1037/a0017925 · Cited by 241
Self-citationYoon, M., & Millsap, R. E. (2007) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705510701301677 · Cited by 219
The Effects of Stress on Cognitive Aging, Physiology and Emotion (ESCAPE) Project
Scott, S. B., Graham-Engeland, J. E., Engeland, C. G., Smyth, J. M., Almeida, D. M., Katz, M. J., Lipton, R. B., Mogle, J. A., Munoz, E., Ram, N., et al. (2015) · BMC Psychiatry · doi:10.1186/s12888-015-0497-7 · Cited by 197 · cited 1×
“…Effects of Stress on Cognitive Aging, Physiology and Emotion project is a prospective measurement burst study (Nesselroade, 1991; Sliwinski, 2008) that tracks variability and change in daily experiences of stress and cognitive function across four 14-day EMA measurement bursts (Scott et al., 2015).”
BackgroundSelf-citationSocial facilitation in motor tasks: a review of research and theory
Strauss, B. (2002) · Psychology of Sport and Exercise · doi:10.1016/s1469-0292(01)00019-x · Cited by 189
Brose, A., Schmiedek, F., Lövdén, M., & Lindenberger, U. (2012) · Emotion · doi:10.1037/a0024436 · Cited by 182
On Multilevel Model Reliability Estimation From the Perspective of Structural Equation Modeling
Raykov, T., & Marcoulides, G. A. (2006) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem1301_7 · Cited by 181
Schmiedek, F., Lövdén, M., & Lindenberger, U. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.01475 · Cited by 173
Nicotine withdrawal in chippers and regular smokers: Subjective and cognitive effects.
Shiffman, S., Paty, J. A., Gnys, M., & Kassel, J. D. (1995) · Health Psychology · doi:10.1037//0278-6133.14.4.301 · Cited by 160
Attentional bias to drug cues is elevated before and during temptations to use heroin and cocaine
Waters, A. J., Marhe, R., & Franken, I. H. A. (2011) · Psychopharmacology · doi:10.1007/s00213-011-2424-z · Cited by 132
Mobile Technologies in the Early Detection of Cognitive Decline
Michèle Allard, A., Husky, M. M., Catheline, G., Pelletier, A., Dilharreguy, B., Amieva, H., Pérès, K., Foubert‐Samier, A., Dartigues, J., & Swendsen, J. D. (2014) · PLoS ONE · doi:10.1371/journal.pone.0112197 · Cited by 119
Schmiedek, F., Li, S., & Lindenberger, U. (2009) · Psychology and Aging · doi:10.1037/a0014685 · Cited by 112
Natural history of nicotine withdrawal
Shiffman, S., Patten, C., Gwaltney, C., Paty, J., Gnys, M., Kassel, J., Hickcox, M., Waters, A., & Balabanis, M. (2006) · Addiction · doi:10.1111/j.1360-0443.2006.01635.x · Cited by 104
Daily fluctuations in positive affect positively co-vary with working memory performance.
Brose, A., Lövdén, M., & Schmiedek, F. (2013) · Emotion · doi:10.1037/a0035210 · Cited by 95
Fluctuations in elementary school children’s working memory performance in the school context.
Dirk, J., & Schmiedek, F. (2015) · Journal of Educational Psychology · doi:10.1037/edu0000076 · Cited by 79
Age-Related Differences in Working Memory Performance in A 2-Back Task
Wild–Wall, N., Falkenstein, M., & Gajewski, P. D. (2011) · Frontiers in Psychology · doi:10.3389/fpsyg.2011.00186 · Cited by 78
Stenneken, P., Bogon, J., Schulte‐Körne, G., Müller, H. J., Schneider, W. X., & Finke, K. (2011) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2011.08.021 · Cited by 69
Keeping It Steady: Older Adults Perform More Consistently on Cognitive Tasks Than Younger Adults
Schmiedek, F., Lövdén, M., & Lindenberger, U. (2013) · Psychological Science · doi:10.1177/0956797613479611 · Cited by 69
Riediger, M., Wrzus, C., Klipker, K. K., Müller, V., Schmiedek, F., & Wagner, G. G. (2014) · Psychology and Aging · doi:10.1037/a0035766 · Cited by 61
Waters, A. J., & Li, Y. (2007) · Psychopharmacology · doi:10.1007/s00213-007-1006-6 · Cited by 59
Ackerman, P. L., & Beier, M. E. (2007) · Journal of Experimental Psychology Applied · doi:10.1037/1076-898x.13.4.249 · Cited by 48
Tiplady, B., Oshinowo, B., Thomson, J., & Drummond, G. B. (2009) · Alcoholism Clinical and Experimental Research · doi:10.1111/j.1530-0277.2009.01049.x · Cited by 46
Ambulant Cognitive Assessment Using a Smartphone
Timmers, C., Maeghs, A., Vestjens, M., Bonnemayer, C. C. W., Hamers, H., & Blokland, A. (2013) · Applied Neuropsychology Adult · doi:10.1080/09084282.2013.778261 · Cited by 37
Investigating the structure and age invariance of episodic memory across the adult lifespan.
Siedlecki, K. L. (2007) · Psychology and Aging · doi:10.1037/0882-7974.22.2.251 · Cited by 35
Cited by
Current advances in digital cognitive assessment for preclinical Alzheimer's disease
Öhman, F., Hassenstab, J., Berron, D., Schöll, M., & Papp, K. V. (2021) · Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring · doi:10.1002/dad2.12217 · Cited by 214
Digital neuropsychology: challenges and opportunities at the intersection of science and software
Germine, L., Reinecke, K., & Chaytor, N. S. (2019) · The Clinical Neuropsychologist · doi:10.1080/13854046.2018.1535662 · Cited by 171
Smith, K. E., Mason, T. B., Johnson, J. S., Lavender, J. M., & Wonderlich, S. A. (2018) · International Journal of Eating Disorders · doi:10.1002/eat.22929 · Cited by 170
Zhaoyang, R., Martire, L. M., & Darnall, B. D. (2020) · Pain · doi:10.1097/j.pain.0000000000001959 · Cited by 105
Jongstra, S., Wijsman, L. W., Cachucho, R., Hoevenaar-Blom, M. P., Mooijaart, S. P., & Richard, E. (2017) · JMIR mHealth and uHealth · doi:10.2196/mhealth.6939 · Cited by 62
Metadata
- Authors
- Sliwinski, M. J., Mogle, J. A., Hyun, J., Munoz, E., Smyth, J. M., & Lipton, R. B.
- Venue
- Assessment
- Publisher
- SAGE Publications
- Published
- 2016-04-15
- Volume
- 25
- Issue
- 1
- Pages
- 14-30
- ISSN
- 1073-1911 (ISSN-L)
- 1552-3489
- OpenAlex
- W2338364226
- Cited by
- 357
- References
- 54
- FWCI
- 23.29
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.